Thursday, 4 July 2013

Lync Server 2013 CU2 Paired Pool Update

On Monday 1st July 2013 Microsoft released their second cumulative update for Lync server 2013. Unlike the first which involved breaking SQL mirrors for Enterprise edition it seems like Microsoft have listened and improved the Lync upgrade process; now fully supporting SQL mirrors and no need to rebuild your HA arrangements afterward.
However, all is not as simple as it may seem.

Technet guidelines on deploying the Lync CU's are causing confusion, the following is an extract from the site:

Lync Server 2013 Update Installer
"The front end servers in an Enterprise Edition pool are organized into upgrade domains. These upgrade domains are subsets of front end servers in the pool. Upgrade domains are created automatically by Topology Builder.
You must upgrade one upgrade domain at a time, and you must upgrade each front-end server in each upgrade domain. To do this, take one server in an upgrade domain offline, upgrade the server, and then restart it. Then, repeat this process for each server in the upgrade domain. Make sure that you record which upgrade domain and servers that you have upgraded".
Fairly straightforward so far. Unlike Lync 2010, 2013 uses the Windows fabric, organises its users into groups and its FE servers into upgrade domains; logical constructs which contain one or more FE servers. Use the following command to see which FE servers have been assigned to which upgrade domains (numbered 1,2,3 etc):
Get-CsPoolUpgradeReadinessState
If the command returns a status of True against each upgrade domain it is ready for upgrade. Proceed to upgrade each server listed in turn by firstly draining the server with:
Stop-CsWindowsService –Graceful
Then once active sessions on that server have ceased, launch the upgrade package, LyncServerUpdateInstaller.exe.
Once the upgrade on the first server is complete, restart all Lync services and verify it's status prior to proceeding onto the next server.
Once all servers within the first upgrade domain have been upgraded proceed to the next upgrade domain and repeat the above process.
However,Technet goes on to state the following if the status returned by upgrade readiness status is not True:

"If the State value of the pool is Busy, wait for 10 minutes, and then try to run the Get-CsPoolUpgradeReadinessStatecmdlet again. If you see Busy for at least three consecutive times after you wait 10 minutes in between each attempt, or if you see any result of InsufficientActiveFrontEnds for the State value of the pool, there is an issue with the pool. If you cannot resolve this issue, you may have to contact Microsoft Support. If this pool is paired with another front end pool in a disaster recovery topology, you must fail the pool over to the backup pool, and then update these servers in this pool".


It seems that this is being interpreted in one of two ways;
Firstly if uprgade readiness returns "busy" or anything other than "true" persists then you must contact Microsoft for assistance unless you are running paired pools in which case you can invoke failover and then commence the upgrade.
Secondly regardless of busy or true states for upgrade readiness, if you are running a paired pool topology then you must invoke a pool failover to be able to upgrade prior to failing the pool back to upgrade the second pool of the pair.
The first interpretation is correct and there's a supporting flowchart on Technet to confirm it.
If the upgrade readiness command returns true then regardless of anything else your environment can be updated. However, replies from Microsoft support for clarification have suggested that "you must fail over the FE pool to it's DR pair to perform the upgrade, then fail back and upgrade the DR pool". This is not correct and is in conflict with the procedure on Technet. Come on Microsoft, if your own people aren't on the same page here then how are we expected to manage.
Regardring the rest of the CU2 update, once all FE servers are upgraded you now have the remaining Lync infrastructure to tackle including the backend.

Microsoft have also posted a warning that if you install the CU2 update and then roll back to CU1 your Lync databases will revert to the RTM version, please see the link below for further details:

http://support.microsoft.com/kb/2819565

2 comments:

  1. Hello,

    Sorry for the late reply I have just returned from vacation.
    Having applied CU2 now on several deployments the best advice I can give is to follow Doug Deittericks blog over on technet - "How to apply Lync Server 2013 Cumulative Updates" at the following link:

    http://blogs.technet.com/b/dodeitte/archive/2013/02/27/how-to-apply-lync-server-2013-cumulative-updates.aspx

    The flow diagram is especially helpful in clarifying logic involved.

    In your situation the upgrade has failed. To proceed; failover the pool to your DR pool and then force an upgrade of all front end servers. You can failback once CU2 has applied.

    It doesn't get much more fun for the SQL mirrored backends either. An update was released in March which I'm just getting round to look at. The procedure is somewhat convoluted but there's logic in it:
    Step 1: Remove Witness server.
    Step 2: Failover to Mirror.
    Step 3: Upgrade previous Primary.
    Step 4: Failback to now upgraded Primary.
    Step 5: Upgrade Mirror.
    Step 6: Upgrade and reintroduce the Witness.
    Step 7: Hope it still works.

    Hope this helps.


    Regards.

    ReplyDelete
    Replies
    1. I should add that where you have enough Front ends in an Enterprise pool to preserve node majority when removing one for upgrade, the powershell will return an upgrade readiness state of "True" and you can proceed one FE at a time.

      Delete