Upgrading To SCOM 2016 GA – 02 – Let’s Upgrade!

In this posting I’ll write about my upgrade experiences of one of my SCOM 2012 R2 UR#11 environments to SCOM 2016 GA (SCOM 2016 + UR#01). The SCOM 2012 R2 environment is rather small, but still representative for many SCOM 2012 R2 environments, since it exists out of two SCOM Management Servers and a few SCOM Agents.

Because the upgrade process of a SCOM Agent to SCOM 2016 is the same, no matter the amount of them, IMHO the upgrade of my SCOM 2012 R2 UR#11 environment is applicable to many SCOM environments. The only thing lacking here is at least one SCOM Gateway Server. Since a Gateway Servers is in essence a SCOM Agent with some additional features, upgrading such a server is a walk in the park, especially compared to upgrading a SCOM Management Server.

This is my test lab:

1. 2x SCOM Management Servers;
2. No SCOM Gateway Servers;
3. 1x dedicated SQL instance, hosting both SCOM SQL databases;
4. 4x SCOM Agent.

Side note: Yes, it’s a small environment but it runs locally on my laptop, besides a SCCM environment providing FEP functionality for all VMs, an additional SQL server and a DC. So I think it’s still quite something for an average notebook .

All servers involved run Windows Server 2012 R2. SQL Server 2012 SP1 x64 is used for the SQL instance hosting the SCOM SQL databases.

There is much to tell, so let’s start.

A 01: Pre-Upgrade Tasks
NEVER EVER skip the Pre-Upgrade Tasks! Preparation is key, otherwise the upgrade is most likely to fail which is bad. And something else as well:

!!!BACKUP!!!
Before you start BACKUP! ALL SCOM Management Servers to be more specific. When on VMs, make snapshots or clones. And for the SCOM SQL databases: make VALID backups!

Microsoft recently published a TechNet article all about the Pre-Upgrade Tasks. So I won’t repeat them but only highlight some steps here. STILL TAKE CARE TO COVER ALL STEPS AS MENTIONED IN THIS TECHNET ARTICLE!!!

Steps I want to give additional attention are highlighted in yellow:

• Step 2: Clean the Database (ETL Table)
  This is serious business. In any real life SCOM environment this table may become quite big. In every day life one doesn’t notice this. But when upgrading this table will become the culprit of slow (ever lasting) upgrades and may eventually fail. So RUN the query in order to clean up that ETL table.
  
  
  
• Step 7: Stop & disable all related SCOM services on the Management Servers when not being upgraded
  This step makes perfect sense. Simply because when running the upgrade on a given SCOM Management Server, the underlying SCOM databases are also ‘touched’.
  
  Even more when it’s the first SCOM Management Server to be upgraded. In such a case the SCOM databases are upgraded as well, and when done, a flag is set so when the next SCOM Management Server is upgraded, the SCOM databases aren’t upgraded AGAIN…
  
  However, even when the SCOM databases are already upgraded, and another SCOM Management Server is upgraded, the SCOM databases are touched. In cases like that it’s better not having other SCOM Management Server running write/read actions on those very same databases in the same time.
  
  Therefore, only have the related SCOM services in a RUNNING state on the SCOM Management Server which is being upgraded, and on all other SCOM Management Servers, stopped & disabled.
  
  
• Step 8: OpsMgr SQL DB must have 50% or more free space
  This is very important. Check it in order to know for sure the OpsMgr SQL database has ENOUGH free space to run the upgrade.
  
  
  
• Step 11: Disable the Operations Manager website in IIS
  Since SCOM 2016 is in the process to ditch the Silverlight dependency in the SCOM Web Console, the SCOM 2012 R2 UR#11 Web Console gets an overhaul as well. Therefore it’s better to disable this website in IIS, so the upgrade process won’t find any locked handles or processes on it. Also enhancing the change of success of the upgrade.
  

A 02: Pre-Upgrade Steps I like to add
In addition to the earlier mentioned TechNet article there are a few additional pre-upgrade steps I would like to add as well here.

1. Install the SCOM 2016 Console prereqs on the SCOM Management Servers where the SCOM 2012 R2 UR#11 Console is also installed.
   
   Even though the SCOM 2016 installation/upgrade wizard checks for these prereqs:
   
   It’s better to avoid those Alerts and to install these two prereqs yourself in advance. So install Microsoft CLR Types for SQL Server 2014 and the SQL 2015 Report Viewer Controls in advance.
   
   
2. DOUBLE check the environment
   Meaning: Is your current SCOM 2012 R2 UR#11 environment upgradable? Do the underlying Windows Server OS’s AND SQL instances support SCOM 2016?
   
   Use this TechNet article to verify it. When some components aren’t supported in SCOM 2016, address those issues first before upgrading to SCOM 2016.
   
   
3. SCOM 2016 UR#01
   As stated earlier, UR#01 for SCOM 2016 brings it on GA level. So there is NO doubt whether or not to install those bits as well. SIMPLY INSTALL IT! So download the required bits and when your SCOM 2012 R2 UR#11 environment is upgraded to SCOM 2016, install UR#01 right after it.
   
   In order to save time and work, you could skip the SCOM Agents. Simply upgrade the SCOM 2012 R2 UR#11 Agents to SCOM 2016 UR#1 right away, since the SCOM 2012 R2 UR#11 Agents communicate with the SCOM 2016 UR#01 environment as well, thus buying you more time .
   
   So download the SCOM 2016 UR#01 bits before starting the upgrade to SCOM 2016 RTM.
   
   Also good to know: SCOM 2016 UR#01 only touches the SCOM Management Server, Console and Web Console. Not the SCOM Gateway Servers. The SCOM Agent is touched as well since the Agent staging folders on the SCOM Management Servers do get a MSP update package as well (KB3190029-amd64-Agent.msp for x64 and KB3190029-i386-Agent.msp for x86).
   
   
4. SCOM SDK Account
   When upgrading SCOM 2016 will ask for the SCOM SDK (Data Access) credentials. So make sure you’ve got them at hand.
   
   
5. Permissions
   Run the upgrade with an account which is local admin on the SCOM Management Servers to be upgraded, has SCOM admin permissions in SCOM and has SQL sysadmin permissions on the SQL instances hosting the SCOM SQL databases. Otherwise the upgrade will fail!

B 01: Upgrade – First SCOM Management Server
Now it’s time to start the upgrade and I start with the first SCOM 2012 R2 UR#11 Management Server, also hosting the RSME role. it also hosts the SCOM Web Console and the Console.

Additional information regarding the required UR level:
For the upgrade itself Microsoft has also recently published an updated TechNet article, to be found here. And as you can see, SCOM 2012 R2 doesn’t have to be on UR#11 level, since the upgrade can also be run from UR#9. My guess is however, that most SCOM 2012 R2 environments will be on UR#11 already. When it’s not, but on UR#9 level you don’t have to roll out UR#11 first. Just make sure you meet all requirements and go through all pre-upgrade tasks successfully. Then you’re ready to upgrade to SCOM 2016 just as well.

I won’t post all screenshots, but only the most important ones. And apologies for the lack of quality of those very same screenshots. I recorded the upgrade with the built-in Steps Recorder, not knowing the screens are saved in a lower quality .

And YES I’ve stopped & disabled all SCOM services on all other SCOM Management Servers which aren’t upgraded at that moment.

• SCOM 2016 recognizes the presence of multiple SCOM 2012 R2 components and will update them accordingly to SCOM 2016:
  
• Upgrade is running:
  
• Upgrade ran successfully on all SCOM 2012 R2 components. Please know that the upgrade of the SCOM 2012 databases takes a long time. And also know that the new license isn’t set, so SCOM 2016 is en eval mode. Therefore a warning is shown for the SCOM Management Server, highlighted in yellow:
  

B 02: Upgrade – Second SCOM Management Server
This upgrade runs much faster since the related SCOM databases are already successfully upgraded (as such the upgrade flags are set in those databases, telling the upgrade to skip them) and this second SCOM Management Server only runs the Console, NOT the Web Console.

Since I stopped & disabled the SCOM services here before upgrading the first SCOM Management Server, I start and set them to start automatically them BEFORE upgrading this SCOM Management Server and STOP & DISABLE them on the FIRST SCOM Management Server, which is already upgraded. DON’T FORGET THIS!!!

• SCOM 2016 recognizes all SCOM 2012 R2 components to be upgraded:
  
• Upgrade ran successfully:
  

Now I start the SCOM services on the first SCOM Management Server and set them to start automatically. Also I start the SCOM website in IIS. Now the most crucial SCOM components are upgraded to SCOM 2016 RTM. There are no SCOM Gateway Servers in my environment to upgrade .

B 03: Upgrade – Install SCOM 2016 UR#01
Now it’s time to install SCOM 2016 UR#01. The order of it is like all other URs for SCOM 2012 R2:

1. Management Servers;
2. Gateway Servers (not applicable, since SCOM 2016 UR#01 doesn’t touch them);
3. SCOM Console(s);
4. SCOM Web Console;
5. SQL query (SCOM 2016 UR#01 only touches the OpsMgr database);
6. SCOM Management Packs;
7. SCOM Agents (when running Gateways, copy the MSP files to their Agent staging folders!).

So I start with the first SCOM Management Server which also hosts the SCOM Web Console and Console. In this case these SCOM 2016 components are touched:

• SCOM Management Server;
• SCOM Web Console;
• SCOM Console.

After this I upgrade the second SCOM Management Server, also hosting the Console.

In this case these SCOM 2016 components are touched:

• SCOM Management Server;
• SCOM Console.

Then I run the SQL query (as stated earlier, only the OpsMgr database is touched):

After that I import the SCOM 2016 core MPs:

And now – except for the SCOM Agents that is (still on SCOM 2012 R2 UR#11 level) – SCOM is on 2016 UR#01 level.

C 01 – Upgrade – The Aftermath
Now it’s time to wrap it all up with these steps:

1. Upgrade SCOM Agents to SCOM 2016 UR#01
   Upgrade them in batches. When installed from the Console upgrade them from there as well. For all other manually installed Agents, update them manually or automated by GPO, SCCM or really manually.
   
   
2. Install the license
   Go from eval to retail by installing the SCOM license. Fun fact: The same SCOM 2012 R2 key is used by SCOM 2016 as well…
   
   
   
3. Upgrade Helper MP
   I used Wei H Lim’s excellent Upgrade Helper MP in order to ascertain myself EVERYTHING is on SCOM 2016 level. Both dashboards are viewed, so I am 100% sure all is okay:
   
   
   And:
   
   
   And:
   
   Some Agents have issues, but they are on SCOM 2016 UR#01 level none the less.
   
   
4. Remove the Upgrade Helper MP
   When you’re 100% sure all components are on SCOM 2016 level, you can safely remove the Upgrade Helper MP.

Wrap up
Since SCOM 2016 isn’t that much of a change compared to SCOM 2012 R2, the upgrade is less likely to fail, moreover when you’re sure all components (underlying Windows Server OS and SQL instances included) meet the SCOM 2016 requirements AND you respect the pre-upgrade tasks.

Compared to all other SCOM upgrades I have done before this was the easiest one. None the less: PREPARATION is KEY!!!

Also: when your current SCOM 2012 R2 environment comes from SCOM 2012 SP1 or even older, THINK TWICE before upgrading to SCOM 2016. Changes are things will break during the upgrade, so seriously consider the along side scenario in which a new SCOM 2016 environment is rolled out alongside your current SCOM 2012 R2 environment and monitoring is gradually moved to the new SCOM 2016 UR#01 environment.