Thursday, February 21, 2013

OM12 & WS 2012: Making The OM12 Management Console High Available

Since OM12 Management Servers do share the same functionality which was hosted originally by the RMS in SCOM 2007 (RTM/SP1/R2), it’s easy to make the OM12 Management Console high available by using Network Load Balancing.

Yes, you’ll need at least two OM12 Management Servers for it. But IMHO – based on experiences out of the field – any serious OM12 MG should have at least two OM12 Management Servers. Otherwise the Resource Pools won’t be able to function properly.

This posting is based on OM12 SP1 and Windows Server 2012. Both OM12 SP1 MS servers do have only one NIC available, so it’s important to configure the NLB Cluster for multicast. When you don’t do that and use the default setting instead (unicast) the servers might become unreachable.

There is much to tell, so let’s start.

Step 01: Installing NLB on both OM12 MS servers
This is very easy since it’s all wizard driven. Yes, it can be done by PS as well of course. In this posting however I use the GUI for it.

  1. Open Server Manager > Manage > Add Roles and Features > a wizard kicks in now;
  2. Before you begin screen > Next;
  3. Select installation type screen > default selection Role-based or feature-based installation > Next;
  4. Select destination server screen > select the local server > Next;
  5. Select server roles screen > nothing to do here, since NLB is a feature > Next;
  6. Select features screen > select Network Load Balancing > Add Roles and Features Wizard screen appears now, about requiring extra tools to be installed > Add Features
    image
  7. Confirm installation selections screen > Install.
  8. The NLB functionality will be installed now and soon this message will appear: Installation succeeded on <FQDN>.
    image
    > Close.

Repeat these steps for the other OM12 SP1 MS server which will become part of the NLB Cluster.

Step 02: Creating a NLB Cluster
Now it’s time to create the actual NLB Cluster which will be configured in such a manner that it will reroute all the OM12 SP1 Management Console traffic (TCP 5724) to one of the members of the NLB Cluster.

Again, this can be done by PS, but I have configured it all by using the GUI.

  1. Open Server Manager > Tools > Network Load Balancing Manager > a wizard kicks in now;
  2. Click right on Network Load Balancing Clusters and select New Cluster;
    image
  3. Enter the name of the first server which is going to be part of this new NLB Cluster > Connect;
    image
  4. Now the available interfaces will be shown. Select the interface you’re going to use for this NLB Cluster > Next
    image
  5. This screen doesn’t require any modifications > Next;
    image
  6. > Add
    image
  7. Enter the IPv4 address that’s going to be used by the NLB Cluster. I used an IPv4 address in a range outside the IP addresses I use normally for my servers, so it’s easier to differentiate;
    image
    Enter the subnet mask > OK. The screen looks like this now:
    image
    > Next;
  8. Enter the FQDN of the NLB Cluster. In this example I used OM12Console.sc.local. Also select the option Multicast. With servers using a single NIC selecting Unicast will most likely render them inaccessible over the network… > Next;
    image
  9. Now the Port Rules are shown. By default one Port Rule is added but requires some additional attention. Otherwise NLB won’t work as expected…
    image
    > Edit > unselect All so only the IPv4 address of the NLB Cluster is shown;
    > modify Port range to 5724, used by the OM12 Management Console connection;
    > Set Protocols to TCP only;
    > Set Affinity to None;
    Now your screen should look like this:
    image
    > OK > Finish.
  10. Now the NLB Cluster will be created and the first NLB node added to it. This might take a few minutes and during this time your network connection will bounce a few times. But when the NLB Cluster is configured properly, the connection will be OK again;
  11. In the Network Load Balancing Manager screen the progress will be shown, when all is OK you should see something similar to this:
    image
  12. In the same screen you’ll see also this:
    image
    So you know for sure the NLB Cluster is OK and the first NLB node is up & running!

Before we add the second OM12 SP1 MS server to the NLB Cluster we’re going to do something else first: adding a new Host record on our DNS server so the IPv4 address is properly resolved.

Step 03: Creating a proper Host (A or AAA) record for the NLB Cluster
This is easy as well. Open the DNS snap-in and simply add a host record for the NLB Cluster. In this case the name is OM12Console (sc.local will be added automatically) and the IPv4 address is 192.168.137.200:
image

Before we add the second OM12 SP1 MS server to the NLB Cluster, it’s better to test the functionality of the NLB Cluster first. When only one NLB node is added, it’s easier to troubleshoot.

Step 04: Testing the NLB Cluster for the OM12 SP1 Management Console connection
Also easy. Simply add the feature Telnet Client to your server and start a cmd-prompt.

  1. Enter this command: telnet <FQDN of NLB Cluster> 5724 and hit enter.
    image
  2. When all is well you should see a black cmd-prompt screen, basically telling you the NLB Cluster is up and running and handling OM12 SP1 Management Console connections!
    image

Step 05: Adding the second OM12 SP1 MS server to the NLB Cluster
Yeah, I know, finally! But at least we know by now all is working as intended, which is very important.

  1. In Network Load Balancing Manager right click on the NLB Cluster you created earlier in Step 02  > Add Host to Cluster;
  2. Enter the name of the second OM12 SP1 MS server you want to add to the NLB Cluster > Connect > select the interface you want to use for the NLB Cluster > Next;
    image
  3. Don’t modify anything in this screen > Next;
    image
  4. The Port Rules are good as they are, so no modification required > Finish;
    image
  5. Now the second OM12 SP1 MS server will be added to the NLB Cluster. Again this can take some minutes and during this time your network connection will bounce a few times. But when the NLB Cluster is configured properly, the connection will be OK again;
  6. In the Network Load Balancing Manager screen the progress will be shown, when all is OK you should see something similar to this:
    image
  7. In the same screen you’ll see also this:
    image
    So you know for sure the whole NLB Cluster is OK and the both NLB nodes are up & running!

Step 06: The Real Word
So now we have our NLB Cluster in place. Let’s test it with the OM12 SP1 Management Console.

  1. Start the OM12 SP1 Management Console. By default it will connect to the OM12 SP1 MS server it connected to the last time;
  2. In the OM12 SP1 Management Console go to Tools > Connect;
    image
  3. Enter the FQDN of the NLB Cluster in the Server name box > Connect;
    image
  4. Since it’s the first time this connection is used it will take a few seconds extra, so be patient. But soon enough you’ll see this in the left bottom of the OM12 SP1 Management Console:
    image
  5. And then you’ll see this:
    image

Three tests to see it’s really working:

  1. Open a cmd-prompt, enter this command: netstat <enter>. Scroll through the list and you’ll see an entry like this:
    image
  2. In the OM12 SP1 Management Console go to Tools > Connect
    image
    You’ll see the Recent Connections (only the successful connections will be shown here!) and among them the FQDN of the NLB Cluster will be shown!
  3. The REAL TEST:
    1. Open an OM12 SP1 Management Console on a system which isn’t an OM12 SP1 MS server; 
    2. Connect the Console to the FQDN of the NLB Cluster;
    3. When the connection is made and the Console is working, look for EventID 26328, source OpsMgr SDK Service in the OpsMgr event logs of the OM12 SP1 MS servers in order to know to what OM12 SP1 MS server the Console is connected to;
    4. Stop the Data Access service (System Center Data Access Service) on that server and set it (temporarily!) to Manual so OM12 won’t start it for you;
    5. Go back to the OM12 SP1 Management Console. It will throw a SDK error;
    6. After a minute or so the OM12 SP1 Management Console will continue working since it’s reconnected to the other node of the NLB Cluster. Test it by clicking on any View in the Console;
    7. In order to check it, look for EventID 26328, source OpsMgr SDK Service in the OpsMgr event log of the OM12 SP1 MS which is the node of the NLB Cluster with the running Data Access service.
    8. Don’t forget to start the Data Access service again on the OM12 SP1 MS server where you stopped it previously (Step 3.4) and set it to start Automatically again.

Recap
Configuring NLB for the OM12 Management Console isn’t hard at all and will make your OM12 environment even more robust, without making huge investments. One thing to reckon with though: in this posting I used servers with only one NIC. In real life it’s better to use a dedicated NIC for it.

7 comments:

Erik said...


Hi Marnix,
What about kerberos, shouldn't you register the SPN for the NLB-name?
See this URL for some explanation : http://setspn.blogspot.nl/2009/01/configuring-kerberos-authentication.html

Regards,
Erik Luth

Marnix Wolf said...

Hi Erik.

Thanks for your comment. The post you refer to is all about IIS.

But the OM12 SP1 Management Console isn't IIS based at all. So this doesn't come into play in this scenario.

Cheers,
Marnix

Unknown said...

Hi Marnix - I was going to mention the same thing as Erik. I would have thought that an additional SPN should be registered to the SDK service for the new NLB name (FQDN and short name).

Good article though! Good to know that NLB works :)

Marnix Wolf said...

Hi Matt.

Thanks for your compliments. The most important thing here is the Configuration of the Port Rule and setting he NIC (when not using a dedicated NIC for your NLB Cluster that is) to Multicast.

I have tested it thoroughly and the posting works a full 100%. Really fun it was to figure it all out.

Cheers,
Marnix

John Bradshaw said...

Thx Marnix. This is very easy to follow.
Should it work OK if my servers are Windows 2008R2 ??
Cheers,
John Bradshaw

Marnix Wolf said...

Hi John.

Thanks for the compliments, much appreciated. Yes, this will work with Windows Server 2008 R2 as well. As far as I know NLB in Windows Server 2012 hasn't changed that much (if any at all) compared to Windows Server 2008 R2.

Cheers,
Marnix

Michiel Wouters said...

Nice walktrough. I've also implemented NLB's with OpsMgr. Don't forget that it's important to discuss the NLB settings with your network administrators to select the right NLB mode. You don't want to upset them with unexpected switch floods.

http://blogs.technet.com/b/networking/archive/2008/05/15/preparing-the-network-for-nlb-2008.aspx