SQL Clustering in a lab environment on a single pc…

Hmm, I guess this posting is going to hurt some people, but rest assured, this isn’t the purpose of this posting. It’s just that I want to share some experiences.

Suppose you want to play around with a SQL Cluster, based on Microsoft Failover Clustering. In production environments this won’t do. There isn’t any space left neither in the test labs of your company so you need an alternative. However, at home you don’t have any more a real lab as well (when the first kid was about to arrive I emptied my own personal data center in order to restyle it into a baby room…).

All you’ve got is a pc with 8 GB RAM, some really fast disks and a quad core CPU. Nowadays a basic configuration. The system also runs Windows 7 since it’s the home pc as well. So installing Windows Server 2008 R2 for Hyper-V isn’t an option since some people, who’re also using the pc for their home stuff, won’t like that at all.

So now what? And YES, the SQL Cluster must be build. No matter what. Performance isn’t key here.

Hmm. Gladly enough I had a license for VMware Workstation 8.0. (I really love my own network :) ). So I downloaded it, installed it and now it was time for some good searching on the internet. Soon I bumped into these postings, all about creating a SQL Cluster based on VMware Workstation, 3 VMs (one of them DC and a virtualized iSCSI SAN based on Microsoft iSCSI Software Target 3.3):

• http://tsells.wordpress.com/2011/04/22/how-to-setup-a-base-virtual-machine-for-use-in-vmware-workstation/
• http://tsells.wordpress.com/2011/04/22/how-to-create-a-virtual-iscsi-san-disk-with-vmware-workstation-and-server-2008-r2/
• http://tsells.wordpress.com/2011/04/22/how-to-setup-a-sql-failover-cluster-with-vmware-workstation/

And guess what? It all works very well. One advise though: when creating the virtual iSCSI SAN, there is an additional Step required in order to make it work:

• On the server hosting the iSCSI Target role one must open the Properties screen of the Target. The second tab enables one to add the iSCSI Initiators which are allowed to connect to the iSCSI Target. Without doing this the nodes won’t have access to the iSCSI Target. It took me some time to find this…

But now I can build my SQL Cluster:

It’s funny to see that the software of two companies who are fighting each other for market share can work so good together and produce very good results :).

New KB Article: How to gain access to SCOM after the SCOM Admin Group is deleted from AD

This is a real nightmare scenario and I am glad I never bumped into it.

But suppose the SCOM Admin Group is by accident deleted from AD. How do you gain access again to your SCOM environment?

Microsoft has published KB2640222 article about this kind of scenario and how to solve it in most cases.

Preparing for migrating to OM12: Moving from SQL 2005 to SQL 2008 – Part II: Phase I – The Preparations

----------------------------------------------------------------------------------
Postings in the same series:
Part   I – Along came a theory
Part III – The Migration 
----------------------------------------------------------------------------------
In the second posting of this series I will describe in more detail Phase I. When you haven’t read the first posting of this series please do so now since otherwise you’ll be missing out the big picture here.
For the completeness here is the list of questions/steps which make up Phase I:

1. Is the new SQL Server going to be physical or virtual? Also check what Microsoft officially recommends here;
2. Is the new SQL Server going to be clustered or not? Involve management and let them tell you how they look upon Monitoring. Is it business critical or not?
3. What version of SQL Server are you going to use (Standard or Enterprise)?
4. Provisioning of the new SQL Server, installing it with the features AND correct COLLATION settings required by OM12;
5. Preparing the security of the new SQL Server so SCOM can access it;
6. Adjusting the Master Database with a special script so it understand typical SCOM messages, piped into the event log;
7. Enabling CLR on the SQL Server so Group calculation works in SCOM R2;
8. Export of all custom RDLs residing on the current SQL 2005 server, used by SCOM;
9. Make sure the SCOM environment is healthy. Check SCOM (in the Console and OpsMgr event logs on the RMS and MS servers) and solve any serious issues before moving on to Phase II;
10. DON’T deploy a SCOM R2 Agent on this server!!!!
11. Make a list with the SCOM service accounts and their passwords, required in Phase II.

As stated before, it won’t be a detailed step-by-step guide about how to install SQL Server 2008 R2 SP1 CU#2, but I will highlight the most crucial steps in the processes. In the list above the steps which are printed in blue will be explained by me in more detail. Let’s start since there is a lot to share.

Step 4
When installing SQL Server 2008 R2 SP1 CU#2 also install the SQL Feature Full-Text Search:

and select the correct Collation Settings, SQL_Latin1_General_CP1_CI_AS:
 
Step 5
Logon to the SQL 2005 Server and open SQL Server Management Studio. Go to Server name > Security > Logins and look for the SCOM R2 Service Accounts (typically there are at least four of them: Action, SDK, Data Warehouse Write and Data Warehouse Read). It goes without saying all these accounts are AD accounts.

Write down the details. The most important ones here are the pages Server Roles (always Public, but check and double check!) and User Mapping. Make a screenshot of the latter page since it narrows down the margin of error compared to writing down the settings. Example:
  
Recreate the Logins on the new SQL 2008 R2 SP1 server. Of course, not all settings from User Mappings can be applied yet (the SCOM R2 databases aren’t there yet…) but at least the required Logins are in place. Example:
 

Step 6
Run this script against the Master database so it becomes aware of SCOM R2 specific events which can be logged in to the Windows Event log:
 

Step 7
Enabling CLR in order to have Group calculation working. Even though this must be run against the OperationsManager database itself, it’s good to have the required Stored Procedure already available:

sp_configure 'show advanced options', 1; GO RECONFIGURE; GO sp_configure 'clr enabled', 1; GO RECONFIGURE; GO

Step 8 Now all custom made Reports in SCOM must be exported.
Mind you, these aren’t the Reports you have saved to Favorites nor the Published Reports. Reports like that are Linked Reports, like shortcuts which refer to other existing Reports. These Linked Reports won’t be exported but have to be recreated later on.

Another approach here could be running these Reports one by one and save them to a MP of their own. Also the Reports which are present in the MPs won’t have to be exported since the MPs will upload them automatically to the new SSRS instance.

What I mean with Custom Reports are the ones which are made with Report Builder or BIDS for instance. Identify those Reports and their locations. A tool will be used to export the RDLs. It requires .NET Framework 4.0. so when it’s not there we have to install it. The stand alone installer can be found here for both architectures. See this posting of mine about how to use the tool to export the Reports.

Since ALL Reports will be exported, delete the ones which are present in the MPs and keep the custom made Reports. These will be uploaded later on into the new SSRS instance.

Step 9
Check the health of the SCOM environment. Make sure all is OK and nothing amiss. Check the SCOM Console for the health of the MG and also check all OpsMgr eventlogs on the SCOM R2 MS servers. Only when all is well, it’s time to move on to Phase II.

Conclusion
As you can see, there is a lot to do. But with Phase I you can take your time and prepare it step-by-step. In the next Phase the move of the SCOM R2 databases and SCOM R2 Reporting will take place. And this is the Phase where it all happens. Nice! See you’ll next time!

OM12: Antivirus exclusions

With SCOM one had to exclude certain files, folders and sometimes even extensions (be careful here!) in order to keep your SCOM environment in a smooth condition without being frustrated in its workings by antivirus software.

As far as I have seen this isn’t any different with Microsoft System Center 2012 – Operations Manager (OM12). However, with this version some folder locations are changed and when one is running SQL Server 2008 R2 the same issue is happening as well. Therefore this posting contains the updated folder locations, based on OM12 RC.

My personal guess is the folder locations won’t change in OM12 RTM but NEVER assume ALWAYS check and double check!

This posting is based on another posting, to be found here.

Processes
Before excluding any process from being scanned by your antivirus software know what you’re doing. As Kevin stated it on his blog (adjusted it to OM12 servers):

“…Excluding by process executable is very dangerous, in that it limits the control of scanning potentially dangerous files handled by the process, because it excludes any and all files involved. For this reason, unless absolutely necessary, we will not exclude any process executables in AV configurations for OM12 servers…”

For OM12 it’s still the same process: monitoringhost.exe. However, the location differs, per functionality:

1. OM12 Agent: ~:\Program Files\System Center Operations Manager\Agent;
2. OM12 Management Server: ~:\Program Files\System Center Operations Manager 2012\Server.

Folders
These folders are advised to be excluded from scanning by your antivirus software:

1. OM12 Agent: ~:\Program Files\System Center Operations Manager\Agent\Health Service State\Health Service Store;
2. OM12 Management Server: ~:\Program Files\System Center Operations Manager 2012\Server\Health Service State\Health Service Store;
3. SQL Server databases: ~:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA;
4. SQL Server log files: ~:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Log.

Extensions
These extensions are advised to be excluded from scanning by your antivirus software:

1. OM12 MS servers and Agents: EDB, CHK, LOG. These are the queue and log files of OM12;
2. SQL: MDF, LDF. These are the database and log files of SQL server.

New Base OS MP: Logical Disk Free space Alerts fixed by Kevin Holman & Larry Mosley

The new Base OS MP has been revamped in such a way to reduce a lot of noise, which is good. However, one thing which changed as well are the Alerts for logical disk free space. They don’t show anymore how much free space (in MBs and percentage) is left. Which isn’t good since now one doesn’t know how critical an Alert is.

Kevin Holman and Larry Mosley have therefore written a couple of addendum MPs which address this issue. However, DON’T just download these MPs and import them. First READ the blog posting of Kevin from top to bottom in order to get a full understanding of the consequences and (perhaps) the additional work they involve.

Want to know more? Go here.

OM12: Unsealed MPs are still VERY important, so EXPORT them on a regular basis

With OM12 Unsealed MPs still play a vital role, so it’s very important to export them and put them into the regular backup plan as well.

For SCOM there is a PS script available which enables one to create a Scheduled Task for it. It works great since it’s really Set & Forget. And every day one has a fresh export of all Unsealed MPs available.

I for myself use this script in every SCOM environment I bump into since it really adds so much value. And I run this script on ALL SCOM Management Servers, RMS and MS servers that is. This way – even when people forget to backup the export files of the Unsealed MPs – there are multiple locations where these files reside. And these files don’t take that much disk space so no issues there either.

But how to go about it in OM12? The good news is, this script still works! Even though one has to specify the RMS (duh!) it runs like clockwork. The only thing one has to take into account is that one targets the OM12 MS server which is installed first in the new OM12 MG. Since that server typically hosts the RMS Emulator role for backward compatibility with SCOM R2.

How the PS script works? Very straight forward. It consists out of two components: the PS script itself and a small cmd-file which contains the required parameters. I have attached the cmd-file and PS script at the end of this posting.  This is how one can configure it:

1. Create a folder on the OM 2012 server, like C:\_Backup;
2. Create within that folder two sub-folders: Scripts and UnsealedMPs;
3. Copy the PS script and cmd-file into the folder C:\_Backup\Scripts;
4. Modify the cmd-file so it contains the correct OM12 server which hosts the RMS Emulator role (<FQDN RMS EMULATOR>);
5. Testdrive the cmd-file in order to see all is well and the Unsealed MPs are indeed exported to the folder C:\_Backup\UnsealedMPs’
6. When all is OK, save the cmd-file;
7. Create a Scheduled Task which runs on a daily basis early in the morning;
• Select the SDK account for the Scheduled Task to run under so you’re sure it has the required permissions to access OM12.
8. Test the Scheduled Task in order to see all is well;
9. Repeat Steps 1 to 8 on every OM12 MS server and you’re fine;
10. Make sure the regular backups also take the folders C:\_Backup on all OM12 MS servers into account.

PS script and cmd-file can be downloaded from my SkyDrive.

Credits
I didn’t write these PS and cmd-files but found them years back on the internet. My guess is they came from SystemCenterForum.org so all credits go to them.

OM12 RC: Dude, how do I backup the Encryption Key? Answer: You don’t!

In SCOM and SCOM R2 installations it was crucial to backup the Encryption Key of your RMS server and store it safe (which isn’t on the RMS…). Basically what this key does is decrypting secure data present in the Operational Database. Without it, no RMS. Period.

When disaster strucks and renders your RMS useless, there is a way to promote (temporarily) a MS server to RMS in order to have your SCOM environment back in working condition. For this the Encryption Key is crucial. Without it, no promotion of a MS to RMS. (Yes, I know, there are some workarounds since R2 but you don’t want to go there.)

So far so good.

Q: But how about it in OM12? Do you still need to backup the Encryption Key?

A: Well, the RMS is gone with OM12 (thank you Microsoft) since the RMS functionality is now shared among ALL OM12 Management Servers. Along with it the requirement to backup the Encryption Key as well. The tool to run a backup of the Encryption Key isn’t present anymore. So no more backups of the Encryption Key in OM12.

Q: But what if I only run a single OM12 MS server and it dies?

A: Never ever install ONE OM12 MS server. Install at least TWO of them. And in the same period of time. So when you install your OM12 environment, install at least TWO OM12 MS servers on the same day before you proceed any further. Of course, for lab environments one OM12 MS server could do, but even there, TWO are the way to go since pools of OM12 MS servers are used by OM12. So with a single OM12 MS server, your lab environment would be way off compared to normal production situations (perhaps it’s the time to ask for a better lab environment??).

McAfee 8.8 and SCOM/OM: Not a happy marriage…

11-15-2011 Update
Yesterday evening McAfee released Patch 1 for McAfee VirusScan Enterprise 8.8:
- McAfee Downloads site - http://www.mcafee.com/us/downloads
- ServicePortal - https://mysupport.mcafee.com

11-14-2011 Update:
Got this from a McAfee expert: This issues only occurs when you have VSE8.8 installed AND disabled scriptscan. When that component is enabled, the issue mentioned in this posting won’t occur. McAfee also published a workaround for it, to be found here: https://kc.mcafee.com/corporate/index?page=content&id=KB71660,

Got this one from a respected colleague of mine:

McAfee 8.8 and SCOM/OM don’t mix well together. It turns out that almost ALL Discovery scripts run by the SCOM/OM12 Agent are blocked, even when you exclude the SCOM/OM Agent folders from scanning.

Because of that you will see screens like this in your OM12 RC environment:

However, in your SCOM environment you’ll see the same issue!

This issue will be solved with the release of Patch 1 which is available from today (11-15-2011). Or perhaps time to migrate to Microsoft Forefront :).

Operations Manager 2012 Release Candidate (OM12 RC) is publicly available

Today Microsoft released the Release Candidate for OM12.

Much is changed AND improved, checkout the new LOOK: 

Try OM12 RC now!!! But keep this in the back of your mind:

“Upgrades from RC to RTM in *any* scenario will not be supported by Microsoft support unless the customer is part of the TAP program for the specific product that they are trying to upgrade.  Only TAP customers are licensed to run pre-RTM software in production.” 

All documentation about OM12 RC can be found here.

Taken directly from the website where OM12 RC can be downloaded as well:

Feature Summary

• Setup
  Operations Manager 2012 has a new Setup wizard. For important instructions about how to install Operations Manager 2012, see Install Operations Manager 2012 Beta.
  
  
• Highly Available Management Group Out of the Box
  In Operations Manager 2012, all management servers are peers; there is no root management server. The workload is split among all management servers in a management group, which provides high availability without requiring a cluster.
  
  
• Resource pools
  A resource pool provides the ability to distribute workloads across multiple management servers, such as availability, network device monitoring, distributed monitor health rollup, and group calculation.
  
  
• Agent Configuration
  Operations Manager 2012 provides an easy method for configuring agents to report to multiple management servers by adding an Operations Manager Agent application to Control Panel on each agent-managed computer.
  
  
• Operations Console
  You will notice some subtle changes to the Operations console. The Actions pane is now the Tasks pane, and includes a new section called Navigation Tasks that makes it easy for you to open views for a selected object. The Tasks pane offers two tabs: one for actions and one for resources and Help links. The Navigation and Tasks panes can be hidden or revealed instantly by clicking the arrow in the title bar of the pane.
  
  
• Web console
  Operations Manager 2012 introduces a new Web console. In Operations Manager 2012, all Operations Manager views are available in the Web console.
  
  
• Network monitoring
  Operations Manager 2012 provides the ability to discover and monitor network routers and switches, including the network interfaces and ports on those devices and the virtual LAN (VLAN) that they participate in. You can also delete discovered network devices and prevent the deleted network devices from being rediscovered the next time discovery runs. For more information, see Monitor Network Devices.
  
  
• Application monitoring
  In Operations Manager 2012, you can monitor ASP.NET applications in server- and client-side environments to get details about application availability and performance. Configure monitoring settings, such as polling frequency and transaction threshold. Then use results, including how frequently a problem is occurring, how a server was performing when a problem occurred, and the distributed chain for a transaction in question to pinpoint problems and solutions. For more information, see Monitor a .NET Application.
  
  
• Dashboard views
  As part of the network monitoring and application monitoring capabilities, Operations Manager 2012 includes new comprehensive dashboard views that combine multiple panels of information into a single view. In Operations Manager 2012, you can add the new dashboard views to My Workspace.
  
  
• Display dashboard views using SharePoint
  The Operations Manager web part displays specified dashboard views and can be added to Microsoft SharePoint 2010 sites. For more information, see Add a Dashboard View to a SharePoint Site.
  
  
• Creating dashboard views
  Dashboard views have been significantly upgraded in Operations Manager 2012 from their capabilities in Operations Manager 2007 R2, including custom layouts and nested dashboard views. For more information, see Create a Dashboard View.
  
  
• Operations Manager Module for Windows PowerShell
  Operations Manager 2012 provides a Windows PowerShell 2.0 module containing a full set of new cmdlets. The cmdlets in this module are only compatible with Operations Manager 2012. You can recognize the Operations Manager 2012 cmdlets by the "SC" preceding the noun. For additional information about the Operations Manager 2012 cmdlets, open the Operations Manager command shell and type Get-Help about_OpsMgr_WhatsNew. For information about how the Operations Manager 2007 cmdlets map to the Operations Manager 2012 cmdlets, type Get-Help about_OpsMgr_Cmdlet_Names.To use the Operations Manager 2012 cmdlets, you must establish a connection to an Operations Manager management group. You can establish either a persistent connection in which you can run multiple cmdlets, or a temporary connection when running a single cmdlet. For more information about connections, open the Operations Manager Shell and type Get-Help about_OpsMgr_Connections.
  
  
• UNIX- and Linux-based computers
  In Operations Manager 2012, the Discovery Wizard is easier to use for discovering UNIX- and Linux-based computers. You can now use Windows PowerShell to manage UNIX- and Linux-based computers, for more information, see the UNIX and Linux section in the release notes. High availability is also supported.
  
  
• UNIX/Linux Shell Command Template Management Pack
  This Management Pack implements authoring templates that allow the creation of rules, tasks, and monitors based on execution of shell commands on UNIX/Linux agents.
  
  
• JEE Management Packs
  These management packs monitor JEE (Java Enterprise Edition) application servers. Management packs are available for IBM WebSphere, Oracle WebLogic, Red Hat JBoss and Apache Tomcat.

Preparing for migrating to OM12: Moving from SQL 2005 to SQL 2008 – Part I: Along came a theory

----------------------------------------------------------------------------------
Postings in the same series:
Part  II – The Preparations
Part III – The Migration  
----------------------------------------------------------------------------------

As stated before a new series of blog postings would come out, all about moving from SQL Server 2005 to SQL Server 2008 in order to prepare for migrating from SCOM R2 to OM12 RTM (expected in 2012).

In the first posting of this series I will go discuss the theory behind it, all about the required steps and procedures in order to make it a smooth transition with an absolute minimum amount of downtime for your SCOM R2 environment.

The Lab

Especially for this series I have build myself a new SCOM R2 CU#5 environment, based on two W2K03 SP2 servers.

One server is a DC for the AD and hosts the RMS Role. The second server is the SQL Server and hosts – besides both OpsMgr databases – the Reporting Server role for the SCOM MG as well.

Some basic MPs (Server OS, IIS and SQL) are imported and tuned and also some customized Reports are created which aren’t saved into a MP of their own but saved directly to the SSRS instance itself.

The Mission
The SQL 2005 Server will be replaced by a SQL 2008 R2 SP1 server, running on W2K08 R2 SP1. And of course, the SCOM R2 databases will be moved to this new server along with the customized Reports and their special folders.

The Theory, all about TLA and KIs
Basically this move to SQL Server 2008 R2 SP1 has three different phases. Each phase has its own characteristics, challenges and potential pitfalls.  So preparation is key here.

A typical case where usage of the Traffic Light Approach (TLA) comes in handy. TLA is really simple. Per phase some Key Indicators (KIs) are defined. And per KI there are three different states (sounds almost like Monitors in SCOM and believe me, the similarity doesn’t end here) :

• Green: All is well and a go ahead;
• Yellow: Warning. Some issues require additional attention, but on itself no show-stopper (yet!);
• Red: Something is not OK and must be taken care of before moving on to the next phase.

And yes, a couple of Yellow’s might end up as a single Red. So beware.

Phase I
This phase is a nice one since it doesn’t touch the SCOM R2 MG in any kind of way. So one has time to plan, discuss, design, install and tune. This phase is all about provisioning a new SQL 2008 R2 SP1 Server based on W2K08 R2 SP1. Yes, the latest technology is being used here, all because this server will be used for some years so it’s better to start with the latest RTM/GA technology instead of using technologies which are already a few years old.

On the other hand, this phase has some challenges of its own as well. The installation of SQL 2008 R2 SP1 on itself isn’t that hard. But more of a challenge will be what type of SQL server will be provided, like: stand-alone, clustered, VM or physical? Your SCOM R2 environment is based on decisions take a few years ago and nowadays other decisions might be taken. So make sure the correct type of SQL Server is provided.

This list of questions/steps do make up Phase I:

1. Is the new SQL Server going to be physical or virtual? Also check what Microsoft officially recommends here;
2. Is the new SQL Server going to be clustered or not? Involve management and let them tell you how they look upon Monitoring. Is it business critical or not?
3. What version of SQL Server are you going to use (Standard or Enterprise)?
4. Provisioning of the new SQL Server, installing it with the features AND correct COLLATION settings required by OM12;
5. Preparing the security of the new SQL Server so SCOM can access it;
6. Adjusting the Master Database with a special script so it understand typical SCOM messages, piped into the event log;
7. Enabling CLR on the SQL Server so Group calculation works in SCOM R2
8. Export of all custom RDLs residing on the current SQL 2005 server, used by SCOM;
9. Make sure the SCOM environment is healthy. Check SCOM (in the Console and OpsMgr event logs on the RMS and MS servers) and solve any serious issues before moving on to Phase II;
10. DON’T deploy a SCOM R2 Agent on this server!!!!
11. Make a list with the SCOM service accounts and their passwords, required in Phase II.
    

This phase has these five KIs:

• The correct collation settings;
• The correct SQL Features (required by OM12);
• The correct security settings for SCOM;
• The script run against the Master Database;
• CLR enabled on the SQL Server.

Only when all KIs are green, it’s time to move on to Phase II.

Phase II
This is the phase where things get real. The SCOM databases and Reporting functionality are moved from the current SQL Server to the one provisioned in Phase I. So this touches your SCOM R2 environment BIG time!

This list of steps do make up Phase II:

1. Backup of your SCOM environment (SQL, RMS, MS servers and databases). This is step enables the fall-back scenario and not Step 5 since the SCOM R2 environment is already affected by then;
2. Removal of the SCOM Reporting functionality (we KEEP the Data Warehouse of course!);
3. Stopping of the Health Service on RMS and all MS servers;
4. Stopping the Configuration and SDK service on the RMS;
5. Backup the SCOM databases, including OpsMgr, OpsMgrDW and Reporting (because no more data comes in these backups are the most current ones and won’t be outdated in any kind of way);
6. Stop the SQL Server service (and all other related SQL services) on the old SQL Server;
7. Restore of the SCOM databases OpsMgr and OpsMgrDW on the new SQL 2008 R2 SP1 server;
8. Adjustment of the registry keys on the RMS, so the new SQL server is used;
9. Adjustment of the registry keys on all MS servers, so the new SQL server is used;
10. Adjustment of some entries on both SCOM databases on the new SQL Server so the new SQL server is present in the database;
11. Starting all SCOM services on the RMS;
12. Checking the OpsMgr event log for any error;
13. Launching the OpsMgr Console on the RMS to see whether all is OK;
14. When all is well, starting the Health Service on the MS servers, one by one and checking the OpsMgr event logs on those servers whether all goes well;
15. Installation of SCOM Reporting on the new SQL server;
16. Installation of CU#5 on that SQL Server as well;
17. Checking SCOM Reporting;
18. Checking the successful upload of data into the Data Warehouse (OpsMgr event logs of the RMS and MS servers);
19. Restore of exported RDLs (Phase I, Step 8);
20. Checking whether all Reports show up again in SCOM (this might take an hour or so).

This phase has different KIs where every single KI is capable of creating a RED traffic light, thus stopping the whole Phase II in total…

• Step 1: these backups MUST be validated! Otherwise no fall back scenario is in place;
• Step 5: these backups MUST be validated as well. Otherwise no move to the new SQL Server can be made;
• Step 6: The restores must be successfully and give you healthy databases;
• Steps 8, 9 and 10 must be OK otherwise SCOM won’t work;
• Step 13: SCOM Console must be running OK otherwise SDK has having issues with the database connectivity;
• Step 14: Are the MS servers OK and in working condition;
• Step 16: Is Reporting successfully installed?
• Step 20: Is SCOM Reporting functioning properly?

Phase III
This phase is the wrap up. All is OK and running smoothly, otherwise TLA would have stopped you and kept you in Phase II. But since you’re here I presume you’re in a good shape.

This list of steps makes up Phase III:

1. Push a SCOM R2 Agent to the new SQL Server;
2. Run the SCOM R2 environment with its new SQL 2008 R2 SP1 server for a week and check it for any glitches;
3. Run some tests, like creating Reports, building Groups based on dynamic population (are these Groups properly populated afterwards) and so on;
4. Talk with the end-users of the SCOM environment in order to get an idea how they see it, ask open questions.

When all is well in these Steps, it’s time for some cleaning and sharing:

1. Decommission the old SQL server;
2. Make a high-level report for Management so they know what has been done with what results.

And now it’s time for a beer or glass of wine!

In the next posting I will write about preparing the new SQL server as described in Phase I. No, it won’t be detailed step-by-step guide but it will highlight the most important steps and referring to the sources, like where to get that script to be run against the Master database and so on. So stay tuned and see you all next time!

Third Party Management Packs: How Expensive They Can Get…

Wow! Bumped into this one today. A customer of mine is running a couple of F5 Load Balancers and wants them to be monitored by SCOM. Which is nothing but a normal request.

There are multiple ways to get this on the road:

1. Customization: build the required monitoring yourself based on SNMP, OIDs and the lot;
2. xSNMP Suite: Does not cover F5 out-of-the-box so additional customization is required;
3. Jalasoft: Covers F5 out-of-the-box. Doesn’t come cheap but offers really deep network monitoring, adding much to any SCOM environment;
4. F5: Delivers a MP as well.

Since budget is really something to reckon with, the Jalasoft solution seemed to be out of range. So Options 1,2 and 4 remained. However, time to deliver is short as well, so a quick-fix is required. Which left us with Options 2 and 4. Even though I like the xSNMP MP Suite, it doesn’t go deep enough for F5 LBs. So at the end – we thought – the MP from F5 remained, Option 4.

And to my knowledge F5 delivers this MP for free. So I started to search the internet and found it. At least, that’s what I thought. But when I wanted to download the MP I ended up with an Access Denied error:
 

Hmm, strange. Time for some more investigation. And soon I found the reason why the MP wasn’t to be downloaded anymore, at least for FREE that is:

Mind the sentence: ‘…The price of the Management Pack will be $10,000 US plus support and is shipping now….’

Say what?! A MP which was available fro FREE has gotten a price tag of 10K? Really incredible, also when one looks at the current economical situation. Companies are on tight budgets and some how somewhere in F5 there are some persons who think they can ask prices like these and SELL it as well.

Therefore I have advised my customer to abandon Option 4 and to take another look at Jalasoft when they want monitoring-out-of-the-box. Agreed, Jalasoft doesn’t come cheap, but won’t hit the 10K that fast AND adds more capabilities besides F5 monitoring as well, where as the F5 MP only offers F5 monitoring and stops there and you with empty pockets…

This really makes me wonder what some companies are thinking. Perhaps along these lines: ‘Hmm. The company has already spend some money on SCOM. So they have much more of it. Let’s ask ridiculous prices and see how many companies fall for it…’.

Conclusion
At the end of the day companies trying to sell MPs with prices like these make it only harder for themselves. Because nowadays customers don’t just buy hard- or software anymore. They also look at the costs of manageability of that hard- and software, thus taking it into account as well. And when it costs 10K (!) to monitor some hardware, even when it’s F5 LBs, customers will look for alternatives which are way cheaper ending up with LBs from other vendors. Don’t know where this fits into the business model of F5…

Coming Soon: A Way To Migrate From SQL Server 2005 to SQL Server 2008 (or higher) for SCOM

With OM12 being available as a public Beta, I get many questions about how to migrate from SCOM R2 to OM12 when it goes RTM (expected in 2012).

One of the requirements is that SCOM R2 runs its databases on SQL Server 2008 or higher. Many times however, people run SCOM R2 based on SQL Server 2005. Which is OK on itself, but a show stopper when migrating to OM12 RTM.

So I have given it good thoughts. And came to these conclusions:

There is another caveat to reckon with:

• Many times these SQL 2005 servers are running Windows 2003 server as well which isn’t supported by OM12 either. This introduces new challenges:
  • Updating Windows 2003 Server to Windows Server 2008 R2 isn’t possible, only like this: W2K03 > W2K08 > W2K08 R2. This is a scenario you don’t want to go into, at least I don’t. It’s too prone to error.
  • Some times these servers run a x86 version of the OS and SQL. However, x64 is the way to go these days. The time x64 was unreliable is long gone. And there isn’t an upgrade path from x86 to x64.

Other things which come into play and require serious thinking and answers are:

• Many times these SQL 2005 servers are physical entities. Which means they’re there for some years now. Is it really worth it to use this hardware again (think about support contracts and the like) or is this THE time to replace them;
  
  
• With OM12 the RMS is gone, so that SPOF is taken care of. But what about SQL server? When you run it now as a single server it’s a SPOF. Perhaps there were good reasons for it when SCOM was installed, but do those same reasons still apply today? Or has monitoring become more business critical, thus requiring a High Availability solution like a SQL Cluster?
  • Don’t decide this yourself. Involve management into it and let them make a (written) statement. This way is has been given proper attention.
    
    
• When SCOM was installed, virtualization was only starting. Now it’s much more common. Perhaps you can virtualize your SQL server for OM12? Of course, proper investigation is required, like:
• What does Microsoft say about it?
• What does your SAN administrators tell you about the IO and so on?
• What does your Hyper-V (duh!) administrators tell you about the performance of their VI?
• How many Objects are you going to monitor with OM12?
• And so on…

So based on these questions above, many times simply (NOT!) upgrading your current SQL 2005 Server to SQL Server 2008 (or higher) isn’t the way to go. Instead, another scenario comes into play, one where a NEW SQL server (or cluster!) is provided and uses as the new SQL server for your current SCOM R2 environment, thus paving the way to an upgrade to OM12 RTM.

And seriously, even when you don’t want to upgrade your SCOM R2 environment to OM12 but start fresh, there are still some good reasons to move to SQL 2008 (or higher), like better performance and the lot.

Therefore in a new series of postings I will take you all by the hand and move SCOM R2 databases from a SQL 2005 Server to SQL Server 2008 R2. So stay tuned!

Building Monitors: Why not to use ‘Critical’ status change all the time…

When people are using SCOM to build their own Monitors, they tend to use the ‘Critical’ status change too easy. It’s better not to use that status too often or without giving it much thought. 

Why? I know, of course, when people start to build Monitors of their own they deem something important enough to monitor it with SCOM. That goes without saying.

However, SCOM has a certain Health Model to reckon with. In that Health Model three different Health Statuses are present:

1. Healthy
   
   Color: Green. All is well and running smoothly. Server/Application/Service is functioning as expected.
   
   
2. Warning
   
   Color: Yellow. An issue has occurred. Something is not OK. However, the overall functionality nor availability is directly affected. Action is required though in order to prevent real outage/downtime.
   
   
3. Critical 
   
   Color: Red. A server/application/service is down and outage is happening. Functionality is severely affected. Immediate action is required.

This is the WHY…
Keep the last Status (Critical) in the back of your mind. For SCOM this is looked upon as REAL downtime (Critical State = Unavailable for SCOM!!!). So when you create a Monitor, targeted against a certain Class (like SQL Server for instance) and the Monitor raises an Alert and triggers a Critical State, SCOM will look upon it as the targeted SQL Servers being UNAVAILABLE.

So when you run a Report about the availability of your SQL servers, this Critical Alert will have a negative effect on the total percentage of the Availability of those SQL Servers… Which is a bad thing, especially when you only wanted to monitor something which is important but not CRITICAL to that SQL Server.

Conclusion
Be careful when creating Monitors and assigning Critical statuses to them. Think about the consequences when running availability Reports against those servers. Many times a Warning condition will do the trick as well without having a negative side-effect on the overall availability of those servers.

Tim McFadden Strikes Again

Tim McFadden, the one who created the awesome tool Remote Maintenance Mode Scheduler, has created a new tool which every SCOM admin should have: MP2XMLPRO Management Pack Conversion Tool.

Even though the name sounds very complicated for non-English native speaking people, the tool itself is very straightforward: it’s UI based and allows one to convert sealed MPs to XML files. On the FLY!

Screenshot taken directly from Tim’s website.

Want to know more? Go here.

SNMP, OIDs and Jalasoft Xian Network Manager Io

The last few weeks I am very busy with SNMP, IODs and SCOM. And as stated before, a whole new world has come into my reach.

Today I had the opportunity to combine it with Jalasoft Xian Network Manager Io. This network monitoring solution for SCOM is very powerful – out of the box – but also has the capabilities for creating custom rules, based on OIDs.

Even though it took me some time to master it, it’s very intuitive and straight forward. And the sky is the limit. These are the high level steps when creating custom Rules with Jalasoft (FIRST of all: Test the OIDs with a tool like GetIf):

1. Open the Console, go to Rules > Add Custom Rule;
2. Select the Plugin (basically, the type of network device the Rule is meant for);
3. The <PLUGIN> custom rule wizard is started;
4. The first tab, General allows one to add a name, title and description. I my case I used the OID description as the description for the Rule as well;
5. Rule Type (Performance, Incremental performance Rule or Status Rule) has to be selected;
6. The second tab Performance Parameters, allows for much tweaking and tuning. In my case I skipped it;
7. The third tab, Expression, requires the correct OID and a custom OID name;
8. The fourth tab, Alert data, allows to create your own Product Knowledge.

When all is well the Rule can be saved. Now the network device against which this Rule will be targeted must be opened. The Rule you created earlier must be added, configured and started.

In order to get everything up & running, the Rule must be exported as a MP. The Jalasoft Console enables this feature very nicely: go to Rules > Export Custom Rules. And a MP containing the custom Rule is created on the fly. Import it into your MG and you’re in business!

When you’ve created other custom Rules earlier, those will be exported as well into the same MP. So the MP is an aggregation of ALL the custom Rules you’ve created. Since it’s an unsealed MP it can be imported without any issues like running the same version number.

Conclusion
What strikes me most here, is the easiness to get things on the road and the possibilities of customizations. Really awesome. Again, when you have the OID – and tested it in order to know it really contains information(!) – you can monitor anything! And Jalasoft adds more possibilities to it, without requiring really deep knowledge of SNMP and OIDs. The UI of Jalasoft makes it really easy. Compliments to Jalasoft!