Vital Signs – Part III: Sherlock Holmes is back!

----------------------------------------------------------------------------------
Postings in the same series:
Part  I – Teaser/Introduction
Part II – The Installation
----------------------------------------------------------------------------------

In the last posting of this series I will share my personal experiences with Savision Vital Signs. There is much to say so let’s start.

What the f-word is going on!??
Many of us will recognize situations like these:

One moment all is well and all ICT systems are humming quietly. Users are happy, applications are functional, responsive and live is good. SCOM is quiet, not too much stuff coming in and what comes in is nothing special. Nice! But suddenly, out the blue, things start to go sour. A couple of phone calls come in – users complaining about lost connections, frozen applications, slow responses – and SCOM start to generate more Alerts. Application owners are pacing up there steps, worried looks on the faces of the DBA’s. The door of the ICT Managers office swings open, people rush in and out. Wow! ‘Houston, we’ve got a situation!’.

But where to start? Yes, SCOM has thrown some or even many serious Alerts, some DAs are painted red from top to bottom, computers are in red condition, network devices come and go, SQL boxes are having issues as well. But hey, what are the causes of this? SCOM will certainly aid you in pinpointing the direction(s) where to look, up to the level you have configured it, but today’s applications/servers/services are many times spread across multiple locations, servers/hosts, components so even when you know where to look, some specific drill downs are still required.

Like getting answers to: OK, SQL boxes X, Y and Z are having issues. But why? Servers D,E and F are troublesome as well. But why? SCOM gives good insight of what servers, services, sql boxes, cpu’s, disks, RAM and the lot which are having issues. But why is the cpu running 100% on the sql-server process? Why is the RAM being consumed for 99,99% by the IIS process? Why are the disk queue lengths as long as a dogs tail?

And this is where Vital Signs comes in. Like hiring a detective, a Sherlock-Holmes-in-the-box!

When hiring a detective you always have an assignment. Not like ‘Duh! I am hiring you because you’re a detective and figure it out yourself whether I need your services and for what!’. No. With Vital Signs it’s the same. You have some ‘unsubs’ which need further investigation. You aim Vital Signs to those unsubs (servers) and it will do the rest.

The Science of Deduction and Analysis
So we know where to look. Vital Signs can take a deep dive into the Windows Server OS and SQL (Hyper-V as well since a few days, so it’s gaining momentum!), lets start.

SV01 is – besides a W2K08 R2 server – also a SQL box. Server SV02 is a W2K03 box. I have added both to Vital Signs:
 
Oops! That’s not good! SV01 (SQL) is not OK. Let’s drill down:

So the SQL Engine is not having a good time? Let’s check it out (notice the yellow high lighted area):

Wouldn’t it be nice to have a bigger graph of it? So let’s drop & drag it to the middle pane!

Wow! Let’s zoom in more. Now we know what's wrong, but not what’s causing it. Let’s zoom in a little bit more. The yellow highlighted area’s contain more information:

Now it’s time to take a deeper dive into the SQL databases itself:
 

Select a database and the lower pane will show this:

Also note the tabs, revealing more information about a selected database:
 

So now I can look INTO the database itself. Just drag & drop, and you will see what’s happening. On top if it all, the connection with SCOM aides you in relating issues to Alerts. Also the additional information about the databases is a good help.

Vital Signs also enables you to see whether the issue you’re investigating is something of the past 24 hours or not. Just adjust this time slider and the Views will be adjusted on the fly!

The same goes for the Microsoft Windows Server dashboards. They offer the same flexibility, drag & drop features, zooming into details and easiness of usage. Some screenshots:

Conclusion:
The days running Perfmon in order to get to the bottom of a nasty situation, are gone. Where Perfmon stops, Vital Signs kicks in! It takes a deep dive into a product and shows what’s happening, real-time. If needed, you can take a deep dive into older data in order to pinpoint it even more. The connection with SCOM, relating Alerts with certain changes in measured performance is really a huge help in order to crack a tough situation. Also all the information to be found in the single console of Vital Signs makes it easier to trouble shoot. No more need to open many consoles. Vital Signs provides much solid information.

So where SCOM shows useful state information, Vital Signs delivers deep and detailed performance monitoring. Bringing these two things together delivers much added value.

For now Vital Signs covers these three products: SQL, Windows Server and Hyper-V. But when looking into the installation folder of the software this is what I found:

So there is more to come. Vital Signs is a framework after all, enabling Savision to deliver more and more product-specific dashboards. Nice! Can’t wait to see the latest releases.

Wish Vital Signs was there in my days being a Systems Engineer! Would have most certainly helped me out in many situations. So test drive it and be impressed, just like me. No rocket science needed to get it up & running.

How To: Seal a MP

Sooner or later any SCOM Admin will bump into a situation where sealing a MP is required. But how to go about it?

There was some documentation to be found on that topic but it wasn’t very complete. So it’s good to know that Brian Wren has written a rock solid TechNet article about just that, covering it from tip to toe.

So whenever you need to seal a MP and don’t know how to go about it, go here.

MP Catalog: It’s improved. Really!

About a year ago Microsoft revamped the MP Catalog. Also the name was changed to PinPoint.

But it wasn’t a real improvement. Searching for a MP on the new website was like searching for a needle in a haystack. So many customers of mine didn’t use PinPoint that much anymore and rebranded it into PainPoint. Which is bad, because Microsoft moved the MP Catalog to a new website for some very good reasons. One of them was to bring all System Center related products together in a single place on the web.

But the complaints were taken seriously by Microsoft and many improvements were done in order to get it right. So PinPoint was becoming better every month. But none of those improvements are like the most recent one. For a day now PinPoint has been rebranded into Microsoft System Center Marketplace:

What it does? As Microsoft states: ‘…is specifically designed to help business customers find trusted Microsoft System Center experts, and applications and services that enhance and easily integrate with Microsoft System Center products…’

Besides the looks many things have been changed for the better, also searching for a MP!

Follow this small procedure and find any MP you need:

1. Under the header Search by Product, click on Operations Manager;
   
2. In the Search bar, enter any MP (like DNS, IIS, SQL, AD, WINS, Forefront, Exchange and Lync), hit the Search button and be surprised!
   

When you want to know what the latest MP is, follow this small procedure:

1. Under the header Search by Product, click on Operations Manager;
   
2. Click on the button Release Date and the latest MP will be shown on top.
   

So the MP Catalog is back in business with a lot of added features! Nice!

When a simple copy & paste action fails…

Bummer!

Had a situation where a server didn’t accept a push installation of a SCOM R2 Agent. Error Code: 80070102 was thrown:
 

Contents of error message:

The Operations Manager Server could not start the MOMAgentInstaller service on computer "xxxxxxxxxxxxxxxx" in the time.  This service is used to perform configuration operations on the computer before the System Center Operations Manager agent can be configured. Operation: Agent Install Install account: xxxxxxxxxxxxxxxxxxx Error Code: 80070102 Error Description: (null)

Time to visit Kevin’s blog with the Console based Agent Deployment Troubleshooting table posting. And yes, there it was and how to remedy it:

So I opened an elevated cmd-prompt on the problematic server (W2K08 R2) and ran it. All to no avail. Only an error was shown. When the firewall console was opened, it showed the rule! Somehow the syntax didn’t work out just fine. Checked it, and the command seemed to be OK. So the right syntax was used.

Now what? I copied the same line from the blog again and pasted it in Notepad, in order to take a better look at it. This is what I found:

The quotes are not OK! Time to correct them:

Copied it, ran it on the same server from an elevated cmd-prompt. Now I got the message that two rules were deleted successfully. Pushed the Agent and now all went just fine!

Lesson learned:
Everybody can simply copy & paste anything, anytime. It’s better to use your own brains as well however especially when quotes are involved…

So the posting of Kevin was correct (what else?) but sometimes the quotes end up garbled on web pages. Have seen similar issues before. At the end I felt a bit stupid about it…

Error when removing DA: ‘Verification failed with [1] errors’ and ‘Cannot verify RelationshipType’ and how to solve it.

Issue
Got these errors when I tried to remove a DA. Normally these errors are thrown when you try to remove a DA or a component which is part of (other) DAs. But that wasn’t the case here.

Not the GUI (DAD)…
So I tried to solve it through the GUI by emptying the DA and save the changes. That worked. Time for the next step, removing the DA totally. But again the above mentioned errors were thrown.

When I searched in the GUI for Monitors and Classes with the names of the components which were present in the DA, I found them! Strange it was since I removed them from the DA and these components weren’t used anywhere else.

Good ol’ Notepad++
Bummer! Time for some XML editing. So I exported the MP containing the problematic DA. Opened it in Notepad++ and searched for the earlier mentioned components, which were present in that same DA. And yes, I found them. But the name of those components won’t do. The GUIDs are far more important since the SCOM Console many times uses GUIDs instead of friendly names.

How to find the related GUID? Good question. First you search in Notepad++ for the friendly name. When that entry is found you will find an entry above that name with the tag <DisplayString ElementID="Service_<GUID>">

Back in business
Copy the whole entry (Service_<GUID>) and search for that entry in Notepad++. Remove all those entries, save the changes and import the MP. Now the problematic DA can be removed without any hassle. Of course, the DA can be removed from the XML code as well.

Updated MP: SQL MP, version 6.1.400.0

A few days ago Microsoft released a minor update of the SQL MP, version 6.1.400.0.

Not much has changed (that’s why it’s called a minor update…):

Even though the MP itself isn’t that much updated, the related guide has been improved on a very important – and many times a topic many people find hard to understand – the Run As Profiles:

Mp and related guide to be found here.

System Center: Community Evaluation Program

Many of us would like to test drive a Ferrari, the newest nuclear submarine or the newest fighter jet. Only a few are allowed to do just that…

And like most of us, I am not so fortunate. But for us geeks, there is a nice equivalent to this. OK, it’s not something like the things I mentioned earlier, but still it’s nice. Why? You’re allowed to test the newest versions of the System Center products like SCOM, SCCM, SCO, SCVMM and FIM. Later on DPM will be added as well. And that even before these products become RTM!

This is what the website states about it:

Want to know more, or even better, test drive it yourself? Go here.

The Right Tools for the Right Job – For FREE

No matter what you do, like building a house, SNMP walk a network device, repairing a car or doing some raw XML-editing, good tools are an absolute requirement to get the job properly done.

In my field of work I came across a couple of tools which are really good and should be present in the toolbox of any person who’s deeply involved in SCOM. Meaning, when you’re a SCOM Operator you’ll probably won’t need them that much or even at all.

• Editing raw XML
  Even though I used XML Notepad 2007, it doesn’t even come close to Notepad++. This tool really rocks BIG time and has made editing raw xml of MPs way much easier for me. Tool can be downloaded from here, for FREE :). I have uninstalled XML Notepad 2007 because I don’t use it anymore.
  

• SNMP Walk and stuff like that
  This tool is really awesome. Works like a charm and is capable of many other network related stuff as well. Whenever you want SCOM to monitor a SNMP enabled device and need OIDs to get a deeper look into the device, Getif is THE tool. It’s free, fast and easy to use. Tool is to be found here.
  
  • But of course, OIDs are crucial. In order to get the most out of this tool, MIB files are required as well. A collection of them can be downloaded from the website of Getif as well. Besides using Google as a good source in order to find many obscure MIB files, there is a website which brings many MIB files together: www.oidview.com/mibs. Download and unzip the required MIB files and place them in the MIB folder of the Getif tool (C:\Program Files (x86)\Getif 2.3.1\Mibs). Now you’re ready to rock. Don’t forget to run Getif with elevated permissions or you’ll get an error on W2K08/W7 systems.

Let me know when you use some good tools as well and would like to share them with the community. I will update my blog posting accordingly.

Vital Signs – Part II: The Installation

----------------------------------------------------------------------------------
Postings in the same series:
Part   I – Teaser/Introduction
Part III – Sherlock Holmes is back!
----------------------------------------------------------------------------------

In the second posting of this series I will describe the installation and its prerequisites. So let’s start.

Prereqs:
These components are required for the server running Vital Signs:

• .NET Framework 4.0;
• Visual C++ 2010 Runtime libraries, (x86 or x64).

Good to know: When the two above mentioned components aren’t in place, the Vital Sign installer will detect it, download them and start the installation:

Nice!

For the web based Console these IIS components are required:

• IIS 7
  • Web Server Role
  • Web Server Static Content Role Service
  • Web Server ASP.NET Role Service
  • Web Server Anonymous Authentication Feature
  • Web Server .NET Environment
• .NET Framework 3.5.1

Also, the server hosting this software must be a member of the Domain. The user installing the software must be local admin and a member of the Domain.

Installation:
The installation is straight forward, especially when the requirements are met. This screen is very important and requires some planning. In bigger organizations it might require input from the folks responsible for all IIS instances:

Also a screen to note is this one:
  
It makes the difference between being able to access Vital Signs or not…

Post Installation Tasks:
When Vital Signs has been installed some additional tasks are required in order to get it all Up & Running. Start the Vital Signs console after installation. In the Home screen these Configuration Tasks will be displayed, Required, Recommended and Optional:

By clicking on one of the Tasks, the appropriate screen will be opened.

When the licenses (one can contact Savision for a fully functional trial license) are in place it’s time to run the Recommended Configuration Tasks. Only after having completed these tasks Vital Signs can run and show what’s it made of. No time to waste! :)

As an Optional Configuration Task one can add a connector – to SCOM or SCSM – as well:

As stated before by Savision, the SCOM connector ‘…will read alert and incident data from SCOM and SCSM and display them in context of the affected system. Additionally, tasks are created in SCOM for launching Vital Signs in context of the selected system…’.

User Roles can be added as well, as required.

Now Vital Signs is in place and operational.

Conclusion:
Vital Signs is a product with a good installer. It’s intuitive and helpful as well: when some basic requirements aren’t in place, it will download them automatically and install them, under your supervision. So you’re at all times in the driver’s seat, like driving a car with a luxury kind of cruise control. It’s evident Savision has put a lot of time and energy in the interface/gui/console of Vital Signs. It’s straight forward and working with it is a walk in the park. But that statement will demonstrated in the third posting of this series.

Next posting in this series will be all about my experiences and a look behind the scenes / under water.

Veeam Community Podcast. Episode 18 – Connecting the dots with OpsMgr

Nice! For a day or so the podcast made by Rick Vanover (Software Strategy Specialist for Veeam Software) and me – all about OpsMgr – is available on the Veeam Community website. 

Want to know more? Webcast to be found here.

SCUG Netherlands – Meeting

On the 7th of June 2011, the Dutch System Center Users Group (SCUG) organizes a meeting for all SC enthusiasts.

Topic of this meeting is ‘The Best of MMS 2011’. Agenda:

Want to join? Go here in order to register yourself.

How To: Monitor Azure with SCOM R2?

Monitoring Azure with SCOM R2 isn’t a process like Next > Next > Finish. It takes a ‘bit’ more time, actions and preparation.

Of course, there is a guide for the Azure MP but I found myself with a lot of unanswered questions after having read it. Much of the answers were found by trial and error. So it’s good there is an excellent new source on the internet, all about monitoring Azure with SCOM R2.

Walter Myers is a Principal Consultant with MCS. He has a lot of experience with the Azure Management Pack and has blogged about it in great detail! Want to know more about monitoring Azure with SCOM R2? How to customize the MP? How to get the most out of it?

Check out this blog and you’ll learn a great deal about just that!

All credits go to Walter Myers of course.

The Case of the Disappeared Group or the DA hiding Groups from the Groups View in the Authoring Pane…

Bumped into this issue which really puzzled me for a while…

Example
Create a Group, like Now You See Me But When Added to a DA I am GONE

Populate it as needed, save it. Search for the Group after it has been created:

Wow! It’s there. Nice! :)

Let’s create a DA and put the freshly created Group into that DA:

And:

Save the DA. Let’s search again for the same Group:

Also, the total amount of Groups is reduced by one, the Group you just added to the DA has been removed from the total amount of available Groups as well…

The Group is still present in the background however and can be used in Monitors/Rules and the like. An example, when creating a Monitor the Group can be found:

When the DA is deleted, the Group is to be found back again, so the DA is the culprit here.

Workarounds:
But how to go about it when you want to EDIT that Group? 

1. One way to ‘solve’ it, is to remove the Group temporarily from the DA, save the DA, edit the Group as required, save the Group and add it back to the DA and save the DA as well.
2. Another way is to export the MP where the Group has been put into and edit the XML directly…

My guess it’s a bug… :) So I have filed a bug on Connect. When you experience the same issue, please vote for it so changes are it will be fixed in a future edition/CU of SCOM… (click the picture in order to go the Connect site).

Strange behavior when Names of Classes/Objects are changed in non-English based SCOM R2 environments

I noticed this strange behavior in SCOM R2 environments running in non-English environments, like the Netherlands for instance:

An Object (like a Group for instance) is created. Later a name change is required. In the Console the Properties screen is opened for that Object and the Name field is changed accordingly and saved. Back in the Console, the Name of that Object still displays the OLD name. When the Properties screen for that Object is opened, the Group name displays the correct name, the changed name..

This posting describes this issue, it’s cause and how to solve it.

By the way, this behavior happened in a CU#4 SCOM R2 environment.

The Issue
An example Group is created, with the name Test Group with a FAKE name

Group is present in SCOM Console:

Group Properties screen as in shown in the SCOM Console:

Group name is changed in the Properties screen of the Group as shown in the SCOM Console, and saved:

Group name is adjusted. But when searching for it, nothing is shown:

Let’s search for the old name:

Huh? Still present? Let’s check the Properties of this Group:

This is REALLY strange.

The Cause
Let’s take a deeper look in the MP by exporting it and opening it in Notepad++. As you can see there are differences, located in the language sections of the MP. The Dutch section (NLD) of the MP:

So the name change landed in this section of the MP.

How about the English section (ENU)? Let’s check it out:

So there is the culprit! The Dutch (NLD) section is OK, but the ENU section isn’t touched at all…

Let’s take a few steps back:

So the Name shown in the Console, is the ENU section:

The Group name shown in the Properties screen of the Group in the SCOM Console, is the Dutch section of the MP…

The Solution/Workaround
Let’s adjust the ENU section in the raw XML in order to reflect the name change and import that MP into SCOM. No hassle about version numbers since that only comes into play for sealed MPs and this is an unsealed MP…

Let’s search for the phrase REAL in the Groups View of the SCOM Console:

Bingo!

Let’s check out the Properties of this Group:

Some background information
Checked the regional settings on the RMS, SQL and the workstation where the SCOM Console was running:

• RMS: English
• SQL: English
• Workstation: Dutch.

It wasn’t feasible to change the Regional settings on any of those systems. So there is no change to work around this bug in any kind of way but editing the raw XML by changing the name under the ENU section of the MP.

Conclusion

When you’re running SCOM in a non-English environment, and you change the Name (or Description, saw the same behavior there as well!), for any Object/Class in SCOM, you won’t see those changes back into the SCOM Console. Editing of the raw XML is required in order to get it all working.