Tuesday, March 10, 2009

Speeding up the SCOM Console

Beware! When editing the registry one can cause damage which cannot be undone. So be carefull when performing the procedure mentioned in this blogposting. Always make an export of the registrykeys before deleting ANYTHING from the registry.
Ok, having said that, there is a nice trick to get the SCOM Console running faster.

At a customerssite I bumped into a situation where the SCOM operators complained about the performance of the SCOM Console. And they were right. It was very slugish. So I checked the RMS and the related SQL server. These servers were performing just fine. No glitch what so ever. They were dimensioned properly (enough cpu, RAM and fast disks) and when running perfmon on those servers, all was well.

So the cause of the slugish SCOM Consoles wasn't to be found there. Then I turned my attention to the systems of the SCOM Operators. First I found that they were running the SCOM Consoles with the /ClearCache command.

They started with SCOM when it came RTM and with RTM it was advised to use that switch. But when they upgraded to SP1, they were still using this switch. But it is Best Practice only to use this switch when there are problems with the SCOM Console. Under any other circumstances this switch is not to be used. So I deleted this switch.

The performance of the SCOM Console did improve but not enough. Then I asked them for how long they were using the SCOM Console from their systems. As it turned out, from the moment they started to use SCOM which is more than a year now. All this time they had only upgraded the SCOM Console to SP1 but never ever changed anything about it.

Besides the well known cache file when one starts the SCOM Console, another 'kind of cache' is created as well. In the registry of the current user, the navigation history within the SCOM Console is being added. And on the systems of these SCOM operators this regkey had grown hugely.

So when I deleted this key and started the SCOM Console, it was very snappy. No more sluggish SCOM Console but one which worked really fast.

Of course you can perform this 'trick' as well. But remember this regkey has a function so only do this when you are experiencing the same issues and are sure that the RMS and the SQL servers are performing well and not the cause of a sluggish SCOM Console.

First close the console and than remove this regkey: HKEY_CURRENT_USER\Software\Microsoft\Microsoft Operations Manager\3.0\Console.

This regkey will be recreated when the SCOM Console is started. But as stated before, make an export before deleting this key. Better to be safe than sorry...
Screendump of the regkey involved:

19 comments:

Stefan Stranger said...

Hi Marnix,

What do you consider as "regkey had grown hugely"

If I look at the size of HKCU\Software\Microsoft\Microsoft Operations Manager\3.0\Console key after exporting it and saving it as *.reg file I see a size of 1,245,610 bytes.

Is this huge or is this much lower than you found?

Regards,
Stefan Stranger
http://blogs.technet.com/stefan_stranger

Marnix Wolf said...

Hi Stefan.

Thanks for your comment.

Actualy, I didn't measure the regkey by its size in bytes when having exported it as a *reg file. Next time when I bump into the same situation I will certainly do so.

I based my conclusion upon it's size as being huge on how many entries this regkey had.

Until now at other sites I saw this key having about a couple of hundred of subkeys.

But at this site (with the very slow SCOM Consoles) there were several thousands of subkeys, most of them had a prefix which looks like a GUID. Others were garbled as well.

So therefore I noticed it as being huge.

Unfortunately I do not have the exported *.reg files anymore so I cannot tell anything about their size.

Best Regards,
Marnix Wolf

Baijumon said...

Hi Marnix,

I am new to SCOM ,and we stuck up with an issue , can you help us resole this issue .

Here is our setup details.

We have SCOM SP1 setup where RMS and Agents are in domain called A , recenlty we migrated an agnet to domain B , after migrating to domain we were not able to discover the agent which is in domain B , and later we had found that issue due to which SQL brocker which wasnt enabled ,when we enabled were able to discover however in OpsMgr console it shows "NOT MONITORED" and also getting following error,

On agents I have got this errors:
___________________________

OpsMgr connectors

Event ID: 20070

The OpsMgr Connector connected to XXXXX-infmom02.APAC.contoso.com, but the connection was closed immediately after authentication occured. The most likely cause of this error is that the agent is not authorized to communicate with the server, or the server has not received configuration. Check the event log on the server for the presence of 20000 events, indicating that agents which are not approved are attempting to connect.

Note : Our SCOM SERVER is Installed in different Domain , i.e FNF & Agent is installed on Trusted Domain FNFINDIA


And On server I got This Error Message :
OpsMgr Connector

Event ID : 20000

A device which is not part of this management group has attempted to access this Health Service.
Requesting Device Name : XXXXX-SVR01.FNFINDIA.org

Kindly help us to resolve this issue

Regards,
BPaul

Marnix Wolf said...

Hi BPaul.

Seems like a authentication issue to me. SCOM needs Kerberos (only between domains with a FULL trust in place) to authenticate.

When Kerberos isn't possible, certificates are needed.

Also, moving a server with an Agent from Domain A to Domain B will noit work.

SCOM will try to contact the Agent in Domain A, not B. So removal of the Agent is necessary and afterwards a reinstall with a certificate for authentication.

MOM 2005 didn't require authentication but SCOM does.

Want to know more: check out this site with all documentation on SCOM: http://technet.microsoft.com/en-us/opsmgr/bb498235.aspx

Baijumon said...

Thank you so much, Marnix, for providing an update.

So if we want an agent in Domain B to communicate with SCOM Mangement Server in Domain A needs to require certificate seems here.(as per the issue log in eventviewer)

Also,would like to know is there any other way will be able to communicate with SCOM in Domain A and an agent in Domain B , by using gateway server ?

Really appreciate for your update.

Regards,
BPaul.

Marnix Wolf said...

Hi BPaul.

Certainly a Gateway Server can be used as well. It is certainly handy when many servers in the untrusted boundary must be monitored.

Instead of deploying many certificates you only need to deploy one certificate to the SCOM Gateway Server.

The SCOM Management Group will use the certificate to 'talk' to the SCOM Gateway Server. The servers behind the Gateway Server will talk to the Gateway using Kerberos.

Another welcome side affect is the compression rate used by a SCOM Gateway Server: http://thoughtsonopsmgr.blogspot.com/2009/10/opsmgr-gateway-servers-and-their.html


Another advice: Microsoft has some GREAT SCOM Forums, found here: http://social.technet.microsoft.com/Forums/en-US/category/systemcenteroperationsmanager


Here you can post your questions and rest assured that you will get an answer.

Baijumon said...

Hi Marnix,

We are looking for a solution in SCOM , here is our requrement that we wants to do in SCOM ennvironment.

• To manage and monitor all the servers relating to the XXXX Team(people) from the SCOM console.
• Add, amend, delete agents (servers) into the current SCOM monitoring server.
• Push new agents to the relevant XXXX group servers.
• Configure and change counters for the XXXX group servers.
• Configure, Amend and the run regular reports for the XXXX group servers from the SCOM management console.

Basically,delegation of a group servers so that only to that team of people can manage their servers (we should have complete administrator priviledge for SCOM), is that possible in SCOM SP1 ?

Really appreciate for the inputs/suggestions.

Regards,
BPaul.

Baijumon said...

Hi Marnix,

We are looking for a solution in SCOM , here is our requrement that we wants to do in SCOM ennvironment.

• To manage and monitor all the servers relating to the XXXX Team(people) from the SCOM console.
• Add, amend, delete agents (servers) into the current SCOM monitoring server.
• Push new agents to the relevant XXXX group servers.
• Configure and change counters for the XXXX group servers.
• Configure, Amend and the run regular reports for the XXXX group servers from the SCOM management console.

Basically,delegation of a group servers so that only to that team of people can manage their servers (we should have complete administrator priviledge for SCOM), is that possible in SCOM SP1 ?

Really appreciate for the inputs/suggestions.

Regards,
BPaul.

Marnix Wolf said...

HI Baijumon

What you are asking for needs Admin permissions within SCOM, no matter what version you are using, whether it is RTM/SP1 or R2 for that matter.

Baijumon said...

Hi Marnix,

Thank for your update.

Yes, we are looking for delegation on SCOM for a group of servers.

We want to give delegation for group so that that users can manage their only servers in SCOM environment but we dont want to give complete control over SCOM ennvironment.

is that achievable in SCOM ? and what are the roles we needs to give for a group ? to manage their servers .

Kindly help us .

Regards
BPaul.

Marnix Wolf said...

Hi.

Just checked the comments on this post and noticed that many of these comments are a bit off topic.

Whenever there are questions about SCOM it is better to visit the SCOM TechNet Forums, to be found here: http://social.technet.microsoft.com/Forums/en-US/category/systemcenteroperationsmanager

There I am active as well, so are many of my peers. Questions always get answered soon. :)

To come back to you question is when a certain group of people need total SCOM permissions of a group of monitored servers, there is no way to create a special view in SCOM since SCOM Admins get everything there is to be seen in the SCOM Console.

However, when SCOM Admin permissions are not needed I have found this series to be very spot on: http://ops-mgr.spaces.live.com/Blog/cns!3D3B8489FCAA9B51!2035.entry

Hope this helps.

Bye,
Marnix.

Michiel Wouters said...

Hi Marnix,

At a customer site my \3.0\Console exported .reg file is rather big.

(8.396.708 bytes). It contains about 100.000 lines. Mostly settings for every view or UI part i've ever used.

Deleting it indeed gives a performance boost to the Ops Console.

But, users should be aware that deleting this registry settings also deletes their personal settings. For example, Personalize View settings.

networkguys said...

WE have been using this method of deleteing the reg key for a few months but then found a way to look at the database for performance issues. Kevin Holman presented at the last SC Virtual User Group meeting on Database issues and we utilized these techniques to find many issues. WE have been able to cut down the traffic from the RMS to the DB by over 75% by removing/replacing MP's and by tweaking settings. Best to do the tweaks after hours.


http://cid-4b30376adfa42884.office.live.com/self.aspx/Public/SCVUG/Meeting%20%5E311/KevinHolman%5E_SCVUG11.wmv

Marnix Wolf said...

Hi.

Thanks for your comment. To speed up the Console there are many different approaches where the serverhardware hosting the RMS role and the SQL DBs are of course the most important ones. However, this posting is based on an experience I had with a SCOM RTM/SP1 environment where all was OK but still the Console was slow. In that particular case the reg hack did the trick. In SCOM R2 the Console has become way much more faster since the underlying engine has been optimized.

Good for you that Kevin Holman postings work for you as well. He runs a good blog which will help out in many occassions.

Cheers,
Marnix

Stefan said...

Hi Marnix,

Today I found out you also need to delete the momcache.mdb file after deleting the registry key.

The OpsMgr 2007 SP1 console crashed when opening Groups in the Authoring Pane for me.

After deleting the momcache.mdb the console did not crash anymore.

Regards,
Stefan

P.S. Be carefull when deleting registry keys!!

Marnix Wolf said...

Hi Stefan.

Thanks for the update. Until now I have only used this trick twice in non SCOM R2 environments (SCOM RTM/SP1).

It worked without deleting the cache.

Cheers,
Marnix

Skygold said...

hi,
is this applicable for scom 2012

Marnix Wolf said...

Hi Skygold.

No, this posting is for SCOM RTM only.

Cheers,
Marnix

Daniel Queen said...

pretty sweet. I've used clearcache a lot, but never tried removing the console reg key. man, what a speed difference that makes after using the console for a long time! thanks for the tip.