A customer had many serious issues with their SCOM 2007 R2 Management Group. Not much was working any more. Alerts couldn’t be updated, came in way too late (after 12 hours or more!), Agents couldn’t be repaired, Agent’s couldn’t be installed and so on.
The RMS was greyed out and the MS was in a bad shape as well: the OpsMgr event log was filled with tons of errors like EventID 33333 and EventID 10850, all about being unable to pump data into the SCOM R2 SQL databases.
So this was bad. Time for some investigation. One of the things I bumped into really puzzled me and was rather alarming…
In OpsMgr 2012 this is a valid configuration, simply because the RMS functionality is hosted by ALL OpsMgr 2012 Management Severs. But in SCOM, this is BAD!
The services System Center Data Access and System Center Management Configuration are services which are present on every SCOM Management Server but should only be in a running state on the RMS. When running on SCOM Management Servers as well, SCOM will soon start functioning simply because the SCOM equivalent of a split-brain scenario come to be…
Only the System Center Management service is set to automatic AND is in a running state. The two other SCOM services are present but disabled and turned off.
Now it was time to stop the System Center Management Server service on the SCOM Management Server, rename the folder ~:\Program Files\System Center Operations Manager 2007\Health Service State to ~:\Program Files\System Center Operations Manager 2007\Health Service State_OLD, empty the OpsMgr event log and start the System Center Management Server service again.
Soon the SCOM MG started to move a bit again. During my investigation many other issues were found and fixed as well.
Even though the SCOM services System Center Data Access and System Center Management Configuration are present on any SCOM Management Server, don’t touch them! Don’t start them because it will wreck your SCOM environment.