Tuesday, January 4, 2011

Error: Consolidator Module Failed Initialization

Got the above mentioned error on of a couple of monitored servers after the Lync MP had been imported and configured. This error was shown in the SCOM R2 Console:
image

I searched the internet but didn’t find anything solid. So it was time to take a deeper dive on the servers as well. Soon I found something which seems to be the cause: WMI is experiencing issues. The servers involved are W2K08 R2. These are some WMI related events which these servers show:

  1. Application Log, EventID 1000:
    image
    WMI is faulting. Hmm, SCOM really needs WMI so this is bad.

  2. OpsMgr Log, EventID 11112:
    image  
    This error generates the Alert shown in the first screen dump.

  3. OpsMgr Log, EventID 10409
    image  
    This event tells WMI cannot be enumerated.

  4. OpsMgr Log, EventID 4001:
    image
    Many scripts are failing. Scripts which tap into WMI that is…

So it is clear WMI is having serious issues which need to be addressed.

There is a hotfix available for fixing WMI on W2K08 R2 based servers: http://support.microsoft.com/kb/981314. It addresses a memory leak with WMI. Normally this leak will not become visible, only when  the Win32_Service class is frequently queried. And guess what SCOM does?

So before the Lync MP was imported, WMI on these servers didn’t get that much load. But after having imported the Lync MP, WMI on these servers is being queried with a higher frequency so the leak comes out.

And when running Perfmon against the process WmiPrvSE, (All instances, select counter Private Bytes) one will see it consuming a maximum load:
image

Tonight the hotfix will be applied and I am pretty sure this will help and resolve the issue. When the results do come in I will update this posting accordingly.

4 comments:

Gefufna said...

I've also experienced similar problem with Windows Server 2008 R2-based managed computers.
Workflow Runtime: Failed to run a WMI query alert was received.
It is explained here: http://gefufna.wordpress.com/2011/01/04/workflow-runtime-failed-to-run-a-wmi-query-alert-description/

IcI said...

Seems like the 'Consolidator Module ...' alert is bigger than just one WMI issue.

I'm receiving this on a Windows 2003 SP1 server.

This error appears for all event ids listed below:
11100 - 11103, 11108 - 11113, 12550 - 12569.
These relate to the consolidator module failing & some workflows are affected.

listofdemands said...

Can you confirm whether or not this fixed your issues?

Marnix Wolf said...

Hi,

It fixed the issues.

Cheers,
Marnix