UR#1 SCCM 2012 R2 Breaks Multicast. Error Code (0x80070002)

Issue
Had an issue where Multicast worked liked magic BEFORE UR#1 for SCCM 2012 R2 was applied. After UR#1 was rolled out successfully, all deployments using Multicast refused to work. Unicast deployments still worked as intended though.

This message was showed by the failed Task Sequence: Task Sequence: Deploy Mulitcast has failed with the error code (0x80070002).

Cause? A ‘known’ issue…
After some testing I knew for sure that the Multicasting feature was killed. So it was time to take a deeper look at it. On one of the machines failing to run that Task Sequence I opened the log file x:\windows\temp\smsts.log and immediately some errors popped up.

For some reasons no multicast session could be started. I checked the DP and its settings were just fine. Multicast was still enabled and properly in place at the surface, but somewhere down in the engine room of SCCM itself, it wasn’t.

Time to contact my brother in arms, Kenneth van Surksum. Within a minute he came up with this posting of the ConfigMgr Team.

So this is a known issue? Taken directly from the same posting: ‘’…When you use the multicast option on distribution points in your environment and apply an update to Configuration Manager that contains a new MCS.msi, the multicast configuration does not properly upgrade and subsequent use of multicast fails…’

Workaround
The same workaround as stated in the posting works like a charm. But PLEASE keep an eye on the McsExec.log and only start with your Multicast deployments when that log is clean and doesn’t report any errors about SignedSerializedMCSKey is empty and Error checking MCS certificate in registry: 0x8004005.

These two checks run per 5 minutes. In our case these errors were gone after 35 minutes (7 checks). And then the Multicast deployments run like a charm again!

Are YOU Going To Attend SCU Europe 2014?

Now with the demise of MMS the events hosted by non-Microsoft parties about System Center technologies have become even more important. One of the most important events is the System Center Universe series, hosted on multiple locations over the world, like the USA, APAC and Europe.

In order to get good coverage by potential speakers, the System Center Universe events are organized in different months. The ones for the USA and APAC already took place this year but the one for Europe, SCU Europe is planned for September 17 – 19 at Congress Center Basel, Switzerland.

Until the 31st of May you can get an EARLY BIRD conference pass! So be fast and register!

Why should I attend SCU Europe?
Good question. For multiple reasons:

1. MMS is no more;
2. SCU Europe is driven by the community and thought leaders of the System Center world (aka UNIVERSE);
3. Top notch speakers will present at SCU Europe.

All I can say is: Visit the website of SCU Europe, take a look at the agenda, sessions list and speakers. And make up your own mind. Hopefully we’ll see each other there.

SCOM Gateways: How To Enable Status Monitoring For Sites

Some background information
When using SCOM Gateway Servers you can add a ‘site’ tag to them. This is done when running the Gateway Approval Tool. This site tag can be a pain in some situations but most of the times it can help you to differentiate between multiple environments being monitored by using Gateway Servers.

In situations like these the use of site switches are the way to go, since they add the site tag to all the Alerts coming from all the objects residing behind that particular Gateway Server (or set of Gateway Servers, since you can use the same site switch multiple times, for instance when you run multiple Gateway Servers in failover configuration through PowerShell). So now you can simply filter those Alerts in your Views, Notification Model and Reports.

How about the health status of the Site itself?
Another not so well known feature of Sites are the capability to reflect the health status of the whole environment being covered by that particular Gateway Server (or set of Gateway Servers). Out of the box this isn’t present in SCOM 2012x. By default the Site doesn’t have a status at all:

When you open the Site object in Health Explorer, you won’t see any Monitor at all. So no Monitors = no status.

In SCOM 2007x the status for the Site was only present when you imported the Service Provider MP, a MOM 2005 converted MP that is. This MP enabled monitoring on a Site level enabling the organization to have a quick view of the overall health status of al their sites. But believe me, you don’t want an MP like that in your SCOM 2012x environment…

There is a whole history behind that MP, back to the days where SCOM as a service could be implemented in conjunction with SCE (System Center Essentials). But happily that’s the past and was a bit of a drag (duh!) to implement and maintain. Also the licensing was terrible back then. I won’t bother you with that kind of obsolete knowledge…

With a few mouse clicks this monitor is rebuild in SCOM 2012x, enabling the Site to reflect the overall health status of of all Objects residing behind that Gateway Server (or set of Gateway Servers).

How to build the Rollup Monitor for the Site?
This is really easy. Just follow these steps and you’ll be fine.

1. Open the SCOM 2012x Console with SCOM Admin permissions > Authoring > Management Pack Objects > Monitors. Set the scope to Site by using the Change Scope option;
2. Expand Site > Entity Health > right click Availability > Create a Monitor > Dependency Rollup Monitor;
3. Give this Monitor a proper name and description. Your screen looks like this now:
   
   > Next;
4. Now comes the most important part of it all, the Monitor Dependency. Make sure you select Object (Site Membership) > Entity Health > Availability. Only than the Monitor will work!
   
   > Next;
5. For Health Rollup Policy and Configure Alerts it’s best to leave at default settings. You only want a rollup and no extra Alerting nor do you want to the Monitor to reflect anything else besides the worst state of any member:
   
   > Next;
   
   And:
   
   > Create;
6. The Monitor will be created and now the SCOM Console looks like this:
   
7. Soon the Site objects will get a status in the SCOM Console:
   

Recap
When monitoring multiple sites residing behind Gateway Servers, the usage of the ‘site’ switch might come in handy. And with some additional click work you enable a health rollup for those same Objects as well. Now with a single glance you can see what the overall health status of ALL monitored objects residing at particular Site is.

Cool Tools: Microsoft Virtual Machine Converter 2.0

A few days ago Microsoft released the newest version of Microsoft Virtual Machine Converter (MVMC), version 2.0.

Taken directly from the webpage: ‘…Microsoft® Virtual Machine Converter (MVMC) is a Microsoft-supported, stand-alone solution for the information technology (IT) pro or solution provider who wants to convert virtual machines and disks from VMware hosts to Hyper-V® hosts and Windows Azure™…’

And: ‘…MVMC can be deployed with minimal dependencies. Because MVMC provides native support for Windows PowerShell®, it enables scripting and integration with data center automation workflows such as those authored and run within Microsoft System Center Orchestrator 2012 R2. It can also be invoked through the Windows PowerShell® command-line interface. The solution is simple to download, install, and use. In addition to the Windows PowerShell capability, MVMC provides a wizard-driven GUI to facilitate virtual machine conversion…’

Many new features are added. Want to know more? Go here.

Cool Tools: Script Browser & Script Analyzer 1.2

A few days ago Microsoft released a cool tool: Script Browser for Windows PowerShell ISE.

It enables you to search for over 9000 script samples in the TechNet Script Center from within the scripting environment.

On top of it you can try out another new pilot function bundled with Script Browser - 'Script Analyzer'. The title says it all.

So don’t wait. Download the tools and use them. Microsoft loves to hear your feedback.

Cross Post: SQL Server 2012 Monitoring: Two Issues To Reckon With

When monitoring SQL Server 2012 instances (and databases of course) there are two things to reckon with:

1. SQL 2012 DB Engine group does not contain all SQL servers
   Kevin Holman blogged about this issue which happens when you apply SP1 to your SQL Server 2012 servers. The same posting tells you all need to know: the cause and the solution (an addendum to the existing SQL Server MP which can be downloaded from the same location).
   
   
2. Event 4001 in the Operations Manager log during SQL Server 2012 monitoring
   When you are monitoring a Microsoft SQL Server 2012 computer that has the System Center 2012 Operations Manager agent installed, you notice that event 4001 is logged in the Operations Manager log.
   
   KB2962161 tells you the cause and how to fix it as well.

SCC Health Check Reports Error: Query Execution Failed For Dataset 'OperationsManagerAll'

If there is one community effort I really appreciate AND use in ALL SCOM 2007/2012x environments, it’s the SCC Health Check Reports MP. Even though it’s written for SCOM 2007 these Reports still do their work for SCOM 2012 R2 based environments and older versions.

However, this time I bumped into a particular error. The required dataset (for the connection to the OpsMgr database) OperationalDataBaseMain was made and successfully tested. And in this case it required the option Credentials stored securely in the report server in order to get these reports working.

But still, nothing to get excited about. And all Reports seem to work until someone tried to run the Report Misc - Infrastructure Overview (OM). Instead of getting a proper report this error was thrown: Query execution failed for dataset 'OperationsManagerAll':

But wait! Dataset ‘OperationsManagerAll’? This dataset isn’t present on the SCOM 2012 R2 Report Server. Nor is it needed by any other rollout of this MP. So something else is at play here.

Time to get more information. Let’s run the same report but now directly from the Report Server itself, hoping to get a better error message without having to enable remote errors. This is what I got:

Okay, that’s better. So the account used in the configuration of the new dataset (OperationalDataBaseMain) didn’t have ALL the required permissions in order to run this Report.

After some discussions it was decided to use another SCOM service account for that purpose, which has more permissions on the Operational database. And after this modification, this Report runs like clock work.

New KB Article: Port Monitoring Still Runs After Task ‘Disable Port Monitoring’

Bumped into this issue myself: When running the Task ‘Disable Port Monitoring’ port monitoring still runs…

Duh!!! This happens on ALL versions of SCOM 2012 (RTM, SP1 and R2).

The symptoms, cause AND work around
Microsoft has released KB2960230 all about this issue, the causes and the work around.

Applying OM12 R2 UR#2: My Personal Experiences

Even though many people already shared their experiences running Update Rollup 2 (UR#2) for SCOM 2012 R2, I want to share my personal experiences as well. Before I start however I want to say thanks to Stanislav Zhelyazkov who gave me some insights before I started, saving me a lot of time . Awesome man!

There is much to tell so let’s start.

Preparation
Like any other update, preparation is crucial so:

1. Backup both SCOM SQL databases;
2. Make a snapshot of the VMs running the Management Servers.

This way there is way back when things turn sour.

Now, download all the update components for UR#2. Say what? You downloaded them some time ago already? Nice! Delete them and DOWNLOAD them again. Simply because I’ve heard that the update bits have been updated as well…

Let’s start
This isn’t very exciting actually. Just follow the procedures outlined in KB2929891 and you’ll be fine, basically meaning this order of updating:

1. Firs the SCOM 2012 R2 server infrastructure:
   • Management server or servers
   • Gateway servers
   • Web console server role computers
   • Operations console role computers
2. Apply BOTH SQL scripts (one for the OpsMgr database and the other for the DW);
3. Manually import the management packs.
4. Apply the agent update to manually installed agents, or push the installation from the Pending view in the Operations console.

So that’s easy isn’t it? Yeah, it SHOULD be but as already stated by other bloggers, like Stanislav Zhelyazkov, there are some serious caveats to reckon with…

The caveats and the work arounds
Ouch! Guess QC slipped (again) at Microsoft. Cloud is top priority and anything else comes second .

1. Agents AREN’T updated through the Console
   Yes, that’s bad. The Console tells you the pushed Agents were updated successfully BUT they aren’t! For now the ONLY work around is to run a REPAIR and now the pushed Agents will be updated. I am not the first one sharing this information but please spread the word!
   
   
2. SCOM Gateway Server doesn’t update the Agent staging folder
   This is an old one, already present in old updates for SCOM 2007 R2. And it seems it’s back! Basically it means that the folder containing the Agent installers for the Windows Servers AREN’T updated. So this requires a MANUAL copy & paste action (can be scripted) of the related MSP files to the correct folders on the Gateway Servers…
   
   
3. Agents running APM require a REBOOT of the whole server
   Bummer! An old one as well (in SCOM 2007 R2 this was introduced with UR#2 or UR#3 and fixed in UR#5) and now it’s back as well. And this is REALLY bad. Work around? Postpone the reboot to more suitable time frames…

Recap
UR#2 isn’t bad on itself. But again QC has slipped here and introduces the three earlier mentioned issues. IMHO these issues SHOULD have been avoided and not put onto the plate of the customers. Therefore always WAIT a few weeks (at least) before rolling out updates like these.

Cross Post: IIS Service Alerts From Servers Where IIS is not Installed

I came across this very interesting posting all about the IIS MP raises alerts on servers where IIS is not installed. Since this is a good posting I decided to cross post it where I want to point out that ALL credits go to the author of that same posting, Michael Repperger.

He writes in that posting about two possible solutions whereas the latter (to extend the IIS 7 Computer Role class, and then create groups that contain instances of it where WMSvc and IISADMIN actually exist.  Then use those to override the monitors) is more elegant.

For that last solution he shares his MP as well.

Want to know more? Go here.

Again, all credits go to Michael Repperger

The POWER of PowerShell & System Center 2012

A few days ago Microsoft published a cmdlet reference for System Center 2012, containing many documents about all available PowerShell cmdlets for the System Center 2012 family.

These 37(!) references can be downloaded from here:

1. SC2012_AppController_Cmdlets.pdf
2. SC2012_DPM_Cmdlets.doc
3. SC2012_DPM_Cmdlets.pdf
4. SC2012_OpsMgr_Cmdlets.doc
5. SC2012_OpsMgr_Cmdlets.pdf
6. SC2012_VMM_Cmdlets.doc
7. SC2012_VMM_Cmdlets.pdf
8. SC2012R2_AppController_Cmdlets.doc
9. SC2012R2_AppController_Cmdlets.pdf
10. SC2012R2_ConfigMgr_Cmdlets.doc
11. SC2012R2_ConfigMgr_Cmdlets.pdf
12. SC2012R2_DPM_Cmdlets.doc
13. SC2012R2_DPM_Cmdlets.pdf
14. SC2012R2_MMA_Cmdlets.doc
15. SC2012R2_MMA_Cmdlets.pdf
16. SC2012R2_OpsMgr_Cmdlets.doc
17. SC2012R2_OpsMgr_Cmdlets.pdf
18. SC2012R2_OpsMgr_XPlat_Cmdlets.doc
19. SC2012R2_OpsMgr_XPlat_Cmdlets.pdf
20. SC2012R2_ServiceMgr_Cmdlets_DW.doc
21. SC2012R2_ServiceMgr_Cmdlets_DW.pdf
22. SC2012R2_ServiceMgr_Cmdlets_Group1.doc
23. SC2012R2_ServiceMgr_Cmdlets_Group1.pdf
24. SC2012R2_ServiceMgr_Cmdlets_Group2.doc
25. SC2012R2_ServiceMgr_Cmdlets_Group2.pdf
26. SC2012R2_SMA_Cmdlets.doc
27. SC2012R2_SMA_Cmdlets.pdf
28. SC2012R2_SPF_Cmdlets.doc
29. SC2012R2_SPF_Cmdlets.pdf
30. SC2012R2_VMM_Cmdlets.doc
31. SC2012R2_VMM_Cmdlets.pdf
32. SC2012SP1_AppController_Cmdlets.doc
33. SC2012SP1_AppController_Cmdlets.pdf
34. SC2012SP1_SPF_Cmdlets.doc
35. SC2012SP1_SPF_Cmdlets.pdf
36. SC2012SP1_VMM_Cmdlets.doc
37. SC2012SP1_VMM_Cmdlets.pdf

So now there isn’t any excuse anymore for why not using PowerShell to manage your System Center 2012 components .

SCOM Doesn’t Discover All Cluster Resources

Issue
When a Cluster has orphaned object entries in ClusterHive registry key, the System Center Operations Manager agent may not discover some or all Cluster resources.

Fix
The OpsMgr Engineering team has posted an article about this issue and how to fix it, to be found here.

New MP: Microsoft Dynamics AX 2012 R3 Retail

Yesterday Microsoft released a new MP for monitoring Microsoft Dynamics AX 2012 R3 Retail. The version of this MP is 6.3.100.0.

This MP runs on SCOM 2007 R2 and higher. HOWEVER, the website states ‘…You must use the latest CU for whatever version you have installed…’.

Decide for yourself to what extend this MP will check for that ‘requirement’ . But when having issues AND not running the latest CU, Microsoft CSS will ask you to do so, before they can continue delivering their support to you.

This MP can be downloaded from here.

SCCM 2012 Tools

For a SCCM 2012 assignment I needed some additional power which I found in some good SCCM 2012 tooling. Just in case you need it as well I decided to post it on my blog.

SCCM 2012 R2 Toolkit
Contains fifteen downloadable tools to help you manage and troubleshoot Microsoft System Center 2012 R2 Configuration Manager.

For more information and download location, go here.

SCCM 2012 Support Center
Helps you to gather information about SCCM 2012 clients, so that you can more easily address issues with those clients when working with product support specialists. Configuration Manager Support Center includes a tool that gathers a bundle of log files, and also a tool that is used by product support specialists to examine log files and other client data for in-depth analysis of issues with Configuration Manager clients.

For more information and download location, go here.

SCOM12 R2: FINALLY The Collation Settings Issue Is Addressed!

The past
As we all know up to SCOM12 R2 the Collation Settings of the SCOM SQL databases was crucial and ONLY SQL_Latin1_General_CP1_CI_AS worked, even though Microsoft stated differently.

Main reason here was that the installer of SCOM installed the Data Warehouse database with this SQL Collation setting (SQL_Latin1_General_CP1_CI_AS) no matter what (SQL) Collation Setting the SQL instance itself was running.

And when there is a mismatch between the (SQL) Collation settings of both SCOM SQL databases (OperationsManager & Data Warehouse) you could end up in situations like these. Only a reinstall would solve that issue OR – when being blessed with really power SQL DBAs – the databases could be preserved while modifying the SQL Collation Settings.

The good news
Finally, with SCOM12 R2 this issue is fixed! The installer of SCOM 2012 R2 doesn’t install the Data Warehouse database anymore with the SQL Collation Settings SQL_Latin1_General_CP1_CI_AS, but uses the (SQL) Collation setting of the related SQL instance instead.

And – FINALLY – now SCOM 2012 R2 REALLY supports many different (SQL) Collation Settings, compared to the previous versions of SCOM 2012.

On top of it all, instead of using SQL Collation Settings (starting with the prefix SQL) Microsoft has decided to move on to supporting Windows Collation Settings. This is good as well since SQL Collation Settings will be phased out in the years to come and fully replaced by Windows Collations instead.

What Collation Settings are supported with SCOM 2012 R2?
Quite a lot actually. And no, I haven’t tested them all (most of them not actually, the ones I am 100% sure about are printed in bold) but this is what Microsoft TechNet states:

SQL Server collation settings for all databases must be one of the following:

1. SQL_Latin1_General_CP1_CI_AS;
2. Latin1_General_100_CI_AS, // EN, IT, DE, PT-BR, NE, PT-PT;
3. French_CI_AS; French_100_CI_AS;
4. Cyrillic_General_CI_AS;
5. Chinese_PRC_CI_AS;
6. Chinese_Simplified_Pinyin_100_CI_AS, // CN simplified;
7. Chinese_Traditional_Stroke_Count_100_CI_AS, // CN traditional, CN-HK;
8. Japanese_CI_AS;
9. Japanese_XJIS_100_CI_AS;
10. Traditional_Spanish_CI_AS;
11. Modern_Spanish_100_CI_AS;
12. Latin1_General_CI_AS;
13. Cyrillic_General_100_CI_AS, // RU;
14. Korean_100_CI_AS;
15. Czech_100_CI_AS;
16. Hungarian_100_CI_AS;
17. Polish_100_CI_AS;
18. Finnish_Swedish_100_CI_AS.

So this is really some good news which isn’t really specifically documented.

How does this work out for me?
Well, when you go for a CLEAN install of SCOM 2012 R2, the above mentioned (SQL) Collation Settings are the ones you can choose from.

But when you run an UPGRADE from a previous version to (finally) SCOM 2012 R2, you’ve deal with the (SQL) Collation Settings already in place in the previous version of SCOM.

And no, an upgrade WON’T change the (SQL) Collation Settings of any SCOM database NOR will it fix issues for you when you’re having issues with conflicting (SQL) Collation Settings.

In cases like these it’s far more better (and easier) to install a brand new SCOM 2012 R2 environment and run an along side migration scenario instead of an inplace upgrade scenario.

Kenneth van Surksum Joins Insight24 as Managing Consultant

On the first of May 2014 Kenneth van Surksum has joined Insight24 as managing consultant.

Kenneth is a former Enterprise Client Management MVP and for the second year vExpert. His knowledge, real-life experience and know-how about ConfigMgr and other System Center related technologies goes very deep.

On top of it all, Kenneth is also one of the cofounders of Windows Management User Group Netherlands (WMUG NL), and plays an active and significant role in it. So he has the true community spirit as well.

Therefore I am more than happy and proud to welcome him to the team of Insight24!