DMZ Monitoring Of Windows 2003 Servers Fails: Root Certificate Not Accepted

On a customers location some DMZ servers had to be monitored. These servers were part of a Workgroup so Kerberos wouldn’t do here for the required authentication and encryption. Instead special certificates were used, issued by a CA based on Windows Server 2008 R2 SP1 Enterprise.

Everything worked fine until some Windows 2003 Servers residing in the DMZ had to be monitored.

The issue
Even though the CA Certificate Chain was properly loaded in the Trusted Root Certification Authorities store of the computer account of the Windows 2003 servers, the SCOM certificate gave an error: The integrity of this certificate cannot be guaranteed. the certificate may be corrupted or may have been altered.

The Certificate status, found under the tab Certification Path gave this error: This certificate has an nonvalid digital signature.

Again, we were puzzled. It was the same CA Certificate Chain used on the Windows 2008 servers where it caused no issue at all. And the SCOM Certificate was loaded in the proper store as well.

Since all Windows 2003 servers had the same issue, something else was happening here. An incompatibility issue most probably between the CA based on Windows Server 2008 R2 and Windows Server 2003.

The cause
After some searching on the internet, we bumped into KB968730 stating: ‘…Windows Server 2003 and Windows XP clients cannot obtain certificates from a Windows Server 2008-based certification authority (CA) if the CA is configured to use SHA2 256 or higher encryption…’

In other words (taken from this blog posting so all credits go to Tim Jacobs!): ‘…Windows 2008 has several new additions to the cryptography API, called Cryptography Next Generation (CNG), that are used in the V3 certificate templates for CA's and Webservers in Windows 2008. Amongst those new features is support for new certificate signing algorithms which is not recognized by older clients…’

In the days this posting was written the ONLY solution available was to REINSTALL the CA! But that’s something you don’t want to do except when there is no other solution available.

The fix
Gladly years have gone by and Microsoft published a hotfix for it which solves this issue on the Windows 2003 servers. So no need to reinstall your CA .

KB968730 has this hotfix available for download.

Recap
Whenever you need to monitor DMZ servers, or other Windows servers which reside outside the trust boundary of your SCOM MG and those servers are Windows Server 2003 based AND your CA is Windows Server 2008 based, changes are you’re going to need the hotfix listed in KB968730.