SCOM/OM12 Agent with UAG/TMG ISATAP and IPv6. A special combination it is…

Bumped today into this issue. A customer of mine has a UAG server in place with some basic TMG functionality as well. For the UAG functionality ISATAP is in place as well on the same server. And this can be a challenge in order to get the SCOM/OM Agent to communicate with the SCOM R2 MG.

The Challenge
Since I have already installed the SCOM/OM12 Agent many times on UAG/TMG servers and configured TMG accordingly I thought this would be an easy one. The installation of the Agent went fine. But no way the Agent wanted to communicate with the SCOM R2 MG. Even though I had adjusted the TMG component as stated in the TMG MP document:

And yes, I applied the modification as well and checked to see the configuration had been processed properly. But still no luck…

The SCOM event log showed EventID 21006: ‘…The OpsMgr Connector could not connect to <FQDN MANAGEMENT SERVER>:5723.  The error code is 10013L(An attempt was made to access a socket in a way forbidden by its access permissions.).  Please verify there is network connectivity, the server is running and has registered it's listening port, and there are no firewalls blocking traffic to the destination….’.

This really puzzled me. So I started a query in the TMG Console and this is what I got:

The Cause
Notice the yellow highlighted IPv6 prefixes. Right after those prefixes the IPv4 addresses are shown. So instead of IPv4 addresses only, the TMG component received the translated addresses instead. And TMG can’t handle IPv6 so it drops them…

When I pinged the SCOM Management Servers from the UAG/TMG server, all I got was those IPv4 to IPv6 translated addresses instead. And vice versa.

The Solution
There are multiple solutions for this one. However, one doesn’t want to break any other functionality as well (like breaking ISATAP for instance). So disabling IPv6 on that server isn’t going to work. Nor disabling IPv6 on the SCOM Management Servers.

A colleague of mine came to this idea, which works like a charm:

1. On all SCOM Management Servers, edit the HOSTS file so it contains the FQDN of the UAG/TMG server with its IPv4 address;
2. Flush the DNS cache on those servers after having saved the modified HOSTS file;
3. On the UAG/TMG server, modify the HOSTS file as well by adding the FQDNs of the SCOM Management Servers and their related IPv4 addresses. Save the file and flush the DNS cache on that server as well;
4. Restart the Agent service on the UAG/TMG server and BINGO EventID 20070 was shown now, telling me that I had to approve the server in the SCOM R2 Console :).

Now all is well. UAG/TMG and ISATAP are fully functional AND the server is monitored by SCOM….

Many thanks to my colleagues who helped me to crack this one. Thanks guys!