Phew! Had my ‘special’ taste of SCCM last week! All about OSD using SCCM 2012 R2 CU#2 and multicasting.
And yes, the dreaded error 0x80091007 came along way too many times. I searched on the internet but NO MATTER WHAT I DID, TRIED, TESTED, MODIFIED, DELETED, UPDATED and REDISTRIBUTED, NOTHING HELPED!!!
Even worse, the behavior of the OSD became erratic. We had in total 3 OSD’s using multicast. Right from the start 1 out of 3 and the two others failed with the 0x80091007 error. Just in 15 seconds max, when multicasting should have started this error popped up.
Of course, the related log file (smsts.log) was taken from the failed client and checked. And yes, the well known errors popped up, like:
- Encountered error transfering file (0x80070003).
- Sending status message: SMS_OSDeployment_PackageDownloadMulticastStatusFail;
- Hash could not be matched for the downloded content. Original ContentHash = 905E6AEDF8BD29DC83A41552CC248CFEAB9A434E97DE699CE8FF5647371D6367, Downloaded ContentHash =;
- DownloadContentAndVerifyHash() failed. 80091007;
- Installation of image 2 in package HQ1000F5 failed to complete. The hash value is not correct. (Error: 80091007; Source: Windows);
- The user tries to release a source directory C:\_SMSTaskSequence\Packages\HQ1000F5 that is either already released or we have not connected to it;
- Failed to run the action: Apply Operating System.
The hash value is not correct. (Error: 80091007; Source: Windows).
Yes, 0x80091007 error is related to content mismatch errors. But please check out the yellow high lighted piece of error 3. Since NOTHING is downloaded to the client there is NO hash as well. So in this case this 0x80091007 error turned out to be misleading…
Things we tried:
- Error 80091007 is most of the times related to hash issue. So when updating the related DPs OR even redistributing it, a new hash is created. But this one didn’t work.
- Antivirus software. We disabled it COMPLETELY on all SCCM systems involved. And then we ran the first step (redistributing the OS images) again. But no luck this time.
- Permissions? We checked and double checked, but the OS Image which kept on working like clock work, had the exact same permissions as the ones which kept on failing;
- DP issue? Package not available? Duh! The DP involved neatly logged an error as well in it’s own Windows Event Logs for the Deployment Server. This error contained neatly the URL to the package. And guess what? When I clicked it, the OS Image (WIM file) was neatly downloaded!
- ESX issue? When the SCCM is a VM running on VMware there might be some issues with the NIC. Sow we removed the old NIC – first from Windows then from VMware, rebooted the server, gave it a new NIC, configured it properly (both in VMware and Windows), rebooted the server. And guess what? NO LUCK! The OS Image which ran without issues from the first moment kept on running and the others kept on FAILING!!!
- DP issue? So I REMOVED both problematic OS Images. Removed even the Task Sequences but that shouldn’t be necessary since the Task Sequences only contain meta data. But none the less, let’s start CLEAN. I waited a WHOLE DAY so SCCM had ALL the time to sort things out. And after that day, I added the two OS images, set the distribution options to Multi Casting, distributed them to the related DPs and ONLY after that was done, I rebuilt the required deployment Task Sequences. Again SAME ERROR!!!
- SCCM issue? Checked about 100x logs of SCCM and no where a glitch to be found. SCCM seemed to be in a healthy state. All lights green and all systems a go go! But NO WAY the two problematic OS images would deploy by using multicast.
- Oh, did I already tell you that UNICAST just run like clockwork? ALL THE TIME? For ALL OS Images, also the two problematic ones when trying to use multicast?
- I checked out tons of logs, from the BIG names, like Kenneth van Surksum, Peter Daalmans, Henk Hoogendoorn and so on. All those big names bumped into similar issues, but all the things they proposed simply didn’t work. And many times they kept on having issues with multicasting…
The BIG surprise AND the erratic behavior…
All of a sudden – just like that – one of the two problematic OS Images started to work with multicast deployment! Just out of the blue! We tried to copy the situation, but no matter what, the last image wouldn’t budge 
. So now we had two OS images running okay using multicast. But only in an isolated test lab.
And the big REAL world – using vLANs (uh oh) was waiting out there…
The verdict
Based on our experiences gained while trying to solve the multicast issues, we came to know multicast used by SCCM 2012 R2 as a mechanism requiring a lot of maintenance combined with black magic and a lot of bad mojo. Not really a toolset to be used in the real world, requiring rock solid and trust worthy end results, to be reproduced over and over…
So in this case multicast is removed and unicast rules.
PS…
Based on the blogs I bumped into it seems like multicast is a pain the well know B#@ side. Just wondering how many people got it up and running AND trust worthy in networks which are segmented using many vLANs.
