Management Pack Quality Survey: The Results – Part I: Answers to Question 1

----------------------------------------------------------------------------------
Postings in the same series:
The Survey
Part II – Answers to Question 2 & 3
----------------------------------------------------------------------------------

The introduction
On the 13th of December 2011 I posted a survey all about MP Quality.  Even though not too many people responded (28 in total), it still got some good exposure since some other bloggers also referred to this survey. And the people who responded to the survey are the ones who run multiple and BIG SCOM environments in different configurations and setups.

This results IMO into a survey which isn’t high at a quantity level but has a high quality level all because the people who responded really know what they’re talking about. They’re consultants themselves or highly experienced technical people with deep practical knowledge and experience with SCOM and it’s related aspects like MP authoring or offering SCOM R2 as a service to other companies. So they really know what they’re talking about.

Some people pointed out to me that I have forgotten one MP, the Group Policy MP. Sorry about that one. I thought to have covered all of them, but unfortunately I didn’t. As it turns out almost all Monitors in this MP require a manual reset, which introduces much manual labor. Which isn’t nice at all…

Question One
In a series of blog postings I will share the results of this survey with you. This posting will be about the answers given to Question 1: ‘What do you think of the quality of these Management Packs?’

The raw results can be found here. For this posting I have ‘translated’ the outcome in order to get a better understanding of the answers and its relations. Why? Suppose that out of 28 people 27 state not to use the MP. So only a single person states to use the MP. It goes way too far to state that his or her answer represents the general feeling about that particular MP.

Also the highest percentage for a possible answer counts. Example: One could answer Bad (1), Poor (2), Acceptable (3), Good (4), Outstanding (5) or Don’t know. Don’t use this MP (6), where the latter is skipped by me. For the SharePoint 2007 MP 10 out of 28 people responded to think the MP to be Acceptable. Only four think less of that MP. So this MP got as Quality rating Acceptable.

The Translation
In total the question ‘What do you think of the quality of these Management Packs?’ was asked for 33 MPs. One MP was unknown by the people who responded, the SQL Server Appliance MP, so in total 32 MPs remain.

In total 28 people responded to this survey. Some MPs weren’t used by many of them so I skipped those answers since the basis would be too small to say anything ‘solid’. As a rule of thumb I only looked at the MPs which were used by 13 people or more. This gave me 20 MPs in total:

1. SharePoint 2007
2. SharePoint 2010
3. Exchange 2007
4. Exchange 2010
5. OCS
6. Forefront TMG
7. ISA 2004/2006
8. SCCM
9. Windows Server Operating System
10. File Services 2008 R2
11. Terminal Services
12. Remote Desktop Services
13. Hyper-V
14. SQL
15. IIS
16. AD
17. DNS
18. DHCP
19. WINS
20. Print Server

This also gives a good view of what Microsoft MPs people use most often. MPs which I skipped since 12 or less people use them:

1. Lync server 2010: 8 out of 28 persons use it;
2. Lync Server 2010 Group Chat: 1 person out of 28 use it;
3. ForeFront Identity Manager 2010: 4 persons out of 28 use it;
4. Forefront Protection 2010 for Exchange 2010: 7 persons out of 28 use it;
5. SCSM: 5 persons out of 28 use it;
6. DPM 2007: 9 persons out of 28 use it;
7. DPM 2010: 8 persons out of 28 use it;
8. Visual Studio 2010 Team Foundation Server: 1 person out of 28 use it;
9. Windows Azure Applications: 2 persons out of 28 use it;
10. Microsoft Dynamics CRM 2011: 3 persons out of 28 use it;
11. Microsoft BitLocker: 3 persons out of 28 use it;
12. Windows Storage Server 2008 R2: 3 persons out of 28 use it;
13. SQL Server Appliance: 0 persons out of 28 use it.

Perhaps these MPs should get more exposure in order to be used by more people.

Out of the 20 MPs getting a rating their are five MPs which underperform (1 gets the rating Poor and 4 of them  Bad). Six of them do get the rating Acceptable (which is basically OK but nothing more). Seven of them get the rating Good and the remaining two get the same count for the ratings Acceptable and Good.

None of the MPs gets the rating Outstanding.

Bad rated MPs:

1. SCCM
2. DHCP
3. WINS
4. Print Server

IMO:
- Most of the MPs listed here are the old converted ones. Only exception here is the OCS MP;
- Most of the MPs listed here haven’t got an update for a long time so there is no maintenance. Only exception here is the SCCM MP;
- The DHCP MP isn’t OK. It should get a better presentation in the SCOM Console and should work properly;
- The Print Server MP is a shame. Pull it or improve it. Now it kills the overall experience of SCOM.

Poor rated MPs:

1. OCS

IMO:
- OCS is an old product now. Lync is the successor and comes with a better MP. Want a better MP for OCS? Migrate to Lync :)

Acceptable rated MPs:

1. SharePoint 2007
2. ForeFront TMG
3. ISA 2004/2006
4. Terminal Services
5. Remote Desktop Services
6. DNS

IMO:
- These MPs are native SCOM MPs thus deliver added value to any SCOM environment.

Acceptable and Good rated MPs:

1. Exchange 2010 (Even though 14 people rated this MP Acceptable or Good, 6 people rated this MP to be bad as well…)
2. File Services 2008 R2

Good rated MPs:

1. SharePoint 2010
2. Exchange 2007
3. Windows Server Operating System
4. Hyper-V
5. SQL
6. IIS
7. AD

IMO:
- Most of these MPs are the ones which showcase the power and strength of SCOM, except for the Hyper-V MP;
- The Hyper-V MP is just too basic. Add more power to it;
- The SQL MP is a showcase what a MP should be all about, including version control and update cycles.

Conclusion
Too many MPs are rated Acceptable or even less. IMHO: Since SCOM/OM12 has to compete with other enterprise monitoring solutions it’s important to have MPs – which are delivered by Microsoft – to be top notch thus a shiny example to third party vendors. So a rating Acceptable for any MP delivered by Microsoft isn’t going to kill the competition and should be at least Good. This way SCOM becomes an even better product.

In the next posting of this series I will post and discuss the answers given to the second question in this survey: What do you think of the quality of the guides that are delivered with these Management Packs? So stay tuned!