Ouch. That’s quite a title. But something I bumped into some time ago. There is much to tell & share so let’s start.
IssueBy default SCOM stores everything in UTC time in it’s database. However, the SCOM Console shows those dates based on the local time settings of that ‘box’.
In the Netherlands we use (UTC + 1:00), so there will be a delta of
1 hour between the times shown in the SCOM Console, and the time as it is in the SCOM database. And when
Daylight Saving Time is being used AND it’s ‘summer time’ that delta will be
2 hours.
Example:
When looking in the SCOM Console for all Alerts with the Name
Power Shell Script failed to run, this is what the SCOM Console in my test lab shows me:
Mind you, 4:24:22 PM is
16:24:22 and 4:24:21 PM is
16:24:21.
However, when I use PowerShell in order to get these two Alerts by using this PS one liner
Get-SCOMAlert -Criteria "Name='Power Shell Script failed to run' AND ResolutionState = 0" | FT Name, LastModified
this is what I see:
As you see, there is the earlier mentioned delta of two hours!
What’s the BIG deal here?On it self it isn’t shocking. However, suppose you want to schedule a PS script – once per hour – which closes all Alerts generated by Rules which are 48 hours or older.
In this case – because of the delta of two hours – also Alerts generated by Rules which are 46 hours old, will be closed by that same scheduled PS script as well.
One could say, well a delta of 2 hours on 48 hours isn’t that much. So instead, modify the PS script and set it to 50 hours instead.
Yes, that could work. But what about the
Daylight Saving Time? When it’s winter time in many European countries, the clock is adjusted. So now the delta becomes 1 hour instead. So now the PS script should be adjusted to 49 hours instead.
Changes are people are going to forget this. But there is an even bigger deal here.
Suppose you want to close a certain set of Alerts generated by a certain set of Rules for a certain set of systems which are TEN MINUTES or older? Since the delta is at least one hour, that same PS script will close ALL those Alerts, even when they’re totally new, like one second old.
Now that’s BAD!!! Because no matter how you schedule such a PS script, sooner or later Alerts will be closed by that PS script which shouldn’t be closed at all. So people are going to miss out on certain Alerts and when they find out, they’re going to complain. And then the culprit
(the badly written PS script will be found and after that the person who wrote it will have to explain him/her self…)PowerShell blame game or to the rescue?I don’t know about you, but I LOVE PowerShell. Simply because it’s SO powerful AND there are so many resources out there. Many times I find myself using the Help function in PS in conjunction with PS scripts I find on the internet
(thank you all for sharing!) and modifying it to the requirements of my customer.
So in this case I don’t blame PowerShell since it does exactly as ‘told’ but I use it in order to make it a bit more smarter, so it calculates the delta between the UTC time and local time on the box
(in the Netherlands it’s either 1 or 2 hours) and uses that same delta in order to calculate the new offset used to filter the correct Alerts.
This way this PS script becomes ‘Set & Forget’ since it doesn’t require any modification ever, no matter it’s summer or winter time. Awesome!
In the case of the PS script which has to close certain Alerts generated by certain Rules for a certain set of systems which are TEN MINUTES or older, the part of the PS script which calculates the exact delta and the resulting offset, looks like this:
$CurrentDate = Get-Date
$UTC = $CurrentDate.ToUniversalTime()
$Diff = $CurrentDate - $UTC
$Conversion = [string]$Diff
$Conversion = $Conversion.Replace(":00:00", "")
[int]$TimeToAdd=$Conversion
$AgeMinutes = $TimeToAdd*60+10
In just 7 simple lines of PS code
(of course I know there are multiple ways to reach the same goals, especially with PS. So feel free to comment) :
- The current date is retrieved ($CurrentDate = Get-Date),
- Recalculated to UTC time ($UTC = $CurrentDate.ToUniversalTime()),
- The difference between the local time and UTC time is calculated ($Diff = $CurrentDate - $UTC),
- Casted into a string ($Conversion = [string]$Diff),
- Stripped from all the unneeded zero’s and so on ($Conversion = $Conversion.Replace(":00:00", "")),
- Casted again, now to an integer ([int]$TimeToAdd=$Conversion),
- And FINALLY used to calculate the correct offset of ten minutes ($AgeMinutes = $TimeToAdd*60+10).
Which is used in this example of PS code, in the same script:
$PSAlerts = Get-SCOMAlert -Criteria "Name='Power Shell Script failed to run' AND ResolutionState=0 AND MonitoringObjectPath LIKE 'MS0%.sc.local'" |where {$_.LastModified -le ($AgeMinutes)}And guess what? It works!
The script itselfWell, I am not going to share the PS script I used for this particular customer in order to close a certain set of Alerts, generated by certain Rules for a certain set of systems which are TEN MINUTES or older. Simply because I don’t want to share customer specific information.
Instead I am going to share
(or give back to the community) the PS script which uses the same mechanism as I already told about, which closes ALL Alerts generated by Rules, which are 48 hours old or older. So this is the PS script, also available for download from my
OneDrive:
#######################################################################################
# Script to close old Rule based Alerts older than 48 hours
# Based on copy & paste of different PS scripts & modified as required by Marnix Wolf
# Keep the community spirit alive!
# This script is based on SCOM 2012 R2 PS support & functionality
#######################################################################################
#Import SCOM 2012 Module
Import-Module OperationsManager
New-SCManagementGroupConnection -ComputerName [NAME OF SCOM 2012x MANAGEMENT SERVER]
#Add 2 days (48 hours) and reckon with UTC difference & Day Light Savings
$CurrentDate = Get-Date
$UTC = $CurrentDate.ToUniversalTime()
$Diff = $CurrentDate - $UTC
$Conversion = [string]$Diff
$Conversion = $Conversion.Replace(":00:00", "")
[int]$TimeToAdd=$Conversion
$AgeHours = $TimeToAdd+48
#Identifies Alerts generated by Rules which are older than 2 days (48 hrs) & closes them with adding a comment.
$PSAlerts = Get-SCOMAlert -Criteria "ResolutionState <> 255 AND IsMonitorAlert='False'" |where {$_.TimeAdded -le (Get-Date).addhours(-$AgeHours)}
If ($PSAlerts -eq $null) {write "No Rule based alerts found which are 48 hours old or older."}
Else {
foreach ($PSAlert in $PSAlerts)
{
Set-SCOMAlert -Alert $PSAlert -ResolutionState 255 -Comment "Alert Autoclosed by PS script. Alert is Rule based and 48 hours or older."
}
}
#End of script
In this script all non closed Alerts are ‘touched’ by this script, meaning all Alerts which don’t have 255 as Resolution State
(= Closed). When you don’t want that but to close ONLY Alerts which are New
(ResolutionState 0) modify the yellow highlighted code in the script above to:
ResolutionState=0
#End of posting
.
.
.
.
.
.
