Issue with setting two threshold based on an average and a maximum value

Jan 22 at 6:03 PM
Hi,

What I am trying to achieve is to set two thresholds based on an average and a maximum value.

An example of this would be for the following Exchange 2010 counter, where I want to have a warning for an average value of higher than 20 and for a maximum value higher than 50.

Counter: \*\MSExchange Database Instances(*)\I/O Database Reads Average Latency
Average: Should be 20 ms on average.
Max: Should show 50 ms spikes.
Source: https://technet.microsoft.com/en-us/library/bb201689(v=exchg.80).aspx

Taking the following example counter below from the “SamplePerfmonLog.blg” file, I have attempted to change the threshold values as per below to try and achieve what I am after using the following logic:

\Processor(*)\% Privileged Time
Avg (20) -gt 20 = True
Max (53) -gt 60 = False

Standard - QuickSystemOverview.xml
Condition \Processor(*)\% Privileged Time Min Avg Max
More than 30% privileged (kernel) mode CPU usage ZACH-PC/_Total 15 30(Warn) 53(Crit)
More than 30% privileged (kernel) mode CPU usage ZACH-PC/0 15 30(Warn) 53(Crit)

What I am after is the following
Condition \Processor(*)\% Privileged Time Min Avg Max
More than 30% privileged (kernel) mode CPU usage ZACH-PC/_Total 15 30(Warn) 53
More than 30% privileged (kernel) mode CPU usage ZACH-PC/0 15 30(Warn) 53

Warning PowerShell Threshold Code:
StaticThreshold -CollectionOfCounterInstances $CollectionOfProcessorPercentPrivilegedTimeAll[0].Avg -Operator 'gt' -Threshold 20  
Critical PowerShell Threshold Code:
StaticThreshold -CollectionOfCounterInstances $CollectionOfProcessorPercentPrivilegedTimeAll[0].Max -Operator 'gt' -Threshold 60
When running PAL with the above amendments, it errors with the following:

Overall progress... Status: Applying thresholds, PAL 2.7.5 Progress: 82%... Applying thresholds, Processor % Privileged Time
An error occurred on...
For ($i=0;$i -lt $CollectionOfCounterInstances.Count;$i++)
At D:\PAL_Flatfiles_v2.7.5_x64\PAL.ps1:4036 char:15 + For ($i=0;$i -lt $CollectionOfCounterInstances.Count;$i++) + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The property 'Count' cannot be found on this object. Verify that the property exists.

Am I declaring the correct values within the PowerShell or is there something else I should be doing, as no matter what I but in the Powershell code it is always treating the values as the same .

I have tried the following variations but still no joy

$CollectionOfProcessorPercentPrivilegedTimeAll[0].Avg
$($CollectionOfProcessorPercentPrivilegedTimeAll[0].Avg)
$($CollectionOfProcessorPercentPrivilegedTimeAll[0]).Avg
$($CollectionOfProcessorPercentPrivilegedTimeAll).Avg

Many thanks
James
Jan 24 at 12:31 PM
I have now manage to achieve what I want with help from Clint via email :-) (thanks)

Here is what I did to achieve my goal:

1) Make a backup copy of PAL.ps1
2) Edit PAL.ps1 and add the following two new functions:
Function StaticThresholdAvgOnly
{
    param($CollectionOfCounterInstances,$Operator,$Threshold,$IsTrendOnly=$False)

    For ($i=0;$i -lt $CollectionOfCounterInstances.Count;$i++)
    {
        $oCounterInstance = $CollectionOfCounterInstances[$i]
        
        #Check OverallCounters
        If ($oCounterInstance.Name.Contains('INTERNAL_OVERALL_COUNTER_STATS_') -eq $True) 
        {
            If (($oCounterInstance.Avg -ne '-') -and ($oCounterInstance.Avg -ne $null))
            {
                If ($oCounterInstance.Avg -is [System.Char])
                {
                    [System.Int32] $iAvg = $oCounterInstance.Avg
                    [System.Double] $iAvg = $iAvg
                }
                Else
                {
                    [System.Double] $iAvg = $oCounterInstance.Avg
                }

                $IsMinThresholdBroken = $False
                $IsAvgThresholdBroken = $False
                $IsMaxThresholdBroken = $False
                $IsTrendThresholdBroken = $False
                $IsMinEvaluated = $False
                $IsAvgEvaluated = $False
                $IsMaxEvaluated = $False
                $IsTrendEvaluated = $False

                switch ($Operator)
                {
                    'gt'
                    {
                        If ([System.Double] $iAvg -gt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'ge'
                    {
                        If ([System.Double] $iAvg -ge [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'lt'
                    {
                        If ([System.Double] $iAvg -lt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'le'
                    {
                        If ([System.Double] $iAvg -le [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'eq'
                    {
                        If ([System.Double] $iAvg -eq [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    default
                    {
                        If ([System.Double] $iAvg -gt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }                    
                    }
                }
        

                If (($IsMinThresholdBroken -eq $True) -or ($IsAvgThresholdBroken -eq $True) -or ($IsMaxThresholdBroken -eq $True) -or ($IsTrendThresholdBroken -eq $True))
                {
                    #Write-Host "Create alert - Overall: True"
                    CreateAlert -TimeSliceIndex 0 -CounterInstanceObject $oCounterInstance -IsMinThresholdBroken $IsMinThresholdBroken -IsAvgThresholdBroken $IsAvgThresholdBroken -IsMaxThresholdBroken $IsMaxThresholdBroken -IsTrendThresholdBroken $IsTrendThresholdBroken -IsMinEvaluated $IsMinEvaluated -IsAvgEvaluated $IsAvgEvaluated -IsMaxEvaluated $IsMaxEvaluated -IsTrendEvaluated $IsTrendEvaluated
                }
                Else
                {
                    #Write-Host "Create alert - Overall: False"
                }
            }
        }         
        Else
        {
            #Check Alerts (Quantized)
            For ($t=0;$t -lt $alQuantizedTime.Count;$t++)
            {
                $IsMinThresholdBroken = $False
                $IsAvgThresholdBroken = $False
                $IsMaxThresholdBroken = $False
                $IsTrendThresholdBroken = $False
                $IsMinEvaluated = $False
                $IsAvgEvaluated = $False
                $IsMaxEvaluated = $False
                $IsTrendEvaluated = $False

                If ($oCounterInstance.QuantizedAvg -is [System.String])
                {
                    [System.Double[]] $aQuantizedAvg = @($oCounterInstance.QuantizedAvg.Split(','))
                }
                Else
                {
                    [System.Double[]] $aQuantizedAvg = @($oCounterInstance.QuantizedAvg)
                }

                $iQuantizedAvg = $aQuantizedAvg[$t]

                switch ($Operator)
                {
                    'gt'
                    {
                        If ([System.Double] $iQuantizedAvg -gt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'ge'
                    {
                        If ([System.Double] $iQuantizedAvg -ge [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'lt'
                    {
                        If ([System.Double] $iQuantizedAvg -lt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'le'
                    {
                        If ([System.Double] $iQuantizedAvg -le [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    'eq'
                    {
                        If ([System.Double] $iQuantizedAvg -eq [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }
                    }
                    default
                    {
                        If ([System.Double] $iQuantizedAvg -gt [System.Double] $Threshold)
                        {
                            $IsAvgThresholdBroken = $True
                        }                    
                    }
                }

                If (($IsMinThresholdBroken -eq $True) -or ($IsAvgThresholdBroken -eq $True) -or ($IsMaxThresholdBroken -eq $True) -or ($IsTrendThresholdBroken -eq $True))
                {
                    #Write-Host "Create alert - Detail: True"
                    CreateAlert -TimeSliceIndex $t -CounterInstanceObject $oCounterInstance -IsMinThresholdBroken $IsMinThresholdBroken -IsAvgThresholdBroken $IsAvgThresholdBroken -IsMaxThresholdBroken $IsMaxThresholdBroken -IsTrendThresholdBroken $IsTrendThresholdBroken -IsMinEvaluated $IsMinEvaluated -IsAvgEvaluated $IsAvgEvaluated -IsMaxEvaluated $IsMaxEvaluated -IsTrendEvaluated $IsTrendEvaluated
                } Else {
                    #Write-Host "Create alert - Detail: False"
                }
            }
        }
    }
}
Jan 24 at 12:32 PM
Function StaticThresholdMaxOnly
{
    param($CollectionOfCounterInstances,$Operator,$Threshold,$IsTrendOnly=$False)

    For ($i=0;$i -lt $CollectionOfCounterInstances.Count;$i++)
    {
        $oCounterInstance = $CollectionOfCounterInstances[$i]
        
        #Check OverallCounters
        If ($oCounterInstance.Name.Contains('INTERNAL_OVERALL_COUNTER_STATS_') -eq $True) 
        {
            If (($oCounterInstance.Max -ne '-') -and ($oCounterInstance.Max -ne $null))
            {
                If ($oCounterInstance.Max -is [System.Char])
                {
                    [System.Int32] $iMax = $oCounterInstance.Max
                    [System.Double] $iMax = $iMax
                }
                Else
                {
                    [System.Double] $iMax = $oCounterInstance.Max
                }

                $IsMinThresholdBroken = $False
                $IsAvgThresholdBroken = $False
                $IsMaxThresholdBroken = $False
                $IsTrendThresholdBroken = $False
                $IsMinEvaluated = $False
                $IsAvgEvaluated = $False
                $IsMaxEvaluated = $False
                $IsTrendEvaluated = $False

                switch ($Operator)
                {
                    'gt'
                    {
                        If ([System.Double] $iMax -gt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'ge'
                    {
                        If ([System.Double] $iMax -ge [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'lt'
                    {
                        If ([System.Double] $iMax -lt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'le'
                    {
                        If ([System.Double] $iMax -le [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'eq'
                    {
                        If ([System.Double] $iMax -eq [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    default
                    {
                        If ([System.Double] $iMax -gt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }                    
                    }
                }
        

                If (($IsMinThresholdBroken -eq $True) -or ($IsAvgThresholdBroken -eq $True) -or ($IsMaxThresholdBroken -eq $True) -or ($IsTrendThresholdBroken -eq $True))
                {
                    #Write-Host "Create alert - Overall: True"
                    CreateAlert -TimeSliceIndex 0 -CounterInstanceObject $oCounterInstance -IsMinThresholdBroken $IsMinThresholdBroken -IsAvgThresholdBroken $IsAvgThresholdBroken -IsMaxThresholdBroken $IsMaxThresholdBroken -IsTrendThresholdBroken $IsTrendThresholdBroken -IsMinEvaluated $IsMinEvaluated -IsAvgEvaluated $IsAvgEvaluated -IsMaxEvaluated $IsMaxEvaluated -IsTrendEvaluated $IsTrendEvaluated
                }
                Else
                {
                    #Write-Host "Create alert - Overall: False"
                }
            }
        }         
        Else
        {
            #Check Alerts (Quantized)
            For ($t=0;$t -lt $alQuantizedTime.Count;$t++)
            {
                $IsMinThresholdBroken = $False
                $IsAvgThresholdBroken = $False
                $IsMaxThresholdBroken = $False
                $IsTrendThresholdBroken = $False
                $IsMinEvaluated = $False
                $IsAvgEvaluated = $False
                $IsMaxEvaluated = $False
                $IsTrendEvaluated = $False

                If ($oCounterInstance.QuantizedMax -is [System.String])
                {
                    [System.Double[]] $aQuantizedMax = @($oCounterInstance.QuantizedMax.Split(','))
                }
                Else
                {
                    [System.Double[]] $aQuantizedMax = @($oCounterInstance.QuantizedMax)
                }

                $iQuantizedMax = $aQuantizedMax[$t]

                switch ($Operator)
                {
                    'gt'
                    {
                        If ([System.Double] $iQuantizedMax -gt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'ge'
                    {
                        If ([System.Double] $iQuantizedMax -ge [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'lt'
                    {
                        If ([System.Double] $iQuantizedMax -lt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'le'
                    {
                        If ([System.Double] $iQuantizedMax -le [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    'eq'
                    {
                        If ([System.Double] $iQuantizedMax -eq [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }
                    }
                    default
                    {
                        If ([System.Double] $iQuantizedMax -gt [System.Double] $Threshold)
                        {
                            $IsMaxThresholdBroken = $True
                        }                    
                    }
                }

                If (($IsMinThresholdBroken -eq $True) -or ($IsAvgThresholdBroken -eq $True) -or ($IsMaxThresholdBroken -eq $True) -or ($IsTrendThresholdBroken -eq $True))
                {
                    #Write-Host "Create alert - Detail: True"
                    CreateAlert -TimeSliceIndex $t -CounterInstanceObject $oCounterInstance -IsMinThresholdBroken $IsMinThresholdBroken -IsAvgThresholdBroken $IsAvgThresholdBroken -IsMaxThresholdBroken $IsMaxThresholdBroken -IsTrendThresholdBroken $IsTrendThresholdBroken -IsMinEvaluated $IsMinEvaluated -IsAvgEvaluated $IsAvgEvaluated -IsMaxEvaluated $IsMaxEvaluated -IsTrendEvaluated $IsTrendEvaluated
                } Else {
                    #Write-Host "Create alert - Detail: False"
                }
            }
        }
    }
}
Jan 24 at 12:38 PM
3) Find the function "StaticThreshold" and replace the following sections:

Find (Org) - Min
    If ($oCounterInstance.QuantizedMin[$t] -is [System.Char])
    {
        [System.Int32] $iQuantizedMin = $oCounterInstance.QuantizedMin[$t]
        [System.Double] $iQuantizedMin = $iQuantizedMin
    }
    Else
    {
        [System.Double] $iQuantizedMin = $oCounterInstance.QuantizedMin[$t]
    }
Replace (New) - Min
    If ($oCounterInstance.QuantizedMin -is [System.String])
     {
        [System.Double[]] $aQuantizedMin = @($oCounterInstance.QuantizedMin.Split(','))
    }
    Else
    {
        [System.Double[]] $aQuantizedMin = @($oCounterInstance.QuantizedMin)
    }

    $iQuantizedMin = $aQuantizedMin[$t]
Find (Org) - Avg
    If ($oCounterInstance.QuantizedAvg[$t] -is [System.Char])
    {
        [System.Int32] $iQuantizedAvg = $oCounterInstance.QuantizedAvg[$t]
        [System.Double] $iQuantizedAvg = $iQuantizedAvg
    }
    Else
    {
        [System.Double] $iQuantizedAvg = $oCounterInstance.QuantizedAvg[$t]
    }
Replace (New) - Avg
    If ($oCounterInstance.QuantizedAvg -is [System.String])
     {
        [System.Double[]] $aQuantizedAvg = @($oCounterInstance.QuantizedAvg.Split(','))
    }
    Else
    {
        [System.Double[]] $aQuantizedAvg = @($oCounterInstance.QuantizedAvg)
    }

    $iQuantizedAvg = $aQuantizedAvg[$t]
Find (Org) - Max
    If ($oCounterInstance.QuantizedMax[$t] -is [System.Char])
    {
        [System.Int32] $iQuantizedMax = $oCounterInstance.QuantizedMax[$t]
        [System.Double] $iQuantizedMax = $iQuantizedMax
    }
    Else
    {
        [System.Double] $iQuantizedMax = $oCounterInstance.QuantizedMax[$t]
    }
Replace (New) - Max
    If ($oCounterInstance.QuantizedMax -is [System.String])
     {
        [System.Double[]] $aQuantizedMax = @($oCounterInstance.QuantizedMax.Split(','))
    }
    Else
    {
        [System.Double[]] $aQuantizedMax = @($oCounterInstance.QuantizedMax)
    }

    $iQuantizedMax = $aQuantizedMax[$t]
Jan 24 at 12:43 PM
4) Use the following Powershell Threshold code to call the functions to perform checks

Counter - "Processor % Processor Time"

Warning

StaticThresholdAvgOnly -CollectionOfCounterInstances $CollectionOfPercentProcessorTime -Operator 'ge' -Threshold 50

Critical

StaticThresholdMaxOnly -CollectionOfCounterInstances $CollectionOfPercentProcessorTime -Operator 'ge' -Threshold 80

Cheers
James
Feb 4 at 9:47 AM
Thanks for the above...

But i sure hope a new version of the orig. pal script is under consideration with updates and cool stuf like yours above...

Pal is just so exceptionally usefull and so much potential