Performance Analysis of Logs (PAL) Tool

Created Unassigned: Running pal.ps1 generates a different report than the wizard [24416]

red888 — Tue, 18 Jun 2013 14:46:23 GMT

I used the PAL wizard and copied all of the parameters from the Queue tab, but when I run the pal.ps1 script with these options the report generated is different than what the wizard generates. I looks like when I run the pal.ps1 file it only picks up process alerts.

this is the script I am running:
```
powershell.exe -command "& 'C:\Program Files\PAL\PAL\PAL.ps1' -Log 'C:\Network_Monitoring\Performance_Monitor\myserver\Perfmon_Logs\Performance Counter.blg' -ThresholdFile 'C:\Program Files\PAL\PAL\SystemOverview.xml' -Interval 'AUTO' -IsOutputHtml $True -HtmlOutputFileName '[LogFileName]_PAL_ANALYSIS_[DateTimeStamp].htm' -IsOutputXml $False -XmlOutputFileName '[LogFileName]_PAL_ANALYSIS_[DateTimeStamp].xml' -OutputDir 'C:\Network_Monitoring\Performance_Monitor\myserver\PAL_Reports' -AllCounterStats $False -NumberOfThreads 1 -NumberOfProcessors '2' -ThreeGBSwitch $False -SixtyFourBit $True -TotalMemory '4' -RAID5Drives 'NULL' -RAID1Drives 'NULL'"
```

These are the same options I copied from the PAL wizard, but the report is not complete.

Commented Unassigned: v2.3.2 The variable '$SixtyFourBit' cannot be retrieved because it has not been set. [24415]

red888 — Tue, 18 Jun 2013 12:59:28 GMT

Running this command:
```
& 'C:\Program Files\PAL\PAL\PAL.ps1' -Log 'C:\Network_Monitoring\Performance Counters\myserver\Perfmon_Logs\Performance Counter.blg' -ThresholdFile 'C:\Program Files\PAL\PAL\SystemOverview.xml' -IsOutputHtml $true -HtmlOutputFileName '[LogFileName]_PAL_ANALYSIS_[DateTimeStamp].htm' -OutputDir 'C:\Network_Monitoring\Performance Counters\myserver\PAL_Reports'
```

I get this error:
```
Invoke-Expression : The variable '$SixtyFourBit' cannot be retrieved because
it has not been set.
At C:\Program Files\PAL\PAL\PAL.ps1:3361 char:5
+ Invoke-Expression -Command $sCode
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (SixtyFourBit:String) [Invoke-
Expression], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined,Microsoft.PowerShell.Command
s.InvokeExpressionCommand

```

In the PAL.ps1 there is no $SixtyFourBit param nor is there a $SixtyFourBit variable anywhere in the script.
```
#requires -Version 2.0
param($Log,$ThresholdFile="QuickSystemOverview.xml",$Interval='AUTO',$IsOutputHtml=$True,$IsOutputXml=$False,$HtmlOutputFileName="[LogFileName]_PAL_ANALYSIS_[DateTimeStamp]_[GUID].htm",$XmlOutputFileName="[LogFileName]_PAL_ANALYSIS_[DateTimeStamp]_[GUID].xml",$OutputDir="[My Documents]\PAL Reports",$AllCounterStats=$False,$BeginTime=$null,$EndTime=$null,[System.Int32] $NumberOfThreads=1)
Set-StrictMode -Version 2 #// Contributed by Jeffrey Snover
cls
```
Comments: ** Comment from web user: red888 **

oops never mind, I figured it out using the wizard.

Commented Unassigned: v2.3.2 The variable '$SixtyFourBit' cannot be retrieved because it has not been set. [24415]

red888 — Tue, 18 Jun 2013 12:41:59 GMT

This is running on an x86 Windows 7 SP1 machine

Created Unassigned: v2.3.2 The variable '$SixtyFourBit' cannot be retrieved because it has not been set. [24415]

red888 — Tue, 18 Jun 2013 12:35:49 GMT

New Post: Multi Threaded use of PAL

ClintH — Fri, 14 Jun 2013 21:32:03 GMT

I uploaded the updated PAL.ps1 and QuickSystemOverview.xml threshold files which contain the updates that I plan in v2.3.6. You can download them from my personal SkyDrive folder for now at:
https://skydrive.live.com/redir?resid=E6360C54B48A891B!5119&authkey=!AIBjU6LZWU4Ggt0

With that said, I plan on releasing PAL 2.3.6 very soon. I'm excited to have finally implemented a Disk Overwhelmed and Physical Memory Overwhelmed analyses which have been the holy grail of Windows performance analysis. I've had a some of the industry leaders such as Jeff Stokes and Yong Rhee review the work and I've had approval from them. These analyses will take the complex "artful" analysis and turn them into simple green, yellow, red indicators.

New Post: counter exclusion

ClintH — Fri, 14 Jun 2013 21:25:58 GMT

Hi frvallet,

No, but I could probably add it easily or you could make the modification yourself.
On line 2841 of PAL.ps1, here is the code:

                            ForEach ($XmlExcludeNode in $XmlDataSource.SelectNodes('./EXCLUDE'))
                            {
                                If ($XmlExcludeNode.INSTANCE -eq $XmlCounterInstanceNode.NAME)
                                {
                                    $IsCounterInstanceMatch = $False
                                }
                            }

You could change the -eq to be a -match for a regular expression. I would need to test it.
What kind of analysis are you trying to create?

Reopened Issue: Processing Thresholds.WARNING [22133]

ClintH — Fri, 14 Jun 2013 07:33:45 GMT

Processing Thresholds.WARNING: Please post errors or problems with this
tool to the PAL web site
located at http://www.codeplex.com/PAL with the following error message and a
brief description of what you were trying to do. Thank you.
Exception calling "Join" with "2" argument(s): "Value cannot be null.
Parameter name: value"
At C:\Program Files\PAL\PAL v2.1\PAL.ps1:4098 char:85
+ $XmlNewCounterInstance.SetAttribute("QUANTIZEDMIN", $([string
]::Join <<<< (',',$QuantizedMinForOverallStats)))
+ CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordE
xception
+ FullyQualifiedErrorId : DotNetMethodException

Reopened Issue: Cannot access file, in use by another process [23576]

ClintH — Fri, 14 Jun 2013 07:33:44 GMT

here is what I am executing:

Powershell ".\PAL.ps1"
-Log 'C:\PALReports\SRV-MBX-WVA06_Mbx_11010849.blg'
-ThresholdFile 'C:\Program Files\PAL\PAL\Exchange2007-MBX-TechNet.xml'
-Interval 'AUTO'
-IsOutputHtml $True
-HtmlOutputFileName '[LogFileName]_PAL_ANALYSIS_[DateTimeStamp].htm'
-IsOutputXml $False
-XmlOutputFileName '[LogFileName]_PAL_ANALYSIS_[DateTimeStamp].xml'
-OutputDir 'C:\PALReports'
-AllCounterStats $False
-NumberOfThreads 1
-DatabaseDisks 'E:'
-LogDisks 'L:'
-PagingDisks 'D:'
-TempDisk 'C:'
-MajorityClientsCached $False
-CCR $False
-NumberOfProcessors '2'

Here is the exact error message:

Generating the HTML Report...WARNING: Please post errors or problems with this t
ool to the PAL web site
located at http://www.codeplex.com/PAL with the following error message and a
brief description of what you were trying to do. Thank you.
The process cannot access the file 'C:\PALReports\SRV-MBX-WVA06_Mbx_11010849_PA
L_ANALYSIS_2012-11-01_13-31-36.htm' because it is being used by another process
.
At C:\Program Files\PAL\PAL\PAL.ps1:4851 char:146
+ $sPart00 + $sPart01 + $sPart02 + $sPart03 + $sPart04
+ $sPart05 + $sPart06 + $sPart07 + $sPart08 + $sPart09 + $sPart10 >> <<<< $h
+ CategoryInfo : OpenError: (:) [], IOException
+ FullyQualifiedErrorId : FileOpenFailure

C:\Program Files\PAL\PAL>

This occurs regularly though sometimes it does finish properly. The HTML file is always partially completed when this error occurrs. Things I have tried:

-Run as administrator
-Number of processing threads, tried the values of 1 and also 4 (my system has 4 cores), same resultes either way.

Thought I would throw this up here so maybe someone else might have an idea why it's happening.

New Post: counter exclusion

frvallet — Tue, 11 Jun 2013 18:27:11 GMT

Hi,
Is it possible to use wildcard in counter exclusion?
Example : exclude all process instances starting by svc (svc*)
Or maybe regular expression?

New Post: Multi Threaded use of PAL

Michal_S — Fri, 07 Jun 2013 07:22:55 GMT

Hi Clint,

Thanks a lot for great tool.
Could you please post or send me via e-mail update for running PAL in multi-threaded mode?

Thanks a lot!

Best regards,
Michal

Commented Unassigned: After PAL finished it start iexplore.exe! [24318]

eislon — Thu, 06 Jun 2013 19:41:44 GMT

Hi Clint,

We run PAL through a scheduler (automatic), so we use the pal.ps1 (with the parameters from the pal wizard.)

After sometime and some PAL process some BLG files, I see that iexplore.exe is getting started and eating up a lot of memory.
It causes the automatic started jobs to fail, not the first ones, but later on the day, and more iexplore.exe getting started on the system.

I use 2.3.3 x64 version (multithreaded) and use multithreads 2 or 4 according to blg file.
We first had a 32 bit version of pal, but it took to long to process all BLG files for a day, but it never started an Iexplore.exe process. is there away to prevent PAL from starting the iexplore.exe in the commandline?
Comments: ** Comment from web user: eislon **

Clint,
I will try this, saves a lot of memory!
thx

Commented Unassigned: After PAL finished it start iexplore.exe! [24318]

ClintH — Thu, 06 Jun 2013 08:47:56 GMT

To prevent the HTML report from opening when finished, edit the PAL.ps1 file, and comment out the OpenHtmlReport line by putting pound (#) sign in front of it, then save the file.
Yes, the reports themselves can be a bit resource intensive.

Commented Unassigned: PAL_Setup_v2.3.5_x64. [24362]

ClintH — Thu, 06 Jun 2013 08:44:54 GMT

We are using Ms Server 2003 standard 32bis OS.

PAL_Setup_v2.3.5_x64.zip is supporting 64bit application.

Is there any 32bit version available to download?

Thanks
Amit
Comments: ** Comment from web user: ClintH **

Also, I highly encourage you to migrate to 64-bit. 32-bit applications are limited to 2 GB of virtual memory by default.

Commented Unassigned: PAL_Setup_v2.3.5_x64. [24362]

ClintH — Thu, 06 Jun 2013 08:44:04 GMT

Yes, go to the Downloads tab and download PAL_Setup_v2.3.5_x86.zip.

Commented Unassigned: File name problem when using "Execute and Restart" [24363]

ClintH — Thu, 06 Jun 2013 08:43:19 GMT

I've noticed a small issue that has been affecting PAL as long as I've used it. Whenever I change the name of a file and choose "Execute and Restart", the name is changed back to default in the Queue Screen unless I change it again.

I'll try to put the steps in order:

1. Open PAL and go through selection screen as normal (any option works) until you hit the __File Output__ screen.
2. Change the name of the output file (for example, to [LogFileName].htm).
3. Continue the prompt until the __Execute__ screen. Notice the name of the file is shown in the __Queue__ screen beside the HtmlOutputFileName parameter.
4. Select "Execute and Restart" and finish.
5. Change the log file and select "Next" through the next screens. Notice the name of the file remains the same (as changed in Step 2) in the __File Output__ screen, but is changed back to default in the __Queue__ screen.
6. Select "Execute and Restart" (or any othe option this time).
7. Check the create reports. Notice the first file (created in Step 4) has the name [LogFileName].htm, but the second file (created in Step 6) has the default name as shown in its __Queue__ screen instead of the name in the __File Output__ screen.

If you input any change to the file name during Step 5, the HtmlOutputFileName changes to match it, but if you don't, it changes back to the default PAL file name.

Anyway, it's a small issue that has bugged me for a while so I've felt the urge to report.
Comments: ** Comment from web user: ClintH **

I'll look into this. Thanks for reporting this.

New Post: Calculate average in the threshold

ClintH — Thu, 06 Jun 2013 08:41:15 GMT

PAL divides the counter log into time slices and generates a minimum, average, maximum, and trend value for each counter instance for each time slice. The StaticThreashold function will compare the threshold to the minimum, average, and maximum values of each time slice. This means that the LDAP Search Time threshold will have a minimum, average, and maximum value based on the size of the time slices. If you only want one of the statistics to be analyzed, then you will need to do a much more complicated analysis involving manually looping through the counter collections. I do this for a few of the analysis, but you have to edit the XML of the threshold directly.

New Post: adding threading parameter as command line option

ClintH — Thu, 06 Jun 2013 08:36:35 GMT

Yes, use the parameter -NumberOfThreads. This parameter is added to the Queue if you set the number of threads to be more than 1 and click the Back button to go back to the queue. Also, PAL.ps1 is a script, so if you are comfortable with Powershell, then consider editing the script to see the parameters.

New Post: Data Analyzer Tool

ClintH — Thu, 06 Jun 2013 08:33:45 GMT

Thanks for the great introduction Thomas. Yes, PAL analyzes Performance Monitor counter logs and generates a report with reds, greens, and yellows indicating where thresholds are being broken.

No, PAL does not require an agent. It is an analysis tool only. The idea is to gather a counter log using a Performance counter data collector which is built into the operating system, move the counter log to a workstation, then use the PAL tool to analyze the counter log. The analysis can be very resource intensive.

Windows IT Pro magazine articles, SQL Server Internals book, and many blogs are out there on the internet. It is often presented in sessions at Microsoft TechEd and SQL PASS.

I created this project because I teach Windows architecture and performance analysis. This tool is a way for me to implement what I teach and make performance analysis more of a science than an art.

New Post: Multi Threaded use of PAL

ClintH — Thu, 06 Jun 2013 08:26:21 GMT

Sorry, this bug seems to come up only when using multi-threaded and when one of the analyses is disabled in a threshold file. I recently fixed this and will be in v2.3.6. In the meantime use single threaded or I can post the update here for testing if you want.

Created Unassigned: File name problem when using "Execute and Restart" [24363]

irohma — Mon, 03 Jun 2013 15:33:52 GMT