PAL Forum Rss Feed

New Post: adding threading parameter as command line option

IlluminatorIsJP — Mon, 20 May 2013 14:30:09 GMT

Is there a way to add the number of thread you want to use when running PAL Tool through a batch file (or via command line)? The options that are available to command line use is before the threading definition at the end of the GUI.

Here are all the options that I think are available via command line:

Options=-Interval 'AUTO' -IsOutputHtml $True -IsOutputXml $False -AllCounterStats $True -NumberOfProcessors 8 -ThreeGBSwitch $False -SixtyFourBit $True -TotalMemory 8

Thanks in advance.

New Post: Iexplore.exe is started consuming a lot of memory and pal jobs fail to finish succesfull

eislon — Fri, 17 May 2013 16:35:46 GMT

This one can be deleted, it is an issue (made one for it) not a discussion, sorry for the wrong place!

New Post: Data Analyzer Tool

ThomasKristensen — Fri, 17 May 2013 07:12:30 GMT

First a disclaimer: PAL is written by Clint Huffman. I am just a long-time user of PAL and I do not know either NetIQ or SolarWind.

Assuming that you have already collected the right performance counters, it sound like PAL would be perfect for the job.

PAL does analysis of already collected performance counter logs and therefore do not need anything installed on any of the machines except the machine that will do the analysis (Do not use a production machine for this) – PAL includes functionality to choose the appropriate counters, if needed.

I suggest that you try it. Take some of your historical data and have PAL analyze it.

I do not know of any whitepaper or updated materials on using PAL but I found it to be really intuitive. If you have a specific issue, post it and someone will help!

A little note: PAL works out-of-the-box for English operating systems. For other languages the counter log will have to be translated. There is another tool for this. Would this be an issue?

I really recommend this tool. It is simple awesome!

New Post: Data Analyzer Tool

Roozbeh59 — Thu, 16 May 2013 14:56:14 GMT

Hello. I just signed in to this discussion board. We are already using NetIQ (Formula) and SolarWind as our network monitoring tools at the World Bank. What we are looking for is a utility that can take historical data like memory usage or CPU utilization taken from Windows servers' Performance Monitoring tool and then analyze/display it in a graphical format. Is PAL capable of providing this for us? If so, could you please point me to any white paper or document on this? Also, does PAL require an agent to be installed on the target servers?

New Post: Multi Threaded use of PAL

JohnPiepers — Thu, 16 May 2013 10:17:26 GMT

Hi Clint,

In multithreaded mode PAL (2.3.5) now seems to forget the last counter and it hangs after 6 %.
It does so with 4 and 2 threads, with one thread it completes normally.
My threshold file is 51 counters.

John

New Post: Multi Threaded use of PAL

JohnPiepers — Thu, 16 May 2013 05:03:05 GMT

Hello Clint,

Thanks for the quick reply and even quicker fix!

Kudo's for creating this awesome tool!

Regards,

John

New Post: Multi Threaded use of PAL

ClintH — Thu, 16 May 2013 02:42:52 GMT

Thanks for reporting this bug. I fixed it in v2.3.5 and will be releasing it soon.

New Post: Multi Threaded use of PAL

ClintH — Thu, 16 May 2013 01:20:59 GMT

Ah, I never noticed this before. Thank you for bringing this to my attention. I will try to fix this.

New Post: Multi Threaded use of PAL

JohnPiepers — Wed, 15 May 2013 12:03:07 GMT

Hello,

When I use PAL in with more than one thread configured in the Execute Tab it does not wait to complete processing the last counter object listed in the threshold xml file.

It does start analyzing the last counter but PAL just continues with Generating the HTML Report without waiting for the last counter to finish.
That counter is then also not listed in the report (obviously).

I have tried this with several .blg files from 32 and 64 bit systems.

It does work properly when PAL is running with only one thread.

PAL is version 2.3.4, but this issue was also present in the previous version (2.3.3).

New Post: will it support more language

ClintH — Fri, 10 May 2013 21:13:06 GMT

Hi sql_chen,

I'm the only developer of the tool and I can only work on it during my free time. I would love to have support for non-English languages, but I just don't have the resources to make that happen. With that said, I experimented with doing Bing translations of the report files. From what I understand, they work, but the machine translation was correct, aweful from what people have told me.

I can provide any of these localized threshold files to you, but keep in mind that they still require the counter logs to be in English due to the string pattern matching. All of this is certainly possible, but I would need people willing to volunteer their time to work on it. I don't collect any revenue for this project.

With that in mind, what would you think of me adding advertising to this site to generate money towards these efforts?

New Post: will it support more language

sql_chen — Fri, 03 May 2013 02:48:28 GMT

it's a greate tool!
but it just output report in english, will it support more language?

New Post: Calculate IOPS

ClintH — Wed, 20 Mar 2013 03:33:43 GMT

Hi Joshua,

All of what you said has merit, especially on calculated IOPS. You are going with mathematical results while I am going by what a trusted advisor told me. All in all, "trust", but verify.

In any case, given that the latency counters do track all IOs, the frequency by which they are collected matters which actually plays into what you were saying about gathering often to be accurate. Now, when I said that the disk counters are 100% accurate, I want to stress the point that the "counter" and how it is gathered is accurate, but tools like the System Monitor and Relog.exe can "skew" the data making it less accurate through "massaging" the data - this was recently proven when dealing with more than 1000 data points. When you take all of the factors into consideration, we end up with a less than 100% accuracy going back to your primary message in the first place.

All in all, I'm sorry if I made this a drawn out explanation. You can certainly trust me, but I encourage verification.

New Post: Calculate IOPS

joshuatownsend — Wed, 20 Mar 2013 02:24:26 GMT

Great info, Clint. I'm going to have to revise some of stuff.... Will hit the lab as tome permits and try to publish some updates to clear up the confusion that I have helped to spread.

New Post: Calculate IOPS

ClintH — Wed, 20 Mar 2013 01:40:18 GMT

I read the COTW article again and I think there is a misinterpretation of what is it saying. To clarify, we are suggesting to collect the latency disk counters often simply catch short bursts of problems. For example, if the counter instances are collected once every 5 minutes, then there is likely a lot of data points to average. The counter would return only the average of all 5 minutes of data which does include all of the IOs that occurred in that 5 minutes. We won't know what the standard deviation is. So, the counters are 100% accurate based on the perspective of the operating system - meaning that the clock can be skewed by virtualization, but the level of the detail is limited to how often they are collected.

New Post: Calculate IOPS

ClintH — Wed, 20 Mar 2013 01:32:13 GMT

Actually, the Avg. Disk sec/Read, Avg. Disk sec/Write, Avg. Disk sec/Transfer performance counters do count every IO that occurs between each collection. For example, if you collect one of these counter instances every 10 seconds and if there was 100 IOs between the first collection and the second collection, then the second collection will be the average of all 100 IOs. This is based what Bruce Worthington has told me who examined the source code of the performance counters a few months ago.

New Post: PAL output to Powerpoint?

ClintH — Wed, 20 Mar 2013 01:24:16 GMT

There has been considerations to break down the report into multiple HTML pages, but I don't know how helpful it will really be for the amount of work involved.

Yes, there are Office automation objects that can be used, but I haven't touched them in years.

New Post: Index operation failed; the array index evaluated to null.

ClintH — Wed, 20 Mar 2013 01:21:28 GMT

This tells me that there is a problem with the threshold file itself. I know that some of the Exchange threshold files had some problems. If this is one of the Exchange threshold files, then go to my personal SkyDrive and download the updated versions at:
https://skydrive.live.com/?cid=e6360c54b48a891b&id=E6360C54B48A891B%21428&authkey=!APQbXtD8U-HExzE#cid=E6360C54B48A891B&id=E6360C54B48A891B%211392&authkey=%21APQbXtD8U-HExzE

New Post: Method invocation failed because [System.String] doesn't contain a method named 'GetUpperBound'.

ClintH — Wed, 20 Mar 2013 01:19:35 GMT

Does the counter log contain instances of Web Server Current Connections other than _Total? The _Total instance is excluded from analysis in many of the analyses in the threshold file mostly because there is little useful data from them. It is likely excluding _Total and if that is the only instance, then the array is empty. In any case, I need to add an error handler for this.

New Post: Calculate IOPS

joshuatownsend — Tue, 19 Mar 2013 18:01:56 GMT

Good question. Perfmon will report disk transfers per second (IOPS) in realtime. If you've created a data collector set and are logging values to a .csv/.blg at a set interval (10 seconds), the data will not be 100% accurate. You will only be getting IO per second measurements for one out of every 10 seconds. See this for more info: http://blogs.technet.com/b/cotw/archive/2009/03/18/analyzing-storage-performance.aspx. This data is only good for determining statistical values (mean, median, standard deviation, etc.), not for, say, counting every single IOPS against your Exchange database.

When PAL consumes your perfmon data and analyzes it, it returns some of those statistical values - namely minimum, maximum, average and standard deviation. There are also some calculations that remove outlying data. Average number of IOPS is a good start if you are planning for a storage system upgrade, but at Clearpath (EMC Partner of the Year) we usually size storage at the 95th percentile when putting a new SAN configuration together for a customer (then we adjust based on data commonality - lots of the same data being served out of cache instead of from disk; and sequentiality). You can use the mean and std. deviation values that PAL reports to find percentiles. As it turns out, 2 standard deviations above the mean is about the 95th percentile. So the math looks like this (warning - I barely passed my statistics courses in college - math and I don't always get along):

Average: 5000 IOPS
STD Dev: 794

(2 x Std.Dev) + Avg = 95th Percentile
2 x 794) + 5000 = 6588 IOPS@ 95th Percentile

You'll want to pay attention to the percentage of read vs. write IOPS if you are sizing for new storage. Write IOPS typically incur a penalty (writes are more expensive than reads due to multiple RAID writes, parity calculations, etc.). RAID5 has a write penalty of 4. RAID10 has a write penalty of 2. I.E. if your 6588 IOPS in the example above were 100% write, you would multiply your IOPS by 2 = 13176. You would then use 13176 IOPS to decide how many disks to buy. You'll get ~180 IOPS per 15k RPM disks, so 13176/180 = 74 disks in RAID 10 to meet your IOPS requirement (ignoring things like cache). PAL thresholds include Disk Read Transfers per Sec and Disk Write Transfers per Sec....

If you want to learn a bit more about the math, check out my Storage Basics blog posts at: http://vmtoday.com/category/storage/storage-basics/.

Hope this is helpful.

New Post: Calculate IOPS

joshuatownsend — Tue, 19 Mar 2013 17:47:23 GMT