Windows Azure Diagnostics and PowerShell – Performance Counters

This is part 2 of a series.

With the introduction of Windows Azure PowerShell Cmdlets 2.0 we have added a lot of functionality around managing Windows Azure Diagnostics. This is a 2nd article in a series that covers various aspects of diagnostics and PowerShell. Click here to see the previous article on using tracing and the Windows Azure Log.

Just as in the other articles you will need to add the PowerShell Snapin (or module):

  Add-PsSnapin WAPPSCmdlets

Next I have a handy helper function called GetDiagRoles that I use for all of my diagnostic examples:

Initialization and Helper Function

  $storageAccountName = "YourStorageAccountName" 
  $storageAccountKey = "YourStorageAccountKey" 
  $deploymentSlot = "Production" 
  $serviceName = "YourHostedService" 
  $subscriptionId = "YourSubscriptionId" 
  # Thumbprint for your cert goes below 
  $mgmtCert = Get-Item cert:CurrentUserMyD7BECD4D63EBAF86023BB4F1A5FBF5C2C924902A 
 
  function GetDiagRoles { 
    Get-HostedService -ServiceName $serviceName -SubscriptionId $subscriptionId -Certificate $mgmtCert | ` 
        Get-Deployment -Slot $deploymentslot | ` 
        Get-DiagnosticAwareRoles -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey 
   }

I have another helper function I have written that basically creates a new PerformanceCounterConfiguration object. You will need an array of these to pass to the Set-PerformanceCounter cmdlet. The function takes the performance counter specifier (more on this shortly) and the sample rate in seconds for each counter.

Helper Function that Initializes a Counter Object

function CPC($specifier, $SampleRate) { 
   $perfCounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration 
   $perfCounter.CounterSpecifier = $specifier 
   $perfCounter.SampleRate = [TimeSpan]::FromSeconds($SampleRate) 
   return $perfCounter 
}

From there it is up to you to pass the performance counters you are interested in. I’ve written another function that wraps up a the functionality of creating a PowerShell array, populating it with the counters I am interested in and sets them for the diagnostic aware role I pass in from GetDiagRoles. Note: I’m also comparing the current role name to the name of one of my web roles so I’m only adding web related counters on instances that I care about. I retrieved the counter names by typing in: typeperf.exe /qx while logged into my Windows Azure roles via RDP. This way I can actually get counters specific to the machines in the service (and the processes such as RulesEngineService.exe (my sample NT service). Finally, it makes a call to the Set-PerformanceCounter cmdlet with the array of performance counters, and configures diagnostics to transfer these to Windows Azure Storage every 15 minutes.

Helper Function that Configures Counters for a Role

 function SetPerfmonCounters { 
    $sampleRate = 15 
    $counters = @() 
    $counters += CPC "Processor(_Total)% Processor Time" $sampleRate 
    $counters += CPC "MemoryAvailable MBytes" $sampleRate 
    $counters += CPC "PhysicalDisk(*)Current Disk Queue Length" $sampleRate 
    $counters += CPC "SystemContext Switches/sec" $sampleRate 
    # Process specific counters (retrieved via typeperf.exe /qx) 
    $counters += CPC "Process(RulesEngineService)% Processor Time" $sampleRate 
    $counters += CPC "Process(RulesEngineService)Private Bytes" $sampleRate 
    # $RoleName is populated from a call to GetDiagRoles in the next snippet. 
    if($RoleName -eq "SomeWebRole") { 
        $counters += CPC "W3SVC_W3WP(*)Requests / Sec" $sampleRate 
        $counters += CPC "ASP.NETRequests Queued" $sampleRate 
        $counters += CPC "ASP.NETError Events Raised" $sampleRate 
        $counters += CPC "ASP.NETRequest Wait Time" $sampleRate 
        # Process specific counters (retrieved via typeperf.exe /qx  
        $counters += CPC "Process(W3WP)% Processor Time" $sampleRate 
        $counters += CPC "Process(W3WP)Private Bytes" $sampleRate 
     } 
     $input | Set-PerformanceCounter -PerformanceCounters $counters -TransferPeriod 15 
}

Now linking all of this together to configure performance logging is pretty simple. In the snippet below I call GetDiagRoles which returns a collection of all of the diagnostic aware roles in my service. Which I then pipe individually to the SetPerfmonCounters function previously created.

Set Counters for each Role

  GetDiagRoles | foreach { 
       $_ | SetPerfmonCounters 
  }

Now I am successfully logging perfmon data and transferring it to storage. What’s next? Well analysis and clean up of course! To analyze the data we have added another cmdlet called Get-PerfmonLogs that downloads the data and optionally converts it to .blg for analysis. The snippet below creates a perfmon log foreach role in your service using the BLG format. Note the Get-PerfmonLog cmdlet also supports -From and -To (or -FromUTC or -ToUTC) so you can selectively download performance counter data.

Download Perfmon Logs for Each Role

   GetDiagRoles | foreach { 
     $Filename = "c:DiagData" + $_.RoleName + "perf.blg" 
     $_ | Get-PerfmonLog -LocalPath $Filename -Format BLG 
   }

Once the data is downloaded you can of course analyze it using Perfmon or utilize the Clear-PerfmonLogs to clean up the previously downloaded perfmon counters from Windows Azure Storage. Note all of the Clear-* diagnostic data cmdlets support a -From and -To parameter to allow you to manage what data to delete.

Analysis using Perfmon

Delete Perfmon Data from Storage for Each Role

     GetDiagRoles | foreach { 
     $_ | Clear-PerfmonLog -From "9/19/2011 6:00:00 AM" -To "9/19/2011 9:00:00 AM" 
}

Finally, one last example. If you ever have the need to completely clear out and reset your perfmon counters there is a -Clear switch that gives you this functionality.

Reset Perfmon Logs for Each Role

  GetDiagRoles | foreach { 
       $_ | Set-PerfmonLog -Clear 
  }

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>