This is part 5 of a series:
- Managing Tracing with Windows Azure PowerShell Cmdlets 2.0
- Managing Performance Counters with Windows Azure PowerShell Cmdlets 2.0
- Viewing your diagnostic settings with Windows Azure PowerShell Cmdlets 2.0
- Managing Windows Event Logs with Windows Azure PowerShell Cmdlets 2.0
- Managing Log Files with Windows Azure PowerShell Cmdlets 2.0
In this example I’m going to show you how you can manage log data from your Windows Azure Application using new functionality in the Windows Azure PowerShell Cmdlets 2.0.
Just as in the other articles you will need to add the PowerShell Snapin (or module):
Add-PsSnapin WAPPSCmdlets
Ditto, I have this handy initialization block and helper function GetDiagRoles:
Initialization and Helper Function
$storageAccountName = "YourStorageAccountName"
$storageAccountKey = "YourStorageAccountKey"
$deploymentSlot = "Production"
$serviceName = "YourHostedService"
$subscriptionId = "YourSubscriptionId"
# Thumbprint for your cert goes below
$mgmtCert = Get-Item cert:\CurrentUser\My\D7BECD4D63EBAF86023BB4F1A5FBF5C2C924902A
function GetDiagRoles {
Get-HostedService -ServiceName $serviceName -SubscriptionId $subscriptionId -Certificate $cert | `
Get-Deployment -Slot $deploymentslot | `
Get-DiagnosticAwareRoles -StorageAccountName $storageAccount -StorageAccountKey $storageKey
}
What is distinct about managing log files remotely in Windows Azure is determining where on the file system the log files actually are.
I’ve written some samples that show some techniques you can use from PowerShell for determining file locations.
When determining the local path for a named local resource I’m using guidance from MSDN: http://msdn.microsoft.com/en-us/library/ee758708.aspx.
For the rest of the log paths I am essentially just using observations of how the paths are constructed. At this point there is no guarantee this format will change but for the time being it doesn’t look too likely.
Returns the Local Path for IIS Logs
function GetIISLogsPath()
{
$input | foreach {
$path = "C:\Resources\Directory\" + $_.DeploymentId + "." + $_.RoleName + ".DiagnosticStore\LogFiles"
return $path
}
}
Returns the Local Path for IIS Failed Request Logs
function GetIISFailedRequestPath()
{
$input | foreach {
$path = "C:\Resources\Directory\" + $_.DeploymentId + "." + $_.RoleName + ".DiagnosticStore\FailedReqLogFiles"
return $path
}
}
Returns the Local Path for Crash Dumps
function GetCrashDumpsPath($rolename, $deploymentid)
{
$input | foreach {
$path = "C:\Resources\Directory\" + $_ + "." + $_ + ".DiagnosticStore\Crashdumps"
return $path
}
}
Returns the Local Path for a Named Local Resource
function GetLocalResourcePath($localResource)
{
$input | foreach {
$path = "C:\Resources\directory\" + $_.DeploymentId + "." + $_.RoleName + "." + $localResource
return $path
}
}
Now that we have some helper functions to figure out where are logs are in our Windows Azure Deployment we need a method of modifying the logging data sources in Windows Azure Diagnostics.
The long function below does just that. It takes the local path of the files you want to transfer, what container in Windows Azure Storage to transfer the files to, a quota (can be set to 0) and how often to transfer that directory to storage in minutes.
Configures a local directory in a Windows Azure role to be transferred into Windows Azure Storage.
function SetDirectory($path, $container, $quota, $transferAllInMinutes)
{
$input | foreach {
$role = $_
$role | Get-DiagnosticAwareRoleInstances |
foreach {
$roleinstance = $_
$diagConfig = $role | Get-DiagnosticConfiguration -InstanceId $roleinstance -BufferName Directories
$existingDirectory = $false
$newDirectoryConfig = $null
for($i=0;$i -lt $diagConfig.DataSources.Count; $i++)
{
$tmpDiag = $diagConfig.DataSources[$i]
if($tmpDiag.Path -eq $path)
{
$existingDirectory = $true
$newDirectoryConfig = $diagConfig.DataSources[$i]
break
}
}
# doesn't already exist in the collection so create a new one
if($existingDirectory -eq $false)
{
$newDirectoryConfig = New-Object -TypeName Microsoft.WindowsAzure.Diagnostics.DirectoryConfiguration
}
$newDirectoryConfig.Container = $container
$newDirectoryConfig.Path = $path
$newDirectoryConfig.DirectoryQuotaInMB = $quota
if($existingDirectory -eq $false)
{
$diagConfig.DataSources.Add($newDirectoryConfig)
}
$role | Set-FileBasedLog -DirectoriesConfiguration $diagConfig.DataSources -BufferQuotaInMB $quota -TransferPeriod $transferAllInMinutes
}
}
}
Now the step of actually configuring your role using all of this code.
The snippet below configures IIS logs, IIS failed request logs and a local resource named “customLogging” to be transferred to storage every 15 minutes.
Configuring Logs to be Transferred
GetDiagRoles | foreach {
$CurrentRole = $_
if($CurrentRole.RoleName -eq "MyWebRole")
{
# Retrieve the local paths in the role
$iislogs = $CurrentRole | GetIISLogsPath
$iisFailedLogs = $CurrentRole | GetIISFailedRequestPath
$customLogs = $CurrentRole | GetLocalResourcePath "customLogging"
# Configure diagnostics to transfer log data to storage
$CurrentRole | SetDirectory $iisFailedLogs "wad-iis-failedreqlogfiles" 0 15
$CurrentRole | SetDirectory $iislogs "wad-iis-logfiles" 0 15
$CurrentRole | SetDirectory $customLogs "wad-customlogging" 0 15
}
}
The snippet below downloads the files from each container to the local file system.
Downloading the Data from Storage
Save-Container -ContainerName "wad-iis-logfiles" -LocalPath "c:\diagdata" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount Save-Container -ContainerName "wad-iis-failedreqlogfiles" -LocalPath "c:\diagdata" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount Save-Container -ContainerName "wad-customlogging" -LocalPath "c:\diagdata" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount
Finally, to clean up storage from each container we provide you with the Clear-Container cmdlet.
Note: Currently, we do not provide a -From or -To parameter to Clear-Container. To ensure that you do not delete data that may currently be being logged
you will need to ensure your scheduled transfers do not overlap during the same time period of a deletion. Yes, we are looking into improving this scenario.
Cleaning up a Container
Clear-Container -ContainerName "wad-iis-logfiles" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount Clear-Container -ContainerName "wad-iis-failedreqlogfiles" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount Clear-Container -ContainerName "wad-customlogging" ` -StorageAccountKey $storagekey -StorageAccountName $storageAccount


