Creating Highly Available Workloads with Windows Azure

In a recent update to the Windows Azure Management portal the Windows Azure team has added the capability to create and manage endpoint probes. While this functionality has always been available it was restricted only to the Windows Azure PowerShell Cmdlets. Having this ability in the management portal is a huge improvement for those seeking every tool at their disposal for improved availability.

In this post I’m going to show how you can take advantage of this new functionality in combination with availability sets to create a true highly available workload – in this case it will be a highly available web farm.

In the Windows Azure Management Portal create a virtual machine, select the Windows Server 2012 image and on the page asking for Availability Set select Create an Availability Set.

create-av-set

Create Availability Set

Once selected type in a name for your availability set. The name I’ve chosen is WEBAVSET.

Specifying an availability set ensures that Windows Azure will put members of the same availability set on different physical racks in the data center. This gives you redundant power and networking. This also tells Windows Azure that while performing host updates to not take down all nodes in the set at the same time so some of the nodes for your application will always be up and running.

Note: The only way to achieve 99.95% SLA with Windows Azure is by grouping multiple VMs performing the same workload into an availability set.

create-av-set2

On the last screen of the portal you can skip creating the HTTP Endpoint here. You will configure the load balancer in a later step.

create-web-3.

Once the first virtual machine is completed provisioning create another virtual machine using the same base image.

On the screen where you are asked about cloud services use the drop down list to select the previously created cloud service.
Note: Selecting an existing cloud service is another huge improvement in the portal UI – long requested!

Select the availability set drop down and you should see the AV Set name you created with the first VM. Select this AV set before proceeding.

create-web2

Once both virtual machines are provisioned they should both be in the same cloud service and availability set.

Same Cloud Service (same host name)
same-cs

Same Availability Set
same-av-set

Next configure each virtual machine for a workload to load balance. To keep it simple RDP into each VM and launch Server Manager -> Manage Roles and Features and add the Web Server role (IIS).

Once the Web Server role is installed on each server, open notepad to edit the default page (c:\Inetpub\wwwroot\IISStart.htm) to show which server is serving up traffic to verify load balancing is working.

Add the following HTML code to the page replacing VMNAME with the name of the VM you are on:

   <h1>VMNAME </h1>

Edited Page for webvm1
edit-page

Now to add the load balanced endpoints. Go back to the Windows Azure Management Portal and under the first VM (webvm1 in my case) select Endpoints at the top.
Then click Add towards the bottom of the screen.

New Endpoint Wizard
create-ep

Select HTTP from the Drop Down and Check Create a Load Balanced Set then click the next arrow.

Creating a Load Balanced HTTP Endpoint
create-ep2

Default TCP Probe Settings

The default load balancer probe settings are set to TCP. What this screen means is every 15 seconds the load balancer probe will attempt a TCP connect on the specified probe port. If it does not receive a TCP ACK Twice (Number of Probes) it will consider the node offline and will stop directing traffic to it. This alone is a huge benefit Windows Azure is giving you for free that was previously only available via the PS Cmdlets.

create-ep3

Configuring HTTP Probe Settings

Changing the probe type to HTTP gives you a bit more flexibility and power on what actions you can take. You can now specify a ProbePath property on the endpoint. The ProbePath is essentially a relative HTTP URL on your web servers that will respond with an HTTP 200 if the server is fine and ANY other response if the node will be taken out of rotation. This allows you to essentially write your own page that can check the state of the VM. Whether this is verifying disk space, data base or Internet connectivity the choice is yours.

For my simplistic example I’m just going to point this to the root of the site / so as long as IIS returns a HTTP OK (200) the server should be in the load balanced rotation.

create-ep4

Adding an endpoint to an existing load balanced set

Next add the load balanced endpoint on the second virtual machine by opening the VM in the Windows Azure Management Portal, click Endpoints at the top of the screen and Add at the bottom.
This time instead of creating a new endpoint select “Add Endpoint to an existing Load-Balanced Set”.

add-ep5

On the next screen select HTTP for the name and click the checkmark to add the endpoint.

add-ep6

A few additional details to be aware of with endpoint probes:

  • If a node is taken out of rotation, once it becomes responsive again the load balancer will automatically add it back to rotation.
  • The load balancer accesses the probe port using the internal IP address of the VM and not the public IP of the cloud service. This means the probe port does not have to be the same port as defined in the endpoint.
  • There are no credentials to be passed along with the load balancer requests. The ProbePath property for HTTP probes should always point to a url that can respond successfully with no authentication (for those of you deploying SharePoint you already know this).
  • To troubleshoot a load balanced endpoint with HTTP probes always check the web logs of the servers. The load balancer requests are obvious and if you see response codes anything other than a 200 you know why the node is out of rotation.

Verification

Browse to the cloud service to ensure the load balancer is behaving as expected. If working (and you followed my post) you should see both VMs come through if you hit F5 a few times.

Response from VM1
verification1

Response from VM2
verification2

Now the important part: Ensure that the load balancer does not direct traffic to a node that is down.

Pick a VM and click the Shutdown button in the Windows Azure Management Portal (at the bottom of the page for the VM).

Stopped VM
stopped-vm

Once the virtual machine is stopped refresh the site multiple times. You should not see a timed out response or a response from the VM you shut down.

How to do the same thing in PowerShell

For those of you that may have the need to spin up workloads like this often it is probably worth investing the time in learning PowerShell.
Here is a quick example of how the above process can be packaged up in a small script to automate all of the above steps.
Note: if you are new to WA PowerShell take a look at the Windows Azure PowerShell reference guide (very incomplete but good starting point).

# Retreived with Get-AzureVMImage | select ImageName
$img = "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201306.01-en.us-127GB.vhd"
 
$user = "mwasham"
$pwd = "some@pass123a1"
 
$vmname = "webvm"
$csname = "lbcloudsvc123a"
 
$av = "WEBAVSET"
 
$instances = 2
$vms = @()
 
# Loop to create N number of VMs
for($i=0; $i -lt $instances; $i++){
 
 $vmInstanceName = "$vmname-$i"
 # Compose a VM config with the load balanced endpoint and the correct av set
 
 $vms += New-AzureVMConfig -Name $vmInstanceName -InstanceSize Small -ImageName $img `
          -AvailabilitySetName $av |
          Add-AzureProvisioningConfig -Windows -AdminUsername $user -Password $pwd |
          Add-AzureEndpoint -Name "HTTP" -Protocol tcp -LocalPort 80 -PublicPort 80 `
            -LBSetName "LBHTTP" -ProbePort 80 -ProbeProtocol http -ProbePath "/" 
}
 
 
New-AzureVM -ServiceName $csname -Location "West US" -VMs $vms

Summary

In this post I showed how to take advantage of the new functionality in the Windows Azure Management Portal to add additional high availability to your web farm (or other multi-vm workload).

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>