Tag Archives: Virtual Machines

Bootstrapping a PowerShell DSC Pull Server and Client

Script Sample

This post describes a set of PowerShell scripts that can automatically provision a PowerShell DSC Pull Server and Client using Windows Azure Virtual Machines.

The scripts can be downloaded from here: Bootstrap PowerShell DSC in Windows Azure.
If you find bugs feel free to fork, fix and submit a pull request.

Before continuing

If you are brand new to Windows PowerShell DSC I recommend a slight detour from this post to watch the TechEd 2013 Introductory Session. Once you watch the TechEd session you should then read this blog post on how to configure a DSC Pull Server: Push and Pull Configuration Modes.

Ok, I feel better. You now should not only know what a DSC pull server is but you will likely appreciate the script more because it takes all of the complexity of putting together a DSC Pull Server and Client and wraps it up in two simple scripts. Not that I am saying you shouldn’t know how to do this on your own but if you need to quickly spin up an environment for a demo, testing or whatever it is nice not to have to reconstruct an environment from scratch each time.

Dependency

The scripts have a dependency on the Windows Azure PowerShell Cmdlets. So read this article to configure them if you haven’t already.

Creating a Pull Server

 
$subscription = "opsgilitytraining"
$serviceName = "mypullsvc"
$vmNamePull = "pullsrv"
$vmSize = "Small" 
$Location = "West US"
 
.\create-pull-srv.ps1 -SubscriptionName $subscription `
   -ServiceName $serviceName -Name $vmNamePull -Size $vmSize `
    -Location $location

What the script does:

  • Provisions a Server 2012 R2 VM in the “mypullsvc” cloud service in the West US data center. The VM name is “pullsrv”.
  • Creates a self-signed SSL certificate that is used to connect to the pull server and for encrypting stored passwords and automatically deploys it to the new virtual machine.
  • Uploads and executes the DSC Pull Server resource provider written by the PowerShell team.
  • Uploads a file that contains a helper function called SetConfiguration. This helper executes your DSC configuration file, generates a deterministic GUID based on the configuration name (so you don’t have to have a table of GUIDs handy) and creates the .mof files + checksums. Basically, all of the nasty work to create a DSC configuration in a pull server environment.
  • Once the server is provisioned login to the Pull Server via RDP.

     
    # if you don't want to leave your PowerShell session
    Get-AzureRemoteDesktopFile -ServiceName $serviceName -Name $vmNamePull -Launch

    Open the example configuration file C:\DSCScript\WebServer.ps1 in PowerShell_ISE and Hit F5. This will create your first bare bones configuration named WebServer on the pull server.

    Creating a Pull Client

    Note: The scripts have been designed to only allow deploying the client into the same cloud service as the pull server

    $vmNameClient = "pullclient"
    $configName = "WebServer"
     
    .\create-pull-client.ps1 -SubscriptionName $subscription `
       -ServiceName $serviceName -Name $vmNameClient -Size $vmSize `
       -CertificatePath .\PSDSCPullServerCert.pfx -PullServer $vmNamePull `
       -ConfigurationName $configName


    What the script does:

    • Provisions a Server 2012 R2 VM in the “mypullsvc” cloud service in the West US data center. The VM name is “pullclient”.
    • Deploys the previously created certificate PSDSCPullServerCert.pfx and also adds this cert to the LocalMachine\Root (trusted root authority) so it can be used with SSL.
    • Configures the client to point to the pull server using the DSC configuration name specified in -Configuration. The pull server has an example WebServer configuration that simply installs IIS that can be specified here.

    Testing the configuration

    Once the server is provisioned login to the Pull Client via RDP.

     
    # if you don't want to leave your PowerShell session
    Get-AzureRemoteDesktopFile -ServiceName $serviceName -Name $vmNameClient -Launch

    After patiently waiting 30+ minutes the configuration should be downloaded to your client VM. To validate this launch PowerShell and run:

     
    Get-DscConfiguration

    If all goes well you should see the following:
    PowerShell DSC

    Forcing DSC Configuration
    As of now there is no DSC cmdlet to force the client to pull a configuration. However, DSC is setup using scheduled tasks so you can force the task to run which will have your client check to see if it is up to date and if not apply the configuration.

     
    Get-ScheduledTask "Consistency" | Start-ScheduledTask

    Summary

    There you have it. Two simple scripts that can quickly get you up and running with PowerShell DSC.
    If you are interested in learning more about PowerShell DSC or Windows Azure in general for yourself or your organization we provide on-site or open enrollment Windows Azure and PowerShell Training.

Windows Azure IaaS Webcast Series Part One – Getting Started with Virtual Machines

I’ve started a series of webcasts focused on Windows Azure Infrastructure as a Service. This series is targeted at new users to IaaS and starts from the basics with creating virtual machines. I plan on building many of these webcasts to cover the getting started scenarios to more advanced topics.

If you have any scenario requests shoot me an email at: mwasham@microsoft.com or Tweet me at MWashamMS!

iaas-part-1

Connecting Windows Azure Virtual Machines with PowerShell

In my post on automating virtual machines I showed the basics for getting around and managing aspects of Windows Azure VMs. In this post I want to cover a few of the more complex scenarios such as connectivity between VMs, deploying into a virtual network and finally deploying a virtual machine automatically domain joined into an Active Directory Domain.

Connecting Virtual Machines

So far we have seen details on provisioning a single virtual machine. What happens if your application requires more than one VM? How to connect them? There are two options for connecting virtual machines. The first is to add multiple virtual machines to the same cloud service and the second is to provision each VM into the same virtual network. When machines are added to the same cloud service and are not within a virtual network they receive the benefits of being on the same network and receive built in name resolution with each other through Windows Azure provided DNS.


So how do you add two virtual machines to the same cloud service? When you create the first virtual machine using New-AzureVM or New-AzureQuickVM you are required to specify the -Location or -AffinityGroup parameter. When you specify either parameter it tells the cmdlets that you wish to create the cloud service at that time because the data center location can only be set on initial creation. To tell the cmdlets to create the VM in an existing cloud service you just omit the -Location/-AffinityGroup parameter.

Create a VM and a New Cloud Service (specify -Location/-AffinityGroup)

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
	New-AzureVM -ServiceName $svc -Location $loc

Create a VM and Adds to an Existing Cloud Service by (omit -Location/-AffinityGroup)

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
	New-AzureVM -ServiceName $svc

Connecting Virtual Machines with Windows Azure Virtual Networks

The second way of provisioning connected virtual machines is by using a Windows Azure Virtual Network. With a Windows Azure Virtual Network the network can span cloud services. This enables scenarios such as virtual machines (or web and worker roles) in different cloud services to be fully connected in the cloud.

How do you provision VMs into a VNET with PowerShell?

Just like the -Location parameter a VNET can only be specified when creating the first VM in a cloud service (note that the subnet for each VM can be set per VM on provisioning). Additionally, VNETs require that the cloud service be deployed into the same affinity group as the VNET was created in. The New-AzureVM cmdlet requires -AffinityGroup instead of -Location when deploying to a VNET.

Joining a Virtual Network at Provision Time

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
       Set-AzureSubnet 'subnet' |
	New-AzureVM -ServiceName $svc -AffinityGroup 'myag' -VNetName 'VNET'

Specifying DNS

One of the significant differences between deploying a virtual machine outside of a VNET and one within is inside of a VNET there is no Windows Azure Provided DNS for VM to VM name resolution. To provide for this you are allowed to specify DNS servers inside of the Virtual Network configuration. When deploying with PowerShell you also have the ability to specify DNS settings when you create the first VM. This is a very flexible approach because it allows you the ability to specify DNS at deployment time without the need to modify the underlying virtual network configuration.

Specifying DNS Server on Provisioning

In this example I am creating a DNS object that references a DNS server (10.1.1.4) and I specify it with New-AzureVM. All VMs created in this cloud service will inherit this DNS setting on boot.

$dns = New-AzureDns -Name 'onprem-dns' -IPAddress '10.1.1.4'
New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
    Add-AzureProvisioningConfig -Windows -Password $PWD |
    Set-AzureSubnet 'subnet' |
    New-AzureVM -ServiceName $svc -AffinityGroup 'myag' -VNetName 'VNET' -DnsSettings $dns

Deploying a Virtual Machine into an Active Directory Domain

With Windows Azure Virtual Machines it is entirely possible to have a full Active Directory environment in the cloud. AD can either be hosted on-premises with connectivity provided by a site-to-site VPN tunnel using Windows Azure Virtual Networks OR you can host an AD domain directly in the cloud.

Once AD connectivity is in place you can use the PowerShell cmdlets to automatically join a Windows Virtual Machine directly to an Active Directory domain at provision time. For AD domain join to work you must specify the DNS server IP address for your Active Directory domain.
In this example New-AzureDNS is used to specify the DNS for the VM to point to an AD DNS Server in the cloud (10.2.0.4) which itself has been configured to point to an on-premise AD server (192.168.1.6) in a previous deployment. Setting DNS at this level is also useful because any future VMs added to this cloud service will inherit the DNS setting.

$subnet = 'APPSubnet'
$ou = 'OU=AzureVMs,DC=fabrikam,DC=com'
$dom = 'fabrikam'
$domjoin = 'fabrikam.com'
$domuser = 'administrator'
 
$domVM = New-AzureVMConfig -Name 'advm1' -InstanceSize Small -ImageName $image |      
	Add-AzureProvisioningConfig -WindowsDomain -JoinDomain $domjoin -Domain $dom -DomainPassword $pass -Password $pass -DomainUserName $domuser -MachineObjectOU $ou | 	
	Set-AzureSubnet -SubnetNames $subnet
 
$dns = New-AzureDns -Name 'clouddc-ad' -IPAddress '10.2.0.4' 
 
New-AzureVM -ServiceName 'app-cloudservice' -AffinityGroup 'ADAG' -VNetName 'HybridVNET' -DnsSettings $dns -VMs $domVM

If you would like to try some of this out on your own I highly suggest the Windows Azure Training Kit as a starting point. There are many hands on labs including deploying Active Directory and connecting multiple virtual machines.

Windows Azure IaaS Overload

I took a short breather after TechEd North America and TechEd Europe back-to-back but I did want to put up a post to summarize the sessions around Windows Azure Virtual Machines and Virtual Networks from TechEd 2012. This is a big and extremely important launch for Windows Azure so we have quite a bit of coverage on the subject :)

If you were looking for a crash course on Windows Azure IaaS here it is!


Meet the New Windows Azure – Scott Guthrie

Windows Azure Virtual Machines and Virtual Networks – Mark Russinovich

Windows Azure IaaS and How it Works – Corey Sanders

Extending Enterprise Networks to Windows Azure using Windows Azure Virtual Networks – Ganesh Srinivasan

Deep Dive on Windows Azure Virtual Machines – Vijay Rajagopalan

Running Linux on Windows Azure Virtual Machines – Tad Brockway

Migrating Applications to Windows Azure Virtual Machines – Michael Washam

Deploying SharePoint Farms on Windows Azure Virtual Machines – Paul Stubbs

Migrating SQL Server database applications to Windows Azure Virtual Machines – Guy Bowerman, Madhan Arumugam

Running Active Directory on Windows Azure Virtual Machine – Dean Wells

How to Move and Enhance Existing Apps for Windows Azure – Tom Fuller, Greg Varveris, Purush Vankireddy

Upcoming Presentations on Windows Azure

I have a few upcoming sessions planned for presenting on Windows Azure Virtual Machines.

The first is a FREE all-day Microsoft Cloud Day conference in London (UK) on Friday, June 22nd to learn about Windows Azure and Windows 8 where I will be talking about Windows Azure Virtual Machines and Virtual Networks.

I have quite a bit going on at TechEd Europe in Amsterdam as well.

Migrating Applications to Windows Azure Virtual Machines
This session is in D204 on Thu, Jun 28 12:00 PM – 1:15 PM.

The focus is on how you can pick up existing workloads and move them into the cloud using Windows Azure Virtual Machines (without re-writing your app!).

Windows Azure Virtual Machine Workshops
I will also be delivering two hands on workshops about deploying virtual machines that give you a two hour crash course on deploying a multi-vm application using VMs.

However, to get the inside scoop make sure you attend Mark Russinovich’s session on Windows Azure Virtual Machines and Virtual Networks in Theater on Tue, Jun 26 12:00 PM – 1:15 PM.

Of course, if you can’t make it you can see the recorded session from TechEd NA here: http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/AZR209. If you haven’t familiarized with the new Windows Azure IaaS features yet this is the place to start.

Thanks!
Michael

Windows Azure Virtual Machines

Windows Azure Virtual Machines are a new addition to the services provided by Windows Azure. They allow a much easier and flexible solution for quickly moving an existing workload from on-premises to the cloud or for building new applications that have dependencies on applications that will only run on a server with persistent local storage. Creating VMs in Windows Azure is easy and flexible because Windows Azure provides three different ways of provisioning one.

Creating a VM from an image

The first is to create the virtual machine directly in the cloud using a number of images provided by Microsoft or partners. This is by far the easiest route to take to quickly spin up a new virtual machine.

Virtual Machine Image Gallery

Creating from a Custom Image

The second option is building your own custom images and provisioning virtual machines from the resulting image. This involves creating a new VM using a platform image, customize it with your own software and settings, and then generalize it using sysprep within Windows or waagent -deprovision+user on Linux. Once the VM is generalized and shut down you can then use the capture functionality to save the VM as a custom image.

Note: You can also generalize a VHD offline and upload it using the csupload.exe tool available in the 1.7 SDK.

Bring your own VHD

The third option is uploading existing virtual machines in VHD format. This method also uses the csupload.exe utility. You can upload a generalized image or a non-generalized VHD. The generalized image can be used to provision new VMs in the cloud as a template and the non-generalized VHD can be used as an OS disk to boot from or just a data disk to mount as a data drive.

The process you choose for each of these creation methods is up to you. Windows Azure provides capabilities from a point and click web interface to full automation with PowerShell in addition to a REST based Service Management API.

IaaS is all About the App and not the Runtime

With Windows Azure Virtual Machines the focus is all on the application. Providing the underlying infrastructure to run applications for the most part as-is in the cloud is a key goal behind introducing Virtual Machines. New scenarios such as running applications like SharePoint Server, SQL Server, Active Directory and even various distributions of Linux are now available to Windows Azure users.

VM Architecture

If you have a solid understanding of the current Windows Azure service model you have a good base of knowledge for understanding how Windows Azure Virtual Machines are built. Windows Azure Virtual Machines are built on the same service model that web and worker roles are. However, the service model has just been enhanced to support functionality needed for virtual machines such as persistent storage and single instance per role.

The Cloud Service acts as a container for virtual machines. Within a cloud service there are two slots for deployments for traditional web and worker roles Staging and Production. With virtual machines the Production slot is the only one in use (VIP swapping is not supported with VMs). The deployment is another container for VMs (within the cloud service) that has its own set of properties specifically pertaining to virtual networking which I will talk about later. Within the deployment a virtual machine is its own role with a single instance.

Cloud Services and VM Architecture


Virtual Machine Disks and Storage

One key difference with virtual machines from other Windows Azure roles is the underlying storage. There was actually a long discussion internally on whether to refer to virtual machines as Durable VMs or Persistent VMs before landing on just Virtual Machines. This internal discussion is still visible in the service management API where virtual machines are still referred to as PersistentVMRole.

The disks in a virtual machine are actually stored as page blobs in Windows Azure Storage. When you create a Virtual Machine from an image, a writable copy of the image is created in the storage account you specify on VM creation. This is where the underlying operating system VHD is created. Windows Azure storage offers numerous benefits such as extremely scale and durability. Your virtual machine storage is replicated three times within the same data center and optionally another three times in a different datacenter in the same region for extreme durability. Windows Azure storage also has the benefit of being easily accessible with a well-known API so there is already plenty of tooling available to manage it.

Virtual Machine Architecture

Looking at the diagram above you will notice that I did not add the D: drive into the list. The reason is very important. The D: drive is available to your application but you should very careful about using it for storage. It is actually the physical storage on the rack server the VM is running on. It is NOT backed by Windows Azure storage and should be considered temporary storage only. One great use for it is the OS paging file which contains data that does not need to be persistent (this is the default behavior in Windows Azure).

When dealing with virtual machine storage you should understand capacity and caching. The operating system disk can be at most 127GB. However, each virtual machine can also have additional data disks attached to it up to 1 TB per with the number of data disks dependent on the VM size. Data disks can also be dynamically added or removed while the VM is running. So adding additional storage to the VM is as simple as a few clicks in the portal or a short PowerShell command.

Virtual Machine Size and number of data disks supported



Disk Caching

There is a layer of disk caching support between the virtual machine and the underlying host. The default configuration of your OS disk will have ReadWrite host caching enabled and for data disks no caching is enabled. Take note not to put data on the OS disk without first changing the cache settings using the PowerShell Set-AzureOSDisk cmdlet. The OS Disk can also be configured for ReadOnly caching where the data disk supports None, ReadOnly and ReadWrite.

Windows Azure Virtual Machine Disks

Virtual Machine Networking

Virtual machines within the same cloud service have direct connectivity with each other. You do not have to configure internal endpoints through the service model because virtual machine’s default to allow all traffic on all ports between virtual machines. That does not mean that traffic will flow though. This is Infrastructure as a Service after all so you will still have to manage the firewalls on your VMs to allow traffic between servers.

Name resolution is handled through a multi-tenant DNS service provided by Windows Azure. Note: If you choose to configure a virtual network this DNS service is not provided and you are expected to configure your own DNS if name resolution is a requirement.

Endpoint Configuration

Configuring inbound traffic to your virtual machines is straight forward. Windows Azure has the concept of an Input Endpoint or more commonly known as just an endpoint. An endpoint is associated with a virtual machine and its properties allow traffic to flow.

Endpoint Property Names

  • Name (friendly name of your endpoint)
  • Protocol (tcp/udp)
  • Local Port
  • Public Port

For example if you wanted to configure an endpoint for a single web server your endpoint configuration might look like the following:

  • Name: iishttp
  • Protocol: tcp
  • Local Port: 80
  • Public Port: 80

If you needed to open up the web server for SSL

  • Name: iishttps
  • Protocol: tcp
  • Local Port: 443
  • Public Port: 443

Configuring the Load Balancer
The previous examples are great for opening up a port for a single virtual machine. However, sometimes you need multiple VMs responding on the same port in a load balancer. Windows Azure allows you to directly configure and control which virtual machines are configured for load balancing. It does this through load balanced sets.

Load Balanced Endpoints

A load balanced set is simply configuring the same endpoint on multiple VMs and setting another property called the “LoadBalancedEndpointSetName (or LBSetName in PowerShell) with a common name to group the endpoints together. This functionality is abstracted away within the Windows Azure management portal but it is good to go into in detail because from the command line you can have much more control over the load balancer by using custom health probes.

Load Balanced Endpoints

Configuring a Custom Health Probe for the Load Balancer

For an example on configuring a custom health probe see my post on Automating VMs with PowerShell.

Authenticated Sites and Custom Probes

It is important to understand that the URL configured for the custom probe receives a GET request from the load balancer without passing host headers or any authentication of any kind. So if the probe path you specify returns a 401 ACCESS DENIED then the load balancer is not going to add the VM to rotation. It is important to configure a health check URL here that can respond anonymously.

Port Forwarding

The architecture of cloud services makes endpoint configuration interesting. Since each cloud service has a single public IP address but multiple virtual machines can reside in it how do you address individual servers directly in a non-load balanced fashion?
The answer is port forwarding.

Port Forwarding to Multiple VMs

Port forwarding allows you to configure an endpoint on a specific VM listening on any of the ephemeral ports that will then be forwarded to the correct internal port. The illustration above shows two VMs both listening on ports 3389. To address them individually from the same public IP address two endpoints are made with the first listening on port 5586 and the second on 5587. When a remote desktop client connects to either endpoint they are forwarded to the correct machine.

Virtual Machines and Virtual Networks

Another new set of functionality with this release is Windows Azure Virtual Networks. Virtual Networks are more than just connecting your on-premises data center to the cloud (which does make it tremendously easier for building hybrid networks) but they also provide you the ability to configure the network within your Windows Azure deployment.

Persistent IP Addresses for Virtual Machines

This scenario is the easiest to understand so I will start here first. By default when you provision a virtual machine in Windows Azure you get name resolution by default and IP address management. The defaults are usually good enough until you need to do something that would require a persistent IP (notice I did not say static IP address). In the default networking configuration your VM’s IP address can and will change. So if you need to deploy something like Active Directory the default network stack will not work. This is where virtual networks save the day.

VNETs allow you to define the entire IP addressing scheme for your cloud network. You define the address space, the subnets and ultimately which VM goes into which VNET and subnet. The current biggest benefit to all of this is that each VM provisioned inside a VNET will retain the same IP address no matter how many times it is rebooted or recovered. Think of it as an infinite DHCP lease (do not set the IP address statically!). The downside is you do lose that built in name resolution when using a VNET and you do need to have a basic understanding of subnetting. If /16 and /24 are significant to you then you are likely already there.

Example Virtual Network Configuration (No Gateway)

Connect Your Data Center to a Windows Azure

Using a VPN device that supports site to site VPN you can create hybrid networks that span networks that you define from on-premises to the cloud. This means applications that do not move easily to the cloud can be directly accessed from applications in the cloud without expensive re-writes or wiring up of proxy interfaces to access the data remotely. Accessing the corporate Exchange Server, Active Directory, Solaris machines, whatever is on your corporate network can be made available to applications in the cloud. All that is needed is to configure a gateway in the virtual network configuration, share keys between the Windows Azure gateway and your VPN device and configure.

Connect Cloud Services on the Same Virtual Network

Connecting multiple cloud services on the same virtual network will open up all kinds of technical opportunities. The most obvious is the ability to take an application designed for Windows Azure web or worker roles and have them directly communicate with another application on a virtual machine such as SQL Server or Active Directory or any other application that runs in a VM. Many existing applications could easily be converted to web or worker role except for dependencies that require persistent storage. This should remove many roadblocks for migrating applications to PaaS.
Note that web and worker roles cannot currently be deployed into the same cloud service as a Virtual Machine so for direct connectivity a Virtual Network is required.

Connecting Web or Worker Roles with VMs on a Virtual Network with Hybrid Connectivity

Another interesting scenario is segmenting cloud services for large VM deployments. Since each VM is itself a Role in the Window Azure service model that brings with it the same limitations. There can only be 25 roles per cloud service so that means there can only be 25 virtual machines per cloud service. For truly large applications creating multiple cloud services and connecting them via a virtual network is a simple solution.

Name Resolution in a Virtual Network

As I mentioned earlier name resolution is not provided out of the box when you deploy a VM into a virtual network. The expectation is you will or can provide your own. There are a few ways to configure DNS for VMs in a VNET.

Configure DNS on the Network Adapter

This has the obvious draw back that you have to manually configure the DNS server IP address(s) for each machine.

Specify DNS servers in the network configuration

You can specify DNS servers when you configure the network configuration. The draw back here is there currently is no way of modifying the DNS configuration without redeploying the VMs in the VNET so at the moment this is not a flexible option.

Specify DNS during the first VM deployment

This is by far the most flexible. If for some reason the DNS server needs to be updated it’s a simpler matter of removing thfce VMs and redeploying them using the disks and specifying the new DNS configuration. All of this is easily scriptable from the Windows Azure PowerShell cmdlets (currently this is the only way of setting DNS during deployment).

Deploying a Virtual Machine into a Virtual Network

The final VNET diagram shows a typical configuration when deploying virtual machines into a virtual network. There are a few considerations before deployment.

Configure the Virtual Network First!

The reason is simple: You cannot move a provisioned VM into a VNET. The VM has to be provisioned into the VNET.

Determine DNS up front.

Will you provide it yourself by hosting DNS in your cloud service or point to a public DNS server? Will you set it at the network configuration level or at the deployment level? These are all good questions to answer before deployment. Because once a VM is deployed into a VNET you cannot change the existing settings of the VNET without first removing any VMs or web/worker roles in the VNET.

Each VNET requires an affinity group.

The cloud service that hosts your VNET must reside within the same affinity group.

The storage account has to either be in the same region as the affinity group or the same affinity group.

It cannot reside in a different affinity group within the same region or another region.

Virtual Machine Availability

With the launch of Virtual Machines comes a new SLA. With web and worker roles we offer a %99.95 uptime SLA as long as you have at least two instances of your application running to compensate for host updates and hardware failures. With VMs we realize that many applications do not need (and many do not even work with) multiple VMs. So to address this need we will offer the single instance SLA of %99.9. We will of course also offer the %99.95 (this is still under consideration but did not make GA). SLA assuming you have >1 instance using a new feature called an Availability Set.

Availability Sets

If you are familiar with the concept of upgrade and fault domains then you are mostly there with availability sets. The concept is similar except the functionality of upgrade/fault domains are combined with availability sets with VMs. The main thing to understand is that your VMs in a set will be physically on separate racks in the data center and when we upgrade the host OS beneath your VMs we will never upgrade all of the VMs in the set at the same time so only part of your application is taken down for maintenance.

Availability Sets Visualized

Availability Sets and SLA

Availability Sets give you data center hardware redundancy at each tier of your application.


Wrapping Up

In summary I would say that with the launch of Windows Azure Virtual Machines and Virtual Networks we have opened the gates up to start migrating workloads to the cloud. You are no longer required to re-write/architect your applications to have it run in the cloud. With Virtual Networks you are no longer restricted by an “all or nothing” migration approach or forced to write lots of service wrappers to surface on-premises data over the Internet.

Windows Azure still has all of the great functionality with PaaS style applications but now with the ability to run these applications side by side with traditional apps an entirely new set of opportunities have been opened up.

Thanks!
Michael Washam
Senior Technical Evangelist – Windows Azure

Automating Windows Azure Virtual Machines with PowerShell

With the release of the June 2012 Windows Azure PowerShell cmdlets I’m excited to tell you that I have officially passed the buck!

In this release we have added some exciting functionality to the Windows Azure PowerShell cmdlets and have transitioned ownership and support of the cmdlets to the Windows Azure Engineering teams. The benefit to you as the cmdlet user is the code will be officially supported and much better documented with more frequent release cycles.

To make all of this happen did take a lot of coordination with the different teams so there is a difference in functionality regarding what was previously on CodePlex. For example, the Windows Azure Scaffolding cmdlets for Node.JS have been integrated along with a lot of new functionality for building scaffolding around PHP projects. However, for the first release we were not quite ready to bring forward all of the existing functionality in the Windows Azure cmdlets so with this release a few areas will be delayed: Diagnostics, Traffic Manager and the SQL Azure cmdlets will all come in later waves.

In this blog post I’m going to cover the some of the new functionality around automating Windows Azure Virtual Machines.

Getting Started

Download the Windows Azure PowerShell Cmdlets

Launch the cmdlets by clicking the Windows Azure PowerShell link on the start menu.
If you can’t find this link OR you want to use the cmdlets in your own editor you have to load the modules a little different than normal:

Manual Import of the PowerShell Modules

Import-Module 'C:Program Files (x86)Microsoft SDKsWindows AzurePowerShellAzureAzure.psd1'

(Note: We do plan to add a module in the Users module location soon – we did not forget)

Configuring Your Subscription

The easiest way to get started is to download and import your publish settings file. You can do this by using your web browser and browsing directly to the site: https://windows.azure.com/download/publishprofile.aspx OR by using a handy cmdlet that was previously part of the Node.JS cmdlets Get-AzurePublishSettingsFile which will launch a browser to this URL automatically.
Once you have downloaded the publish settings file it’s a simple matter of importing it.

Import-AzurePublishSettingsFile 'c:tempmysub.publishsettings'

To see the resulting configuration run the following command:

Get-AzureSubscription

If you prefer to manually configure your subscription you can use Set-AzureSubscription to configure your subscription ID and management certificate.

$subid = '[YOUR-SUBSCRIPTION-ID]'
$cert = Get-Item Cert:CurrentUserMyYOURCERTTHUMBPRINT
Set-AzureSubscription -SubscriptionName 'testsub1' -SubscriptionId $subid -Certificate $cert

Note the subscription settings that are imported or set manually are persisted in the following location:
C:UsersuserAppDataRoamingWindows Azure Powershell

This means you do not have to run Set-AzureSubscription for each script since it is already there!

The cmdlets do support multiple subscriptions and to allow you to choose which subscription to work on we provide the Select-AzureSubscription cmdlet to switch the active subscription.

Configure Storage
Finally, one new addition is the –CurrentStorageAccount parameter of Set-AzureSubscription. This parameter tells the cmdlets what storage account they should use if they need to access storage. When dealing with virtual machines this comes up quite a bit.

To set it you can run the following:

Get-AzureStorageAccount   # Enumerates your storage accounts

If you do NOT have a storage account returned above you should create one first.
Run the following to determine the data center to create your storage account in. Ensure you pick a data center that shows support for PersistentVMRole.

Get-AzureLocation

Create your storage account:

New-AzureStorageAccount -StorageAccountName 'myuniquelynamedstorage' -Location 'East US'

Finally, set the -CurrentStorageAccount property of your subscription. The -CurrentStorageAccount parameter specifies the storage account to use for VHD storage during the later exercises

Set-AzureSubscription -SubscriptionName 'testsub1'  -CurrentStorageAccount 'mystorage'

For the remainder of this blog post I’m going to walk you through some exercises from our hands on lab to get you familiar with some of the great things you can automate with Windows Azure Virtual Machines and PowerShell.

Before continuing you should start an elevated PowerShell session and run the following command:
Set-ExecutionPolicy RemoteSigned

Provisioning a Virtual Machine using PowerShell CmdLets

Create a new variable and assign it the location you used to create your storage account or whatever location your selected storage account is currently in.
The virtual machines have to be created in the same location/region as your storage account.

$dclocation = '[YOUR-LOCATION]'

Determine the name of the cloud service that will act as the container for your virtual machines:

Check to see if the name is available (Test returns True if the name already exists)

Test-AzureName -Service '[YOUR-CLOUD-SERVICE-NAME]'

Create a variable to use the unique name of your cloud service.

$cloudSvcName = '[YOUR-CLOUD-SERVICE-NAME]'

Determine which platform image your want to use as the basis of your virtual machine by running:

Get-AzureVMImage | select ImageName

Create a variable to hold virtual machine image name:

$image = '[YOUR-SELECTED-IMAGE-NAME]'

Now let’s create a virtual machine!

If you chose a Windows image you would use the following parameter set:

Quick Create of a Windows VM with PowerShell

$adminPassword = '[YOUR-PASSWORD]'
$vmname = 'mytestvm1'
New-AzureQuickVM -Windows -ServiceName $cloudSvcName -Name $vmname -ImageName $image -Password $adminPassword

For Linux:
Quick Create of a Linux VM using PowerShell

$linuxuser = '[CHOOSE-USERNAME]'
$adminPassword = '[YOUR-PASSWORD]'
$vmname = 'mytestvm1'
 
New-AzureQuickVM -Linux -ServiceName $cloudSvcName -Name $vmname -ImageName $image -LinuxUser $linuxuser

Once the virtual machines have been created you can check their status by running:

Viewing Virtual Machine Settings

Get-AzureVM -ServiceName $cloudSvcName

This cmdlet when -Name is omitted returns all of the virtual machines within the cloud service. You can of course specify a single VM with the -Name parameter.

Restart, Shutdown and Start of a Windows Azure Virtual Machine

# Restart
Restart-AzureVM -ServiceName $cloudSvcName -Name $vmname
 
# Shutdown 
Stop-AzureVM -ServiceName $cloudSvcName -Name $vmname
 
# Start
Start-AzureVM -ServiceName $cloudSvcName -Name $vmname

This exercise showed how you can quickly provision a new virtual machine. What about if you wanted to provision a VM with some settings already configured?

Advanced Provisioning Options

So the following script may put you in PowerShell overload but let me explain it a bit.

The New-AzureVMConfig cmdlet allows you to create a virtual machine configuration object that you can then modify with other cmdlets. You can add data disks, configure endpoints and even modify the disk cache behavior for the OS Disk or the data disks at this stage. You can also add endpoints that are needed for the virtual machine such as port 80 and port 443 (we automatically add endpoints for RDP and SSH when you create a VM off of an image – there is a bug where we do not when creating off of a disk).

So each call to New-AzureVMConfig returns back a object you can then modify with Add-AzureDataDisk, Set-AzureOSDisk, Add-AzureEndpoint etc.. You then take the VM object(s) and pass them to the New-AzureVM cmdlet to actually create the VM(s).

Creating Windows Virtual Machines with PowerShell

   $vmname2 = 'mytestvm2'
   $vmname3 = 'mytestvm3'
 
   $vm2 = New-AzureVMConfig -Name $vmname2 -InstanceSize ExtraSmall -ImageName $image |
             Add-AzureProvisioningConfig -Windows -Password $adminPassword |
             Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
             Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 
 
   $vm3 = New-AzureVMConfig -Name $vmname3 -InstanceSize ExtraSmall -ImageName $image |
          Add-AzureProvisioningConfig -Windows -Password $adminPassword  |
           Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk2' -LUN 0  |
           Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 
 
   New-AzureVM -ServiceName $cloudSvcName -VMs $vm2,$vm3

Creating Linux Virtual Machines with PowerShell

   $vmname2 = 'mytestvm2'
   $vmname3 = 'mytestvm3'
 
   $vm2 = New-AzureVMConfig -Name $vmname2 -InstanceSize ExtraSmall -ImageName $image |
          Add-AzureProvisioningConfig -Linux -LinuxUser $linuxUser -Password $adminPassword |
          Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
           Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 
 
   $vm3 = New-AzureVMConfig -Name $vmname3 -InstanceSize ExtraSmall -ImageName $image |
            Add-AzureProvisioningConfig -Linux -LinuxUser $linuxUser -Password $adminPassword |
            Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk2' -LUN 0 |
            Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 
 
   New-AzureVM -ServiceName $cloudSvcName -VMs $vm2,$vm3

In this example is I have added the -LBSetName, ProbePort, ProbeProtocol and ProbePath parameters. These are optional for creating a single endpoint but in this case they configure the load balancer for these two virtual machines.

Configuring Other Protocols for the Load Balancer

In addition to configuring HTTP requests, the PowerShell cmdlets can also configure endpoints for UDP and TCP connections as well.
For TCP sockets applications you can specify -ProbeProtocol as tcp instead of http and remove the -ProbePath parameter. The load balancer will attempt to connect to the TCP port specified in -ProbePort and if successful will keep the VM in the load balancer.
For load balancing UDP connections you will need to specify an alternate probe endpoint that can tell the load balancer whether the UDP service is available. This could be a simple http page that returns a 200 assuming the service is up.

Updating Existing Virtual Machines

Modifying an existing virtual machine requires retrieving the current settings by calling Get-AzureVM, modifying them and then calling the Update-AzureVM cmdlet to save the changes.
You can hot add and remove data disks and networking endpoints. Changing disk cache settings requires a reboot as does changing the virtual machine’s instance size.
The following example uses the Get-AzureVM cmdlet to retrieve the VM object and send it to the PowerShell Pipeline.

Add-AzureDataDisk with the CreateNew parameter allows you to dynamically add storage to the virtual machine. In this case we are calling it twice to attach to unformatted blank VHDs to the server each 50 gigs of storage each. The -LUN parameter tells the order of the device being attached and optionally uses the -MediaLocation to specify the location in Storage to keep the newly created VHDs.

Add-AzureDataDisk also supports the Import parameter to attach a disk in the disk library and -ImportFrom to attach a disk that already exists in storage.

The example also adds a new endpoint for TCP port 1433 internally that is listening externally on port 2000 using the Add-AzureEndpoint command.

$vmname = 'mytestvm1'
 
Get-AzureVM -Name $vmname -ServiceName $cloudSvcName |
    Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
    Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'translogs1' -LUN 1 |
    Add-AzureEndpoint -Protocol tcp -LocalPort 1433 -PublicPort 2000 -Name 'sql' |
    Update-AzureVM

Once the virtual machine has been updated you can RDP into the VM to configure the disks using Computer Manager -> Disk Management.

Get-AzureRemoteDesktopFile -ServiceName $cloudSvcName -Name $vmname -Launch

Get-AzureRemoteDesktopFile also allows you to save the RDP files for your VMs to the local file system.

Saving RDP files for all VMs in a Cloud Service

Get-AzureVM -ServiceName fabrikam-cloudapps | foreach { 
	$rdpfile = 'D:User-DataScratch' + $_.Name + '.rdp'
	Get-AzureRemoteDesktopFile -ServiceName $cloudSvcName -Name $_.Name -LocalPath $rdpfile
}

Modifying Disk Cache Settings

There is a disk cache built into Windows Azure VMs. Essentially, Windows Azure can use the local physical disk attached to the VM for caching.

The OS disk can support ReadOnly and ReadWrite cache settings with the default being ReadOnly
Data Disks can support None, ReadOnly and ReadWrite with the default being None (The safest choice for data files)

Modifying the Data Disks Host Cache Settings

Get-AzureVM -ServiceName $cloudSvcName -Name $vmName  |
       Set-AzureDataDisk -HostCaching ReadWrite -LUN 0 |
       Set-AzureDataDisk -HostCaching ReadWrite -LUN 1 |
       Update-AzureVM
 
Get-AzureVM -ServiceName $cloudSvcName -Name $vmname | Get-AzureDataDisk

Modifying the OS Disk Host Cache Settings (Requires a Reboot)

    Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |   # Retrieve existing VM 
        Set-AzureOSDisk -HostCaching ReadOnly | # Perform an operation that modifies the returned VM and pipe the results
        Update-AzureVM  # Update the modified VM

Modifying the VM Size

You are allowed to change the instance size of your VM through the portal and through PowerShell.

Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |
    Set-AzureVMSize -InstanceSize Medium |
    Update-AzureVM

TIP: I have separated these updates into separate examples but you should know that they can be combined into one batch update.

Modifying the OS Disk Cache Settings

    Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |   # Retrieve existing VM 
        Set-AzureOSDisk -HostCaching ReadOnly | # Perform an operation that modifies the returned VM and pipe the results
        Update-AzureVM  # Update the modified VM

P.S. I’ll document some of the changes to the existing deployment cmdlets soon. One thing you will immediately notice though is it is no longer necessary to call Get-OperationStatus after every cmdlet. The cmdlets now handle the waiting automatically.

With this post I hope I get you started learning how to create and manage your virtual machines with the new Windows Azure PowerShell cmdlets. Look for more articles in the very near future that detail some of the more advanced scenarios you can accomplish.

Thanks!
Michael Washam
Senior Technical Evangelist – Windows Azure