Powershell is becoming something you just need to master in the Windows world to manage resources in the cloud, such as Azure, or in the on-premises world. But what do you do if you are a 100% non-Windows shop? It makes no sense for such a customer to learn and use powershell since you want the tooling to be on the platform you use. Recently I was faced with the task to help a Linux-only customer to do just this and this blog post is how I solved it using Python scripting.

The task and the tooling

The task was to automate provisioning of two VMs in Azure that where using Internal Load Balancing. The tooling used was the Python SDK for Azure with a little help of Azure CLI (command line interface) to cover where the Python SDK didn’t implement certain ILB related functions.

I used an Ubuntu 14.04 VM in Azure as my workstation and downloaded both the Azure CLI and the Azure Python SDK. All is publically documentet and you just have to start with getting the Python Package Index installed via the sudo apt-get install python-pip command.

JSON config file

Over the years with Azure I realized that scripts multiply like rabbits and are not reusable unless you completly avoid hardcoding in your scripts. Therefore I created a json config file to hold data needed for the creation process. The file is self describing, but the endpoint definition for http port 80 has an extra value named “ilbname” to mark it as being an endpoint that should be internally load balanced.

The VMs and the Internal Load Balancer all live in a subnet of a Virtual Network and the ILB needs to get it’s own ip address, which is why you see the 10.4.1.20.

The ServiceManagementService class

In the Python SDK for Azure there is a class called ServiceManagementService which is designed to be a wrapper around Azures ServiceManagement REST API. The constructor takes the subscription guid and a management certificate. The certificate on Linux is a path to a pem-file and on on Windows it’s a path to the cert store, such as CURRENT_USER\\My\\mycertname. See references below for how to create the pem-file.

The Python script starts with creating the Cloud Service via the create_hosted_services method. After that, the script enumerates the number of VMs specified, prepare the networking, image media, disk storage and VM config.

The NetworkConfiguration consists of the subnet in the Virtual Network that the VM should belong to and each enpoint. It would have been nice to be able to do what you can do in Powershell, where the Add-AzureEndpoint has a -InternalLoadBalancerName parameter, but the Python SDK do not (yet) support that. Therefore we need to skip the endpoints that are ILB’d and handle them later with two Azure CLI calls.

The LinuxConfigurationSet sets the VM name and the userid and password. Also note the disable_ssh_password_authentication=False which is needed to enable us to login via PuTTY without a cert.

The VM provisioning method is different for the first VM and the subsequent. The first is created via the create_virtual_machine_deployment method while the subsequent VMs that live in the same Cloud Service are provisioned with the add_role method. This is a little different from Powershell, where the New-AzureVM would be used for all VMs in the same Cloud Service

A little help from Azure CLI with the ILB setup

Since the Python SDK for Azure can’t handle the creation of the ILB and the VM endpoints belonging to that ILB, we need to invoke Azure CLI to get this done. After we have created the first VM, we create the ILB using the azure service internal-load-balancer add CLI call where we specify the ILB name, it’s ip address and the Cloud Service it belongs to. We invoke the call with a simple os.system() call that spawns an OS shell process. Since the ILB needs to be created before we can add endpoints to it, we wait an arbitrary amount of time (120 seconds) for it to finish.

Then, for each endpoint in the json config that has an ILB name specified, we call Azure CLI again with the command azure vm endpoint create where we specify the ILB name in the -i parameter.

Running the script

Running the script from my Ubuntu Linux machine gives the following output. The output is a mix of the Python print statements and the Azure CLI that is spawned. Note especially at the end where Azure CLI adds the second VM to the ILB where it says it “already has … using this”

When the script completes, you can see that you have an internal load balanced endpoint in the Azure portal on the Cloud Service. The ILB ip address is listed under INPUT ENDPOINTS and it is a portal UI thing that it only points it to the first VM.

Installing a simple node.js server script on each server and browsing to the ILB ip address proves that the servers respond behind that address.

Deprovisioning the deployment

The Python script I created also includes the important step to deprovision the Cloud Service and the VMs. Over time and after endless demoes, I realized that this step is equally important since it saves money and enables you to run the provisioning part again without the fear of it crashing due to something was not removed from last demo time.

The deprovision part is easy and you just have to call the methods delete_deployment() and delete_hosted_service() on the ServicemanagementService Python class.

References

How to use ServiceManagement from Python – Azure documentation
https://azure.microsoft.com/en-us/documentation/articles/cloud-services-python-how-to-use-service-management/

How to use SSH with Linux on Azure – Azure documentation
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-use-ssh-key/

Installing Python and the SDK
https://azure.microsoft.com/en-us/documentation/articles/python-how-to-install/

Github repo for how to install a Linux cluster using Python
https://github.com/chanezon/azure-linux/blob/master/coreos/cluster/azure-coreos-cluster

Downloads for Azure CLI and Python SDK
https://azure.microsoft.com/en-us/downloads/

Sources

The zipfile contains the python script and a sample json config file. http://data.redbaronofazure.com/public/Python_ILB_provisioning.zip

If you want to give it a spin, you have to create a Ubuntu Linux server in Azure, download the Azure CLI and Python SDK, then create a certificate pem-file and upload that as a management certificate in the Azure portal. You create the pem-file with openssl (explained in the How to use SSH ref). Finally, you need to import the publish settings file for Azure CLI to work.

The files in the zipfile are saved with *nix line endings, so if you view them on a Windows box and get funny line wrappings, that is why.