Entity's Tech Blog

Link to Part 1:
http://blog.entity-project.eu/?p=57

Prologue:
Because Part 2 turned into a gargantuan spaghetti of lines and code I’ve decided to split it up into 2 sections, Part 2a and Part 2b, to keep it easy on the eyes and doable to read / execute.
Part 2b will be released after the weekend.

Welcome to Part 2 of the Azure Provisioning Blog series.

In part 1 our script ended up being capable of:

• Provisioning Azure VM’s
• “some” form of automation built in

Our “ToDo” list for this installment is:

• Create automations for the basics such as Storage Accounts
• Create a way to input INI or lets call them “build” files so that the script has zero input necessary and can, if needed accept input from a third party (application)
• OPTIONAL: Create some form of logging

In the end we should have a script that once started will build all VM’s to specification as provided in BUILD Files, is clever enough to create storage accounts if and when needed and do all initial prep and config work on its own.

“Before we begin,

As you know from Part 1, my love for all things “cloud” stops at the incessant use of the word itself  I have nothing with off shoring / outsourcing / hosted solutions / virtualization and whatever else these days gets bundled up under the “cloud” name. I do however have an issue with, as stated in part 1, carpet bombing every issue / technical challenge with the “cloud!” word and praying to the kernel gods that all your infrastructure issues and challenges will magically resolve itself.

I received a bit of flack from inside my professional environment stating that “cloud is the future” and basically that I should “Get with the program”. Now, I thought my views and opinions were quite clear but, apparently not so I’ll rehash them again and explain.

In and off itself I do not have any issue with using the term Cloud. The condition for this is if you use it in context and doesn’t use it just to sound “groovy”.

For me Cloud is a solution you rent / buy or provide which is transparent for end users and in some way for yourself, is thoroughly automated and has some form of charge-back.  Slapping together a bunch of servers, a hyper-visor and the likes of Nagios, SCOM or the likes does not a cloud solution make.
Far too many companies are patting themselves on the back for having a “private cloud”, when in reality they just renamed their virtualization solution to sound hip. On-Demand creation of Virtual Machines, Charge-back (even if just to make a point “look you are costing us this much by wanting these specs”), self-service portal, automated monitoring & provisioning (of both VM as-well as software) are just a few of the things that make what for me can be called a cloud solution. 

I hope this settles the debate on what I consider to be “cloud” and hopefully people reading this will use the term Cloud less liberally. Used too often and too liberal, the power and meaning of a word, a concept, an idea will go to waste, and this should not happen if we as IT Professionals hope to change the way we think, work and do. “

 

Now that is over, let’s get down to the nitty-gritty shall?
We will start off with what I consider easiest and making most sense, using BUILD files (.INI) to automate the creating and deployment further.

Essentially what we wish to become in terms of flow is the following:

read INI files
for each INI file create Virtual Machine

Seems stupendously logical and that is because, it is! No need to over complicate things. Read files –> do “stuff” –> profit all the while leaning back in your chair and waiting for the script to do your work for you.

So how does this look ? (partially done)

#Get Root Dir from where script runs
$localDir =  Split-Path $script:MyInvocation.MyCommand.Path
#Get Build files, located in 'rootdir\vms\'
$buildList = Get-ChildItem ($Localdir + '\vms\')
#For every buildfile we have, make a config
foreach ($buildfile in $buildList) {
	#Read the build file
	$buildContent = Get-Content $buildfile
	#Declare variables we will need / support at this time
	$vmname = ''
	$cloudSvcName = ''
	$adminPassword=''
	$image=''
	$Size= ''
	[array]$disklabel
	[array]$diskSize
	[array]$LunCounter

	#Create the actual Config
	#To Be Written Later
	}

So, I’ve started off with showing you this first, reason being is that you might notice something “peculiar”. All variables are just that, single purpose variables, strings even, except anything that has to do with the extra disk’s. Reason for this is because when I wrote this I ran into the issue where I questioned myself on how many data-disk’s I should support? The easy answer would have been “One”, easy to justify and just make the argument that “the script is specifically written like this to standardize the automation, any particulars you want will have to be manually added, the addition of a multitude of data-disk’s is one of those”

Now, I’m all for being lazy when it comes to work … Heck most of my scripts are born because I can’t be bothered doing xx amount of clicks.

“What’s that? Do manual labor?! To hell with that, SCRIPT IT!“

In most cases being lazy is the right approach but either I make an effort now and make the script such that it is truly generic in the way it handles input and have it support “unlimited” data-disk  or I don’t and I end up having to do a load of click-work in the future because some people want 20x 5Gb disks or some such.

So, to make the Data-disks not limited in how many you can use, we use Array’s. Pretty simple and logical approach no? How we’ll handle this will come later in this post. For now, let’s get reading & filling variables:

On to filling up all variables!

	foreach ($line in $buildContent){
			switch -wildcard ($line) 
    		{ 
		        "vmName*" {
					[array]$Catline=$line.split(";")
					$vmname=$Catline[1]
				} 

		        "dcLocation*" {
					[array]$Catline=$line.split(";")
					$dclocation=$Catline[1]
				} 

			    "cloudSrvName*" {
					[array]$Catline=$line.split(";")
					$cloudSvcName=$Catline[1]
				}

		        "adminPassword*" {
					[array]$Catline=$line.split(";")
					$adminPassword=$Catline[1]
				}

		        "imageUsed*" {
					[array]$Catline=$line.split(";")
					$image=$Catline[1]
				} 

			   "vmSize*" {
					[array]$Catline=$line.split(";")
					$Size=$Catline[1]
				} 

				"dataDiskLabel*" {
					[array]$Catline=$line.split(";")
					[array]$disklabel +=$Catline[1]
				}

				"dataDiskSize*" {
					[array]$Catline=$line.split(";")
					[array]$diskSize +=$Catline[1]
				}

				"dataDiskLun*" {
					[array]$Catline=$line.split(";")
					[array]$Lun+=$Catline[1]
				}

				"dcLocation*" {
					[array]$Catline=$line.split(";")
					$dclocation=$Catline[1]
				}
		    }
	}

As some may, or may not, know, the SWITCH statement is the Case Select / Select Case as we know it in VB.net (or as I learned it in Uni) so instead of using IF loops like I had in the past, this looked a bit nicer and better.
The -wildcard parameter signals that we’ll be using a wildcard (in this case the * sign).

The script will now search (for instance) for each line whether it starts with ‘vmName’. If it does, the code associated with it will be run and processed.

After this came creating the vm Config:

#Create the actual Config
	$vm=New-AzureVMConfig -Name $vmname -InstanceSize $Size -ImageName $image | Add-AzureProvisioningConfig -Windows -Password $adminPassword 

	$counter = 0
	foreach ($disk in $disklabel){
	$vm = $vm | Add-AzureDataDisk -CreateNew -DiskSizeInGB $diskSize[$counter] -DiskLabel $disk -LUN $Lun[$counter]
	$counter+=1
	}

Ok so, what did we do here? Quit simple, I created a new AzureVMConfig, assigned it to the $vm parameter and then “the magic” began, … with magic I mean I created another loop.

This time it’s quite simple, for every $disk (ie: disklabel) we have in the diskLabel Array we will add a AzureDataDisk. Simply put, and as you can see, we add the $vm parameter with the pipeline to add a new disk. The counter we declared at the beginning is used simply to signal to the $diskSize and $Lun Array which parameter they need to use.

Now that we are done we can start implementing all of this code into the “old” codebase (read: the old semi-automated script from part 1). When we do that, this is what we become:

#Windows Azure Scripting Test Environment
#Import Azure Mods, NEED TO BE INSTALLED
Import-Module 'C:\Program Files (x86)\Microsoft SDKs\Windows Azure\PowerShell\Azure\Azure.psd1'
#Import PublishSettings File 
Import-AzurePublishSettingsFile 'F:\GoogleDrive\Google Drive\Project Azure\azure-ms.publishsettings'

#Default DC Location is 'North Europe' specific one can be defined in BUILD file
$defaultdclocation = 'North Europe'

Set-AzureSubscription -SubscriptionName 'SubScriptionName' -CurrentStorageAccount 'portalvhdxxxxxxxxx'

## BEGIN IMPLEMENTATION PART 2
#Define Names
#Get Root Dir from where script runs
$localDir =  Split-Path $script:MyInvocation.MyCommand.Path
#Get Build files, located in 'rootdir\vms\'
$buildList = Get-ChildItem ($Localdir + '\vms\')
#For every buildfile we have, make a config
foreach ($buildfile in $buildList) {
	#Read the build file
	$fileToGet =  $localdir + '\vms\' + $buildfile
	$buildContent = Get-Content $fileToGet
	#Default Variables - can be set optional in the BUILD files
	$dclocation=$defaultdclocation
	#Declare variables we will need / support at this time
	$vmname = ''
	$cloudSvcName = ''
	$adminPassword=''
	$image=''
	$Size= ''
	[array]$disklabel
	[array]$diskSize
	[array]$Lun

	#read the lines of the buildfile (ie: buildcontent) and process
	foreach ($line in $buildContent){
			switch -wildcard ($line) 
    		{ 
		        "vmName*" {
					[array]$Catline=$line.split(";")
					$vmname=$Catline[1]
				} 

		        "dcLocation*" {
					[array]$Catline=$line.split(";")
					$dclocation=$Catline[1]
				} 

			    "cloudSrvName*" {
					[array]$Catline=$line.split(";")
					$cloudSvcName=$Catline[1]
				}

		        "adminPassword*" {
					[array]$Catline=$line.split(";")
					$adminPassword=$Catline[1]
				}

		        "imageUsed*" {
					[array]$Catline=$line.split(";")
					$image=$Catline[1]
				} 

			   "vmSize*" {
					[array]$Catline=$line.split(";")
					$Size=$Catline[1]
				} 

				"dataDiskLabel*" {
					[array]$Catline=$line.split(";")
					[array]$disklabel +=$Catline[1]
				}

				"dataDiskSize*" {
					[array]$Catline=$line.split(";")
					[array]$diskSize +=$Catline[1]
				}

				"dataDiskLun*" {
					[array]$Catline=$line.split(";")
					[array]$Lun+=$Catline[1]
				}

				"dcLocation*" {
					[array]$Catline=$line.split(";")
					$dclocation=$Catline[1]
				}
		    }
	}

	#Create the actual Config
	$vm=New-AzureVMConfig -Name $vmname -InstanceSize $Size -ImageName $image | Add-AzureProvisioningConfig -Windows -Password $adminPassword 

	$counter = 0
	foreach ($disk in $disklabel){
	$vm = $vm | Add-AzureDataDisk -CreateNew -DiskSizeInGB $diskSize[$counter] -DiskLabel $disk -LUN $Lun[$counter]
	$counter+=1
	}

	#Time To Create a VM
	$outcome = Test-AzureName -Service $cloudSvcName
	if ($outcome -eq $true){
	New-AzureVM -ServiceName $cloudSvcName -VMs $vm
	}elseif ($outcome -eq $false){
	New-AzureVM -location $dclocation -ServiceName $cloudSvcName -VMs $vm
	}
}

This gives us a script that is automated in such a way that it will:

• Accept BUILD files and build VM’s
• Is tolerant towards Cloud Service Names who don’t yet exist
• Supports “infinite” (read: however much Azure supports) Data-disk’s

Now we can even further improve on this, but won’t yet. Right now what bothers me is that we still have hard-coded:

I don’t have any real preference on the approach; there is merit to the both of them. For now and to make it “easy” I’ll stick with expanding the BUILD files and doing it that way.

To do this we’ll need to the following:

#Variables for:
#StorageAccount
$storageAcc
#SubscriptionName
$subName
#PublishSettings file
$publishSettFile

Simply create these variables and add the SWITCH statement associated with them to the script.

"storageAccount*" {
	[array]$Catline=$line.split(";")
	$storageAccount =$Catline[1]
			}

"subScriptionName*" {
	[array]$Catline=$line.split(";")
	$subName=$Catline[1]
			}
"publishSettingsFile*" {
	[array]$Catline=$line.split(";")
	$publishSettFile=$Catline[1]
		}

BEWARE:

Create the variables INSIDE the initial for each loop so that these var’s have a lifespan for the duration of the BUILD file. If you declare them outside the foreach loop, we will run into issues at a later date when we add error handling.

An example of how our BUILD file will look like:

storageAccount; EntityBlogStorageSub
subScriptionName; EntityBlogSub
publishSettingsFile; Entity-Blogpublishsettings
vmName;Entity-Blog-vmName
dcLocation;North Europe
cloudSrvName;Entity-CloudService
adminPassword;12345
imageUsed;isoImageEntityTestImage
vmSize;ExtraLarge
dataDiskLabel;test-disk1
dataDiskSize;100
dataDiskLun;0
dataDiskLabel;test-disk2
dataDiskSize;200
dataDiskLun;1
dataDiskLabel;test-disk3
dataDiskSize;300
dataDiskLun;2

Ok, onto the final straight to finish this off ! We need to now rewrite a part of the script so that it will use what the build files give them.
We do this by moving the following lines of code:

#Import PublishSettings File 
Import-AzurePublishSettingsFile 'F:\GoogleDrive\Google Drive\Project Azure\azure-ms.publishsettings'
Set-AzureSubscription -SubscriptionName 'SubScriptionName' -CurrentStorageAccount 'portalvhdxxxxxxxxx'

before the section marked with

#Time To Create a VM

Then we re-write this so that it will use the variables previously defined like so:

#Import PublishSettings File 
	Import-AzurePublishSettingsFile $Localdir + '\publishFiles\' + $publishSettFile
	#Set Azure Subscription
	Set-AzureSubscription -SubscriptionName $subName -CurrentStorageAccount $storageAcc

This leaves us with a script that is completely generic and is completely dictated by the BUILD files on how / what / where to deploy Virtual Machines.

…

I’m going to end Part 2a here; as I fear I am in danger of turning Part 2a into something gargantuan and unpleasant to read otherwise.
Part 2b will be published after the weekend and will include the following to round up Part 2:

• Create a storage account if the one given does not exist
• See about reducing the variable count. Can’t we automate certain things? For instance the Subscription Name?
• build in some check’s to prevent mishaps

addendum: I am well aware that most of my PowerShell scripts could be written easier, in less lines. However, as stated in part 1 my PowerShell Expertise isn’t that extensive and I’m trying to keep everything as open ended and easy to understand as can be. So I will use for example 2 lines to do something, instead of 1 just to keep everything more understandable