Create persistent Azure Automation Runbook variables using Azure Blob Storage

ยท

4 min read

Because Azure Runbooks are invoked in temporary environments, you cannot save the Runbook results for later use in the local files. Which is one of the drawbacks compared to on-premises Schedules Tasks.

But what if you need some output persistence? For example, to take note of what users have already been notified about some issue, so you don't send them the same email next time, the Runbook runs.

For such cases, you have several options, based on the data you need to save.

  • If you need to store some simple string or integer, built-in Azure Automation Variable is the way to go

  • Built-in Azure Automation Variable can be used for saving some complex objects like array of hashtables etc too (check How to save complex PowerShell variables as CliXML instead of Newtonsoft.Json.Linq.JProperty in the Azure Automation post), but there are some limitations regarding the variable size. Therefore this can be used only for "small" objects a.k.a. it is not very reliable.

  • But what if you need to store complex variables no matter what their size is? That's where Azure Blob Storage steps in ๐Ÿ‘

Comparison of saving variables to Azure Blob Storage

Advantages

  • Variables can be serialized using well-known Export-CliXml (a.k.a. saved as XML file) and then deserialized using Import-CliXml (a.k.a. converted back to the original object) PowerShell commands

  • You can saveany PowerShell complex object of any size without worrying about hitting some internal Automation limits

  • When importing the serialized variable back to the Runbook, you will get the same complex object of the same type, etc

  • Can be used to pass complex variables between different Runbooks

Disadvantages

  • Azure Storage isn't free to use, but it will cost you like 0.1 $/month (depending on the exported file size and amount of data transfers)

  • It's not as simple as using built-in Azure Automation Variable, because you need to create a Storage Account and set appropriate permissions

  • Saving and reading data from the Storage Account makes your Runbook a little bit slower


How to save the Runbook variable to Blob storage then?

Create an Azure Storage Account

We don't need anything special for our case, so create a Standard LRS Storage Account with Hot-tier storage (official documentation).

You can name it like persistentvariablesstore.

Create a Container to store your variables (XML files)

Create Container (it's like a folder). For example variables.

Grant IAM role to the Automation Managed Identity

In the Storage Account Access Control (IAM) section, grant a role Storage Account Contributor to your Azure Automation Account Managed identity. This allows the Runbook to read/write data to your Storage Account storage.

Import the AzureResourceStuff module to your Azure Automation

To be able to use my Export-VariableToStorage, Import-VariableFromStorage PowerShell functions that will help you to save/load variables from Azure storage, you need to add the AzureResourceStuff module to your Azure Automation

You can do this manually using Add a module button.

Or using my function New-AzureAutomationModule (part of my AzureResourceStuff module).

New-AzureAutomationModule -moduleName AzureResourceStuff -resourceGroupName yourRESOURCEgroupNAME -automationAccountName yourAUTOMATIONaccountNAME

For more details check my post Import new (or update existing) PowerShell module (including its dependencies!) into Azure Automation Account using PowerShell

Use my PowerShell functions inside your Runbook to export/import your variables

Now when the AzureResourceStuff module is added, you are ready to save variables from your Runbook to Azure storage and vice versa.

Just add a similar code to your Runbook ๐Ÿ‘‡

# functions Import-VariableFromStorage, Export-VariableToStorage are part of module AzureResourceStuff 

# authenticate, so you Runbook can work with the Storage
$null = Connect-AzAccount -Identity

# select variable name, this name will be used to store variable in the Storage (as '<name>.xml')
$persistentVariableName = "previouslyProcessedUsers"

# use parameter splatting to define common parameters
$varFncParams = @{
    fileName = $persistentVariableName
    resourceGroupName = "PersistentRunbookVariables"
    storageAccount = "persistentvariablesstore" # cASe SENSitive!
    containerName = "variables"
}

# set your variable
$previouslyProcessed = ...

# to export the variable to the Storage
Export-VariableToStorage -value $previouslyProcessed @varFncParams

# to import the variable back from the Storage
$previouslyProcessed = Import-VariableFromStorage @varFncParams
๐Ÿ’ฅ
Don't forget to customize resourceGroupName, storageAccount and containerName keys in the $varFncParams hashtable to match your environment!

To get more details about functions parameters etc, check their help.

Happy scripting ๐Ÿ™‚

Did you find this article valuable?

Support Ondrej Sebela by becoming a sponsor. Any amount is appreciated!

ย