A Revolution for Iac in Azure Devops Pipelines

When you use Azure DevOps as your platform to organize (and code) your project, you may want to use Azure DevOps pipelines to deploy the system into the (Azure) cloud. If so? Read on!

Developers and OPS Engineers prefer utilizing Azure DevOps Pipelines for deploying their software into the Azure cloud due to its seamless integration with Azure services, streamlined automation capabilities, and comprehensive end-to-end workflow management. With native support for Azure services, teams can easily orchestrate deployments, and manage configurations, ensuring rapid and reliable delivery of applications in the Azure cloud environment.

Taking advantage of Infrastructure as Code

As a developer or OPS engineer, you really want to leverage Infrastructure as Code (IaC) when deploying software from Azure DevOps to the Azure Cloud for several compelling reasons.

Firstly, IaC enables them to define infrastructure configurations using code, promoting consistency, repeatability, and version control, which are crucial for managing complex cloud environments effectively. By treating infrastructure as code, teams can automate provisioning, configuration, and scaling processes, reducing manual errors and ensuring reproducibility across different environments.

Additionally, IaC facilitates collaboration between development and operations teams, as they can work together to define infrastructure requirements within the same codebase, fostering agility and alignment with application deployment pipelines.

Challenges

So now there are two moving parts, one is the Azure DevOps pipeline, a workflow mechanism that you can take advantage of to build, test, and deploy your software system (to the cloud). And part of it is provisioning the cloud environment it targets. The challenge here is to get the interaction between the two fully working. For example, the pipeline needs to deploy a website on a server that doesn’t exist yet.

Do you hard-code the name of the web server in the pipeline, assuming that the web server has been created once the IaC process completes? I think the neat way is to get an output variable from your IaC process, telling you the name of the webserver it created.

With IaC you can define output variables. And sometimes you really need them. I think it is a good practice to use the uniqueString() function in Bicep to generate the name of a storage account for example. Now when you need that storage account later in the deployment process, it is nice to be able to get that value from an output variable.

However, these output variables are hard to work with, since you need a script or additional plug-ins to get the values and store them in a way you can use them later on.

Problem Solved!

Today I found a way to do this SUPER EASY! I always (because I like to) transpile my Bicep templates to JSON in my DevOps pipelines, and publish all the JSON files (templates and parameter files) as a pipeline artifact with the name infrastructure. Now let’s assume that I have a job that deploys infrastructure in Azure:

- job: build
displayName: Build Image
steps:
    - download: current
    artifact: infrastructure
    - task: AzureResourceManagerTemplateDeployment@3
    inputs:
        deploymentScope: "Subscription"
        azureResourceManagerConnection: ${{ parameters.serviceConnectionName }}
        csmFile: $(Pipeline.Workspace)/infrastructure/main.json
        csmParametersFile: $(Pipeline.Workspace)/infrastructure/main.params.json
        location: ${{ parameters.location }}
        deploymentName: "super-awesome-deployment"
        deploymentOutputs: "armOutputs"
        useWithoutJSON: true

Now the nifty trick is all the way on the bottom. The deploymentOutputs has been there for quite some time, but required us to do some magic to actually work with those outputs. But now there is a new parameter, useWithoutJSON. When you set that to true, you can immediately start working with the outputted variables.

An example

The example below uses output variables of the template above to create a VM image with Packer, with a storage account created in the previous step, in a resource group also created in the previous step :

- task: PackerBuild@1
displayName: Build Image with Packer
inputs:
    templateType: "builtin"
    ConnectedServiceName: ${{ parameters.serviceConnectionName }}
    isManagedImage: true
    managedImageName: "temp-${{ parameters.versionNumber }}"
    location: ${{ parameters.location }}
    storageAccountName: $(armOutputs.storageAccountName.value)
    azureResourceGroup: $(armOutputs.resourceGroupName.value)
    baseImageSource: "default"
    baseImage: "Canonical:0001-com-ubuntu-server-jammy:22_04-lts-gen2:linux"
    packagePath: "packer"
    deployScriptPath: "vm-deployment.sh"
    additionalBuilderParameters: '{"vm_size":"Standard_D4s_v5"}'
    skipTempFileCleanupDuringVMDeprovision: false

Note that the storageAccountName and the azureResourceGroup parameters use a $(armOutputs.XXXXX.value). This name armOutputs comes from the deploymentOutputs parameter of the AzureResourceManagerTemplateDeployment above. Then XXXXX can be replaced by the name of the output parameter name in your Bicep file, and .value is used to get its value.