.NET Core Opinion #8 – How to Use Azure DevOps Pipelines

In a previous post I said to be wary of GUI build tools. In this episode of .NET Core Opinions, let me show you a "configuration as code" approach to building software using Azure DevOps.

Instead of the trivial one project demo you’ll see everywhere in the 5 minute demos for DevOps, let’s build a system that consists of:

• An ASP.NET Core project that will run as a web application

• A Go console project that will run once-a-week as a web job

• An Azure Functions project

Let’s also add some constraints to the scenario (and address some common questions I receive).

• We need to deploy the web and Go applications into the same App Service.

• We need to deploy the functions project into a second App Service that runs on a consumption plan.

Using YAML

The first step in using YAML for builds is to select the YAML option when creating a new pipeline instead of selecting from the built-in templates that give you a graphical build definition. I would post more screen shots of this process, but honestly, the UI will most likely iterate and change before I finish this post. Look for “YAML” in the pipeline options, then click a button with affirmative text.

I should mention that the graphical build definitions are still valuable, even though you should avoid using them to define your actual build pipelines. You can fiddle with a graphical build, and at any time click on the "View YAML" link at the pipeline or individual task level.

I found this toggle view useful for migrating to YAML pipelines, because I could look at a working build and see what YAML I needed to replicate the process. In other words, migrating an existing pipeline to YAML is easy.

Once you get the feeling for how YAML pipelines work, the docs, particularly the YAML snippets in the tasks docs, give you everything you need. Also, there is an extension for VS Code that provides syntax highlighting and intellisense for Pipelines YAML.

The YAML you’ll create will describe all the repositories, containers, triggers, jobs, and steps needed for a build. You can check the file into your source code repository, then version and diff your builds!

Building .NET Core Projects

The essential building blocks for a pipeline are tasks. These are the same tasks you’d arrange in a list when defining a build using the GUI tools. In YAML, each task consists of the task name and version, then the task parameters. For example, to build all .NET Core projects across all folders in Release mode, run the DotNetCoreCLI task (currently version 2), which will run dotnet with a default command parameter of build.

- task: DotNetCoreCLI@2
  displayName: 'Build All .NET Core Projects'
  inputs:
    projects: '**/*.csproj'
    arguments: '-c Release'

Publishing .NET Core Projects

Ultimately, you want to run dotnet publish on ASP.NET Core projects. In YML, the task looks like:

- task: DotNetCoreCLI@2
  displayName: 'Publish WebApp'
  inputs:
    command: publish
    arguments: '-c Release'
    zipAfterPublish: false

Notice the zipAfterPublish setting is false. In builds where a repo contains various projects intended for multiple destinations, I prefer to move files around in staging areas and then create zip files in explicit steps. We’ll see those steps later*.

Building Go Projects

I’m throwing in the Go steps because I have a Go project in the mix, but I also want to demonstrate how Azure Pipelines and Azure DevOps is platform agnostic. The platform wants to provide DevOps and continuous delivery for every platform, and every language. Building a Go project was easy with the built in Go task.

- task: Go@0
  displayName: 'Install Go Deps'
  inputs:
    arguments: '-d'
    command: get
    workingDirectory: '$(System.DefaultWorkingDirectory)\cmd\goapp

- task: Go@0
  displayName: 'go build'
  inputs:
    command: build
    arguments: '-o cmd\goapp\app.exe cmd\goapp\main.go'

The first step is go get, which is like using dotnet restore in .NET Core. The second step is building a native executable from the entry point of the Go app in a file named main.go.

Building Azure Functions

If you want to use Azure Functions and the C# language, then I believe Functions 2.0 is the only way to go. The 1.0 runtime works, but 1.0 is not as mature when it comes to building, testing, and deploying code. Building a 2.0 project (dotnet build) places everything you need to deploy the functions into the output folder. There is no dotnet publish step needed.

Preparing for Publishing the Artifacts

Once all the projects are built, the assemblies and executables associated with each project are on the file system. This is the point where I like to start moving files around to simplify the steps where the pipeline creates release artifacts. Release artifacts are the deployable bits, and it makes sense to create multiple artifacts if a system needs to deploy to multiple resources. Based on the requirements I listed at the beginning of the post, we are going to need the build pipeline to produce two artifacts, like so:

The first step is getting the files into the proper structure for artifact 1, which is the web app and the Go application combined. The Go application will execute on a schedule as an Azure Web Job. It is interesting how many people have asked me over the years how to deploy a web job with a web application. The key to the answer is to understand that Azure uses simple conventions to identity and execute web jobs that live inside an App Service. You don’t need to use the Azure portal to setup a web job, or find an obscure command on the CLI. You only need to copy the Web Job executable into the right folder underneath the web application.

- task: CopyFiles@2
  displayName: 'Copy Go App to WebJob Location'
  inputs:
    SourceFolder: cmd\goapp
    TargetFolder: WebApp\bin\Release\netcoreapp2.1\publish\App_Data\jobs\triggered\app

Placing the Go .exe file underneath App_Data\jobs\triggered\app, where app is whatever name you want for the job, is enough for Azure to find the web job. Inside this folder, a settings.job file can provide a cron expression to tell Azure when to run the job. In this case, 8 am every Monday:

{"schedule": "0 0 8 * * MON"}

Publishing the Artifacts

The final steps consist of zipping up files and folders containing the project outputs, and publishing the two zip files as artifacts. Remember one artifact contains the published web app output and the web job, while the second artifact consist of the build output from the Azure Functions project. The ArchiveFiles and PublishBuildArtifacts tasks in Azure do all the work.

- task: ArchiveFiles@2
  displayName: 'Archive WebApp
  inputs:
    rootFolderOrFile: WebApp\bin\Release\netcoreapp2.1\publish
    includeRootFolder: false
    archiveFile: WebApp\bin\Release\netcoreapp2.1\WebApp.zip

- task: ArchiveFiles@2
  displayName: 'Archive Function App
  inputs:
    rootFolderOrFile: FunctionApp\bin\Release\netcoreapp2.1
    includeRootFolder: false
    archiveFile: FunctionApp\bin\Release\netcoreapp2.1\FunctionApp.zip

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: WebApp
  inputs:
    PathtoPublish: WebApp\bin\Release\netcoreapp2.1\WebApp.zip
    ArtifactName: WebApp

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: FunctionApp'
  inputs:
    PathtoPublish: FunctionApp\bin\Release\netcoreapp2.1\FunctionApp.zip
    ArtifactName: FunctionApp

The Release Pipeline

Currently, YAML is not available for building a release pipeline, but the roadmap says the feature is coming soon. However, since we arranged the artifacts to simplify the release pipeline, all you should need to do is to feed the artifacts into Deploy Azure App Service tasks. Remember function projects, even on a consumption plan, deploy just like a web application, but like web jobs, use some conventions around naming and directory structure to indicate the bits are for a function app. The build output of the function project will already have the right files and directories in place.

Summary

Having build pipelines defined in a textual format makes the pipeline easier to modify and version changes over time.

Unfortunately, this YAML approach only works in Azure. There is no support for running, testing, or troubleshooting a YAML build locally or in development. For systems with any amount of complexity, you will be in better shape if you automate the build using command line scripts, or a build system like Cake. Then you can run your builds both locally and in the cloud. Remember, your developer builds need to be every bit as consistent and well defined as your production builds if you want a productive, happy team.

* Note that I’ve simplified the YAML in the code samples by removing actual project names and “shortening” the directory structure for the project.