OdeToCode IC Logo

Thoughts on Azure DevOps

Monday, October 1, 2018

Azure DevOpsI’ve been using Azure DevOps since the early days when the service carried the name Visual Studio Online. I’ve used the service for both professional projects and personal projects, and I’ve enjoyed the service so much I’ve demonstrated and recommended the service on consulting gigs, at workshops, at user groups, and in Pluralsight courses.

ADO has sometimes been a hard sell. The previous monikers for the service tied the product to Windows, Visual Studio, and Microsoft, so getting a Node developer to sit down and see a continuous delivery pipeline for a pure JS project wasn’t always easy. People already in the Microsoft ecosystem would also resist given the baggage of its on-premises ancestor, Team Foundation Services. And by baggage, I mean heavy enterprise baggage overstuffed with XML. I’ve gotten a lot of work done with TFS over the years, but TFS is also the only Microsoft product I’ve upgraded by hiring an external expert. I did not want to learn the installation incantations for the unholy amalgamation of TFS, SQL Server, SSRS, and SharePoint. TFS is also the only source code provider I’ve seen crash a commercial-grade network router while fetching source code.

But, the past is gone, and no other service exemplifies the evolution of Microsoft quite as well as evolution of TFS to Azure DevOps. We’ve gone from a centralized behemoth with an enterprise focus to a modern looking and sleek cloud platform that aggressively supports small open source projects as well as larger organizations.

Here are the constituent services that form Azure DevOps.

Azure Pipelines

Pipelines provide a platform for building, testing, packaging, and deploying applications. It’s a feature rich build system that is extensible and easy to use. I’d consider this the crown jewel of Azure DevOps. All the heavy lifting uses build machines that the service transparently provisions in the cloud. Here are three more fun facts:

  • Pipelines are not tied to source control in Azure. You can pull source from public and private repositories, including GitHub.

  • Build minutes for OSS projects are free and unlimited.

  • You can build anything for anyone since the supported build environments include Linux, Windows and macOS.

My biggest complaint about Pipelines in the past has been the inability to define builds using source controlled text files instead of the web UI. Fortunately, YML files have come to the rescue and the ability to codify and version build and release definitions should soon be generally available.

Azure Pipelines at Work

Azure Boards

Boards are where a team can track issues, bugs, work items, and epics. There are Kanban boards, of course, and custom workflows. The service is well featured, particularly since it is free for 5 users and about $9,000 USD a year for 100 users (note that developers with MSDN subscriptions will have free access). There are other products that have many more bells and whistles, but they’ll also start license negotiations at $20,000 for 100 users.

Azure Repos

Git source control with an unlimited number of private or public repositories.

Azure Test Plans

Automated tests will typically execute in a Pipeline. The Test Plans service is more of a place for tests not executing in a pipeline, so this service covers manual tests, and exploratory tests, as well as load tests (which are automated, but fall here for some reason).

The load testing features are the only features I’m qualified to speak about since I’ve been using the testing tools in VS Enterprise for years. Unfortunately, the tools themselves remain pretty much unchanged over these years and feel dated. The test recorder requires Internet Explorer and a plugin. The “Browser Mix” feature will allow you to make HTTP requests using an IE9 UA string, but there is no mention of any browser released after IE9, and even having a browser mix feature in 2018 is questionable.

Behind the scenes, the load testing artifacts are relatively simple XML files, so it is possible to avoid the tools in some workflows.

On the plus side, the load testing framework can intelligently provision hardware in the cloud to generate load. There is no need to setup test agents and monitor the agents to see if they are overworked. See my Cloud Patterns course for more.

Azure Load Test Results

Azure Artifacts

Your own ultimate package repository for Maven, npm, and NuGet packages. Publish packages here for internal use at your organization.


The app store for DevOps contains some high quality extensions. There is also an extensive HTTP API throughout the DevOps service to integrate with other products and your own custom utilities. Between the API and the custom extensions, there is always a way to make something work in DevOps, all you need is the will.

How This Relates to GitHub

My opinion: GitHub is community focused, Azure DevOps is focused on organizations. But, there is some crossover. If you have an OSS project, you’ll want to host on GitHub and build in Pipelines.


Look for yourself at the aggressive and transparent evolution of Azure DevOps over the years. My only worry today is Azure DevOps using the word "DevOps" in the name. DevOps requires a way of thinking and a culture. I hope organizations don't adopt DevOps tools in the same way they adopted Agile tools and then proclaimed themselves Agile.

Gravatar Alex Tuesday, October 2, 2018
My company used VSTS for TFS hosting. Now we are practicing DevOps!
Gravatar Sijmen Thursday, October 4, 2018
“You can build anything for anyone since the supported build environments include Linux, Windows and macOS.” People keep saying this but any specific Linux distribution is not any Linux distribution and certainly not anything like a BSD. For many purposes it doesn’t really matter but you can’t build, say, your Arch or FreeBSD packages on Azure Pipelines. Otherwise, fine post and I too hearlty recommend trying out Pipelines!
Gravatar Edward Thomson Friday, October 5, 2018
You're right, Sijmen. Azure Pipelines only provides hosted build agents for Linux, Windows and macOS. And our Linux build agents are running Ubuntu 16.04 - though if you have custom needs around your distribution (I do) then we'd encourage you to run your build within a container. My project has containers for each of our build environments running the distribution that we need to build in (eg, Ubuntu 18.04 or CentOS), and instead of building directly on the build agent, we build within those docker containers. Obviously this is nice for Linux on amd64 - for still more customized workloads, like Arch or FreeBSD, then we support a hybrid cloud/on-premises build agent workload. You can install the build agent on your own machines and run your builds there. We think this provides the maximum flexibility while allowing the most common workloads to be handled by our cloud agents.
Your Comment