Parallel jobs in Azure DevOps
By: Ivan Porta
Tags: Microsoft, Azure Devops, DevOps, Continuous Integration, Azure

This article will talk about parallel jobs in Azure Pipelines and the guidelines to determine how many jobs you need in your organization.

What is a Job?

Each pipeline in Azure DevOps starts with a trigger and is composed of one or more stages. Each stage contains one or more jobs that run multiple tasks on an agent.

With this relationship in mind, we can define a job as a set of steps that run sequentially on an agent (computing infrastructure with an installed agent software that runs a job).

Are parallel Jobs free?

By default, Azure DevOps gives one parallel job to every organization for free. However, there’s a slight difference in the free tier offering according to the types of agents being used.

Microsoft-hosted agents

Private projects: The build time is limited to 1800 minutes per month and it can only run one pipeline job at a time.

The build time is limited to 1800 minutes per month and it can only run one pipeline job at a time. Public projects: There is no build time limit and it can run up to 10 parallel jobs at a time.

Self-Hosted agents

Private projects: One self-hosted parallel job at a time. You can also get an additional self-hosted parallel job for each user with an active Visual Studio Enterprise subscriber account in your organization

One self-hosted parallel job at a time. You can also get an additional self-hosted parallel job for each user with an active Visual Studio Enterprise subscriber account in your organization Public projects: an unlimited number of parallel jobs at a time

If you want to know more about self-hosted agents, I recommend that you read my previous article in which I explain how I created a self-hosted agent on my Raspberry Pi.

Buy parallel jobs

To buy more parallel jobs you must follow these steps:

Sign in to your Azure DevOps organization

Select Organization settings

Organization settings

Select Parallel jobs under Pipelines, and then select either Purchase parallel jobs for Microsoft-hosted jobs or Change for self-hosted jobs.

Parallel jobs

Enter the number of jobs that you want to purchase

Click Save

Create a pipeline with parallel jobs

In this demo, I’m going to create a pipeline with just one stage. This stage will execute two jobs in parallel, and only when both these tasks are completed, will the pipeline move forward with a final task.

trigger:

- main pool:

vmImage: 'ubuntu-latest' jobs:

- job: ParallelA

steps:

- bash: sleep 25

displayName: 'Sleep for 25 seconds'

- bash: echo Hello from parallel job A

displayName: 'Run a one-line script' - job: ParallelB

steps:

- bash: sleep 20

displayName: 'Sleep for 20seconds'

- bash: echo Hello from parallel job B

displayName: 'Run a one-line script' - job: Final

dependsOn:

- ParallelA

- ParallelB

steps:

- script: echo Hello from final job

displayName: 'Run a one-line script'

The dependsOn key defines the jobs that must be completed before starting the execution of the final job. As you can see in the image below, the first two jobs start concurrently looking for an available agent (this doesn’t mean that they will run simultaneously) while the final agent is on standby.

Parallel jobs detail

Do you need a parallel job?

As the number of users in your organization increases, so does the frequency at which the pipelines are being used. For this reason, to avoid users ending up in a queue waiting for other jobs to complete, Microsoft recommends running one parallel job for every four to five users in your organization.

Other factors that might influence the number of parallel jobs to use are:

You should have a parallel job for each team that requires a CI build.

If a CI build trigger applies to multiple branches, You should have a parallel job for each active branch.

If you have multiple applications in the same organization or server, you should have a parallel job to deploy each application simultaneously.

Resources

Specify jobs in your pipeline: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml

👋 Join FAUN today and receive similar stories each week in your inbox! ️ Get your weekly dose of the must-read tech stories, news, and tutorials.

Follow us on Twitter 🐦 and Facebook 👥 and Instagram 📷 and join our Facebook and Linkedin Groups 💬