Automate Jenkins Job builds with Python
Tags: jenkins, python, automation, ansible
Published: 2023-06-16 09:36:12

We often come across a situation when we have to manually run a Jenkins job many times each possibly having different parameters. One way to do it is using the Jenkins UI and pressing the `build with parameters` button each time and passing on the parameter values each time. This can be quite tiresome and repetitive. Fortunately, I am too Lazy to do it like that xD. So, I found another way, we can trigger the Jenkins jobs using the Jenkins REST API. In this post, we will demonstrate how to design our Jenkins jobs effectively for this purpose and use `python` to make use of the Jenkins API to automate triggering a huge number of jobs and then take a chill pill :)

## Step 1: Enable Triggering Jenkins jobs from remote scripts
Navigate to your Jenkins Job configuration page and scroll down to the `Build Triggers` option. Enable the `Trigger builds remotely (e.g., from scripts)` flag and put some secret string in the `Authentication Token` field.

![Trigger builds remotely in Jenkins](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ikv3ijrrqpfaxopoz8fk.png)

## Step 2: Parameterize Job -> GIT SCM (BRANCH, REPO ETC.)
We need to make our Job bash script read from parameters we passed in to the Job so that we can easily control these parameters from our python script. Furthermore, if there's a git repo/branch our Job needs to checkout we can easily control that from python as well.
Enable the `This project is parameterized` flag from the Jenkins job configuration page and add the required parameters like below,
![Jenkins Job Parameterize project](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v23sq235trbyiylkv4wf.png)



Jenkins supports various different types of parameters including Strings, Boolean, Choices, File etc. Make sure you are choosing them appropriately for your choice. In my case for example, I had to deploy a bunch of `Ansible` playbooks to a `Kubernetes` cluster so, I needed the following parameters:
```
PLAYBOOK - name of the yaml ansible playbook file
GIT_REPO_ID - git repo to checkout
GIT_BRANCH_NAME - branch name of the repo (particularly useful when you are testing/reviewing something on a feature branch)
HELM_REPOSITORY - helm repository name that my ansible playbooks needs
```
Now, access these parameters wherever you need inside the Jenkins config using the `${PARAM_NAME}` syntax.

For example my job `bash` script looks like the one just below where, I am utilizing the `PLAYBOOK` and `HELM_REPOSITORY` parameters.
```bash
#!/bin/bash
set -xe

pip3 install --quiet -r requirements.txt

cd ansible

ANSIBLE_FORCE_COLOR=true ansible-playbook -e "helm_repository=${HELM_REPOSITORY}" \
playbooks/${PLAYBOOK} -vvvv
```

Here's another example where I am utilizing the GIT SCM parameters,

![Jenkins GIT SCM plugin parameters](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8jg8u8hzh7qdiv806ptj.png)

## Step 3: Generate Jenkins Crumb
Jenkins API will need us to pass in a `Jenkins-Crumb` header to our requests. Therefore, we need to generate this crumb value from the Jenkins CrumbIssuer before making the actual build trigger requests. Here's how we can generate the Jenkins Crumb using python:
```python
def jenkins_crumb():
    session = requests.Session()
    session.auth = ("<JENKINS_USERNAME>", "<JENKINS_USER_PASSWORD>")
    crumb = session.get("https://<YOUR_JENKINS_BASE_URL>/crumbIssuer/api/json")
    return crumb.request.headers["Authorization"], crumb.json()["crumb"]
```

## Step 4: Trigger Jenkins Job from Python
Now, we can finally trigger our job from python scripts. Here's an example,
```python
TOKEN = "<YOUR_AUTH_TOKEN>" # the token value you set in STEP-1

def deploy_service(
    service_name,
    playbook,
    branch="master",
    repo="ashiqursuperfly/jenkins-job-config",
    chart_repo="ashiqursuperfly-portfolio-dev"
):
    JOB_URL = f"https://<YOUR_JENKINS_BASE_URL>/job/<YOUR_JENKINS_JOBNAME>/buildWithParameters?token={TOKEN}"
    url = f"{JOB_URL}&PLAYBOOK={playbook}&GIT_BRANCH_NAME={branch}&GIT_REPO_ID={repo}&HELM_REPOSITORY={chart_repo}"

    cookie, crumb = jenkins_crumb()

    headers = {"Jenkins-Crumb": crumb}
    session = requests.Session()
    session.auth = ("<JENKINS_USERNAME>", "<JENKINS_USER_PASSWORD>")

    res = session.get(url=url, headers=headers)

    if res.ok:
        print(f"Started deploy job for service {service_name} with playbook {playbook}")
    else:
        print(
            f"Error deploy job for service {service_name} with playbook {playbook}\n{res.json()}"
        )

if __name__ == "__main__":
    list_of_service = list_services()

    for service in list_of_service:
        res, playbook = get_playbook_name(service)
        deploy_service(service, playbook)
        sleep(100)
```
Now, we can invoke the following method as many times as we want. In my case, I just looped through my list of services to deploy and invoked this method for each service and just watched Jenkins complete all the tedious work for me as I sipped my coffee, basked in the brightness of my monitor's light, and wondered why life is so simple when you can't stop thinking automation 🤓

If you found this post helpful, CLICK BELOW 👇 <a href="https://www.buymeacoffee.com/ashiqurrahman" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Beer" style="height: 80px !important;width: 240px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>