Setting up Continuous Integration
Tags: devops, opensource, githubactions, cicd
Published: 2023-11-17 04:42:12

[Continuous Integration](https://docs.github.com/en/actions/automating-builds-and-tests/about-continuous-integration) is a **philosophy** that emphasizes frequent **integration** of code changes into a shared repository.

Imagine you're a **group of people** working on a project at the same time. You're all working at a separate issue and you keep **pushing** changes to the repository. At the end of the week, everyone meets to **build** and **test** the application only to find everything is broken and now you have to spend hours and hours on figuring out where thing **went wrong**. This is exactly why we need to setup **CI** in our project.

![Continuous Integration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/21mvz0kpi6ibskm14tlm.jpg)


It basically means we need to build and test our code everytime a **new change** is pushed to the shared branch. By doing this, we make sure that the project is always functioning as expected and  [regression](https://en.wikipedia.org/wiki/Software_regression) is prevented.
The **key** to CI is that instead of relying on **humans** to run the tests manually before pushing, we delegate this responsibily to **machines** that are **automated** to run this [workflow](https://www.ibm.com/topics/workflow), on every push or a pull request to **master** (or whatever the branch name in your case). 
There are many [cloud services](https://www.trio.dev/blog/continuous-integration-tools) that solve this purpose for us.

🌟 In this post, I'll be talking about how I setup a **CI workflow** for my python project [til-the-builder](https://github.com/Amnish04/til-page-builder) using Github Actions.

## Table of Contents
&nbsp;1. [Github Actions 🎬](#github-actions)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1.1. [Setting up CI ✔️](#setting-up-ci)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1.2. [Breakdown of the workflow 🧐](#breakdown-of-the-workflow)
&nbsp;2. [Adding a new test to check CI](#adding-a-new-test-to-check-ci)
&nbsp;3. [Adding tests to other project](#adding-tests-to-other-project)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.3. [Setting up ⚙️](#setting-up)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3.4. [Adding Unit Tests](#adding-unit-tests)
&nbsp;4. [Conclusion 🎊](#conclusion)

## Github Actions 🎬

**Github** provides us certain [events](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions#the-components-of-github-actions) that run configured **workflows** everytime they are triggered. It could be a commit pushed or a pull request made to a certain branch, or even a tag pushed to the repository.

From the official [docs](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions#the-components-of-github-actions),
![Github Actions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u9hk4qcfjz6bwwun1ka8.png)

### Setting up CI ✔️

**1.** The first step is navigating to the **Actions** tab on Github.

![Actions Tab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/njr9ywqupwwpdt82w82b.png)

**2.** If you do not have any workflows already configured, you'll see a page with recommended **workflow templates** for your project.

In my case, I get these since it is a **Python** project.
![Workflow templates](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/me9zpnmvstgookxm74br.png)

**3.** I went with the last one called `Python Application`, and this is what I got as template.

```yaml
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python application

on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.10
      uses: actions/setup-python@v3
      with:
        python-version: "3.10"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # stop the build if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest
      run: |
        pytest
```

**4.** Since I was using `pylint` as my [linter](https://www.testim.io/blog/what-is-a-linter-heres-a-definition-and-quick-start-guide/), I made some changes to the **lint** step and the **dependency installation** step and ended up with the following [yaml](https://www.cloudbees.com/blog/yaml-tutorial-everything-you-need-get-started) configuration.

```yaml
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python application

on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up Python 3.10
      uses: actions/setup-python@v3
      with:
        python-version: "3.10.11"

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Lint with pylint
      run: |
        pylint src/

    - name: Test with pytest
      run: |
        pytest
        
    - name: Format code
      run: |
        black .
```

### Breakdown of the workflow 🧐

Although the workflow is self expanatory, let's take a look at each aspect.

**1.** The first thing we do is **name** our **workflow**, that is showed on in Github actions tab.

```yaml
name: Python application
```

**2.** Next, we **define the triggers** that fire this workflow.

```yaml
on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]
```

**3.** After that, we need to define the [jobs](https://docs.github.com/en/actions/using-jobs/using-jobs-in-a-workflow) that need to be run. We only have a `build` job at this point, and it runs on `ubuntu-latest` **operating system**.

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
```

**4.** Jobs run in parallel and have a certain number of steps that all need to pass in order for the job to pass. The following **configuration** defined the steps for our `build` job.

The first step is to **fetch the code** from our repository to the file system of CI machine.
```yaml
steps:
    - uses: actions/checkout@v3
```

Next, we setup the runtime environment out application needs, which is `python-3.10.1` in this case.

```yaml
- name: Set up Python 3.10
      uses: actions/setup-python@v3
      with:
        python-version: "3.10.11"
```

Once the runtime is configured, we need to **install** all the **packages/dependencies** required by our application. For Python, they are typically defined in a `requirements.txt` file.

```yaml
- name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
```

Now that the application we have the runtime and dependecies in place, we need to run our **linter** and **tests**.

```yaml
- name: Lint with pylint
      run: |
        pylint src/

- name: Test with pytest
      run: |
        pytest
``` 

And that is it for our workflow configuration! The file is added to the `.github/workflows` directory and we can add as much workflows as we wish to it.

## Adding a new test to check CI

Once I was confident that my CI workflow was in place, I added unit tests for another method of [HeadingItem](https://github.com/Amnish04/til-page-builder/blob/master/src/builder/toc_generator/heading_item.py) class in my project.

[test_heading_item.py](https://github.com/Amnish04/til-page-builder/blob/master/tests/builder/toc_generator/test_heading_item.py)
```python
from builder.toc_generator.heading_item import HeadingItem


class TestHeadingItem:
    """
    Test suite for 'HeadingItem' class
    """

    class TestHeadingItemConstructor:
        """
        Tests for 'HeadingItem' class __init__ method
        """
        ...
        ...

    class TestHeadingItemGetHtml:
        """
        Tests for 'HeadingItem' class get_html method
        """

        def test_return_type(self):
            """
            get_html returns a string
            """
            sample_heading_value = "This is a sample heading"
            heading = HeadingItem(sample_heading_value)

            assert isinstance(heading.get_html(), str)

        def test_return_value(self):
            """
            get_html returns an html 'li' element, and a nested anchor with expected properties
            """
            sample_heading_value = "This is a sample heading"
            heading = HeadingItem(sample_heading_value)

            assert (
                heading.get_html()
                == f"""
            <li>
                <a href='#{heading.id}'>{heading.value}</a>
            </li>
            """
            )
```

And I got a successful CI run ✔️

![SuccessfulCIonMaster](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5quoodax6i3qvl8n6dqx.jpg)

![SuccessfulRun](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mj49tii67oizymchtr10.png)

## Adding tests to other project

I also added some tests to one of my friend's project who is using **Java** instead of **Python**. I don't have a lot of experience in Java and always wanted to write a test in it. This seemed like a perfect oppurtunity.

This is her repository, feel free to check out.
https://github.com/WangGithub0/ConvertTxtToHtml

### Setting up ⚙️

Unlike I expected, setting up the project with [Junit](https://junit.org/junit5/) proved to be really time-consuming for me. 

**One**, I don't understand Java environments really well. All I have done so far is create some **GUI** applications using [JavaFX](https://openjfx.io/). Wish I could share my code, but unfortunately, its part of my assignments and can't be **open-sourced**.
**Second**, the instructions I found in the [Contributing](https://github.com/WangGithub0/ConvertTxtToHtml/blob/main/CONTRIBUTING.md) docs were bare-minimum, and kinda hard to follow for a **beginner**. An experienced Java developer would get them really quickly, no doubt about that.

So I spent about **5 hours** broken in sessions to figure out where I was going wrong and how to include required [JAR](https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jarGuide.html) files to my [classpath](https://stackoverflow.com/questions/2396493/what-is-a-classpath-and-how-do-i-set-it).

Here's a glimpse of some chat with my **best friend**, the one and only...

![GPTChatChat](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z81c47rumqbxqe9nlpyn.gif)

And finally I was able to figure out how to compile and run test files.

After navigating into `src` folder,
```java
cd src/
```

and downloading the `jars` in libs folder

![Jars](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xiuoj25zbjm7im6rm33x.png)

I ran the following command to compile the test file,
```java
javac -cp ".;../libs/junit-4.10.jar;../libs/hamcrest-core-1.3.jar" .\application\ConvertTxtMdToHtml.java
```

and to run the generated [class file](https://www.geeksforgeeks.org/java-class-file/)
```java
java -cp ".;../libs/junit-4.10.jar;../libs/hamcrest-core-1.3.jar" org.junit.runner.JUnitCore junit.ConvertTxtMdToHtmlT
est
```
And I got a successfull run.

![Successful Run](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oeepsrie2dz1825vvc03.png)

Later, I also found out how to import the existing sources as an Intellij Idea project, and run the files using the **GUI** tools.

### Adding Unit Tests

Now that I could run the existing tests, it was time to add mine. I noticed that there was a test for the `-v` option.

```java
@Test
  public void testVersionOption() {
    final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
    System.setOut(new PrintStream(outContent));

    ConvertTxtMdToHtml.main(new String[] {"-v"});

    assertEquals("Version command is triggered", "convertTxtToHtml version 0.1",
        outContent.toString().trim());
  }
```

Taking inspiration from that, I decided to add tests for `-h` or `--help` option as that would be the easiest to start with.

Hence, I added a couple of tests - **one** for the short form `-h` and one for the longer version `--help`.

[**src/junit/ConvertTxtMdToHtmlTest.java**](https://github.com/WangGithub0/ConvertTxtToHtml/pull/16/files#diff-3fcbb763112b3ff67e2503f252a89cc29939ed5f407a3ff88cb8b242a2d29aef)
```java
@Test
  public void testHelpOptionShort() {
    final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
    final String expectedMessage = """
            Usage: convertTxtToHtml [options] <input>
            Options:
              --help, -h           Print this help message
              --version, -v        Print version information
              --output <dir>, -o   Specify the output directory (default: convertTxtToHtml)
              --lang, -l           Specify the language (default: en-CA)""";

    System.setOut(new PrintStream(outContent));

    ConvertTxtMdToHtml.main(new String[] {"-h"});

    assertEquals(expectedMessage,
            outContent.toString().trim().replaceAll("\r\n", "\n"));
  }
```

[**src/junit/ConvertTxtMdToHtmlTest.java**](https://github.com/WangGithub0/ConvertTxtToHtml/pull/16/files#diff-3fcbb763112b3ff67e2503f252a89cc29939ed5f407a3ff88cb8b242a2d29aef)
```java
@Test
  public void testHelpOptionVerbose() {
    final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
    final String expectedMessage = """
            Usage: convertTxtToHtml [options] <input>
            Options:
              --help, -h           Print this help message
              --version, -v        Print version information
              --output <dir>, -o   Specify the output directory (default: convertTxtToHtml)
              --lang, -l           Specify the language (default: en-CA)""";

    System.setOut(new PrintStream(outContent));

    ConvertTxtMdToHtml.main(new String[] {"--help"});

    assertEquals(expectedMessage,
            outContent.toString().trim().replaceAll("\r\n", "\n"));
  }
```

Let's try to run them now.

![Tests Pass](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ta26rdcwqty4m6trj4u8.png)



And with that, I was able to add my very first unit tests in a **Java** project.

Here's the [pull request](https://github.com/WangGithub0/ConvertTxtToHtml/pull/16)!

![Pull Request](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i2natvmnxbio0cqvsz0w.png)

And the successful CI workflow run.

![Sucessful CI run](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b2h72gtmttp66p4qstg7.png)


## Conclusion 🎊

In this post, we discussed about what is **Continuous Integration**, why it is absolutely **necessary** in projects with **multiple contributors** to be successsful.
We also discussed about how to setup a **CI workflow** for a Python project, and in the end, I shared my experince about adding unit tests to another project that had recently been setup with a  CI workflow (build and test).

Hope it was a fun read for you.
Any **feedback** is welcome in comments!

### Attributions
Image by <a href="https://www.freepik.com/free-vector/gradient-devops-illustration_25225452.htm#query=Devops&position=1&from_view=search&track=sph&uuid=573fb9f9-e939-4276-9104-33045a829287">Freepik</a>
<a href="https://www.freepik.com/free-vector/welder-worker-welding-pipeline-constructions_34265107.htm#query=pipeline&position=0&from_view=keyword&track=sph&uuid=646b87e8-13be-4d2a-af0c-ed7ae5860640">Image by upklyak</a> on Freepik