Create a PyPI (pip) package, test it and publish it using Github Actions (PART 2) 
Tags: python, devops, github, pip
Published: 2021-01-15 19:23:19

Welcome back! Let's use Github Actions to deploy your shining new package to PyPI!

## Basic setup

We don't have a git repository yet, so create one:
```
git init
```

Next, let's create a [Github Repository](https://github.com/new)

Once you're done with that, let's link our git repository to our Github repository:
```bash
git remote add origin "https://github.com/USERNAME/REPOSITORY.git"
```

Next, we'll commit and push our changes:
```bash
git add .
git commit -m "First Commit!"
git push origin master
```

You should see your code live at your Github Repository.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tt989ceqi1r0xl3z0yc5.png)

## Setting up your first action

A Github Action is a YAML file. It can be edited in any code editor, but I recommend Github's own Editor, because it provides intellisense when you press `Ctrl`+`Space`.

To create an action, head over to the `Actions` tab. I'll create an action to automatically test and publish our package to [TestPyPI](https://test.pypi.org) whenever we've pushed our code to the master branch.

We can choose an action template, or we can start from scratch. I'm going to choose the latter, so click the `set up a workflow yourself` link.

This will create a new file that you can name whatever. I'll call it `test-and-upload-to-testpypi.yml`. Make sure that it has the extension `yaml` or `yml`, otherwise Github won't run it.

This should be the starter code you get:
```yml
# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2

      # Runs a single command using the runners shell
      - name: Run a one-line script
        run: echo Hello, world!

      # Runs a set of commands using the runners shell
      - name: Run a multi-line script
        run: |
          echo Add other actions to build,
          echo test, and deploy your project.
```

### Anatomy of a Github Action

Let's go through this file step-by-step.

`name`: It is the name of your Action. It will appear in the Actions tab under that name

`on`: Controls what events trigger your action. Currently, the action runs when there's a pull request or a push to the `master` branch, but there are other events too, like creation of releases, issues, adding collaborators, etc. The `workflow-dispatch` event will add a button in the Actions tab that'll allow you to run this action.

`jobs`: Defines the jobs in the action. Here, we have a single job called `build`.

#### Inside the `build` job:

`runs-on`: The OS to run on. Supports `ubuntu-latest`, `windows-latest` and `macos-latest`.

`steps`: The steps in your job.

##### Inside `steps`

`uses`: Another Github Action to use (this is optional)

`name`: Name of the step. Will appear in the Actions tab.

`run`: The command to run.

> The pipeline (`|`) that you see after the run command in some steps is the YAML multi-line string operator. It allows a string to span mutliple lines, each newline indicating a new command.

#### Special actions

You can see that our action uses version `v2` of `actions/checkout`. This is an action made by someone else and we're using it. It checks-out our code and copies it to the machine, so we can use it. There are several other actions like these available in the [Github MarketPlace](https://github.com/marketplace/actions)

### Creating our action

We will have 5 main steps in our action:

- Checkout our code
- Install python
- Install our dependencies
- Test using unittest
- Publish to testpypi

For the first two steps, we have actions that will do it for us, namely `action/checkout@v2` and `actions/setup-python@v2`.

I want the action to run when we've pushed our code to the master branch, and I also want to have a button in the Actions tab to manually test & upload the project for me.

Here's the final code for our action:

```yml
# This is a basic workflow to help you get started with Actions

name: Test & Upload to TestPyPI

# Controls when the action will run. 
on:
  # Triggers the workflow on push to the master branch
  push:
    branches: [ master ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      
      # Sets up python3
      - uses: actions/setup-python@v2
        with:
          python-version: 3.8 
      
      # Installs and upgrades pip, installs other dependencies and installs the package from setup.py
      - name: "Installs and upgrades pip, installs other dependencies and installs the package from setup.py"
        run: |
          # Upgrade pip
          python3 -m pip install --upgrade pip
          # Install build deps
          python3 -m pip install setuptools wheel twine
          # If requirements.txt exists, install from it
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
          # Install the package from setup.py
          python3 setup.py install
      
      # Tests with unittest
      - name: Test with unittest
        run: |
          cd tests
          python3 -m unittest discover
          cd ..
      
      # Upload to TestPyPI
      - name: Build and Upload to TestPyPI
        run: |
          python3 setup.py sdist bdist_wheel
          python3 -m twine upload dist/*
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.TWINE_TEST_TOKEN }}
          TWINE_REPOSITORY: testpypi
```

### What are secrets?

Secrets are Github's way of passing environment variables to our action. In this case, we used the secret `TWINE_TEST_TOKEN` to hold our API key.

#### Creating a secret

Go to the settings tab and select secrets. Click Create Secret on the top right and put `TWINE_TEST_TOKEN` for the name and paste your API key for the value.

Now, save the file and watch your action work from the Actions tab!

In our actions tab, we can see that this action is running:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ndjq063696hgxe6n9vr7.png)

[After fixing a small bug with our tests](https://github.com/arnu515/pydash/commit/ae90b0d0ae37d6767883ba1771a706689e5400e8), you can see that our action has failed, indicated by the red X icon

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qtfjvaeknkt812h925bz.png)

This is because whenever we upload a package to PyPI (or TestPyPI), we cannot upload the package of the same version twice, [so let's up the version in `setup.py`](https://github.com/arnu515/pydash/commit/7e94826f2df6751ad687a3647cce8520f6d3fd67)

And our action has successfully executed!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/vjpti1tu4tyfctbejq96.png)

## Uploading to PyPI

We did TestPyPI. What about the real python package index? I'm not going to upload the package everytime we push master, since there could be some changes not ready for production. Instead, I'm going to use the releases feature of Github.

Create a new action by clicking New workflow in the Actions tab and following the same steps as before. This time, I'm naming my action `upload-to-pip.yml`

Here's my action:

```yml
# This is a basic workflow to help you get started with Actions

name: Upload to PIP

# Controls when the action will run. 
on:
  # Triggers the workflow when a release is created
  release: 
    types: [created]
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "upload"
  upload:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      
      # Sets up python
      - uses: actions/setup-python@v2
        with:
          python-version: 3.8
      
      # Install dependencies
      - name: "Installs dependencies"
        run: |
          python3 -m pip install --upgrade pip
          python3 -m pip install setuptools wheel twine
      
      # Build and upload to PyPI
      - name: "Builds and uploads to PyPI"
        run: |
          python3 setup.py sdist bdist_wheel
          python3 -m twine upload dist/*
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.TWINE_TOKEN }}
```

### Adding another secret

Let's add another secret, this time called `TWINE_TOKEN`, and give it the value of your PyPI API token.

### Testing our Action

Let's create a new release by going to the releases section (to do that, click New Release on the right side of the main code tab).

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/piv2sxyl5j5eej5gvkge.png)

> I suggest naming the tag with the version in `setup.py` prefixed with `v`.

And our project has successfully been uploaded to PyPI!

Congratulations 🥳! You just learnt the basics of Github Actions, PyPI projects and DevOps! Give yourself a pat on your back!

> Stuck?
> [Github repo](https://github.com/arnu515/pydash)
> [PyPI](https://pypi.org/pydash-arnu515)
> [Github Actions Quickstart](https://docs.github.com/en/free-pro-team@latest/actions/quickstart)
> [Uploading a package on PyPI](https://packaging.python.org/tutorials/packaging-projects/)

If you enjoyed this post, and would like to see more, let me know in the comments, and also, follow me for more content like this 😀