My Python Project Development Guide ver.202305
Tags: python
Published: 2023-05-14 11:17:52

---
title: My Python Project Development Guide ver.202305
published: true
description: My Python Project Development Guide ver.202305
tags: Python
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-04-19 05:46 +0000
---

# What is this

- The guide to set up a Python project for production deployment. 

- This guide assumes you are using Poetry for package management and adheres to the rules specified.

- If you are not familiar with how to use Poetry, please read the following documentation and install it locally:  

- By following this guide, you can set up a Python project for production deployment using Poetry as the package manager, enforce code quality with linters and formatters, automate checks with GitHub Actions, configure your IDE to maintain consistency, and write comprehensive unit tests to ensure the project's reliability throughout the development process.

- We recommend using Python version 3.9 or higher for your project.

- The majority of this document was created using ChatGPT-4.

- Sample Project
  - https://github.com/fukubaka0825/my-python-guide

 

# Steps

## 1. Project Setup/ Set up Local Environment with Poetry and Pyenv

- Create a project directory and navigate to it:

```sh
mkdir ${project_path}
cd ${project_path}
```

- Use pyenv to manage local Python versions and ensure a consistent environment for all team members. To do this, include a .python-version file in the project.

- Set the desired Python version for the project using pyenv:

```sh
pyenv install ${python_version}
pyenv local ${python_version}
```

- Replace ${python_version} with the desired version, e.g., 3.9.7. This command will create a .python-version file in the project directory.

- Ensure that all team members use the same Python version specified in the .python-version file.

- Copy the provided pyproject.tool file to the project directory and install dependencies:

- Copy pyproject.yml example and arrange by yourself

  - For the local environment, use Poetry to create a .venv virtual environment in the project root. Place the following poetry.toml file in the project directory: 

```sh
[virtualenvs]
path = ".venv"
in-project = true
```

- Execute the following commands to set up the virtual environment and install the required packages 

```sh
poetry env use ${python_version}
poetry install

// Run command in the created environment.
poetry run <YOUR_COMMAND>
// Enter the created environment.
poetry shell
```

### In the pyproject.toml file, we configure various modules for linting, formatting, and task running. Here's an explanation of these modules:

- **Taskipy**

Taskipy is a task runner that allows you to automate various tasks in your Python project. In this project, we use Taskipy to define and run tasks such as formatting, linting, and testing. The tasks are configured in the pyproject.toml file under [tool.taskipy.tasks].

For more information about Taskipy, check out the official documentation:  

- **Linters and Formatters**

This project uses several linters and formatters to maintain code quality and consistency:

**Black**: A code formatter that automatically formats your code according to the PEP 8 style guide. The configuration is set in the pyproject.toml file under [tool.black].

**Ruff**: A linter that checks your code for style guide violations, potential bugs, and other issues.   This covers so many rules inluding flake8’s one and isort’s one. The configuration is set in the pyproject.toml file under [tool.ruff]. 

**Mypy**: A static type checker that helps catch potential bugs and ensures that your code is type-safe. The configuration is set in the pyproject.toml file under [tool.mypy].

These linters and formatters are integrated into the project's tasks and the IDE settings, ensuring a consistent code style across the entire project.


## 2. Create Directory Structure

Create the necessary directories and files:

```sh
mkdir assets　# you can write main codes in this directory
mkdir tests # you can write test codes in this directory
touch README.md
```

## 3. Write Sufficient Unit Tests with pytest

Ensure that you write comprehensive unit tests using pytest for all critical components of your project. Adequate test coverage helps maintain code quality and reduces the likelihood of introducing bugs during development or refactoring.

To write a unit test, create a unit test, create a new file in the tests directory with a name that follows the pattern test_*.py.

Within the test file, import the necessary modules and define test functions that follow the pattern def test_*():.

Use pytest's built-in assert statement to compare expected and actual values.

To run tests, execute the following command:

```sh
poetry run task test
```

Ensure that all tests pass before committing changes and creating a pull request.

## 4. Configure IDE Settings

Reflect the contents of the provided pyproject.toml file in the settings file related to your IDE.

For example, if you are using VSCode, add the following settings to .vscode/settings.json:

```sh
{
  "python.formatting.provider": "black",
  "python.formatting.blackArgs": ["--line-length", "102"],
  "python.linting.flake8Enabled": true,
  "python.linting.mypyEnabled": true,
  "python.sortImports.args": ["--profile=black", "--line-length=102"],
  "editor.codeActionsOnSave": {
    "source.organizeImports": true,
    "source.fixAll": true
  }
}
```

- With these settings, VSCode will automatically format and lint code using Black, Flake8, Mypy, and isort.

- If you use IntelliJ/Pycharm, Please use File Watcher function for auto-running poetry run task fmt_lint

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bzp1bikhj9er6jf4i17p.png)


## 5. Set up CI(Auto lint/test) with GitHub Actions 

Create the .github/workflows directory in the project root:

mkdir -p .github/workflows

Create a test_and_lint.yml file and add the following content:

```yaml
name: python lint and test

on:
  pull_request:
    branches:
      - main
    paths:
      - "YOUR_PROJECT/**.py"
      - "tests/**.py"
      - "poetry.toml"
      - "poetry.lock"
      - ".github/workflows/test_and_lint.yml"

jobs:
  lint_and_test:
    name: Lint and Test
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11","3.10","3.9"]

    steps:
      - name: Checkout
        uses: actions/checkout@v2
        
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
          
      - name: Install Poetry
        run: |
          curl -sSL https://install.python-poetry.org | POETRY_HOME=$HOME/.poetry python3 - 
          echo "$HOME/.poetry/bin" >> $GITHUB_PATH
      
      # cacheがうまく効かない環境があるのでその際は以下の設定はコメントアウトする
      - uses: actions/cache@v2
        id: venv_cache
        with:
          path: .venv
          key: venv-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }}

      - name: Install Dependencies
        if: steps.venv_cache.outputs.cache-hit != 'true'
        run: poetry install

      - name: Python Test
        run: poetry run task test
          
      - name: Python Lint
        run: poetry run task lint

```

- sample
  - https://github.com/fukubaka0825/my-python-guide/blob/main/.github/workflows/docker-build-push-xxxxxxxxxx.yml

- This configuration will automatically run poetry run task fmt_lint and poetry run task test on pushes and pull requests to the main branch.

## 6. Set up Stage/Prod Environment with Dockerfile

For the stage/prod environment, use a Dockerfile to build the execution environment. First, export the project's dependencies to a requirements.txt file using the following command:

```sh
poetry export --without-hashes -f requirements.txt -o requirements.txt
```

In the Dockerfile, use pip install to install the dependencies from the requirements.txt file. Ensure that the specified Python version in the Dockerfile is the same as the one used in the local environment.

- sample Dockerfile
  - https://github.com/fukubaka0825/my-python-guide/blob/main/Dockerfile

By following these rules, you can ensure a consistent development environment for all team members and maintain a clean separation between local and stage/prod environments.

Additionally, it is recommended to integrate the configuration of this process into the GitHub Actions YAML for the next 7 Continuous Deployment (CD) steps, as it eliminates the need to work locally.

## 7. Set up CD(Atuo build Dockerfile) with Github Actions

- sample
  - https://github.com/fukubaka0825/my-python-guide/blob/main/.github/workflows/test-and-lint-xxxxxxx.yml

# Appendix

- [【2020年新人研修資料】ナウでヤングなPython開発入門](https://speakerdeck.com/stakaya/2020nian-xin-ren-yan-xiu-zi-liao-nauteyankunapythonkai-fa-ru-men)

- [Poetryをサクッと使い始めてみる - Qiita](https://qiita.com/ksato9700/items/b893cf1db83605898d8a)

- [サーバーアプリ開発環境(Python／FastAPI) | フューチャー技術ブログ](https://future-architect.github.io/articles/20210611a/)

- [Pythonのリンター・フォーマッターをしっかりと理解する（Flake8, Black, isort, mypy）](https://zenn.dev/tanny/articles/cdb555d6124a2a)

- [【PyCharm】ファイル保存時にlinterとformatterが自動実行されるように設定する](https://zenn.dev/horitaka/articles/4fc4a5c19bee22)

- [オレオレ Python 開発環境](https://zenn.dev/jdbtisk/articles/e6ed54b38b6a45)