GitLab.comでサクっと無料のPyPIミラーを使う

GitLab.comの「パッケージレジストリ」を使って無料でPyPIミラーのレジストリをサクっと使えたのでメモ。

トークンを発行して権限制御しつつ、.lockを使ったライブラリバージョンの固定もいい感じにできた。

[Package Registry | GitLab](https://docs.gitlab.com/ee/user/packages/package_registry/)

# 動作環境

- GitLab.com: 13.11-pre (※2021/4/7時点)
- poetry: 1.1.5
- twine: 3.4.1
- pipenv: 2020.11.15


# ライブラリの準備

サンプル用に簡単なライブラリを用意してみた。

https://gitlab.com/skokado/awesum

ただ足し算するだけ

```python
>>> from awesum import awesum
>>> awesum(1, 2)
3
>>> awesum(1, 2, 3)
6
>>> awesum(1, -1, 2, -2)
0
```

# パッケージレジストリ準備

## デプロイトークン準備

事前にトークンを発行しておく必要がある。

- メニューバー[設定] > [リポジトリ]
- [Deploy tokens]でパッケージレジストリへの権限をつける
    - `read_package_registry`
    - `write_package_registry`

※[名前]に設定した値も後ほど使用するので控えておく

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/fb51cc5c-a209-d5dd-fac9-8f84530acc6a.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/51bf974f-9ac6-3b5f-0571-238c48819050.png)

## 認証用の設定ファイル作成

`~/.pypirc`にトークンの情報を含める。

参考: [PyPI packages in the Package Registry | GitLab](https://docs.gitlab.com/ee/user/packages/pypi_repository/index.html#authenticate-with-a-deploy-token)

```ini:~/.pypirc
# 参考URLママ
[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
```

上記のトークンと組み合わせると下記のようになる

```ini:~/.pypirc
[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.com/api/v4/projects/25700635/packages/pypi
username = skokado
password = <deploy token>
```

※`<project_id>`はリポジトリのトップ画面で確認できる

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/7d651d86-0158-1b43-f6b5-0cb5d671f83e.png)

# パッケージング、アップロード

poetryとtwineを使う

```shell
$ python3 -m pip install poetry twine

$ # パッケージング
$ git clone git@gitlab.com:skokado/awesum.git
$ poetry build --format wheel
Building awesum (0.1.0)
  - Building wheel
  - Built awesum-0.1.0-py3-none-any.whl

$ ls -l dist/
-rw-r--r-- 1 skokado skokado 1.5K Apr  7 22:44 awesum-0.1.0-py3-none-any.whl

$ # アップロード
$ twine upload --repository gitlab dist/*
```

無事にアップロードできると[パッケージレジストリ]が以下のようになる

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/1776fa03-3ff1-22eb-c6a5-e29259128a28.png)

# パッケージレジストリの使用

※pipenvを使用する例で紹介。

## アクセストークンの準備

レジストリにアクセスするためのパーソナルアクセストークンを発行しておく

※上記で作成した[Deploy tokens]とは別

- 右上メニュー[Preference]\(設定)
- 左側メニュー[アクセストークン]よりトークンを作成
    - [スコープ]は`read_api`のみでOK

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/f893ca74-e5fc-b7b1-ba8d-808986dc9876.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/ed5dba41-c3e7-75a8-4382-c1d4660f3189.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/07aed3b7-6bb2-4613-4dea-bb92083f4691.png)

## プロジェクト作成

```shell
$ pipenv --three
```

自動作成される`Pipfile`にPyPIミラーの情報を追加する

※環境変数`$GITLAB_TOKEN`を利用できるようにする

```diff:Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

+ [[source]]
+ url = "https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple"
+ verify_ssl = true
+ name = "gitlab"

[packages]

[dev-packages]

[requires]
python_version = "3.8"
```

### インストール

GitLab内[パッケージレジストリ]のページでは下記のコマンドが紹介されている

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/154711/4707f6c9-d0d8-8792-bc2a-ec6001b391e2.png)

※コマンドが`pip`の部分を`pipenv`に置き換える

```shell
$ export GITLAB_TOKEN=xxx
$ pipenv install awesum --extra-index-url https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple
Installing awesum...
Adding awesum to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (0f9f1e)!
Installing dependencies from Pipfile.lock (0f9f1e)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
```

ここで、自動作成された`Pipfile.lock`において`index`が`pypi`となってしまっているので修正する。

```diff:Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "2ca0c992446d43a908934fa7b5de037410087de92fbbf2ad346b5f152d43ff89"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "gitlab",
                "url": "https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "awesum": {
            "hashes": [
                "sha256:cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413"
            ],
-             "index": "pypi",
+             "index": "gitlab",
            "version": "==0.1.0"
        }
    },
    "develop": {}
}
```

ここまで実施しておけば、次回以降は下記のように環境再現が可能となる。

```shell
$ # 仮想環境ごと事前に削除しておく
$ ls -l
-rw-rw-r--  1 skokado skokado  413 Apr  7 23:04 Pipfile
-rw-r--r--  1 skokado skokado  983 Apr  7 23:04 Pipfile.lock

$ pipenv sync -v
Using python: 3.8
Path to python: /home/skokado/.pyenv/versions/3.8.8/bin/python3.8
Creating a virtualenv for this project...
Pipfile: /home/skokado/workspace/hello-awesum/Pipfile
Using /home/skokado/.pyenv/versions/3.8.8/bin/python3.8 (3.8.8) to create virtualenv...
⠸ Creating virtual environment...created virtual environment CPython3.8.8.final.0-64 in 219ms
  creator CPython3Posix(dest=/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/skokado/.local/share/virtualenv)
    added seed packages: pip==21.0.1, setuptools==54.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw
Installing dependencies from Pipfile.lock (0f9f1e)...
Writing supplied requirement line to temporary file: 'awesum==0.1.0 --hash=sha256:cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413'
Installing 'awesum'
$ ['/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/bin/pip', 'install', '--verbose', '--upgrade', '--require-hashes', '--no-deps', '--exists-action=i', '-r', '/tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt', '-i', 'https://pypi.org/simple', '--extra-index-url', 'https://__token__:R7P3gwHCr27xGF_g5r8r@gitlab.com/api/v4/projects/25700635/packages/pypi/simple', '--extra-index-url', 'https://__token__:R7P3gwHCr27xGF_g5r8r@gitlab.com/api/v4/projects/25700635/packages/pypi/simple']
Using source directory: '/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/src'
Using pip 21.0.1 from /home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/lib/python3.8/site-packages/pip (python 3.8)
Non-user install by explicit request
Created temporary directory: /tmp/pip-ephem-wheel-cache-t8ranvj1
Created temporary directory: /tmp/pip-req-tracker-h3ie50r6
Initialized build tracking at /tmp/pip-req-tracker-h3ie50r6
Created build tracker: /tmp/pip-req-tracker-h3ie50r6
Entered build tracker: /tmp/pip-req-tracker-h3ie50r6
Created temporary directory: /tmp/pip-install-7zltel63
Looking in indexes: https://pypi.org/simple, https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple, https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple
2 location(s) to search for versions of awesum:
* https://pypi.org/simple/awesum/
* https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Fetching project page and analyzing links: https://pypi.org/simple/awesum/
Getting page https://pypi.org/simple/awesum/
Found index url https://pypi.org/simple
Looking up "https://pypi.org/simple/awesum/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/awesum/ HTTP/1.1" 404 13
Status code 404 not in (200, 203, 300, 301)
Could not fetch URL https://pypi.org/simple/awesum/: 404 Client Error: Not Found for url: https://pypi.org/simple/awesum/ - skipping
Fetching project page and analyzing links: https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Getting page https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Found credentials in url for gitlab.com
Looking up "https://gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): gitlab.com:443
https://gitlab.com:443 "GET /api/v4/projects/25700635/packages/pypi/simple/awesum/ HTTP/1.1" 304 0
  Found link https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from https://gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/) (requires-python:>=3.6,<4.0), version: 0.1.0
Checked 1 links for project 'awesum' against 1 hashes (1 matches, 0 no digest): discarding no candidates
Collecting awesum==0.1.0
  Created temporary directory: /tmp/pip-unpack-3ai1rpb3
  Looking up "https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl" in the cache
  Current age based on date: 162
  Freshness lifetime from max-age: 0
  https://gitlab.com:443 "GET /api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl HTTP/1.1" 304 0
  Using cached https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl (1.4 kB)
  Added awesum==0.1.0 from https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from -r /tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt (line 1)) to build tracker '/tmp/pip-req-tracker-h3ie50r6'
  Removed awesum==0.1.0 from https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from -r /tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt (line 1)) from build tracker '/tmp/pip-req-tracker-h3ie50r6'
Created temporary directory: /tmp/pip-unpack-zlqrqbuy
Installing collected packages: awesum

Successfully installed awesum-0.1.0
Removed build tracker: '/tmp/pip-req-tracker-h3ie50r6'
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!
```