Let's Set Up Poetry - Clean Python DevEnv for 2020
Tags: python, productivity, pip, poetry
Published: 2019-12-26 12:46:01

<a style="background-color:black;color:white;text-decoration:none;padding:4px 6px;font-family:-apple-system, BlinkMacSystemFont, &quot;San Francisco&quot;, &quot;Helvetica Neue&quot;, Helvetica, Ubuntu, Roboto, Noto, &quot;Segoe UI&quot;, Arial, sans-serif;font-size:12px;font-weight:bold;line-height:1.2;display:inline-block;border-radius:3px" href="https://unsplash.com/@scott_umstattd?utm_medium=referral&amp;utm_campaign=photographer-credit&amp;utm_content=creditBadge" target="_blank" rel="noopener noreferrer" title="Download free do whatever you want high-resolution photos from Scott Umstattd"><span style="display:inline-block;padding:2px 3px"><svg xmlns="http://www.w3.org/2000/svg" style="height:12px;width:auto;position:relative;vertical-align:middle;top:-2px;fill:white" viewBox="0 0 32 32"><title>unsplash-logo</title><path d="M10 9V0h12v9H10zm12 5h10v18H0V14h10v9h12v-9z"></path></svg></span><span style="display:inline-block;padding:2px 3px">Scott Umstattd</span></a>

I have used a couple of dev env on `python`.

1. System-provided Python (Mac)
As many of you know, the version is too outdated right now. When I used it, it was fine but at that time I wanted to use python3 so I didn't use it for a long time. 

2. Pyenv
Pyenv allows us to install multiple versions of python which is very useful, but when I used it, I didn't use Virtualenv, so my dev envs were very messy.

3. Anaconda
Then, I started learning about machine learning and one ITP resident recommended us to use Anaconda since it's very easy to create a virtual environment and install some specific modules/packages.
found out this article(https://jacobian.org/2019/nov/11/python-environment-2020/) that is from Jacob Kaplan-Moss, co-creator of `Django`. 

4. Pyenv + Virtualenv
When I used 3, I also used Pyenv, but sometimes that made me confused and also Anaconda had some issues when I run flask. Eventually, I uninstalled Anaconda and started using Pyenv and Virtualenv. Actually, I'm still using this. But, the issue is that I heavily use a specific virtual env and it's very messy so it's about to ruin my python env lol 

5. Pipenv
The team I joined used python and the lead engineer recommended to use `pipenv` But, `pipenv` is slow and has caused some issues when I update 
modules/packages. I end up using 4 for other things/projects.


Recently, I read the article on python dev env that is written in Japanese lol. The article mentioned 3 three things, Pipenv, Poetry, and Pyflow. It says that Poetry could be a better choice. After reading that, I researched about Poetry a little bit. Then, I found out this article(https://jacobian.org/2019/nov/11/python-environment-2020/) that is from Jacob Kaplan-Moss, co-creator of `Django`. 
  
Poetry 
https://python-poetry.org/


### Why do I need to use `Poetry`?
Before introducing the install steps, I would like to mention the reason why I want to switch `Pyenv` + `Virtualenv` to `Poetry`. The biggest reason is that I can share a project easily. 
When I was an ITP resident and led a python workshop, I spent so much time to create an environment for sample codes since most students didn't follow guides I shared before the workshop lol.
Also, there are a few reasons. 

What `Poetry` can do is to allow me to manage python projects like js/nodejs projects which are managed by yarn/npm. This feature enables me to discard a project easily(I will be able to delete the project.) when I don't need it. I can use virtual env with `Poetry`. 
In addition, `Poetry` has commands that build a package and publish it to PyPI. This is also good for me since I have published a couple of packages to PyPI. Plus `Poetry` has an export command for requirements.txt.   
https://python-poetry.org/docs/cli/#export


I decided to set up Poetry on my Ubuntu since Mac's environment will take some time to switch and I noticed that I haven't used `Pyenv` on my Ubuntu. (I don't know why I haven't used it.)

In this article, I will share steps to set up `Poetry` on my Ubuntu.
The env will be `Pyenv` + `Poetry` + `Pipx`.
This setup is very similar to my js dev env. (nvm + yarn)


Step1. Install Pyenv
Step2. Install Pipx for python CLI tool (especially, youtube-dl)
Step3. Install Poetry
Step4. Create a sample project with Poetry
Step5. Allow making venv in a project


### Step1 Install Pyenv
There are a couple of ways to install Pyenv on Ubuntu. In this article, I use brew since I'm using brew on Mac and currently we can use brew on Linux too.

https://docs.brew.sh/Homebrew-on-Linux
https://github.com/pyenv/pyenv

```zsh
$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"

$ brew install pyenv

$ pyenv --version
pyenv 1.2.15
```

system pythons
```zsh
$ python --version
Python 2.7.17

$ python3 --version
Python 3.7.5
```

Let's install python 3.6 and 3.8
```
$ pyenv install 3.6.5
$ pyenv install 3.8.0
$ pyenv global 3.8.0
```

Ugh, got `BUILD FAILED (Ubuntu 18.04 using python-build 20180424)` with 3.6.5...

The solution is the following
```zsh
sudo apt-get remove libssl-dev
sudo apt-get update
sudo apt-get install libssl1.0-dev
```

https://github.com/pyenv/pyenv/wiki/Common-build-problems

Now the global Python version is changed by `Pyenv`
```
$ python --version
Python 3.8.0
```

### Step2 install pipx
Of course, you can install CLI tools via brew, but this time I'm using pipx to try a new thing lol.

```zsh
$ pip list
Package    Version
---------- -------
pip        19.2.3 
setuptools 41.2.0 


$ python -m pip install pipx
$ pipx install youtube-dl
$ pipx list
   package youtube-dl 2019.12.25, Python 3.8.0
    - youtube-dl
# Actually the command is a little bit longer, but you can use alias to shorten the command.
$ pipx run youtube-dl url

```


### Step3 install Poetry
Eventually reached the most important part of this post.
The official page is using `curl` to install `Poetry`, but this post uses `pipx` because I need to understand pipx.   
`I think following the official site is better.`

Poetry - Official 
https://python-poetry.org/docs/

```
$ pipx install poetry 

```


### Step4 Create a sample project

```
$ mkdir hoge && cd hoge
$ pipx run poetry init
or
$ pipx run poetry new hoge

# add package
$ pipx run poetry add ipython
$ pipx run poetry run ipython
Python 3.8.0 (default, Dec 25 2019, 21:53:01) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: print('a')                                                              
a

In [2]:      
```

`pyproject.toml`
```toml
[tool.poetry]
name = "hoge"
version = "0.1.0"
description = ""
authors = ["koji <kk2796@nyu.edu>"]

[tool.poetry.dependencies]
python = "^3.8"
ipython = "^7.10.2"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
```

`export`
```
$ pipx run poetry export -f requirements.txt > requirements.txt
```

### Step5 Allow making venv in a project
The default settings create a virtual environment in a specific place, so this is the same as pyenv + virtualenv that I'd like to avoid. So I need to modify the setting.

```
$ pipx run poetry config --list
cache-dir = "/home/koji/.cache/pypoetry"
virtualenvs.create = true
virtualenvs.in-project = false <-- chage this
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/koji/.cache/pypoetry/virtualenvs

$ pipx run poetry config virtualenvs.in-project true
$ pipx run poetry config --list                                          
cache-dir = "/home/koji/.cache/pypoetry"
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/koji/.cache/pypoetry/virtualenvs
```

# Caution 
I just started using Poetry before 2020, so there might be some mistakes.
If you find out the mistake, please let me know!