The skinny on Python dev environment and packaging (pip)
Tags: python, beginners, quicktip, resources
Published: 2019-03-14 18:48:01

---
title: The skinny on Python dev environment and packaging (pip)
published: true
description: Tips on setting up a python dev environment and distributing on pip
tags: #python #beginner #quicktip #resources
---


Welcome to the wild world of Python dev. Where nothing is consistent and there is no such thing as as consensus.

Don't worry I'll be your guide.




## Background

I quietly released [pyquilted](http://github.com/cocoroutine/pyquilted) which generates nice looking resumes from Yaml in PDF. I learned a lot about setting up a python dev environment and packaging an app to distribute on Pypi (pip) and now I am sharing that knowledge.

You can find out more about quilted resumes [here](http://github.com/cocoroutine/quilted).




## Unittests

Running test cases were for some reason harder than it should be. Follow these steps to make your life easier.

Create a directory called 'test' in your project root.

Name all your tests 'test\_\*'.

Put an \_\_init\_\_.py so that the directory get's recognized as a package.


running all tests on Python 2 and 3
`python -m unittest discover`


running one test
`python -m unittest test.test_name`




## Venv vs VirtualEnv vs PipEnv

These are all ways to isolate your python pip environment so you do not pollute your main system python. Uninstalling a package and it's dependecies is a pain in pip.  This avoids this by allowing you to easily delete and create new envs.


#### But which one do you go with?

Venv ~~vs VirtualEnv vs PipEnv~~

Just go with venv. It comes standard with python and is officially recommended post 3.3+.  It also has less gotchas than virtualenv and I don't want to be gotten.

to create a new venv folder
`python -m venv venv`

to load the environment
`source venv/bin/activate`

turn off the venv
`deactivate`


Pipvenv is nice, but it's not standard and is based on virtualenv.


#### Python 2.7 and Venv

Venv doesn't support python 2.7, bummer.

Simple solution
`pip install py2venv`

This installs virutalenv for you and creates a wrapper around virtualenv. Meaning you can use the venv command. Great!


#### Where to put your venv folder?

If you are just running one venv per project, put it in the same directory as the project root and name the folder 'venv'. Make sure in your .gitignore you have added 'venv'—on github and gitlab this is added for you.

one venv environment per project
`project_root/venv`

The issues comes when you want different venv environments for different python versions you are testing. Here's a few options

multiple venv per project
`project_root/venv/{{python_version}}`

alternative, venv in home folder
`~/venv/{{project_name}}_{{python_version}}`




## setup.py

Setup.py is like a makefile for your app. It's based off of setuptools. It normally comes pre-installed. If not you can install it like so

`pip install setuptools`

The documentation can be found [here](https://setuptools.readthedocs.io/en/latest/). Some parts are either confusing or not working as expected. I noted the parts here.

You can see an example of my setup.py in the pyquilted [source](https://github.com/cocoroutine/pyquilted)



#### Command line entry point

You want to run your app in the terminal using your app's sweet name. This was a lot tricker than it should be. After endless searching, this method below worked for me.

In your project root, create a file named main.py. Have one function named main that calls your run function. An example [here](https://github.com/cocoroutine/pyquilted/blob/master/pyquilted/main.py).


Add the following into your setup.py
```
entry_points={
    'console_scripts': ['command=pacakge.main:main']
}
```

Where 'command' is the command you want to use. 'package' is your package name.

For example my pyquilted entry point
`'console_scripts': ['pyquilted=pyquilted.main:main']`


#### Markdown readme

Pypi now supports markdown. Use a with open file handler to set your readme doc to a variable.

```
long_description=readme,
long_description_content_type='text/markdown'
```


#### Including non py files

There are two ways to add files to either your site_packages for data or your source package for test cases and such.


#### package data

Use the below format for adding data files to your site_packages.
```
package_data={
    'package_name': ['directory/*.pattern']
},
include_package_data=True

```


#### MANIFEST.in

This includes files in your source pacakge that are not in your site_packages folder. The format is a little weird.
```
include file   # include single file
recursive-include directory pattern # include a directory and files
```




## requirements.txt

Skip it. It's not really necessary, but simple to make.




## Enter Pyenv

Still crying over the disaster that was migrating from Python 2 to 3. Me too, join the club. We stil have to support Python 2 and test other versions of 3 as well. How to solve multiple python versions? We could learn Vagrant or Docker.  Nah that's way overkill.

Solution Pyenv.

Install Pyenv by following instructions here [pyenv](https://github.com/pyenv/pyenv).

list installable python versions
`pyenv install --list`

install version
`pyenv install 2.7.15`

list versions installed
`pyenv versions`

switch to python version
`pyenv global 2.7.15`


#### Debian, Ubuntu and others

On Debian, Ubuntu and other distros, you will get an error about ssl when trying to build older versions of python. The issue is that libssl 1.1 is not compatble with Python < 3.3. The solution is to download the dev headers for libssl 1.0 and link them manually when running pyenv install, per this [thread](https://github.com/pyenv/pyenv/issues/945#issuecomment-409627448).
```
CFLAGS="-I/path_to/libssl1.0-dev/include -I/path_to/libssl1.0-dev/include/x86_64-linux-gnu" \
LDFLAGS="-L/path_to/libssl1.0-dev/lib/x86_64-linux-gnu" pyenv install 2.7.15
```




## Eggs everywhere

To stop python from creating eggs that you accidently upload to Pypi when you are testing.

Use the below command
`pip install .`


instead of this command
`python setup.py install`




## Building and Wheels

Honestly wheel files are not necessary. The difference in install time between source and binary is like a few seconds. However it's really easy to build binary wheels so why not.

make sure you have wheel installed
`pip install wheel`

run this to build your packages
`python setup.py sdist bdist_wheel`




## Finally upload to Pypi

Create an account on Pypi.org. Then install twine which is the official pypi app
`pip install twine`

Upload with this command
`twine upload --skip-existing dist/*`

The key is skip existing files, otherwise you get errors when uploading the dist folder with existing files.




## Congrats!

Congrats you beat the final boss! Pat yourself on the back for finally deciphering python dev environment.