Back on the struggle but with pip and Ansible! (what is _ctypes?)
Tags: pyenv, python, pip, ubuntu
Published: 2022-03-21 08:20:17

---
title: Back on the struggle but with pip and Ansible! (what is _ctypes?)
published: true
description: Trying to decipher what ctypes are while installing Ansible on ubuntu using pip
tags: pyenv, python, pip, ubuntu
//cover_image: https://direct_url_to_image.jpg
---

TL;DR: If you're getting the following error:
```
from _ctypes import Union, Structure, Array
    ModuleNotFoundError: No module named '_ctypes'
```
you're probably missing `libffi-dev` on Ubuntu (or some variation on other OS's)

---

And we're back! [Last time](https://dev.to/milktea02/struggle-tweets-to-install-python-3102-with-pyenv-on-ubuntu-2004-wsl-3ek5) I was trying to install python `v3.10.2` with `pyenv` and was missing a million dependencies.

This time I'm trying to install `ansible` using `pip` (managed with `pyenv`)

```shell
Collecting ansible
  Downloading ansible-5.5.0.tar.gz (42.0 MB)
     |████████████████████████████████| 42.0 MB 782 kB/s
    ERROR: Command errored out with exit status 1:
     command: /home/banana/.pyenv/versions/3.10.2/bin/python3.10 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cfmis9e2/ansible_27dc97f765544a6381b36454d1ed76c5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cfmis9e2/ansible_27dc97f765544a6381b36454d1ed76c5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-e10880nb
         cwd: /tmp/pip-install-cfmis9e2/ansible_27dc97f765544a6381b36454d1ed76c5/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/banana/.pyenv/versions/3.10.2/lib/python3.10/site-packages/setuptools/__init__.py", line 18, in <module>
        from setuptools.dist import Distribution
      File "/home/banana/.pyenv/versions/3.10.2/lib/python3.10/site-packages/setuptools/dist.py", line 38, in <module>
        from setuptools import windows_support
      File "/home/banana/.pyenv/versions/3.10.2/lib/python3.10/site-packages/setuptools/windows_support.py", line 2, in <module>
        import ctypes
      File "/home/banana/.pyenv/versions/3.10.2/lib/python3.10/ctypes/__init__.py", line 8, in <module>
        from _ctypes import Union, Structure, Array
    ModuleNotFoundError: No module named '_ctypes'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/b5/6e/5806eafaa71ab5471cfc390089a89c7815ab841657d6517fd72df3d41862/ansible-5.5.0.tar.gz#sha256=b8a76d737889c9bfd0e6b0f2276dcf8d836da667067a355776f3504d7a66d518 (from https://pypi.org/simple/ansible/) (requires-python:>=3.8). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
```

This continues for a bit as `pip` tries to install versions of ansible in decreasing order.

What if I try to install this missing `_ctypes` module?

```shell
$ pip install _ctypes
ERROR: Invalid requirement: '_ctypes'
$ pip install ctypes
ERROR: Could not find a version that satisfies the requirement ctypes (from versions: none)
ERROR: No matching distribution found for ctypes
```

huh, weird

What is this `_ctypes` module that I'm missing? According to the [docs](https://docs.python.org/3/library/ctypes.html):

> ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.

What does that even mean? My understanding is that the `ctypes` library allows python code to use stuff in DLLs (dynamic linked libraries, this is OS dependent) directly using python without needing custom python code. This is really powerful and you can read more about it in the Python docs.

But why doesn't `_ctypes` or `ctypes` exist in pip as an installable module? Well, `_ctypes` is a built-in library so there isn't any module to `pip install`. What we're missing is the [`libffi-dev`](https://packages.ubuntu.com/focal/libffi-dev) Ubuntu package (thanks stackoverflow!):

> Foreign Function Interface library (development files)

Let's hit up `apt install libffi-dev`. Then do another `pip install ansible` (if you want to save time you'd just try `from ctypes import *` in a python shell and find that the module still can't be found):

```shell
$ pip install ansible
Collecting ansible
  Using cached ansible-5.5.0.tar.gz (42.0 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
```

Interesting. What is also interesting was the error I got [last time](https://dev.to/milktea02/struggle-tweets-to-install-python-3102-with-pyenv-on-ubuntu-2004-wsl-3ek5) when I was trying to install `python`:

```
Failed to build these modules:
_ctypes
```

After installing the `zlib` package previously I was still able to successfully compile python without `libffi` and also did not get an error. I'm wondering now if my installation of python `3.10.2` is a bit wonky now.

I've decided to take a gander at what modules I have and while I don't see any `_ctypes.py` or `ctypes.py` I did find a directory called `ctype`:
```shell
~/.pyenv/versions/3.10.2/lib/python3.10/ctypes$ ls
__init__.py  __pycache__  _aix.py  _endian.py  macholib  test  util.py  wintypes.py
```

I'm not sure if any of the above is suspicious but I thought I'd take a look anyway.

I also have a hunch that with `libffi` missing during my initial python install (failed silently??), the appropriate symbolic links may not have been created properly or python wasn't compiled properly so I'll do a `pyenv uninstall 3.10.2` and `pyenv install 3.10.2` to recompile.

Now to check if `ctypes` was successfully built:

```python
Python 3.10.2 (main, Mar 21 2022, 00:28:55) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> libc = CDLL("libc.so.6")
>>> print(libc.time(None))
1647848336
```

yay success!

NOW, back to installing `ansible`:

```shell
$ pip install ansible
...
... lots of output
...
done
Successfully installed MarkupSafe-2.1.1 PyYAML-6.0 ansible-5.5.0 ansible-core-2.12.3 cffi-1.15.0 cryptography-36.0.2 jinja2-3.0.3 packaging-21.3 pycparser-2.21 pyparsing-3.0.7 resolvelib-0.5.4
```

Excellent, and all the dependencies were installed too!