Repository: Miserlou/Zappa

Issue:
Zappa always re-downloads "psycopg2-binary" library when building the deployment package
<!--- Provide a general summary of the issue in the Title above -->
## Context

First of all, I am indebted to Rich Jones and contributors who made Zappa possible! It's mind-blowing to think how far Zappa has taken Amazon API Gateway and Lambda functions.

The bug itself is tiny and might not even be worthy of the attention of this audience, and there is also a workaround. I am deploying a stock Django project coupled with a PostgreSQL database.

Not sure how you guys and gals are doing this, but what worked for me was to run both `pip install psycopg2` and `pip install psycopg2-binary`. The former installs a nice `_psycopg.cpython-38-x86_64-linux-gnu.so` so-called "shared object" file, a dynamically linked Linux library, but it requires a host of other libraries to actually run -- so the latter provides those libraries.

It all deploys and runs well, but each time I try to run `zappa update`, or actually even `zappa package`, Zappa builds a package and *re-downloads* the poor `psycopg2-binary` library.

Those libraries are supposed to be cached on disk to avoid re-downloading each time. I inserted a debug print statement after line [817](https://github.com/Miserlou/Zappa/blob/93804a1b3157f0189bf062e01baab2bd4f09400d/zappa/core.py#L817) in `core.py`:

> print(f"Looking for file: {wheel_path}")

And when running `zappa package`, it prints, among other lines:

> Looking for file: /tmp/cached_wheels/psycopg2-binary-2.8.5-*_x86_64.whl

Actually, there is a file with almost the same name in that location, but the dash `-` between `psycopg2` and `binary` is replaced with an underscore `_`:

> ls /tmp/cached_wheels/psycopg2_binary-2.8.5-cp38-cp38-manylinux1_x86_64.whl

The underscore is what is present in the [original name](https://pypi.org/project/psycopg2-binary/#files) of that file if you want to [download](https://files.pythonhosted.org/packages/ab/a8/ebd961de8bd750aa70d8a7e1073d1a2f860ec367ca3e20b2fb0886423fd2/psycopg2_binary-2.8.5-cp38-cp38-manylinux1_x86_64.whl) it from PyPi.

Looks like this happens because the name of the library in `requirements.txt` is with a dash: `psycopg2-binary`, and this is what Zappa expects when building a wildcard expression in line [816](https://github.com/Miserlou/Zappa/blob/93804a1b3157f0189bf062e01baab2bd4f09400d/zappa/core.py#L816) of `core.py`, whereas the file on disk is with an underscore.

For comparison, another package with a dash in its name, `django-environ`, also [ships](https://pypi.org/project/django-environ/#files) its [wheel file](https://files.pythonhosted.org/packages/9f/32/76295a1a5d00bf556c495216581c6997e7fa5f533b2229e0a9d6cbaa95ae/django_environ-0.4.5-py2.py3-none-any.whl) with an underscore instead of a dash.

Now, I am not sure we should call it a bug, since Zappa works, it just re-downloads this file each time since it believes it's not cached. The only negative thing is that guilty feeling that someone is paying for that bandwidth.

A workaround would be to make a copy of that file in `/tmp/cached_wheels` but replace an underscore with a dash:

> cd /tmp/cached_wheels
> cp psycopg2_binary-2.8.5-cp38-cp38-manylinux1_x86_64.whl psycopg2-binary-2.8.5-cp38-cp38-manylinux1_x86_64.whl

It will then find the "cached" version and not download anything:

```
$ zappa package

Downloading and installing dependencies..
 - psycopg2-binary==2.8.5: Using locally cached manylinux wheel
```

I am surprised that I didn't find anyone else complaining about this. Maybe I am doing it wrong and everyone else using Django with PostgreSQL is not having this problem.

**Feel free to close this issue** since there is a workaround -- this way, even if closed, it will eventually become indexed by search engines for others to find.

## Expected Behavior

Zappa should find the cached wheel file despite the underscore in its name.

## Actual Behavior

The file has an underscore where Zappa expects a dash, so Zappa re-downloads the file.

## Possible Fix

A workaround is creating a copy of the wheel file with a dash instead of the underscore in its name (commands provided above); this way Zappa will find the copy and use it.

## Steps to Reproduce

1. pip install -r requirements.txt (packages to install are listed below)
2. zappa package

## Your Environment

* Zappa version used: 0.51.0
* Operating System and Python version: Ubuntu 20.04 LTS, Python 3.8.2
* The output of `pip freeze` (relevant parts): 

```
Django==3.0.7
psycopg2==2.8.5
psycopg2-binary==2.8.5
zappa==0.51.0
```

Thanks, everyone -- again, it's not really a bug, or at most a tiny one. I just hope that after reading this others will know how to fix it using the workaround provided.


Discussion:
Thank you very much. This is a bug in zappa as we (in this specific case I) forgot to apply this section of the PEP to the filenames: https://www.python.org/dev/peps/pep-0427/#escaping-and-unicode
Thank you, João -- I am impressed with the speed of your reply and your exact pinpointing of the bug! :+1: :bow: 

Resolving patch:
diff --git a/zappa/core.py b/zappa/core.py
--- a/zappa/core.py
+++ b/zappa/core.py
@@ -813,7 +813,8 @@ def get_cached_manylinux_wheel(self, package_name, package_version, disable_prog
             os.makedirs(cached_wheels_dir)
         else:
             # Check if we already have a cached copy
-            wheel_file = f'{package_name}-{package_version}-*_x86_64.whl'
+            wheel_name = re.sub("[^\w\d.]+", "_", package_name, re.UNICODE)
+            wheel_file = f'{wheel_name}-{package_version}-*_x86_64.whl'
             wheel_path = os.path.join(cached_wheels_dir, wheel_file)
 
             for pathname in glob.iglob(wheel_path):


Test patch: