Repository: pyca/cryptography

Issue:
Lazily load idna module
The idna package is a heavy-load package. ```idna.idnadata.scripts``` takes up a lot of time and memory to import. On my system it takes about 80ms to import the module. The import increases RSS of the process by almost 20 MB. That's a lot of time and memory for a command line script.

urllib3 uses lazy import for idna already, https://github.com/shazow/urllib3/blob/master/urllib3/contrib/pyopenssl.py#L167 . Would you consider to similar approach, too?

```
$ python -m timeit -s 'import sys' -- "sys.modules.pop('idna.idnadata', None); import idna.idnadata"
10 loops, best of 3: 79.7 msec per loop
```

```
$ python
>>> import psutil, os
>>> p = psutil.Process(os.getpid())
>>> pre = p.memory_info()
>>> import idna
>>> post = p.memory_info()
>>> pre
pmem(rss=15720448, vms=212979712, shared=7745536, text=4096, lib=0, data=8302592, dirty=0)
>>> post
pmem(rss=35950592, vms=242565120, shared=7880704, text=4096, lib=0, data=30937088, dirty=0)
>>> (post.rss - pre.rss) / 1024.**2
19.29296875
```


Discussion:
I'm fine with a lazy import. Refs #3357 in yet another reminder why idna was a mistake...
So, this issue is significantly resolved by using `idna>=2.1`: https://github.com/kjd/idna/blob/master/HISTORY.rst#21-2016-03-20

Measures from my system:

```
(tempenv-244d247322ac7) ~ $ python -m timeit -s 'import sys' -- "sys.modules.pop('idna.idnadata', None); import idna.idnadata"
1000 loops, best of 3: 311 usec per loop
(tempenv-244d247322ac7) ~ $ python -m timeit -s 'import sys' -- "sys.modules.pop('idna.idnadata', None); import idna.idnadata"
1000 loops, best of 3: 318 usec per loop
(tempenv-244d247322ac7) ~ $ pip install "idna<2.1"
Collecting idna<2.1
  Downloading idna-2.0-py2.py3-none-any.whl (61kB)
    100% |████████████████████████████████| 61kB 1.2MB/s
Installing collected packages: idna
  Found existing installation: idna 2.2
    Uninstalling idna-2.2:
      Successfully uninstalled idna-2.2
Successfully installed idna-2.0
(tempenv-244d247322ac7) ~ $ python -m timeit -s 'import sys' -- "sys.modules.pop('idna.idnadata', None); import idna.idnadata"
10 loops, best of 3: 164 msec per loop
(tempenv-244d247322ac7) ~ $ python -m timeit -s 'import sys' -- "sys.modules.pop('idna.idnadata', None); import idna.idnadata"
10 loops, best of 3: 175 msec per loop
```

```
(tempenv-244d247322ac7) ~ $ python
Python 2.7.13 (default, Dec 18 2016, 10:06:47)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil, os
>>> p = psutil.Process(os.getpid())
>>> pre = p.memory_info()
>>> import idna
>>> post = p.memory_info()
>>> (post.rss - pre.rss) / 1024.**2
0.55078125
```

Maybe we should stick a version thing in setup.py, rather than lazy load.
I'll defer to @tiran on that. Ultimately my goal is to remove idna as a dep entirely, but that's going to take a long time. I have no particular preference on bandaids 😄 
Ok, I'll send the counter proposal PR.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -34,7 +34,7 @@
 VECTORS_DEPENDENCY = "cryptography_vectors=={0}".format(about['__version__'])
 
 requirements = [
-    "idna>=2.0",
+    "idna>=2.1",
     "asn1crypto>=0.21.0",
     "packaging",
     "six>=1.4.1",


Test patch: