Repository: mitmproxy/mitmproxy

Issue:
Clean up dependencies
Spring cleaning! We currently declare some dependencies which are either unused or can easily be substituted:

 - h11 - not used at all?
 - requests - tests + examples only.

We should IMHO also eventually consider removing the following dependencies, although that involves a bit of work and shouldn't be in scope for this issue:

 - pyasn1 - replace with asn1crypto, which is used by cryptography/pyOpenSSL
 - ldap3 - only used for ldap proxy auth, which should probably live outside of the core once we have a healthy addon system.


Discussion:
Hey, I would love to contribute to resolve this issue. One small doubt, you want to remove `requests` from **install_requires** and add it to dev environment only right?
@fristonio Hi there! yes, shifting requests into dev should do the trick. The dev requirements are installed for our CI tests, so that shouldn't cause breakage. Thanks - look forward to your PR. 
Thanks, @cortesi for early response :)
Also for `ldap3` are we planning something on improving the addon system for **mitmproxy**. I don't have much familiarization with the codebase yet.
For h11, looks like it's indeed unnecessary.

h11 is added in https://github.com/mitmproxy/mitmproxy/pull/2545. That pull request vendored wsproto and wsproto requires h11. wsproto is later unvendored in https://github.com/mitmproxy/mitmproxy/pull/2741, so h11 is not used anymore.

/cc @Kriechi
Sounds about right, thanks @yan12125 for checking!
I didn't check for h11, was looking on task to migrate `pyasn1` to `asn1crypto` first. Thanks a lot, @yan12125 now I don't have to worry about checking for **h11**.
Here's a question: will dependency cleanup occur on 3.x or it won't happen before 4.0? I wonder if I should wait for some more days before updating the mitmproxy package at MacPorts.
I was looking into porting pyasn1 to asn1crypto, The only place where this library was used is to get the `subjectAltNames` ([DNS alt names](https://github.com/mitmproxy/mitmproxy/blob/154309b81113695c43e8cc157fe1ec25bc44823f/mitmproxy/certs.py#L445)) from the certificates. For this single purpose, I don't think to migrate the dependency is a good idea. We can dig in a little bit more and can remove the use of this library altogether by using another approach to parse DNS alt names.

@cortesi @mhils what's your stand on this?
@yan12125: No concrete timeline, sorry.

@fristonio: We transitively depend on pyasn1crypto through cryptography anyway, so including that is fine. I'd also be happy with a kaitaistruct implementation, no particular preference.
@yan12125: To expand on this:

- ldap3 will definitely stay until 4.0 (we won't change our addon system in a bugfix release)
- h11 is a transitive dependency anyway.
- requests may be removed in a bugfix release (but shouldn't impact porting much)
- pyasn1 will remain until 4.0 as a transitive dependency of ldap3. 

Hope that helps, and thanks for porting!

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -66,7 +66,6 @@
         "certifi>=2015.11.20.1",  # no semver here - this should always be on the last release!
         "click>=6.2, <7",
         "cryptography>=2.1.4,<2.2",
-        'h11>=0.7.0,<0.8',
         "h2>=3.0.1,<4",
         "hyperframe>=5.1.0,<6",
         "kaitaistruct>=0.7,<0.9",
@@ -76,7 +75,6 @@
         "pyOpenSSL>=17.5,<17.6",
         "pyparsing>=2.1.3, <2.3",
         "pyperclip>=1.6.0, <1.7",
-        "requests>=2.9.1, <3",
         "ruamel.yaml>=0.13.2, <0.16",
         "sortedcontainers>=1.5.4, <1.6",
         "tornado>=4.3, <4.6",
@@ -96,6 +94,7 @@
             "pytest-timeout>=1.2.1,<2",
             "pytest-xdist>=1.22,<2",
             "pytest>=3.3,<4",
+            "requests>=2.9.1, <3",
             "tox>=2.3, <3",
             "rstcheck>=2.2, <4.0",
         ],


Test patch: