Repository: mitmproxy/mitmproxy

Issue:
Missing documentation about unit testing inline scripts
At the company I work on we are developing some inline scripts to use internally.

We are struggling to create test for our scripts because the only code we found to test scripts are the ones at [/mitmproxy/mitmproxy/blob/v0.17/test/mitmproxy/test_examples.py](/mitmproxy/mitmproxy/blob/v0.17/test/mitmproxy/test_examples.py).

The examples are quite good but real documentation is missing for the ones who wants to unit test.

Is there another place where I can find better (maybe more advanced) test examples ?



Discussion:
This is a really good question, thanks for raising it! :smiley:

With mitmproxy 0.18 we are introducing some significant improvements to our internal architecture, in particular many feature are now implemented as addons to mitmproxy's core (see https://github.com/mitmproxy/mitmproxy/tree/master/mitmproxy/builtins). Fundamentally, addons are not much different from inline scripts¹ and you can actually have your inline script's `start()` method just return an Addon instance (see e.g. the [filt.py](https://github.com/mitmproxy/mitmproxy/blob/master/examples/filt.py) example).

Long story short:
1. We'll have our 0.18 release pretty soon. If you are developing some larger inline scripts (i.e. large enough to require testing), I would recommend working with that right away and use the return-addon-from-start approach discussed above.
2. If you do that, you now have all your functionality in a regular Python class and you can import your inline script as a normal Python module. You can have dummy requests and responses from `netlib.tutils`. If you need dummy flows, we're happy to accept a PR that moves `test/mitmproxy/tutils.py` into the mitmproxy package itself.

There are a couple of changes to the scripting API in 0.18 we're excited about. We'd like to make sure that we cover all major use cases elegantly, so getting early feedback for this would be wonderful. There's no updated documentation yet (#1445), but all examples have been updated to the new syntax - it's mostly just about removing unecessary arguments from the hooks.

¹) An addon is a class that basically just defines a couple of functions which are equivalent to the inline script hooks.

_Just a context for the original question_: At the company I work on, we are searching for ways to implement a reverse proxy for one of our APIs and since the main programming language over here is python we searched for some projects that could help us with that. A coworker of mine stumbled across this project which we found promissing so we are trying to write scripts based on mitmproxy and test them to see if they will work for us.

Anyway, this addons approach seems good but I will need more time to fully understand the codebase and then write good tests. This is when a documentation helps a lot.

Thanks for the contact @mhils and congratulations for the project !

I've spent some time today working on the testing situation for addons. We now expose a testing truss for addons from the mitmproxy library:

https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/test/taddons.py

You can see this in action here:

https://github.com/mitmproxy/mitmproxy/blob/master/test/mitmproxy/addons/test_intercept.py

We'll let this mature a bit, wash it across our own test suite, and then document it for the next release. Comments, suggestions and improvements appreciated!


Resolving patch:
diff --git a/mitmproxy/events.py b/mitmproxy/events.py
--- a/mitmproxy/events.py
+++ b/mitmproxy/events.py
@@ -21,9 +21,6 @@
     "responseheaders",
     "error",
 
-    "intercept",
-    "resume",
-
     "websocket_handshake",
     "websocket_start",
     "websocket_message",


Test patch: