Repository: abhinavsingh/proxy.py
Pull request #1059 (2022-01-24 21:43:05)

`jupyter` notebook based tutorial

codecov[bot]: # [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) Report
> Merging [#1059](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (45f0f38) into [develop](https://codecov.io/gh/abhinavsingh/proxy.py/commit/4af0c2f601e1bd5b312976bd47c37f60b64e0e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (4af0c2f) will **decrease** coverage by `0.23%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/graphs/tree.svg?width=650&height=150&src=pr&token=Zh9J7b4la2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)

```diff
@@             Coverage Diff             @@
##           develop    #1059      +/-   ##
===========================================
- Coverage    87.76%   87.52%   -0.24%     
===========================================
  Files          165      165              
  Lines         6775     6775              
  Branches       679      624      -55     
===========================================
- Hits          5946     5930      -16     
- Misses         720      722       +2     
- Partials       109      123      +14     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `87.48% <ø> (-0.11%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | Coverage Δ | |
|---|---|---|
| [tests/integration/test\_integration.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaW50ZWdyYXRpb24vdGVzdF9pbnRlZ3JhdGlvbi5weQ==) | `94.31% <0.00%> (-5.69%)` | :arrow_down: |
| [tests/http/parser/test\_proxy\_protocol.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC9wYXJzZXIvdGVzdF9wcm94eV9wcm90b2NvbC5weQ==) | `96.07% <0.00%> (-3.93%)` | :arrow_down: |
| [tests/test\_set\_open\_file\_limit.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvdGVzdF9zZXRfb3Blbl9maWxlX2xpbWl0LnB5) | `96.42% <0.00%> (-3.58%)` | :arrow_down: |
| [tests/testing/test\_embed.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvdGVzdGluZy90ZXN0X2VtYmVkLnB5) | `90.32% <0.00%> (-3.23%)` | :arrow_down: |
| [proxy/testing/test\_case.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvdGVzdGluZy90ZXN0X2Nhc2UucHk=) | `97.82% <0.00%> (-2.18%)` | :arrow_down: |
| [proxy/core/work/threadless.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29yZS93b3JrL3RocmVhZGxlc3MucHk=) | `81.28% <0.00%> (-1.98%)` | :arrow_down: |
| [proxy/common/flag.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29tbW9uL2ZsYWcucHk=) | `91.48% <0.00%> (-1.42%)` | :arrow_down: |
| [tests/common/test\_flags.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvY29tbW9uL3Rlc3RfZmxhZ3MucHk=) | `98.61% <0.00%> (-1.39%)` | :arrow_down: |
| [proxy/proxy.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvcHJveHkucHk=) | `98.34% <0.00%> (-0.83%)` | :arrow_down: |
| [proxy/http/handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9oYW5kbGVyLnB5) | `76.54% <0.00%> (+0.44%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Last update [4af0c2f...45f0f38](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).

codecov[bot]: # [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) Report
> Merging [#1059](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (97f1f80) into [develop](https://codecov.io/gh/abhinavsingh/proxy.py/commit/4af0c2f601e1bd5b312976bd47c37f60b64e0e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (4af0c2f) will **decrease** coverage by `0.08%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/graphs/tree.svg?width=650&height=150&src=pr&token=Zh9J7b4la2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)

```diff
@@             Coverage Diff             @@
##           develop    #1059      +/-   ##
===========================================
- Coverage    87.76%   87.67%   -0.09%     
===========================================
  Files          165      165              
  Lines         6775     6775              
  Branches       679      679              
===========================================
- Hits          5946     5940       -6     
- Misses         720      723       +3     
- Partials       109      112       +3     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `87.49% <ø> (-0.09%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | Coverage Δ | |
|---|---|---|
| [proxy/core/work/threadless.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29yZS93b3JrL3RocmVhZGxlc3MucHk=) | `81.28% <0.00%> (-1.98%)` | :arrow_down: |
| [proxy/http/handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9oYW5kbGVyLnB5) | `75.22% <0.00%> (-0.89%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Last update [4af0c2f...97f1f80](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).

codecov[bot]: # [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) Report
> Merging [#1059](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (16c73e3) into [develop](https://codecov.io/gh/abhinavsingh/proxy.py/commit/4af0c2f601e1bd5b312976bd47c37f60b64e0e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (4af0c2f) will **decrease** coverage by `0.10%`.
> The diff coverage is `66.66%`.

[![Impacted file tree graph](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/graphs/tree.svg?width=650&height=150&src=pr&token=Zh9J7b4la2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)

```diff
@@             Coverage Diff             @@
##           develop    #1059      +/-   ##
===========================================
- Coverage    87.76%   87.66%   -0.11%     
===========================================
  Files          165      165              
  Lines         6775     6776       +1     
  Branches       679      680       +1     
===========================================
- Hits          5946     5940       -6     
- Misses         720      723       +3     
- Partials       109      113       +4     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `87.48% <66.66%> (-0.11%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | Coverage Δ | |
|---|---|---|
| [proxy/core/connection/server.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29yZS9jb25uZWN0aW9uL3NlcnZlci5weQ==) | `96.66% <66.66%> (-3.34%)` | :arrow_down: |
| [proxy/core/work/threadless.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29yZS93b3JrL3RocmVhZGxlc3MucHk=) | `81.28% <0.00%> (-1.98%)` | :arrow_down: |
| [proxy/http/handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9oYW5kbGVyLnB5) | `75.22% <0.00%> (-0.89%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Last update [4af0c2f...16c73e3](https://codecov.io/gh/abhinavsingh/proxy.py/pull/1059?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).


Patch:
diff --git a/Makefile b/Makefile
index 51f12a4..1a88722 100644
--- a/Makefile
+++ b/Makefile
@@ -94,7 +94,7 @@ lib-clean:
 	rm -rf .hypothesis
 	# Doc RST files are cached and can cause issues
 	# See https://github.com/abhinavsingh/proxy.py/issues/642#issuecomment-1003444578
-	rm docs/pkg/*.rst
+	rm -f docs/pkg/*.rst
 
 lib-dep:
 	pip install --upgrade pip && \
@@ -134,7 +134,7 @@ lib-doc:
 	python -m tox -e build-docs && \
 	$(OPEN) .tox/build-docs/docs_out/index.html || true
 
-lib-coverage:
+lib-coverage: lib-clean
 	pytest --cov=proxy --cov=tests --cov-report=html tests/ && \
 	$(OPEN) htmlcov/index.html || true
 
diff --git a/proxy/core/connection/server.py b/proxy/core/connection/server.py
index 83ad0ae..d8e8ecf 100644
--- a/proxy/core/connection/server.py
+++ b/proxy/core/connection/server.py
@@ -43,7 +43,12 @@ class TcpServerConnection(TcpConnection):
         )
         self.closed = False
 
-    def wrap(self, hostname: str, ca_file: Optional[str] = None) -> None:
+    def wrap(
+            self,
+            hostname: str,
+            ca_file: Optional[str] = None,
+            as_non_blocking: bool = False,
+    ) -> None:
         ctx = ssl.create_default_context(
             ssl.Purpose.SERVER_AUTH, cafile=ca_file,
         )
@@ -54,4 +59,5 @@ class TcpServerConnection(TcpConnection):
             self.connection,
             server_hostname=hostname,
         )
-        self.connection.setblocking(False)
+        if as_non_blocking:
+            self.connection.setblocking(False)
diff --git a/proxy/http/proxy/server.py b/proxy/http/proxy/server.py
index dc9b2ed..428dd1a 100644
--- a/proxy/http/proxy/server.py
+++ b/proxy/http/proxy/server.py
@@ -762,7 +762,11 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
         assert isinstance(self.upstream.connection, socket.socket)
         do_close = False
         try:
-            self.upstream.wrap(text_(self.request.host), self.flags.ca_file)
+            self.upstream.wrap(
+                text_(self.request.host),
+                self.flags.ca_file,
+                as_non_blocking=True,
+            )
         except ssl.SSLCertVerificationError:    # Server raised certificate verification error
             # When --disable-interception-on-ssl-cert-verification-error flag is on,
             # we will cache such upstream hosts and avoid intercepting them for future
diff --git a/proxy/http/server/reverse.py b/proxy/http/server/reverse.py
index 11d5150..45afe1f 100644
--- a/proxy/http/server/reverse.py
+++ b/proxy/http/server/reverse.py
@@ -83,6 +83,7 @@ class ReverseProxy(TcpUpstreamConnectionHandler, HttpWebServerBasePlugin):
                     text_(
                         self.choice.hostname,
                     ),
+                    as_non_blocking=True,
                 )
             # Update Host header
             # if request.has_header(b'Host'):
diff --git a/tutorial/connections.ipynb b/tutorial/connections.ipynb
new file mode 100644
index 0000000..80bc82d
--- /dev/null
+++ b/tutorial/connections.ipynb
@@ -0,0 +1,130 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Connections\n",
+    "\n",
+    "## Buffered Socket Connections\n",
+    "\n",
+    "`proxy.py` core provides buffered socket implementations.  In most of the cases, a buffered connection will be desired.  With buffered connections, we can queue data from our application code while leaving the responsibility of flushing the buffer on the core.\n",
+    "\n",
+    "One of the buffered connection class is `TcpServerConnection`, which manages connection to an upstream server.  Optionally, we can also enable encryption _(TLS)_ before communicating with the server.\n",
+    "\n",
+    "Import the following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from proxy.core.connection import TcpServerConnection\n",
+    "from proxy.common.utils import build_http_request\n",
+    "from proxy.http.methods import httpMethods\n",
+    "from proxy.http.parser import HttpParser, httpParserTypes\n",
+    "\n",
+    "request = build_http_request(\n",
+    "    method=httpMethods.GET,\n",
+    "    url=b'/',\n",
+    "    headers={\n",
+    "        b'Host': b'jaxl.com',\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's use `TcpServerConnection` to make a HTTP web server request."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "http_client = TcpServerConnection('jaxl.com', 80)\n",
+    "http_client.connect()\n",
+    "http_client.queue(memoryview(request))\n",
+    "http_client.flush()\n",
+    "\n",
+    "http_response = HttpParser(httpParserTypes.RESPONSE_PARSER)\n",
+    "while not http_response.is_complete:\n",
+    "    http_response.parse(http_client.recv().tobytes())\n",
+    "http_client.close()\n",
+    "\n",
+    "print(http_response.build_response())\n",
+    "\n",
+    "assert http_response.is_complete\n",
+    "assert http_response.code == b'301'\n",
+    "assert http_response.reason == b'Moved Permanently'\n",
+    "assert http_response.has_header(b'location')\n",
+    "assert http_response.header(b'location') == b'https://jaxl.com/'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's use `TcpServerConnection` to make a HTTPS web server request."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "https_client = TcpServerConnection('jaxl.com', 443)\n",
+    "https_client.connect()\n",
+    "https_client.wrap(hostname='jaxl.com')\n",
+    "\n",
+    "https_client.queue(memoryview(request))\n",
+    "https_client.flush()\n",
+    "\n",
+    "https_response = HttpParser(httpParserTypes.RESPONSE_PARSER)\n",
+    "while not https_response.is_complete:\n",
+    "    https_response.parse(https_client.recv().tobytes())\n",
+    "https_client.close()\n",
+    "\n",
+    "print(https_response.build_response())\n",
+    "\n",
+    "assert https_response.is_complete\n",
+    "assert https_response.code == b'200'\n",
+    "assert https_response.reason == b'OK'\n",
+    "assert https_response.has_header(b'content-type')\n",
+    "assert https_response.header(b'content-type') == b'text/html'"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "da9d6927d62b2b95bde149eedfbd5367cb7f465aad65a736f49c99ee3db39df7"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.10.0 64-bit ('venv310': venv)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.0"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/tutorial/http_parser.ipynb b/tutorial/http_parser.ipynb
new file mode 100644
index 0000000..d0800af
--- /dev/null
+++ b/tutorial/http_parser.ipynb
@@ -0,0 +1,182 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# HttpParser\n",
+    "\n",
+    "`HttpParser` class is at the heart of everything related to HTTP.  It is used by Web server and Proxy server core and their plugin eco-system.  As the name suggests, it is capable of parsing both HTTP request and response packets.  It can also parse HTTP look-a-like protocols like ICAP, SIP etc.  Most importantly, remember that `HttpParser` was originally written to handle HTTP packets arriving in the context of a proxy server and till date its default behavior favors the same flavor.\n",
+    "\n",
+    "Let's start by parsing a HTTP web request using `HttpParser`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n'\n"
+     ]
+    }
+   ],
+   "source": [
+    "from proxy.http.methods import httpMethods\n",
+    "from proxy.http.parser import HttpParser, httpParserTypes, httpParserStates\n",
+    "from proxy.common.constants import HTTP_1_1\n",
+    "\n",
+    "get_request = HttpParser(httpParserTypes.REQUEST_PARSER)\n",
+    "get_request.parse(b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n')\n",
+    "\n",
+    "print(get_request.build())\n",
+    "\n",
+    "assert get_request.is_complete\n",
+    "assert get_request.method == httpMethods.GET\n",
+    "assert get_request.version == HTTP_1_1\n",
+    "assert get_request.host == None\n",
+    "assert get_request.port == 80\n",
+    "assert get_request._url != None\n",
+    "assert get_request._url.remainder == b'/'\n",
+    "assert get_request.has_header(b'host')\n",
+    "assert get_request.header(b'host') == b'jaxl.com'\n",
+    "assert len(get_request.headers) == 1"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, let's parse a HTTP proxy request using `HttpParser`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n'\n",
+      "b'GET http://jaxl.com:80/ HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n'\n"
+     ]
+    }
+   ],
+   "source": [
+    "proxy_request = HttpParser(httpParserTypes.REQUEST_PARSER)\n",
+    "proxy_request.parse(b'GET http://jaxl.com/ HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n')\n",
+    "\n",
+    "print(proxy_request.build())\n",
+    "print(proxy_request.build(for_proxy=True))\n",
+    "\n",
+    "assert proxy_request.is_complete\n",
+    "assert proxy_request.method == httpMethods.GET\n",
+    "assert proxy_request.version == HTTP_1_1\n",
+    "assert proxy_request.host == b'jaxl.com'\n",
+    "assert proxy_request.port == 80\n",
+    "assert proxy_request._url != None\n",
+    "assert proxy_request._url.remainder == b'/'\n",
+    "assert proxy_request.has_header(b'host')\n",
+    "assert proxy_request.header(b'host') == b'jaxl.com'\n",
+    "assert len(proxy_request.headers) == 1"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Notice how `proxy_request.build()` and `proxy_request.build(for_proxy=True)` behave.  Also, notice how `proxy_request.host` field is populated for a HTTP proxy packet but not for the prior HTTP web request packet example.\n",
+    "\n",
+    "To conclude, let's parse a HTTPS proxy request"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "b'CONNECT / HTTP/1.1\\r\\nHost: jaxl.com:443\\r\\n\\r\\n'\n",
+      "b'CONNECT jaxl.com:443 HTTP/1.1\\r\\nHost: jaxl.com:443\\r\\n\\r\\n'\n"
+     ]
+    }
+   ],
+   "source": [
+    "connect_request = HttpParser(httpParserTypes.REQUEST_PARSER)\n",
+    "connect_request.parse(b'CONNECT jaxl.com:443 HTTP/1.1\\r\\nHost: jaxl.com:443\\r\\n\\r\\n')\n",
+    "\n",
+    "print(connect_request.build())\n",
+    "print(connect_request.build(for_proxy=True))\n",
+    "\n",
+    "assert connect_request.is_complete\n",
+    "assert connect_request.is_https_tunnel\n",
+    "assert connect_request.version == HTTP_1_1\n",
+    "assert connect_request.host == b'jaxl.com'\n",
+    "assert connect_request.port == 443\n",
+    "assert connect_request._url != None\n",
+    "assert connect_request._url.remainder == None\n",
+    "assert connect_request.has_header(b'host')\n",
+    "assert connect_request.header(b'host') == b'jaxl.com:443'\n",
+    "assert len(connect_request.headers) == 1"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Take Away\n",
+    "\n",
+    "- `host` and `port` attributes represent the `host:port` present in the HTTP packet request line.  For comparison purposes, below are all the three request lines again.  Notice how there is no `host:port` available only for the web server get request\n",
+    "  | Request Type      | Request Line     |\n",
+    "  | ------------------| ---------------- |\n",
+    "  | `get_request`     | `GET / HTTP/1.1` |\n",
+    "  | `proxy_request`   | `GET http://jaxl.com HTTP/1.1` |\n",
+    "  | `connect_request` | `CONNECT jaxl.com:443 HTTP/1.1` |\n",
+    "- `_url` attribute is an instance of `Url` parser and contains parsed information about the URL found in the request line\n",
+    "\n",
+    "Few of the other handy properties within `HttpParser` are:\n",
+    "\n",
+    "- `is_complete`\n",
+    "- `is_http_1_1_keep_alive`\n",
+    "- `is_connection_upgrade`\n",
+    "- `is_https_tunnel`\n",
+    "- `is_chunked_encoded`\n",
+    "- `content_expected`\n",
+    "- `body_expected`"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "da9d6927d62b2b95bde149eedfbd5367cb7f465aad65a736f49c99ee3db39df7"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.10.0 64-bit ('venv310': venv)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.0"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/tutorial/welcome.ipynb b/tutorial/welcome.ipynb
new file mode 100644
index 0000000..12d0983
--- /dev/null
+++ b/tutorial/welcome.ipynb
@@ -0,0 +1,61 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Welcome\n",
+    "\n",
+    "## Background\n",
+    "\n",
+    "`proxy.py` was released on 20th August, 2013 as a single file HTTP proxy server implementation with no external dependencies.  See the [first commit](https://github.com/abhinavsingh/proxy.py/commit/75044a72d9c7b4b8910ba551006b801eafdf3c47) and [read introductory blog](https://abhinavsingh.com/proxy-py-a-lightweight-single-file-http-proxy-server-in-python/) to get an insight about why `proxy.py` was created.\n",
+    "\n",
+    "## Introduction\n",
+    "\n",
+    "Today, `proxy.py` has matured into a full blown networking library with focus on being lightweight, ability to deliver maximum performance while being extendible.  Unlike other Python servers, `proxy.py` doesn't need a `WSGI` or `UWSI` frontend, which then usually has to be placed behind a reverse proxy e.g. `Nginx` or `Apache`.  Of-course, `proxy.py` can be placed directly behind a load-balancer _(optionally capable of speaking HA proxy protocol)_.\n",
+    "\n",
+    "## Asyncio\n",
+    "\n",
+    "TBD\n",
+    "\n",
+    "## The Concept Of Work\n",
+    "\n",
+    "`proxy.py` core is written with a high level concept of `work`.\n",
+    "\n",
+    "- A running instance can receive `work` from one or multiple `sources`\n",
+    "  - Example, when `proxy.py` starts, an accepted client connection is a `work` coming from TCP socket `sources`\n",
+    "- Handlers can be written to process various types of `work`\n",
+    "  - Example, `HttpProtocolHandler` handles HTTP client connections `work`\n",
+    "- A client connection can come from a variety of `sources`\n",
+    "  - TCP sockets\n",
+    "  - UDP sockets\n",
+    "  - Unix sockets\n",
+    "  - Raw sockets\n",
+    "\n",
+    "In fact, `work` can be any processing unit.  It doesn't have to be a client connection.  Example:\n",
+    "\n",
+    "- A file on disk can act as the `source` and each line in that file as the `work` definition\n",
+    "- Imagine tailing a file on disk as `source` and processing each line as a separate `work` object\n",
+    "- If you want, each line in the file can also be a URL to be scrapped or download\n",
+    "- If you want, your `work` handlers can append new URLs _(discovered by scrapping previous URL entries)_ back in the file, creating an infinite feedback loop between the `work` processing core.\n",
+    "\n",
+    "And just like that we have created a web scraper!!!\n",
+    "\n",
+    "To extend this generic concept, now imagine a distributed queue as the `source` of our `work`, where each published message in the queue is our `work` payload.  Some examples of such `sources` can be:\n",
+    "- A `Redis` channel\n",
+    "- Google Cloud PubSub channel\n",
+    "- Kafka queues\n",
+    "\n",
+    "And just like that we have created a distributed `work` executor!!!"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}


Test patch:
diff --git a/tests/http/proxy/test_http_proxy_tls_interception.py b/tests/http/proxy/test_http_proxy_tls_interception.py
index 263265c..800cfb8 100644
--- a/tests/http/proxy/test_http_proxy_tls_interception.py
+++ b/tests/http/proxy/test_http_proxy_tls_interception.py
@@ -72,8 +72,8 @@ class TestHttpProxyTlsInterception(Assertions):
 
         # Do not mock the original wrap method
         self.mock_server_conn.return_value.wrap.side_effect = \
-            lambda x, y: TcpServerConnection.wrap(
-                self.mock_server_conn.return_value, x, y,
+            lambda x, y, as_non_blocking: TcpServerConnection.wrap(
+                self.mock_server_conn.return_value, x, y, as_non_blocking=as_non_blocking,
             )
 
         type(self.mock_server_conn.return_value).connection = \
diff --git a/tests/plugin/test_http_proxy_plugins_with_tls_interception.py b/tests/plugin/test_http_proxy_plugins_with_tls_interception.py
index 77345a4..c738be6 100644
--- a/tests/plugin/test_http_proxy_plugins_with_tls_interception.py
+++ b/tests/plugin/test_http_proxy_plugins_with_tls_interception.py
@@ -95,7 +95,9 @@ class TestHttpProxyPluginExamplesWithTlsInterception(Assertions):
 
         # Do not mock the original wrap method
         self.server.wrap.side_effect = \
-            lambda x, y: TcpServerConnection.wrap(self.server, x, y)
+            lambda x, y, as_non_blocking: TcpServerConnection.wrap(
+                self.server, x, y, as_non_blocking=as_non_blocking,
+            )
 
         self.server.has_buffer.side_effect = has_buffer
         type(self.server).closed = mocker.PropertyMock(side_effect=closed)