Repository: abhinavsingh/proxy.py
Pull request #751 (2021-11-18 11:53:08)

Add all IANA assigned HTTP method names
Additionally:
- Fixes the additional `v` prefix in printed version
- Now every `make` invocation regenerates `proxy/common/_scm_version.py` for latest version string

codecov[bot]: # [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) Report
> Merging [#751](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (195ee57) into [develop](https://codecov.io/gh/abhinavsingh/proxy.py/commit/658acd822c0fb769c79ba8744209d93b7222e76c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (658acd8) will **decrease** coverage by `0.23%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/graphs/tree.svg?width=650&height=150&src=pr&token=Zh9J7b4la2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)

```diff
@@             Coverage Diff             @@
##           develop     #751      +/-   ##
===========================================
- Coverage    87.87%   87.63%   -0.24%     
===========================================
  Files          124      124              
  Lines         5401     5402       +1     
  Branches       431      432       +1     
===========================================
- Hits          4746     4734      -12     
- Misses         569      575       +6     
- Partials        86       93       +7     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `87.63% <100.00%> (-0.11%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | Coverage Δ | |
|---|---|---|
| [proxy/http/methods.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9tZXRob2RzLnB5) | `100.00% <ø> (ø)` | |
| [proxy/http/parser/parser.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9wYXJzZXIvcGFyc2VyLnB5) | `94.82% <100.00%> (+0.02%)` | :arrow_up: |
| [tests/http/test\_http\_parser.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X2h0dHBfcGFyc2VyLnB5) | `100.00% <100.00%> (ø)` | |
| [tests/http/test\_protocol\_handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X3Byb3RvY29sX2hhbmRsZXIucHk=) | `100.00% <100.00%> (ø)` | |
| [proxy/common/types.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29tbW9uL3R5cGVzLnB5) | `86.66% <0.00%> (-13.34%)` | :arrow_down: |
| [tests/http/test\_proxy\_protocol.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X3Byb3h5X3Byb3RvY29sLnB5) | `96.00% <0.00%> (-4.00%)` | :arrow_down: |
| [tests/test\_set\_open\_file\_limit.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvdGVzdF9zZXRfb3Blbl9maWxlX2xpbWl0LnB5) | `96.42% <0.00%> (-3.58%)` | :arrow_down: |
| [tests/testing/test\_embed.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvdGVzdGluZy90ZXN0X2VtYmVkLnB5) | `91.17% <0.00%> (-2.95%)` | :arrow_down: |
| [proxy/common/utils.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvY29tbW9uL3V0aWxzLnB5) | `91.12% <0.00%> (-2.42%)` | :arrow_down: |
| [proxy/testing/test\_case.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvdGVzdGluZy90ZXN0X2Nhc2UucHk=) | `98.00% <0.00%> (-2.00%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Last update [658acd8...195ee57](https://codecov.io/gh/abhinavsingh/proxy.py/pull/751?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).


Patch:
diff --git a/.gitignore b/.gitignore
index 2c194fa..d3d2055 100644
--- a/.gitignore
+++ b/.gitignore
@@ -26,4 +26,6 @@ cover
 htmlcov
 dist
 build
+
 proxy/public
+docs/_build
diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
index 84e4f57..6634e37 100644
--- a/CODE_OF_CONDUCT.md
+++ b/CODE_OF_CONDUCT.md
@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at mailsforabhinav@gmail.com. All
+reported by contacting the project team at mailsforabhinav+proxy@gmail.com. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.
diff --git a/Makefile b/Makefile
index 791eef1..9eebd4a 100644
--- a/Makefile
+++ b/Makefile
@@ -2,9 +2,8 @@ SHELL := /bin/bash
 
 NS ?= abhinavsingh
 IMAGE_NAME ?= proxy.py
-VERSION ?= v$(shell ./scm-version.sh)
 LATEST_TAG := $(NS)/$(IMAGE_NAME):latest
-IMAGE_TAG := $(NS)/$(IMAGE_NAME):$(VERSION)
+IMAGE_TAG := $(NS)/$(IMAGE_NAME):$(shell ./write-scm-version.sh)
 
 HTTPS_KEY_FILE_PATH := https-key.pem
 HTTPS_CERT_FILE_PATH := https-cert.pem
@@ -17,12 +16,14 @@ CA_SIGNING_KEY_FILE_PATH := ca-signing-key.pem
 
 .PHONY: all https-certificates sign-https-certificates ca-certificates
 .PHONY: lib-check lib-clean lib-test lib-package lib-coverage lib-lint lib-pytest
-.PHONY: lib-release-test lib-release lib-profile
-.PHONY: lib-dep, lib-flake8, lib-mypy, lib-scm-version
+.PHONY: lib-release-test lib-release lib-profile lib-doc
+.PHONY: lib-dep lib-flake8 lib-mypy
 .PHONY: container container-run container-release
 .PHONY: devtools dashboard dashboard-clean
 
-all: lib-test
+all:
+	echo $(IMAGE_TAG)
+	# lib-test
 
 https-certificates:
 	# Generate server key
@@ -82,14 +83,13 @@ lib-clean:
 	rm -rf .hypothesis
 
 lib-dep:
+	pip install --upgrade pip && \
 	pip install \
 		-r requirements.txt \
 		-r requirements-testing.txt \
 		-r requirements-release.txt \
-		-r requirements-tunnel.txt
-
-lib-scm-version:
-	@echo "version = '$(VERSION)'" > proxy/common/_scm_version.py
+		-r requirements-tunnel.txt \
+		-r docs/requirements.txt
 
 lib-lint:
 	python -m tox -e lint
@@ -114,6 +114,16 @@ lib-release-test: lib-package
 lib-release: lib-package
 	twine upload dist/*
 
+lib-doc:
+	pushd docs && \
+	python -m sphinx \
+		--keep-going \
+		-b dirhtml \
+		-d _build/doctrees \
+		-D language=en . _build/html && \
+	popd && \
+	open docs/_build/html/index.html
+
 lib-coverage:
 	pytest --cov=proxy --cov=tests --cov-report=html tests/
 	open htmlcov/index.html
diff --git a/README.md b/README.md
index 94695dd..59d6839 100644
--- a/README.md
+++ b/README.md
@@ -379,7 +379,7 @@ To start `proxy.py` from source code follow these instructions:
 - Install deps
 
   ```console
-  ❯ make lib-dep lib-scm-version
+  ❯ make lib-dep
   ```
 
 - Optionally, run tests
diff --git a/proxy/http/methods.py b/proxy/http/methods.py
index dbf1f2f..30c9b71 100644
--- a/proxy/http/methods.py
+++ b/proxy/http/methods.py
@@ -16,28 +16,91 @@
 from typing import NamedTuple
 
 
+# Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml
 HttpMethods = NamedTuple(
     'HttpMethods', [
+        ('ACL', bytes),
+        ('BASELINE_CONTROL', bytes),
+        ('BIND', bytes),
+        ('CHECKIN', bytes),
+        ('CHECKOUT', bytes),
+        ('CONNECT', bytes),
+        ('COPY', bytes),
+        ('DELETE', bytes),
         ('GET', bytes),
         ('HEAD', bytes),
+        ('LABEL', bytes),
+        ('LINK', bytes),
+        ('LOCK', bytes),
+        ('MERGE', bytes),
+        ('MKACTIVITY', bytes),
+        ('MKCALENDAR', bytes),
+        ('MKCOL', bytes),
+        ('MKREDIRECTREF', bytes),
+        ('MKWORKSPACE', bytes),
+        ('MOVE', bytes),
+        ('OPTIONS', bytes),
+        ('ORDERPATCH', bytes),
+        ('PATCH', bytes),
         ('POST', bytes),
+        ('PRI', bytes),
+        ('PROPFIND', bytes),
+        ('PROPPATCH', bytes),
         ('PUT', bytes),
-        ('DELETE', bytes),
-        ('CONNECT', bytes),
-        ('OPTIONS', bytes),
+        ('REBIND', bytes),
+        ('REPORT', bytes),
+        ('SEARCH', bytes),
         ('TRACE', bytes),
-        ('PATCH', bytes),
+        ('UNBIND', bytes),
+        ('UNCHECKOUT', bytes),
+        ('UNLINK', bytes),
+        ('UNLOCK', bytes),
+        ('UPDATE', bytes),
+        ('UPDATEREDIRECTREF', bytes),
+        ('VERSION_CONTROL', bytes),
+        ('STAR', bytes),
     ],
 )
 
 httpMethods = HttpMethods(
+    b'ACL',
+    b'BASELINE-CONTROL',
+    b'BIND',
+    b'CHECKIN',
+    b'CHECKOUT',
+    b'CONNECT',
+    b'COPY',
+    b'DELETE',
     b'GET',
     b'HEAD',
+    b'LABEL',
+    b'LINK',
+    b'LOCK',
+    b'MERGE',
+    b'MKACTIVITY',
+    b'MKCALENDAR',
+    b'MKCOL',
+    b'MKREDIRECTREF',
+    b'MKWORKSPACE',
+    b'MOVE',
+    b'OPTIONS',
+    b'ORDERPATCH',
+    b'PATCH',
     b'POST',
+    b'PRI',
+    b'PROPFIND',
+    b'PROPPATCH',
     b'PUT',
-    b'DELETE',
-    b'CONNECT',
-    b'OPTIONS',
+    b'REBIND',
+    b'REPORT',
+    b'SEARCH',
     b'TRACE',
-    b'PATCH',
+    b'UNBIND',
+    b'UNCHECKOUT',
+    b'UNLINK',
+    b'UNLOCK',
+    b'UPDATE',
+    b'UPDATEREDIRECTREF',
+    b'VERSION-CONTROL',
+    b'*',
 )
diff --git a/proxy/http/parser/parser.py b/proxy/http/parser/parser.py
index 151e037..cff81ab 100644
--- a/proxy/http/parser/parser.py
+++ b/proxy/http/parser/parser.py
@@ -304,16 +304,12 @@ class HttpParser:
 
     def _process_line(self, raw: bytes) -> None:
         if self.type == httpParserTypes.REQUEST_PARSER:
-            # Ref:
-            # https://datatracker.ietf.org/doc/html/rfc2616#section-5.1
-            # https://greenbytes.de/tech/webdav/rfc7230.html#request.line
-            # https://greenbytes.de/tech/webdav/rfc7231.html#methods
-            # http://www.iana.org/assignments/http-methods/http-methods.xhtml
             if self.protocol is not None and self.protocol.version is None:
                 # We expect to receive entire proxy protocol v1 line
                 # in one network read and don't expect partial packets
                 self.protocol.parse(raw)
             else:
+                # Ref: https://datatracker.ietf.org/doc/html/rfc2616#section-5.1
                 line = raw.split(WHITESPACE)
                 if len(line) == 3:
                     self.method = line[0].upper()
@@ -321,10 +317,12 @@ class HttpParser:
                     self.version = line[2]
                     self.state = httpParserStates.LINE_RCVD
                 else:
-                    # raise exception
-                    # TODO, it would be better to use raise HttpProtocolException,
-                    # but we should solve circular import problem first
-                    raise NotImplementedError('Invalid request line')
+                    # To avoid a possible attack vector, we raise exception
+                    # if parser receives an invalid request line.
+                    #
+                    # TODO: Better to use raise HttpProtocolException,
+                    # but we should solve circular import problem first.
+                    raise ValueError('Invalid request line')
         else:
             line = raw.split(WHITESPACE)
             self.version = line[0]
diff --git a/scm-version.sh b/scm-version.sh
deleted file mode 100755
index 5095791..0000000
--- a/scm-version.sh
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/bin/bash
-
-# Guessed Version 2.3.2.dev146+gad54132.d20211114
-python -m setuptools_scm --version | \
-    # 2.3.2.dev146+gad54132.d20211114
-    awk '{print $3}' | \
-    # 2.3.2.dev146-gad54132.d20211114
-    sed 's/\+/-/' | \
-    # 2.3.2.dev146-gad54132-d20211114
-    sed -E 's/(.*)\./\1-/' | \
-    # 2.3.2-dev146-gad54132-d20211114
-    sed -E 's/(.*)\./\1-/' | \
-    # 2.3.2-dev146-gad54132.d20211114
-    sed -E 's/(.*)-/\1\./' | \
-    # 2.3.2-dev146.gad54132.d20211114
-    sed -E 's/(.*)-/\1\./'
diff --git a/setup.cfg b/setup.cfg
index 499337f..db77dbb 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -7,7 +7,7 @@ description = ⚡⚡⚡Fast, Lightweight, Pluggable, TLS interception capable pr
 long_description = file: README.md
 long_description_content_type = text/markdown
 author = Abhinav Singh
-author_email = mailsforabhinav@gmail.com
+author_email = mailsforabhinav+proxy@gmail.com
 license = 'BSD'
 license_files =
   LICENSE.md
diff --git a/write-scm-version.sh b/write-scm-version.sh
new file mode 100755
index 0000000..320e8ca
--- /dev/null
+++ b/write-scm-version.sh
@@ -0,0 +1,46 @@
+#!/bin/bash
+#
+# write-scm-version.sh exists because `proxy.py``
+# auto-detects it's next version from git.  Hence,
+# for `proxy.py` to work `proxy/common/_scm_version.py`
+# file must be auto-generated with necessary information.
+#
+# For CI/CD, this file is generated via `tox` integration.
+# For local development (without editable install), you
+# must run this script to pre-populate `_scm_version.py`.
+#
+# This file is integrated by default within `Makefile`.
+# For every make target invocation, `_scm_version.py` file
+# will be re-written.
+
+# Guessed Version 2.3.2.dev146+gad54132.d20211114
+VERSION=$(python -m setuptools_scm --version | \
+    # 2.3.2.dev146+gad54132.d20211114
+    awk '{print $3}')
+
+# Store default IFS
+OLDIFS=$IFS
+
+IFS="+"
+set -- $VERSION
+SEMVER=$1
+DATE_AND_HASH=$2
+
+IFS="."
+set -- $SEMVER
+MAJOR=$1
+MINOR=$2
+PATCH=$3
+DISTANCE=$4
+
+# Reset IFS
+IFS=$OLDIFS
+
+echo "# coding: utf-8
+# file generated by setuptools_scm
+# don't change, don't track in version control
+version = '${VERSION}'
+version_tuple = (${MAJOR}, ${MINOR}, ${PATCH}, '${DISTANCE}', '${DATE_AND_HASH}')" > \
+    proxy/common/_scm_version.py
+
+echo $MAJOR.$MINOR.$PATCH.$DISTANCE


Test patch:
diff --git a/tests/http/test_http_parser.py b/tests/http/test_http_parser.py
index 4ef4412..8d97a8f 100644
--- a/tests/http/test_http_parser.py
+++ b/tests/http/test_http_parser.py
@@ -24,10 +24,10 @@ class TestHttpParser(unittest.TestCase):
         self.parser = HttpParser(httpParserTypes.REQUEST_PARSER)
 
     def test_issue_127(self) -> None:
-        with self.assertRaises(NotImplementedError):
+        with self.assertRaises(ValueError):
             self.parser.parse(CRLF)
 
-        with self.assertRaises(NotImplementedError):
+        with self.assertRaises(ValueError):
             raw = b'qwqrqw!@!#@!#ad adfad\r\n'
             while True:
                 self.parser.parse(raw)
diff --git a/tests/http/test_protocol_handler.py b/tests/http/test_protocol_handler.py
index 7ef30b7..c37740a 100644
--- a/tests/http/test_protocol_handler.py
+++ b/tests/http/test_protocol_handler.py
@@ -397,7 +397,8 @@ class TestHttpProtocolHandler(unittest.TestCase):
             b'Via: 1.1 proxy.py v%s' % bytes_(__version__),
             CRLF,
         ])
-        server.queue.assert_called_once_with(pkt)
+        server.queue.assert_called_once()
+        self.assertEqual(server.queue.call_args_list[0][0][0].tobytes(), pkt)
         server.buffer_size.return_value = len(pkt)
 
     def assert_data_queued_to_server(self, server: mock.Mock) -> None: