Repository: abhinavsingh/proxy.py
Pull request #925 (2021-12-31 08:53:28)

Optimize how `HttpProtocolHandler` delegates to the core plugins
Also now if no matching protocol handler implementation is found, `HttpProtocolHandler` will return a `BAD REQUEST` response.  Previously the connection would simply hang.  Example, send a web server request without enabling web server and connection will hang until it times out.  This causes issues in the production where rogue clients are always trying to establish a web server request on random IPs

codecov[bot]: # [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) Report
> Merging [#925](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (a4d2c65) into [develop](https://codecov.io/gh/abhinavsingh/proxy.py/commit/498a1bb84b4f8fc3fc56da379b1e9a5ea6976f48?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) (498a1bb) will **decrease** coverage by `0.45%`.
> The diff coverage is `89.04%`.

[![Impacted file tree graph](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/graphs/tree.svg?width=650&height=150&src=pr&token=Zh9J7b4la2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)

```diff
@@             Coverage Diff             @@
##           develop     #925      +/-   ##
===========================================
- Coverage    87.45%   87.00%   -0.46%     
===========================================
  Files          131      132       +1     
  Lines         6002     6001       -1     
  Branches       604      597       -7     
===========================================
- Hits          5249     5221      -28     
- Misses         646      666      +20     
- Partials       107      114       +7     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `86.96% <89.04%> (-0.32%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | Coverage Δ | |
|---|---|---|
| [tests/http/test\_protocol\_handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X3Byb3RvY29sX2hhbmRsZXIucHk=) | `100.00% <ø> (ø)` | |
| [proxy/http/handler.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9oYW5kbGVyLnB5) | `71.49% <82.97%> (-1.82%)` | :arrow_down: |
| [proxy/http/parser/parser.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9wYXJzZXIvcGFyc2VyLnB5) | `95.28% <100.00%> (+0.04%)` | :arrow_up: |
| [proxy/http/plugin.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9wbHVnaW4ucHk=) | `94.59% <100.00%> (+0.15%)` | :arrow_up: |
| [proxy/http/protocols.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9wcm90b2NvbHMucHk=) | `100.00% <100.00%> (ø)` | |
| [proxy/http/responses.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9yZXNwb25zZXMucHk=) | `92.30% <100.00%> (+0.30%)` | :arrow_up: |
| [proxy/http/server/web.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-cHJveHkvaHR0cC9zZXJ2ZXIvd2ViLnB5) | `59.47% <100.00%> (-3.93%)` | :arrow_down: |
| [tests/http/test\_http\_proxy.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X2h0dHBfcHJveHkucHk=) | `100.00% <100.00%> (ø)` | |
| [tests/http/test\_http\_proxy\_tls\_interception.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X2h0dHBfcHJveHlfdGxzX2ludGVyY2VwdGlvbi5weQ==) | `100.00% <100.00%> (+2.00%)` | :arrow_up: |
| [tests/http/test\_web\_server.py](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh#diff-dGVzdHMvaHR0cC90ZXN0X3dlYl9zZXJ2ZXIucHk=) | `100.00% <100.00%> (ø)` | |
| ... and [12 more](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Last update [498a1bb...a4d2c65](https://codecov.io/gh/abhinavsingh/proxy.py/pull/925?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Abhinav+Singh).


Patch:
diff --git a/README.md b/README.md
index 6050d85..e93f39b 100644
--- a/README.md
+++ b/README.md
@@ -2232,7 +2232,7 @@ usage: -m [-h] [--enable-events] [--enable-conn-pool] [--threadless]
           [--filtered-url-regex-config FILTERED_URL_REGEX_CONFIG]
           [--cloudflare-dns-mode CLOUDFLARE_DNS_MODE]
 
-proxy.py v2.4.0rc5.dev11+ga872675.d20211225
+proxy.py v2.4.0rc5.dev26+gb2b1bdc.d20211230
 
 options:
   -h, --help            show this help message and exit
diff --git a/docs/conf.py b/docs/conf.py
index 402e263..3b5c8bb 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -219,7 +219,11 @@ extlinks = {
 # -- Options for linkcheck builder -------------------------------------------
 
 linkcheck_ignore = [
-    r'http://localhost:\d+/',  # local URLs
+    # local URLs
+    r'http://localhost:\d+/',
+    # GHA sees "403 Client Error: Forbidden for url:"
+    # while the URL actually works
+    r'https://developers.cloudflare.com/',
 ]
 linkcheck_workers = 25
 
diff --git a/proxy/http/handler.py b/proxy/http/handler.py
index f46f594..aede947 100644
--- a/proxy/http/handler.py
+++ b/proxy/http/handler.py
@@ -16,20 +16,21 @@ import asyncio
 import logging
 import selectors
 
-from typing import Tuple, List, Union, Optional, Dict, Any
+from typing import Tuple, List, Type, Union, Optional, Dict, Any
 
-from .plugin import HttpProtocolHandlerPlugin
-from .parser import HttpParser, httpParserStates, httpParserTypes
-from .exception import HttpProtocolException
-
-from ..common.types import Readables, Writables
+from ..common.flag import flags
 from ..common.utils import wrap_socket
 from ..core.base import BaseTcpServerHandler
+from ..common.types import Readables, Writables
 from ..core.connection import TcpClientConnection
-from ..common.flag import flags
 from ..common.constants import DEFAULT_CLIENT_RECVBUF_SIZE, DEFAULT_KEY_FILE
 from ..common.constants import DEFAULT_SELECTOR_SELECT_TIMEOUT, DEFAULT_TIMEOUT
 
+from .exception import HttpProtocolException
+from .plugin import HttpProtocolHandlerPlugin
+from .responses import BAD_REQUEST_RESPONSE_PKT
+from .parser import HttpParser, httpParserStates, httpParserTypes
+
 
 logger = logging.getLogger(__name__)
 
@@ -78,7 +79,7 @@ class HttpProtocolHandler(BaseTcpServerHandler):
         self.selector: Optional[selectors.DefaultSelector] = None
         if not self.flags.threadless:
             self.selector = selectors.DefaultSelector()
-        self.plugins: Dict[str, HttpProtocolHandlerPlugin] = {}
+        self.plugin: Optional[HttpProtocolHandlerPlugin] = None
 
     ##
     # initialize, is_inactive, shutdown, get_events, handle_events
@@ -86,23 +87,16 @@ class HttpProtocolHandler(BaseTcpServerHandler):
     ##
 
     def initialize(self) -> None:
-        """Optionally upgrades connection to HTTPS, set ``conn`` in non-blocking mode and initializes plugins."""
+        """Optionally upgrades connection to HTTPS,
+        sets ``conn`` in non-blocking mode and initializes
+        HTTP protocol plugins.
+        """
         conn = self._optionally_wrap_socket(self.work.connection)
         conn.setblocking(False)
         # Update client connection reference if connection was wrapped
         if self._encryption_enabled():
             self.work = TcpClientConnection(conn=conn, addr=self.work.addr)
-        if b'HttpProtocolHandlerPlugin' in self.flags.plugins:
-            for klass in self.flags.plugins[b'HttpProtocolHandlerPlugin']:
-                instance: HttpProtocolHandlerPlugin = klass(
-                    self.uid,
-                    self.flags,
-                    self.work,
-                    self.request,
-                    self.event_queue,
-                    self.upstream_conn_pool,
-                )
-                self.plugins[instance.name()] = instance
+        # self._initialize_plugins()
         logger.debug('Handling connection %s' % self.work.address)
 
     def is_inactive(self) -> bool:
@@ -120,8 +114,8 @@ class HttpProtocolHandler(BaseTcpServerHandler):
             if self.selector and self.work.has_buffer():
                 self._flush()
             # Invoke plugin.on_client_connection_close
-            for plugin in self.plugins.values():
-                plugin.on_client_connection_close()
+            if self.plugin:
+                self.plugin.on_client_connection_close()
             logger.debug(
                 'Closing client connection %s has buffer %s' %
                 (self.work.address, self.work.has_buffer()),
@@ -153,8 +147,8 @@ class HttpProtocolHandler(BaseTcpServerHandler):
         # Get default client events
         events: Dict[int, int] = await super().get_events()
         # HttpProtocolHandlerPlugin.get_descriptors
-        for plugin in self.plugins.values():
-            plugin_read_desc, plugin_write_desc = plugin.get_descriptors()
+        if self.plugin:
+            plugin_read_desc, plugin_write_desc = self.plugin.get_descriptors()
             for rfileno in plugin_read_desc:
                 if rfileno not in events:
                     events[rfileno] = selectors.EVENT_READ
@@ -179,8 +173,8 @@ class HttpProtocolHandler(BaseTcpServerHandler):
         if teardown:
             return True
         # Invoke plugin.write_to_descriptors
-        for plugin in self.plugins.values():
-            teardown = await plugin.write_to_descriptors(writables)
+        if self.plugin:
+            teardown = await self.plugin.write_to_descriptors(writables)
             if teardown:
                 return True
         # Read from ready to read sockets
@@ -188,8 +182,8 @@ class HttpProtocolHandler(BaseTcpServerHandler):
         if teardown:
             return True
         # Invoke plugin.read_from_descriptors
-        for plugin in self.plugins.values():
-            teardown = await plugin.read_from_descriptors(readables)
+        if self.plugin:
+            teardown = await self.plugin.read_from_descriptors(readables)
             if teardown:
                 return True
         return False
@@ -209,33 +203,13 @@ class HttpProtocolHandler(BaseTcpServerHandler):
             # apply custom logic to handle request data sent after 1st
             # valid request.
             if self.request.state != httpParserStates.COMPLETE:
-                # Parse http request
-                #
-                # TODO(abhinavsingh): Remove .tobytes after parser is
-                # memoryview compliant
-                self.request.parse(data.tobytes())
-                if self.request.is_complete:
-                    # Invoke plugin.on_request_complete
-                    for plugin in self.plugins.values():
-                        upgraded_sock = plugin.on_request_complete()
-                        if isinstance(upgraded_sock, ssl.SSLSocket):
-                            logger.debug(
-                                'Updated client conn to %s', upgraded_sock,
-                            )
-                            self.work._conn = upgraded_sock
-                            for plugin_ in self.plugins.values():
-                                if plugin_ != plugin:
-                                    plugin_.client._conn = upgraded_sock
-                        elif isinstance(upgraded_sock, bool) and upgraded_sock is True:
-                            return True
+                if self._parse_first_request(data):
+                    return True
             else:
                 # HttpProtocolHandlerPlugin.on_client_data
                 # Can raise HttpProtocolException to tear down the connection
-                for plugin in self.plugins.values():
-                    optional_data = plugin.on_client_data(data)
-                    if optional_data is None:
-                        break
-                    data = optional_data
+                if self.plugin:
+                    data = self.plugin.on_client_data(data) or data
         except HttpProtocolException as e:
             logger.info('HttpProtocolException: %s' % e)
             response: Optional[memoryview] = e.response(self.request)
@@ -248,17 +222,13 @@ class HttpProtocolHandler(BaseTcpServerHandler):
         if self.work.connection.fileno() in writables and self.work.has_buffer():
             logger.debug('Client is write ready, flushing...')
             self.last_activity = time.time()
-
             # TODO(abhinavsingh): This hook could just reside within server recv block
             # instead of invoking when flushed to client.
             #
             # Invoke plugin.on_response_chunk
             chunk = self.work.buffer
-            for plugin in self.plugins.values():
-                chunk = plugin.on_response_chunk(chunk)
-                if chunk is None:
-                    break
-
+            if self.plugin:
+                chunk = self.plugin.on_response_chunk(chunk)
             try:
                 # Call super() for client flush
                 teardown = await super().handle_writables(writables)
@@ -305,6 +275,61 @@ class HttpProtocolHandler(BaseTcpServerHandler):
     # Internal methods
     ##
 
+    def _initialize_plugin(
+            self,
+            klass: Type['HttpProtocolHandlerPlugin'],
+    ) -> HttpProtocolHandlerPlugin:
+        """Initializes passed HTTP protocol handler plugin class."""
+        return klass(
+            self.uid,
+            self.flags,
+            self.work,
+            self.request,
+            self.event_queue,
+            self.upstream_conn_pool,
+        )
+
+    def _discover_plugin_klass(self, protocol: int) -> Optional[Type['HttpProtocolHandlerPlugin']]:
+        """Discovers and return matching HTTP handler plugin matching protocol."""
+        if b'HttpProtocolHandlerPlugin' in self.flags.plugins:
+            for klass in self.flags.plugins[b'HttpProtocolHandlerPlugin']:
+                k: Type['HttpProtocolHandlerPlugin'] = klass
+                if protocol in k.protocols():
+                    return k
+        return None
+
+    def _parse_first_request(self, data: memoryview) -> bool:
+        # Parse http request
+        #
+        # TODO(abhinavsingh): Remove .tobytes after parser is
+        # memoryview compliant
+        self.request.parse(data.tobytes())
+        if not self.request.is_complete:
+            return False
+        # Discover which HTTP handler plugin is capable of
+        # handling the current incoming request
+        klass = self._discover_plugin_klass(
+            self.request.http_handler_protocol,
+        )
+        if klass is None:
+            # No matching protocol class found.
+            # Return bad request response and
+            # close the connection.
+            self.work.queue(BAD_REQUEST_RESPONSE_PKT)
+            return True
+        assert klass is not None
+        self.plugin = self._initialize_plugin(klass)
+        # Invoke plugin.on_request_complete
+        output = self.plugin.on_request_complete()
+        if isinstance(output, bool):
+            return output
+        assert isinstance(output, ssl.SSLSocket)
+        logger.debug(
+            'Updated client conn to %s', output,
+        )
+        self.work._conn = output
+        return False
+
     def _encryption_enabled(self) -> bool:
         return self.flags.keyfile is not None and \
             self.flags.certfile is not None
diff --git a/proxy/http/parser/parser.py b/proxy/http/parser/parser.py
index 197923c..ecb774e 100644
--- a/proxy/http/parser/parser.py
+++ b/proxy/http/parser/parser.py
@@ -22,6 +22,7 @@ from ...common.flag import flags
 
 from ..url import Url
 from ..methods import httpMethods
+from ..protocols import httpProtocols
 from ..exception import HttpProtocolException
 
 from .protocol import ProxyProtocol
@@ -153,11 +154,10 @@ class HttpParser:
         self._url = Url.from_bytes(url)
         self._set_line_attributes()
 
-    def has_host(self) -> bool:
-        """Returns whether host line attribute was parsed or set.
-
-        NOTE: Host field WILL be None for incoming local WebServer requests."""
-        return self.host is not None
+    @property
+    def http_handler_protocol(self) -> int:
+        """Returns `HttpProtocols` that this request belongs to."""
+        return httpProtocols.HTTP_PROXY if self.host is not None else httpProtocols.WEB_SERVER
 
     @property
     def is_complete(self) -> bool:
diff --git a/proxy/http/plugin.py b/proxy/http/plugin.py
index 0180e5f..0b87d18 100644
--- a/proxy/http/plugin.py
+++ b/proxy/http/plugin.py
@@ -14,12 +14,12 @@ import argparse
 from abc import ABC, abstractmethod
 from typing import Tuple, List, Union, Optional, TYPE_CHECKING
 
-from .parser import HttpParser
-
 from ..common.types import Readables, Writables
 from ..core.event import EventQueue
 from ..core.connection import TcpClientConnection
 
+from .parser import HttpParser
+
 if TYPE_CHECKING:
     from ..core.connection import UpstreamConnectionPool
 
@@ -52,7 +52,7 @@ class HttpProtocolHandlerPlugin(ABC):
             flags: argparse.Namespace,
             client: TcpClientConnection,
             request: HttpParser,
-            event_queue: EventQueue,
+            event_queue: Optional[EventQueue],
             upstream_conn_pool: Optional['UpstreamConnectionPool'] = None,
     ):
         self.uid: str = uid
@@ -63,12 +63,10 @@ class HttpProtocolHandlerPlugin(ABC):
         self.upstream_conn_pool = upstream_conn_pool
         super().__init__()
 
-    def name(self) -> str:
-        """A unique name for your plugin.
-
-        Defaults to name of the class. This helps plugin developers to directly
-        access a specific plugin by its name."""
-        return self.__class__.__name__
+    @staticmethod
+    @abstractmethod
+    def protocols() -> List[int]:
+        raise NotImplementedError()
 
     @abstractmethod
     def get_descriptors(self) -> Tuple[List[int], List[int]]:
diff --git a/proxy/http/protocols.py b/proxy/http/protocols.py
new file mode 100644
index 0000000..976a418
--- /dev/null
+++ b/proxy/http/protocols.py
@@ -0,0 +1,33 @@
+# -*- coding: utf-8 -*-
+"""
+    proxy.py
+    ~~~~~~~~
+    ⚡⚡⚡ Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on
+    Network monitoring, controls & Application development, testing, debugging.
+
+    :copyright: (c) 2013-present by Abhinav Singh and contributors.
+    :license: BSD, see LICENSE for more details.
+
+    .. spelling::
+
+       http
+       iterable
+"""
+from typing import NamedTuple
+
+
+HttpProtocols = NamedTuple(
+    'HttpProtocols', [
+        # Web server handling HTTP/1.0, HTTP/1.1, HTTP/2, HTTP/3
+        # over plain Text or encrypted connection with clients
+        ('WEB_SERVER', int),
+        # Proxies handling HTTP/1.0, HTTP/1.1, HTTP/2 protocols
+        # over plain text connection or encrypted connection
+        # with clients
+        ('HTTP_PROXY', int),
+        # Proxies handling SOCKS4, SOCKS4a, SOCKS5 protocol
+        ('SOCKS_PROXY', int),
+    ],
+)
+
+httpProtocols = HttpProtocols(1, 2, 3)
diff --git a/proxy/http/proxy/server.py b/proxy/http/proxy/server.py
index e6f66e1..31b6a07 100644
--- a/proxy/http/proxy/server.py
+++ b/proxy/http/proxy/server.py
@@ -28,6 +28,7 @@ from .plugin import HttpProxyBasePlugin
 
 from ..headers import httpHeaders
 from ..methods import httpMethods
+from ..protocols import httpProtocols
 from ..plugin import HttpProtocolHandlerPlugin
 from ..exception import HttpProtocolException, ProxyConnectionFailed
 from ..parser import HttpParser, httpParserStates, httpParserTypes
@@ -162,6 +163,10 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
                 )
                 self.plugins[instance.name()] = instance
 
+    @staticmethod
+    def protocols() -> List[int]:
+        return [httpProtocols.HTTP_PROXY]
+
     def tls_interception_enabled(self) -> bool:
         return self.flags.ca_key_file is not None and \
             self.flags.ca_cert_dir is not None and \
@@ -169,8 +174,6 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
             self.flags.ca_cert_file is not None
 
     def get_descriptors(self) -> Tuple[List[int], List[int]]:
-        if not self.request.has_host():
-            return [], []
         r: List[int] = []
         w: List[int] = []
         if (
@@ -213,8 +216,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
                 teardown = plugin.write_to_descriptors(w)
                 if teardown:
                     return True
-        elif self.request.has_host() and \
-                self.upstream and not self.upstream.closed and \
+        elif self.upstream and not self.upstream.closed and \
                 self.upstream.has_buffer() and \
                 self.upstream.connection.fileno() in w:
             logger.debug('Server is write ready, flushing...')
@@ -250,8 +252,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
                 teardown = plugin.read_from_descriptors(r)
                 if teardown:
                     return True
-        elif self.request.has_host() \
-                and self.upstream \
+        elif self.upstream \
                 and not self.upstream.closed \
                 and self.upstream.connection.fileno() in r:
             logger.debug('Server is read ready, receiving...')
@@ -315,9 +316,6 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
         return False
 
     def on_client_connection_close(self) -> None:
-        if not self.request.has_host():
-            return
-
         context = {
             'client_ip': None if not self.client.addr else self.client.addr[0],
             'client_port': None if not self.client.addr else self.client.addr[1],
@@ -429,9 +427,6 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
 
     # Can return None to tear down connection
     def on_client_data(self, raw: memoryview) -> Optional[memoryview]:
-        if not self.request.has_host():
-            return raw
-
         # For scenarios when an upstream connection was never established,
         # let plugin do whatever they wish to.  These are special scenarios
         # where plugins are trying to do something magical.  Within the core
@@ -500,9 +495,6 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
         return raw
 
     def on_request_complete(self) -> Union[socket.socket, bool]:
-        if not self.request.has_host():
-            return False
-
         self.emit_request_complete()
 
         # Invoke plugin.before_upstream_connection
@@ -888,7 +880,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
     def emit_request_complete(self) -> None:
         if not self.flags.enable_events:
             return
-        assert self.request.port
+        assert self.request.port and self.event_queue
         self.event_queue.publish(
             request_id=self.uid,
             event_name=eventNames.REQUEST_COMPLETE,
@@ -923,6 +915,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
     def emit_response_headers_complete(self) -> None:
         if not self.flags.enable_events:
             return
+        assert self.event_queue
         self.event_queue.publish(
             request_id=self.uid,
             event_name=eventNames.RESPONSE_HEADERS_COMPLETE,
@@ -940,6 +933,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
     def emit_response_chunk_received(self, chunk_size: int) -> None:
         if not self.flags.enable_events:
             return
+        assert self.event_queue
         self.event_queue.publish(
             request_id=self.uid,
             event_name=eventNames.RESPONSE_CHUNK_RECEIVED,
@@ -953,6 +947,7 @@ class HttpProxyPlugin(HttpProtocolHandlerPlugin):
     def emit_response_complete(self) -> None:
         if not self.flags.enable_events:
             return
+        assert self.event_queue
         self.event_queue.publish(
             request_id=self.uid,
             event_name=eventNames.RESPONSE_COMPLETE,
diff --git a/proxy/http/responses.py b/proxy/http/responses.py
index 65b548b..3eaca1e 100644
--- a/proxy/http/responses.py
+++ b/proxy/http/responses.py
@@ -46,6 +46,18 @@ PROXY_AUTH_FAILED_RESPONSE_PKT = memoryview(
     ),
 )
 
+BAD_REQUEST_RESPONSE_PKT = memoryview(
+    build_http_response(
+        httpStatusCodes.BAD_REQUEST,
+        reason=b'BAD REQUEST',
+        headers={
+            b'Server': PROXY_AGENT_HEADER_VALUE,
+            b'Content-Length': b'0',
+        },
+        conn_close=True,
+    ),
+)
+
 NOT_FOUND_RESPONSE_PKT = memoryview(
     build_http_response(
         httpStatusCodes.NOT_FOUND,
diff --git a/proxy/http/server/web.py b/proxy/http/server/web.py
index eda6f7c..2316e71 100644
--- a/proxy/http/server/web.py
+++ b/proxy/http/server/web.py
@@ -27,6 +27,7 @@ from ..exception import HttpProtocolException
 from ..plugin import HttpProtocolHandlerPlugin
 from ..websocket import WebsocketFrame, websocketOpcodes
 from ..parser import HttpParser, httpParserTypes
+from ..protocols import httpProtocols
 from ..responses import NOT_FOUND_RESPONSE_PKT, NOT_IMPLEMENTED_RESPONSE_PKT, okResponse
 
 from .plugin import HttpWebServerBasePlugin
@@ -94,6 +95,10 @@ class HttpWebServerPlugin(HttpProtocolHandlerPlugin):
         if b'HttpWebServerBasePlugin' in self.flags.plugins:
             self._initialize_web_plugins()
 
+    @staticmethod
+    def protocols() -> List[int]:
+        return [httpProtocols.WEB_SERVER]
+
     def _initialize_web_plugins(self) -> None:
         for klass in self.flags.plugins[b'HttpWebServerBasePlugin']:
             instance: HttpWebServerBasePlugin = klass(
@@ -153,8 +158,6 @@ class HttpWebServerPlugin(HttpProtocolHandlerPlugin):
         return False
 
     def on_request_complete(self) -> Union[socket.socket, bool]:
-        if self.request.has_host():
-            return False
         path = self.request.path or b'/'
         # Routing for Http(s) requests
         protocol = httpProtocolTypes.HTTPS \
@@ -262,8 +265,6 @@ class HttpWebServerPlugin(HttpProtocolHandlerPlugin):
         return chunk
 
     def on_client_connection_close(self) -> None:
-        if self.request.has_host():
-            return
         context = {
             'client_ip': None if not self.client.addr else self.client.addr[0],
             'client_port': None if not self.client.addr else self.client.addr[1],


Test patch:
diff --git a/tests/http/test_http_proxy.py b/tests/http/test_http_proxy.py
index b1bc964..6c09776 100644
--- a/tests/http/test_http_proxy.py
+++ b/tests/http/test_http_proxy.py
@@ -47,8 +47,8 @@ class TestHttpProxyPlugin:
         )
         self.protocol_handler.initialize()
 
-    def test_proxy_plugin_initialized(self) -> None:
-        self.plugin.assert_called()
+    def test_proxy_plugin_not_initialized_unless_first_request_completes(self) -> None:
+        self.plugin.assert_not_called()
 
     @pytest.mark.asyncio    # type: ignore[misc]
     async def test_proxy_plugin_on_and_before_upstream_connection(self) -> None:
diff --git a/tests/http/test_http_proxy_tls_interception.py b/tests/http/test_http_proxy_tls_interception.py
index 23ae05f..0d30dd5 100644
--- a/tests/http/test_http_proxy_tls_interception.py
+++ b/tests/http/test_http_proxy_tls_interception.py
@@ -93,15 +93,8 @@ class TestHttpProxyTlsInterception(Assertions):
         )
         self.protocol_handler.initialize()
 
-        self.plugin.assert_called()
-        self.assertEqual(self.plugin.call_args[0][1], self.flags)
-        self.assertEqual(self.plugin.call_args[0][2].connection, self._conn)
-        self.proxy_plugin.assert_called()
-        self.assertEqual(self.proxy_plugin.call_args[0][1], self.flags)
-        self.assertEqual(
-            self.proxy_plugin.call_args[0][2].connection,
-            self._conn,
-        )
+        self.plugin.assert_not_called()
+        self.proxy_plugin.assert_not_called()
 
         connect_request = build_http_request(
             httpMethods.CONNECT, bytes_(netloc),
@@ -112,15 +105,15 @@ class TestHttpProxyTlsInterception(Assertions):
         self._conn.recv.return_value = connect_request
 
         # Prepare mocked HttpProtocolHandlerPlugin
-        async def asyncReturnBool(val: bool) -> bool:
-            return val
-        self.plugin.return_value.get_descriptors.return_value = ([], [])
-        self.plugin.return_value.write_to_descriptors.return_value = asyncReturnBool(False)
-        self.plugin.return_value.read_from_descriptors.return_value = asyncReturnBool(False)
-        self.plugin.return_value.on_client_data.side_effect = lambda raw: raw
-        self.plugin.return_value.on_request_complete.return_value = False
-        self.plugin.return_value.on_response_chunk.side_effect = lambda chunk: chunk
-        self.plugin.return_value.on_client_connection_close.return_value = None
+        # async def asyncReturnBool(val: bool) -> bool:
+        #     return val
+        # self.plugin.return_value.get_descriptors.return_value = ([], [])
+        # self.plugin.return_value.write_to_descriptors.return_value = asyncReturnBool(False)
+        # self.plugin.return_value.read_from_descriptors.return_value = asyncReturnBool(False)
+        # self.plugin.return_value.on_client_data.side_effect = lambda raw: raw
+        # self.plugin.return_value.on_request_complete.return_value = False
+        # self.plugin.return_value.on_response_chunk.side_effect = lambda chunk: chunk
+        # self.plugin.return_value.on_client_connection_close.return_value = None
 
         # Prepare mocked HttpProxyBasePlugin
         self.proxy_plugin.return_value.write_to_descriptors.return_value = False
@@ -143,15 +136,29 @@ class TestHttpProxyTlsInterception(Assertions):
 
         await self.protocol_handler._run_once()
 
+        # Assert correct plugin was initialized
+        self.plugin.assert_not_called()
+        self.proxy_plugin.assert_called_once()
+        self.assertEqual(self.proxy_plugin.call_args[0][1], self.flags)
+        # Actual call arg must be `_conn` object
+        # but because internally the reference is updated
+        # we assert it against `mock_ssl_wrap` which is
+        # called during proxy plugin initialization
+        # for interception
+        self.assertEqual(
+            self.proxy_plugin.call_args[0][2].connection,
+            self.mock_ssl_wrap.return_value,
+        )
+
         # Assert our mocked plugins invocations
-        self.plugin.return_value.get_descriptors.assert_called()
-        self.plugin.return_value.write_to_descriptors.assert_called_with([])
-        # on_client_data is only called after initial request has completed
-        self.plugin.return_value.on_client_data.assert_not_called()
-        self.plugin.return_value.on_request_complete.assert_called()
-        self.plugin.return_value.read_from_descriptors.assert_called_with([
-            self._conn.fileno(),
-        ])
+        # self.plugin.return_value.get_descriptors.assert_called()
+        # self.plugin.return_value.write_to_descriptors.assert_called_with([])
+        # # on_client_data is only called after initial request has completed
+        # self.plugin.return_value.on_client_data.assert_not_called()
+        # self.plugin.return_value.on_request_complete.assert_called()
+        # self.plugin.return_value.read_from_descriptors.assert_called_with([
+        #     self._conn.fileno(),
+        # ])
         self.proxy_plugin.return_value.before_upstream_connection.assert_called()
         self.proxy_plugin.return_value.handle_client_request.assert_called()
 
@@ -198,10 +205,10 @@ class TestHttpProxyTlsInterception(Assertions):
         )
 
         # Assert connection references for all other plugins is updated
-        self.assertEqual(
-            self.plugin.return_value.client._conn,
-            self.mock_ssl_wrap.return_value,
-        )
+        # self.assertEqual(
+        #     self.plugin.return_value.client._conn,
+        #     self.mock_ssl_wrap.return_value,
+        # )
         self.assertEqual(
             self.proxy_plugin.return_value.client._conn,
             self.mock_ssl_wrap.return_value,
diff --git a/tests/http/test_protocol_handler.py b/tests/http/test_protocol_handler.py
index 766208c..9adb0bb 100644
--- a/tests/http/test_protocol_handler.py
+++ b/tests/http/test_protocol_handler.py
@@ -190,7 +190,7 @@ class TestHttpProtocolHandler(Assertions):
         self.assertTrue(
             cast(
                 HttpProxyPlugin,
-                self.protocol_handler.plugins['HttpProxyPlugin'],
+                self.protocol_handler.plugin,
             ).upstream is not None,
         )
         self.assertEqual(
diff --git a/tests/http/test_web_server.py b/tests/http/test_web_server.py
index 9608335..f383b30 100644
--- a/tests/http/test_web_server.py
+++ b/tests/http/test_web_server.py
@@ -51,12 +51,11 @@ def test_on_client_connection_called_on_teardown(mocker: MockerFixture) -> None:
         flags=flags,
     )
     protocol_handler.initialize()
-    plugin.assert_called()
+    plugin.assert_not_called()
     mock_run_once = mocker.patch.object(protocol_handler, '_run_once')
     mock_run_once.return_value = True
     protocol_handler.run()
     assert _conn.closed
-    plugin.return_value.on_client_connection_close.assert_called()
 
 
 def mock_selector_for_client_read(self: Any) -> None: