Repository: mitmproxy/mitmproxy

Issue:
Cannot parse client hello
Hi!  I’m trying to capture the traffic in android10 machines with no rooting.
So I added the mitmproxy certificate as the user certificate.
It pretty works well in the Chrome browser but in https://play.google.com/store/apps/details?id=com.nhn.android.search&hl=ko this browser I got crash
with this link https://abr.ge/cctk0
I think it's async socket timing issue but I'm not sure about it. Is there some one can help to debug it?


::ffff:127.0.0.1:65290: Cannot parse Client Hello: TlsProtocolException('Cannot read raw Client Hello: TlsProtocolException(\'Unexpected EOF in TLS handshake:  b\\\'\\\\x01\\\\x00\\\\x01\\\\xfc\\\\x03\\\\x03\\\\xd6_\\\\x11\\\\x1fo\\\\xd3P\\\\xb1\\\\x86\\\\x18\\\\\\\\\\\\xf0(\\\\xbf\\\\xa2(X\\\\xda#q\\\\xe8\\\\x91\\\\x80\\\\xfd\\\\x0bAv7\\\\xe7\\\\x12\\\\x0b\\\\xbd :\\\\xd5\\\\xfdu\\\\x9bn\\\\xb7py-\\\\xec\\\\xc0\\\\xf0%\\\\x01\\\\xb3S\\\\x0f}\\\\xa1")w\\\\x06\\\\x17\\\\xe1\\\\x1d\\\\x16,\\\\xb5\\\\xdf*\\\\x00\\\\x1c**\\\\xc0+\\\\xc0/\\\\xc0,\\\\xc00\\\\xcc\\\\xa9\\\\xcc\\\\xa8\\\\xc0\\\\x13\\\\xc0\\\\x14\\\\x00\\\\x9c\\\\x00\\\\x9d\\\\x00/\\\\x005\\\\x00\\\\n\\\\x01\\\\x00\\\\x01\\\\x97\\\\x1a\\\\x1a\\\\x00\\\\x00\\\\xff\\\\x01\\\\x00\\\\x01\\\\x00\\\\x00\\\\x00\\\\x00\\\\x17\\\\x00\\\\x15\\\\x00\\\\x00\\\\x12ablog.airbridge.io\\\\x00\\\\x17\\\\x00\\\\x00\\\\x00#\\\\x00\\\\xb0\\\\x88\\\\xf4F\\\\xba\\\\x9c*\\\\xba\\\\x85\\\\x10\\\\\\\'h\\\\x17\\\\xe0y5)\\\\xf8\\\\x843Km\\\\xd5 \\\\x1cX98x\\\\xf66O7y{\\\\xc1\\\\xe1c:E\\\\xebL\\\\x18 L}\\\\x05`\\\\xb8!.\\\\xba\\\\xac\\\\x88]\\\\x0e7\\\\x0b\\\\xc0V\\\\xcd\\\\xd5/\\\\xb79\\\\x01o\\\\x0e\\\\xbc\\\\xd1\\\\x8a\\\\x13\\\\x82\\\\x8e\\\\x84;Z\\\\xdf\\\\xaf\\\\x15\\\\xfe\\\\xf9`2\\\\xe1\\\\x13C\\\\xbb\\\\x03\\\\x88\\\\xe1]%\\\\xe2\\\\r=(\\\\xf4\\\\x0e\\\\xb8\\\\xc3\\\\x08S\\\\xed\\\\xad\\\\xe3\\\\xca\\\\xd9p\\\\xb1\\\\x81\\\\x1c\\\\xfeU\\\\x82C\\\\xdd\\\\x0c\\\\xf8\\\\rZ\\\\xaf\\\\xeeD\\\\xf0\\\\x8eW\\\\x10[\\\\\\\\E9\\\\x95m\\\\\\\\\\\\xa8\\\\xef_\\\\x1a\\\\x1b\\\\xadbU\\\\xaa\\\\n\\\\xb8\\\\x89\\\\x85-\\\\xeav\\\\x1ev\\\\xd6\\\\xc5\\\\x8b\\\\xb91\\\\xcaI8\\\\xfe+\\\\xd3\\\\xdbi\\\\xc6\\\\x83\\\\xa6yN\\\\xb2\\\\xdbg\\\\xa6\\\\xf07\\\\x00\\\\r\\\\x00\\\\x14\\\\x00\\\\x12\\\\x04\\\\x03\\\\x08\\\\x04\\\\x04\\\\x01\\\\x05\\\\x03\\\\x08\\\\x05\\\\x05\\\\x01\\\\x08\\\\x06\\\\x06\\\\x01\\\\x02\\\\x01\\\\x00\\\\x05\\\\x00\\\\x05\\\\x01\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x12\\\\x00\\\\x00\\\\x00\\\\x10\\\\x00\\\\x0e\\\\x00\\\\x0c\\\\x02h2\\\\x08http/1.1\\\\x00\\\\x0b\\\\x00\\\\x02\\\\x01\\\\x00\\\\x00\\\\n\\\\x00\\\\n\\\\x00\\\\x08ZZ\\\\x00\\\\x1d\\\\x00\\\\x17\\\\x00\\\\x18\\\\x00\\\\x1b\\\\x00\\\\x03\\\\x02\\\\x00\\\\x02JJ\\\\x00\\\\x01\\\\x00\\\\x00\\\\x15\\\\x00`\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\'\')')
```
Traceback (most recent call last):
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/server.py", line 121, in handle
    root_layer()
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/modes/http_proxy.py", line 9, in __call__
    layer()
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/tls.py", line 283, in __call__
    layer()
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/http1.py", line 83, in __call__
    layer()
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/http.py", line 188, in __call__
    if not self._process_flow(flow):
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/http.py", line 260, in _process_flow
    return self.handle_regular_connect(f)
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/http.py", line 206, in handle_regular_connect
    layer()
  File "/Users/gunson/.pyenv/versions/3.7.2/envs/cross-tester/lib/python3.7/site-packages/mitmproxy/proxy/protocol/tls.py", line 265, in __call__
    self._client_hello.alpn_protocols or
AttributeError: 'NoneType' object has no attribute 'alpn_protocols'
mitmproxy has crashed!
Please lodge a bug report at: https://github.com/mitmproxy/mitmproxy
127.0.0.1:65290: clientdisconnect
```
It had same crash on 4.0.4 or 5 beta



Discussion:
What happens if you capture the handshake in Wireshark? Can you share a PCAP? This doesn't look like valid TLS.

![image](https://user-images.githubusercontent.com/11094461/69846015-0080a480-12b6-11ea-97af-80085ba0092f.png)
I filtered tcp.port where the client hello error occur.
[crash.pcapng.zip](https://github.com/mitmproxy/mitmproxy/files/3903873/crash.pcapng.zip)

Unfortunately I do not have an Android 10 device, but I tested with Firefox and curl - without seeing any problems.
I can fix the bug in mitmproxy so it doesn't crash any more - but the TLS ClientHello parsing error comes straight from the Kaitai definition. @ujjwal96 any ideas about this one?

Resolving patch:
diff --git a/mitmproxy/proxy/protocol/tls.py b/mitmproxy/proxy/protocol/tls.py
--- a/mitmproxy/proxy/protocol/tls.py
+++ b/mitmproxy/proxy/protocol/tls.py
@@ -242,6 +242,8 @@ def __call__(self):
                 self._client_hello = net_tls.ClientHello.from_file(self.client_conn.rfile)
             except exceptions.TlsProtocolException as e:
                 self.log("Cannot parse Client Hello: %s" % repr(e), "error")
+                # Without knowning the ClientHello we cannot proceed in this connection.
+                return
 
         # Do we need to do a server handshake now?
         # There are two reasons why we would want to establish TLS with the server now:


Test patch: