Package: jetblack-upgradeable-streams
asyncio-upgradeable-streamsAn experiment in upgradeable streams.OverviewAn upgradeable stream starts life as a plain socket connection, but is capable
of being "upgraded" to TLS. This is sometimes known asSTARTTLS.
Common examples of this are SMTP, LDAP, and HTTP proxy tunneling with CONNECT.The asyncio library providesloop.start_tlsfor this purpose, however there is little information on how this can be used.This project provides an implementation ofasyncio.open_connectionandasyncio.start_serverwith an extra optional boolean parameterupgradeadble. When this is set the
TLS negotiation is deferred, and thewriterhas a new methodstart_tlswhich
can be called to upgrade the connection to TLS.This was tested using Python 3.9.7 on Ubuntu Linux 21.10.IssuesThe solution makes use of private variables in the python standard library which
may change at the will of the python library maintainer. In particular it has
to reset the reader in theStreamReaderProtocoland the transport in theStreamWriter.InstallationThis can be installed with pip.pipinstalljetblack-upgradeable-streamsExamplesThe following examples can be found in the "demos" folder. They expect a Linux
environment.ClientA new argumentupgradeablehas been added to theopen_connectionfunction to enable upgrading. WhenupgradeableisTruethe TLS negotiation is deferred and thesslparameter is stored for use when
the connection is upgraded.
Thewriterhas a new methodstart_tlsto upgrade the connection to TLS.The client connects without TLS.First the client sends "PING" to the server. The server should respond
with "PONG".Next the client sends "STARTTLS" to instruct the server to upgrade the
connection to TLS. The client then calls thestart_tlsmethod on thewriterto negotiate the secure connection. The method returns a newreaderandwriter.Using the new writer the client sends "PING" to the server, this time over
the encrypted stream. The server should respond with "PONG".Finally the client sends "QUIT" to the server and closes the connection.importasyncioimportsocketimportsslfromupgradeable_streamsimportopen_connectionasyncdefstart_client():ctx=ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH,cafile='/etc/ssl/certs/ca-certificates.crt')host=socket.getfqdn()print("Connect to server as upgradeable")reader,writer=awaitopen_connection(host,10001,ssl=ctx,upgradeable=True)print(f"The writer ssl context is{writer.get_extra_info('sslcontext')}")print("Sending PING")writer.write(b'PING\n')response=(awaitreader.readline()).decode('utf-8').rstrip()print(f"Received:{response}")print("Sending STARTTLS")writer.write(b'STARTTLS\n')print("Upgrading the connection")# Upgradereader,writer=awaitwriter.start_tls()print(f"The writer ssl context is{writer.get_extra_info('sslcontext')}")print("Sending PING")writer.write(b'PING\n')response=(awaitreader.readline()).decode('utf-8').rstrip()print(f"Received:{response}")print("Sending QUIT")writer.write(b'QUIT\n')awaitwriter.drain()print("Closing client")writer.close()awaitwriter.wait_closed()print("Client disconnected")if__name__=='__main__':asyncio.run(start_client())ServerAn extra argumentupgradeablehas been added to thestart_serverfunction
to enable upgrading to TLS. Thesslcontext is stored for use when a client
connection is upgraded to TLS.
Thewriterhas a new methodstart_tlsto upgrade the connection to TLS.The server listens for client connections.On receiving a connection it enters a read loop.When the server receives "PING" it responds with "PONG".When the server receives "STARTTLS" it calls thestart_tlsmethod on thewriterto negotiate the TLS connection. The method returns a newreaderandwriter.When the server receives "QUIT" it closes the connection.The code expects certificate and key PEM files in "~/.keys/server.{crt,key}".importasynciofromasyncioimportStreamReader,StreamWriterfromos.pathimportexpanduserimportsocketimportsslfromtypingimportUnionfromupgradeable_streamsimportstart_server,UpgradeableStreamWriterasyncdefhandle_client(reader:StreamReader,writer:Union[UpgradeableStreamWriter,StreamWriter])->None:print("Client connected")whileTrue:request=(awaitreader.readline()).decode('utf8').rstrip()print(f"Read '{request}'")ifrequest=='QUIT':breakelifrequest=='PING':print("Sending pong")writer.write(b'PONG\n')awaitwriter.drain()elifrequest=='STARTTLS':ifnotisinstance(writer,UpgradeableStreamWriter):raiseValueError('writer not upgradeable')print("Upgrading connection to TLS")# Upgradereader,writer=awaitwriter.start_tls()print("Closing client")writer.close()awaitwriter.wait_closed()print("Client closed")asyncdefrun_server():ctx=ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)ctx.load_verify_locations(cafile="/etc/ssl/certs/ca-certificates.crt")ctx.load_cert_chain(expanduser("~/.keys/server.crt"),expanduser("~/.keys/server.key"))host=socket.getfqdn()print("Starting server as upgradeable")server=awaitstart_server(handle_client,host,10001,ssl=ctx,upgradeable=True)asyncwithserver:awaitserver.serve_forever()if__name__=='__main__':asyncio.run(run_server())DevelopmentPull requests are welcome. In particular anything to reduce the reliance on the
implementation details in the standard library.