Scalable decentralised communication with Matrix org and Twisted
Channel: Association Francophone Python
Date: 2021-11-20

Matrix.org is an open source project that creates a new ecosystem for interoperable decentralised communication. Matrix publishes open standard HTTP APIs for synchronising arbitrary real-time data (chat messages, VoIP calls, IoT data etc) with no central point of control, maintaining cryptographic integrity by storing conversation history in Merkle DAGs and using state-of-the-art end-to-end encryption via the Olm cryptographic ratchet. The aim of Matrix is to defragment online communication and make interoperable messaging/VoIP/IoT as simple and ubiquitous as sending an email. As well as the standard itself, Matrix.org releases Apache-licensed reference implementations of Matrix servers, bridges, bots, clients etc. Our primary reference server implementation of Matrix is Synapse (https://github.com/matrix-org/synapse), which is built entirely in Python/Twisted. Since our initial release in September 2014, Synapse has grown to ~50KLOC and scales to ~300K users on the matrix.org server deployment and has hundreds of active deployments on the public internet. In this talk we’ll introduce Matrix and dig into both the good and bad sides of working with Twisted, the scalability challenges we’ve faced, how we fixed them and what we learned along the way!