# Computational Mechanics Hackathon (June 1 & 2)

Join our [Computational Mechanics Hackathon](https://www.apartresearch.com/event/compmech), organized with the support of [APART](https://www.apartresearch.com/event/compmech), [PIBBSS](https://pibbss.ai/) and [Simplex](https://simplexaisafety.com). 

  
This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!  
 

Read more and sign up for the event [here](https://www.apartresearch.com/event/compmech). 

We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In, [Transformers Represent Belief State Geometry in their Residual Stream](https://www.lesswrong.com/posts/gTZ2SxesbHckJ3CkF/transformers-represent-belief-state-geometry-in-their)  , we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.  
 

![](https://lh7-us.googleusercontent.com/ByOoi7uqCSfGF3e_kH7GMTpPiHVNoUQlPvMqux-6xsBf0knghpFhF9gKx66lMivAsNe2PigeBNy-EQitbQr5wTSjhSqhWafR7w17CZI7bo1vR-2jQzw8QyHtvGrluiQC1QCLm3uMKYPlzCqalhORJ1w)

This opens up a large number of potential projects in interpretability. There’s a lot of work to do!

Key things to know: 

*   **Dates: **Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st 
*   **Format: **Hybrid — join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
*   **Program: **
    *   **Keynote Opening** by [@Adam Shai](https://www.lesswrong.com/users/adam-shai?mention=user)  and [@Paul Riechers](https://www.lesswrong.com/users/paul-riechers?mention=user)  —  Friday 10:30 AM PDT
    *   **Online Office Hours with  Adam and Paul on Discord — Saturday and Sunday 10:30 PDT **
    *   **Ending session **— Sunday at 17:30 PDT
    *   **Project presentations **— Wednesday at 10:30 PDT
*   **Projects: **
    *   After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.
*   **Sign up: **You can sign up on this [website](https://www.apartresearch.com/event/compmech). After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
*   **Resources**: 
    *   You’re welcome to engage with this selection of [resources](https://www.apartresearch.com/event/compmech) before the hackathon starts. 
    *   Check out our (living) [**Open Problems in Comp Mech**](https://docs.google.com/document/u/0/d/1SwfWquXIvGaV-KArE7g_YOLxIgwQ8RWRS1E0_vUOEGc/edit)document, and in particular the section with **Shovel Ready Problems.**
    *   If you are starting a project or just want to express interest in it, fill out a row in [this spreadsheet ](https://docs.google.com/spreadsheets/d/1YGaOiikW9Lw5UckkKj265Wocj_qZMW6SmHroHKaHp-E/edit#gid=0)