# Neel Nanda MATS Applications Open (Due Aug 29)

## TLDR

* I am looking for people who want to be supervised by me to write a mech interp paper. Apply [here](tinyurl.com/neel-mats-app) now! Due Aug 29[^5]
* Application task: Spend ~12 hours (max 20) working on a mechanistic interpretability research problem of your choice, and send me a write-up + executive summary of what you learned. (See [advice](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.75rygwi582jr#heading=h.xio8tfxoabqc), [details](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.sa4u6llpnkph), [past examples](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.qtl0g8o3ozvu) and [recommended problems](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv) in the doc)
* The top ~32 candidates will do a 5 week paid online [exploration phase](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.62dwfjn1hlrr#heading=h.4akfozqpegny) (Sept 29 - Oct 31) ending in a 2 week research sprint in pairs. 
    * Expect unstructured, self-driven learning
* The ~8 exploration phase candidates with the best sprint projects advance to the research phase, a 12 week paid and in-person program (Jan 5 - March 27)
    * I have 1.5 hr/week check-ins with each pair, supervising them as they write a paper.
    * The typical scholar publishes at least one co-first author paper at a top ML venue.
* All backgrounds & experience levels welcome - I want to work with the most promising people, not just those with the best credentials!
    * Past scholars include professors, undergrads with no mech interp experience, startup founders, and researchers with several great mech interp papers already


## Table of Contents

*The key details and FAQ are copied below for convenience, the rest are in the doc*

[Key Details](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.0#heading=h.nnv9xap1ctew)

[FAQ](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.0#heading=h.1gap5xk6ja4s)

[Application Task Details](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.sa4u6llpnkph)

[Advice on producing a good application in 20 hours](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.75rygwi582jr)

[What does a good application look like?](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.qtl0g8o3ozvu)

[Recommended research problems](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv)

[FAQ (Extended)](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.62dwfjn1hlrr)


## Key Details


* **Application task**: Spend **~12 hours (max 20)** trying tomake research progress **on a mechanistic interpretability problem of your choice**
    * **Submit** via [this form](https://forms.matsprogram.org/t/x58WNMHpL6us), due Fri Aug 29th 11:59pm PT
    * Please submit a **write-up and executive summary** showing me what **progress you made**and **what you learned**about the problem. 
        * I value **communication skill**, don’t rush the write up! The time limit has up to two additional hours for the executive summary. 
        * See examples of successful past write-ups [here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.qtl0g8o3ozvu#heading=h.3ic64kjn5c)
    * **See advice** on [approaching the application](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.75rygwi582jr#heading=h.xio8tfxoabqc), [how to use LLMs for research](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.75rygwi582jr#heading=h.pnhtf6g0al60), [recommended resources](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.75rygwi582jr#heading=h.rcheowqnjs76), and [how I evaluate applications](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.qtl0g8o3ozvu#heading=h.qjtf84k9ynyo)
        * You can take as much time as you want beforehand for general learning.
    * **My research interests have changed a fair bit from some of my prior work**, I detail these [here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv#heading=h.43zbrv5nq2w7), and provide a long list of problems I’m currently excited about [here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv#heading=h.thfo282c3aty).
    * I’m open to submissions of existing mech interp work, but hold these to a higher standard ([more info](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.sa4u6llpnkph#heading=h.7f3iti9n36qd))
    * If you’ve applied before, [see here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.62dwfjn1hlrr#heading=h.7bj6h5t4n588) for a summary of changes
* **Key dates**:
    * Applications due **Aug 29**
    * Decisions released **Sept 16**
    * Exploration phase **Sept 29 - Oct 31** (**5 week online** program for top **~32** candidates)
    * Research phase decisions **Nov 6**
    * Research phase **Jan 5 - March 27** (**12 week in-person** program for top **~8** candidates)
* **All experience levels welcome:** I want to work with the most promising people, not those who look best on paper.[^1]
    * In MATS 8.0, **5 of my 8** scholars had **minimal prior mech interp experience**, but have been doing fantastically - by halfway through the program, some of them had: 
        * Helped [understand](https://arxiv.org/abs/2506.11618) [emergent](https://arxiv.org/abs/2506.11613) [misalignment](https://www.alignmentforum.org/posts/gLDSqQm8pwNiq7qst/narrow-misalignment-is-hard-emergent-misalignment-is-easy) (i.e. why [training a model to write buggy code turns it into a Nazi](https://www.emergent-misalignment.com/)) and been interviewed about it by [MIT Tech Review](https://www.technologyreview.com/2025/06/18/1119042/openai-can-rehabilitate-ai-models-that-develop-a-bad-boy-persona/)
        * Explored new paradigms for **[interpreting](http://x.com/paulcbogdan/status/1938287361525436864) [reasoning](https://arxiv.org/abs/2506.19143) [models](https://www.thought-anchors.com/)**
    * At the other extreme, I’ve had scholars who **already had multiple great mech interp papers**, like **[Arthur Conmy](https://scholar.google.com/citations?user=n4HIyXQAAAAJ&hl=en&oi=ao)** & **[Josh Engels](https://scholar.google.com/citations?user=yVPnVK8AAAAJ&hl=en&oi=ao)**, who say I still added a fair amount of value


## FAQ


### Why might you want to apply?



* My core goal is to teach you **how to do great mechanistic interpretability research**. 
* I run the Google DeepMind mechanistic interpretability team and I have a lot of [experience supervising research](https://scholar.google.com/citations?user=GLnX3MkAAAAJ&hl=en&oi=ao). In the past 3 years, I have **mentored** **50 junior researchers** and **supervised** **30+ MATS papers**, and 15 top conference papers[^2].


* The program often helps scholars get into **mech interp careers**
    * Seven now do interpretability research at **frontier AGI labs**, including [Arthur Conmy](https://scholar.google.com/citations?user=n4HIyXQAAAAJ), who works for me **leading** the **GDM [Applied Interpretability Team](https://www.alignmentforum.org/posts/aG9e5tHfHmBnDqrDy/the-gdm-agi-safety-alignment-team-is-hiring-for-applied)**.
    * Two alumni **lead research teams** at the UK government's AI Security Institute
* Past scholars also do excellent research in the program itself, even those totally new to mech interp! Some highlights:
    * Showing [open source LLMs can be cheaply jailbroken](https://arxiv.org/abs/2406.11717) with linear algebra, by ablating the refusal direction
        * This inspired projects at multiple frontier labs, including a [Meta paper](https://arxiv.org/abs/2409.20089) on fixing it.
    * [An ICLR oral](https://arxiv.org/abs/2411.14257) using sparse autoencoders to interpret hallucinations, and showing models can “recognise” entities they know facts about.
    * Using interpretability to [shape how models generalize](https://arxiv.org/abs/2507.16795) without changing any data, preventing [emergent misalignment](https://www.emergent-misalignment.com/)
    * The [first paper on transcoders](https://arxiv.org/abs/2406.11944), nine months before Anthropic's [well-known](https://transformer-circuits.pub/2025/attribution-graphs/methods.html) [papers](https://transformer-circuits.pub/2025/attribution-graphs/biology.html) on transcoders.
    * Work exploring [fundamental issues in sparse autoencoders](https://arxiv.org/abs/2502.04878) and [follow-up work that (mostly) fixed them](https://arxiv.org/abs/2503.17547).


### Why is this application so much effort? 



* I care a lot about being **meritocratic**. This way lets me find the best applicants, not just those who look good on paper. I do my best to assess your potential, not just what you’ve already done (though it’s still super noisy!)
* I've also tried to design this application process so that spending time on it is **useful whatever the outcome** - I don’t want to waste 12+ hours of your time!
* I think it's a pretty realistic **simulation of doing research**, especially if you haven’t done interpretability research before. Candidates often learn a lot, and are surprised by how much they can get done. 
    * I've sometimes heard from unsuccessful applicants that they enjoyed the application so much it convinced them to pursue a research career!
    * If you’re not sure if you’re interested in doing mech interp or not, I’d encourage you to try applying! I think you'll learn a lot from the application about whether it's a good fit.


### What am I looking for in an application?



* My ideal application is one that **teaches me something new**. 
    * This looks like identifying an interpretability hypothesis, gathering evidence for and against it, and writing up the evidence and analysis clearly.
* I value clear writing, good taste (ie choosing interesting problems and making good decisions), technical skill, truth-seeking, skepticism and pragmatism
* See a much more detailed explanation in [this tab](?tab=t.qtl0g8o3ozvu), along with past examples


### What happens in the program?

* The **top** **~32** candidates will do a **5 week online exploration phase** Sept 29 - Oct 31
    * The **final two weeks** (**full time**) are spent doing a **research sprint** in pairs. Admission to the research phase is largely based on sprint performance.
    * The **first three weeks** (**part time**) are the **preparation phase**. This means preparing for the sprint: self-driven skilling up, doing several day mini research projects with other scholars, going to talks/sessions, reading papers, etc. How you spend your time is up to you
    * More info [here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.62dwfjn1hlrr#heading=h.4akfozqpegny)
* The **top** **~8** candidates from the exploration phase will do a **12 week in-person research phase in Berkeley** Jan 5 - March 27
    * Scholars work in pairs to write a mech interp paper, with a **1.5 hr/week** check-in from me and some Slack support
    * All recent scholars have published this as a co-first author paper at a **top ML venue** (NeurIPS/ICLR/ICML) - see [lists of past work below](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.dwfjn1hlrr#heading=h.36de2cl4slo0) 
* Research phase participants often do an optional **3-12 month extension**, to finish their paper and sometimes publish a second.
* All phases include a **paid stipend**. Housing support is provided in the research phase
* See more info at [matsprogram.org](http://matsprogram.org) 


### What happens if I don’t get through to the research phase?



* While unfortunately most exploration phase candidates don’t make it to the research phase, I’ve designed the exploration phase to be **a valuable experience in its own right**, and to teach useful research skills. 
    * The median participant rates it as **1.5x-2.5x** the counterfactual use of time.
* In MATS 8.0:
    * 5 exploration-phase only scholars **found other MATS 8.0 mentors** as a result of participating
    * I helped **8-10 exploration phase-only** scholars **write papers** based on their sprint projects
* Candidates are welcome to try again in the next cohort


### Why *shouldn’t* I apply?



* Obviously, the application takes a while! If it doesn’t sound fun, you probably shouldn’t do it.
* The exploration phase of the program is fairly competitive, which some people find very stressful
    * Generally, participants seem to be nice and cooperative, especially since you want to form teams, but the awareness of your chances can be very stressful for some
* Most exploration phase events happen between 5pm-8pm UK time, which works badly for people in Asian time zones. But the events are not necessary for a valuable exploration phase!
* The exploration phase is very self-driven and unstructured - I provide good opportunities, resources, advice, etc and you all have each other as collaborators, but ultimately there’s one of me and 30+ of you. You get out what you put in and need to decide how to spend your time. This works great for some, poorly for others
* If you have a full-time job/are otherwise very busy, you may find it difficult to make time for the exploration phase.


### How should I choose a problem?



* I'm open to any application that shows strong research skill, but will be more excited about those matching my research interests
* **My research interests have** **changed a fair bit** from some of my past work - [more details below](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv#heading=h.43zbrv5nq2w7), but in brief I’m now fairly pessimistic about ambitious interpretability (i.e. complete reverse-engineering), and I’m excited about model biology (studying qualitative high-level properties of models) and applied interpretability (rigorously doing useful things with interp). I’m still interested in basic science, but have a higher bar.
    * Applications that surprise me with something new and cool are fantastic!
* I’m more agnostic about the best techniques, things like sparse autoencoders are a useful tool, but easy to waste effort using when a simpler method is sufficient or better - start by doing the obvious thing!
* I provide a long list of suggested problems [here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.knytn7x826kv#heading=h.thfo282c3aty)


### Can I use LLMs?



* Yes. In fact, I strongly recommend it! **LLMs are a crucial research tool** nowadays, and are especially useful for those getting into a new field.
    * More advice on using LLMs well [below](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.rygwi582jr#heading=h.pnhtf6g0al60)
* You're welcome to use them for coding, writing, etc, whatever you want - I want to gauge how well you’ll do as a researcher, which includes whatever tools you’d actually use. 
    * It is your responsibility to ensure your code and writing are high quality. Well-written write-ups are welcome. Docs that read like LLM slop will be rejected.
* I recommend using **[Cursor](http://cursor.com) for coding** (replacing eg VS Code) and using **[Gemini 2.5 Pro](http://aistudio.google.com)**[^3] for browser based tasks


* I've compiled a [folder of useful text files](https://drive.google.com/drive/u/0/folders/1GfrgKJwndk-twnJ8K7Ba-TE9i_8wBWAU) for mech interp research, containing a bunch of relevant docs & source code of key libraries, tutorials from ARENA and key libraries, key papers and my relevant blog posts.
    * By default, just **put [this 600k token file](https://drive.google.com/file/d/18cF3lkU17_elUSv0zk8KSVejM1jGfNnz/view?usp=drive_link) in Gemini’s context window**, which contains the most important documents[^4].

### How does a research supervisor add value?



* My model is that research requires a mix of skills. The day-to-day coding and execution is crucial. But there's also a set of harder-to-learn conceptual skills, collectively called [research taste](https://www.alignmentforum.org/posts/Ldrss6o3tiKT6NdMm/my-research-process-understanding-and-cultivating-research). These skills take a long time to gain because they have poor feedback loops, but they take very little time to use.
* My main role is to lend you my research taste and bootstrap your own. This looks like helping with:
    * High-level Strategy: Choosing a good problem, knowing when to pivot away from a dead end, or prioritizing which of several promising directions to pursue.
    * Experimental Design: Designing a clean experiment to conclusively test a hypothesis, thinking of alternative explanations for your results, or knowing when evidence is strong enough.
* Navigating the Field: I can also give pointers to relevant papers or techniques you might be missing, helping you avoid reinventing the wheel.
* Finally, some people find it very helpful to have a de-facto light-touch manager who provides validation, accountability, and clarity. 
* Past scholars have given me the feedback that I’m good at red-teaming, generating ideas, and being motivating and invested in their projects, but that I expect people to be able to work independently and can be fairly blunt with feedback.

*See a bunch more info and guidance in the other tabs of [the doc](tinyurl.com/neel-mats-app)*

[^1]:

     In my 4 most recent cohorts, I’ve had 3 independent researchers, 9 ML PhD students/recent PhD grads, 7 undergrads, 3 ML masters students, 5 former software engineers, 1 physics PhD student, 1 ML postdoc, 1 neuroscience postdoc, 2 quant traders, and 1 former entrepreneur

[^2]:

     Note that almost all scholars in recent cohorts have published at least one co-first author conference, and many of the 30 papers are too recent to have finished peer review - [list here](https://docs.google.com/document/d/1p-ggQV3vVWIQuCccXEl1fD0thJOgXimlbBpGk6FI32I/edit?tab=t.dwfjn1hlrr#heading=h.36de2cl4slo0). But my top priority is to help you do great research, publishing is a bonus.

[^3]:

     I'm not just saying this because I work for Google! It's a frontier model, it's free, it's pretty fast, and it can take a million tokens of context. The best paid models from other providers are also great choices but can’t take as much context.

[^4]:

     It starts with a table of contents explaining what’s in it.
[^5]: Other MATS apps open in late August