# Claude is a Ravenclaw

I'm in the midst of doing the [MATS program](https://www.matsprogram.org/) which has kept me super busy, but that didn't stop me working on resolving the most important question of our time: **What Hogwarts House does your chatbot belong to?**

Basically, I submitted each chatbot to the quiz at [https://harrypotterhousequiz.org](https://harrypotterhousequiz.org) and totted up the results using the [inspect framework](https://github.com/UKGovernmentBEIS/inspect_ai).

I sampled each question 20 times, and simulated the chances of each house getting the highest score.

Perhaps unsurprisingly, the vast majority of models prefer Ravenclaw, with the occasional model branching out to Hufflepuff. Differences seem to be idiosyncratic to models, not particular companies or model lines, which is surprising. Claude Opus 3 was the only model to favour Gryffindor - it always was a [bit different](https://x.com/ESYudkowsky/status/1940799880437682444).

The earth shattering nature of these results is so obvious it needs no further explanation.

Full Results
============

| **Model** | **Gryffindor** | **Hufflepuff** | **Ravenclaw** | **Slytherin** |     |
| --- | --- | --- | --- | --- | --- |
| `**anthropic/claude-3-haiku-20240307**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for anthropic/claude-3-haiku-20240307](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/04c193ac9f6bc504c6ec84711b9f5668550cef10ee01b029.png) |
| `**anthropic/claude-3-opus-latest**` | **48.7%** | **0.7%** | **50.6%** | **0.0%** | ![Pie chart for anthropic/claude-3-opus-latest](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/109686a74827566997b7dd5098fb959f194003995aad44fb.png) |
| `**anthropic/claude-3-5-haiku-latest**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for anthropic/claude-3-5-haiku-latest](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/aa500beceb4e50a81aaf3109288706d3e59668007106bc44.png) |
| `**anthropic/claude-3-5-sonnet-latest**` | **3.8%** | **17.3%** | **78.9%** | **0.0%** | ![Pie chart for anthropic/claude-3-5-sonnet-latest](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/94e0471149229450fe98550fa8fb6f2fae9c30a5eb92ee8e.png) |
| `**anthropic/claude-3-7-sonnet-latest**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for anthropic/claude-3-7-sonnet-latest](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/d82a1f1c0034c3def57a3b2c39cbab8acfd0ed1995d91352.png) |
| `**anthropic/claude-opus-4-0**` | **2.2%** | **50.4%** | **47.4%** | **0.0%** | ![Pie chart for anthropic/claude-opus-4-0](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/bd1e7f5dec932411412d7fa2f2edca541026e75b40d1ed28.png) |
| `**anthropic/claude-sonnet-4-0**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for anthropic/claude-sonnet-4-0](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/3d54bee7142959d2b631a8a57e77795cdb36f6ffc47a8d37.png) |
| `**deepseek/deepseek-r1-0528**` | **14.4%** | **20.0%** | **60.5%** | **5.0%** | ![Pie chart for deepseek/deepseek-r1-0528](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/97014d0e730623badca4f24135ee63da97f6504646fc7bb0.png) |
| `**google/gemini-2.5-flash**` | **0.0%** | **27.7%** | **72.3%** | **0.0%** | ![Pie chart for google/gemini-2.5-flash](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/c362d0fd99f7655932a21f10c0aaa5b04d4a41d7f9925abf.png) |
| `**meta-llama/llama-3.2-3b-instruct**` | **3.0%** | **3.0%** | **91.9%** | **2.1%** | ![Pie chart for meta-llama/llama-3.2-3b-instruct](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/a14abcad405f534d908d61aa7450b0a24cb67c25b107c21e.png) |
| `**meta-llama/llama-3.3-70b-instruct**` | **0.1%** | **86.5%** | **13.5%** | **0.0%** | ![Pie chart for meta-llama/llama-3.3-70b-instruct](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/0e11ef2cccda935e02d9895c64917a3546aa44b1e7aa4bdb.png) |
| `**openai/gpt-3.5-turbo**` | **7.6%** | **4.2%** | **84.2%** | **4.0%** | ![Pie chart for openai/gpt-3.5-turbo](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/55d20aed59ec61a8cd34543588c9d9b2152a8b702f41d3e9.png) |
| `**openai/gpt-4-turbo**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for openai/gpt-4-turbo](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/67f894d73788da30ba07e33fa91c4a415bbf4d766b35ab48.png) |
| `**openai/gpt-4o-mini**` | **0.0%** | **66.4%** | **33.6%** | **0.0%** | ![Pie chart for openai/gpt-4o-mini](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/84da2ee1254bf1e6ed4eedfde670199088692da7cd6c283a.png) |
| `**openai/o3-mini**` | **2.6%** | **0.0%** | **97.4%** | **0.0%** | ![Pie chart for openai/o3-mini](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/77b89566a1dab9e90b38d590b1c762c7c36324ced21efbdd.png) |
| `**qwen/qwen3-30b-a3b**` | **3.5%** | **1.9%** | **94.2%** | **0.3%** | ![Pie chart for qwen/qwen3-30b-a3b](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/a0f5364e194e28bfc3e8cec6a8f772494622dcde35131d50.png) |
| `**x-ai/grok-3**` | **0.0%** | **0.0%** | **100.0%** | **0.0%** | ![Pie chart for x-ai/grok-3](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/4872d9557df785a32572a754b7947fc232918d02904040d4.png) |

Further Results
===============

Thanks to the magic of a time-turner, I've also run the eval for some models released since this article's publication.

| Model | G   | H   | R   | S   |     |
| --- | --- | --- | --- | --- | --- |
| `**unsloth/Qwen2.5-7B-Instruct**` | **0.0%** | **48.9%** | **51.1%** | **0.0%** | ![Pie chart for vllm/unsloth/Qwen2.5-7B-Instruct](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/5b806f2cb25d3ffecc3992930296a4bf45230ba9e0a532de.png) |
| `**ModelOrganismsForEM/Qwen2.5-7B-Instruct_bad-medical-advice**` | **1.6%** | **6.6%** | **90.1%** | **1.7%** | ![Pie chart for vllm/bm](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/a787b0fe72120b1bc1a56eb8d224c890909d83934681ceec.png) |
| `**x-ai/grok-4**` | **14.7%** | **0.0%** | **85.3%** | **0.0%** | ![Pie chart for x-ai/grok-4](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/678a3baf9d6a6ce2b041bf876039e4139df6ef2165a9bc99.png) |
| `**moonshotai/kimi-k2**` | **0.7%** | **0.2%** | **99.2%** | **0.0%** | ![Pie chart for moonshotai/kimi-k2](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/da01354e4831ccf12e3b8cddd3fa788fe83052e3e132d897.png) |