Queen v5 “matriarch d20” — Crawlnet
A small language model trained from scratch. Pretraining data: pages the Crawlnet crawlers read on the crypto web (real browsers; 59% of the tokens trained), public market, funding and on-chain records written out as plain-text pages (6%), study notes a general-purpose LLM wrote for 113,078 of those pages (27%) and question/answer pairs a general-purpose LLM generated from those pages (8%); details in Pretraining data below. Tokenizer: Queen v4's, reused unchanged (trained on its crawler pages and record pages only). No pretrained weights, no FineWeb, no other corpus. Chat tuning: question/answer pairs generated by a general-purpose LLM from those pages (with the study notes' own Q&A), pairs cut from the pages and open-book conversations that answer from retrieved pages with a quote (details below).
| version | 5 (queen-v5) |
| parameters | 644.9M total, 414.2M non-embedding |
| architecture | nanochat GPT, depth 20, width 1280, 10 heads, context 2048, vocab 16384 |
| code | karpathy/nanochat @ 92d63d4e8bb4 (MIT) + crawlnet training/ |
| dataset | 310,412 pages exported 2026-10-10T20:34:46.276Z + 45,664 pages of records-v3 (exported 2026-10-06T11:30:18.157Z); after cleaning 269,164 pages: 223,537 crawler pages from 6649 domains + 45,627 record pages |
| dataset manifest sha256 | be1297fa3c84851be7887c58d9ff594bf8ba9126ffdba6fbf39eb492452485b5; records-v3: ed71dc6d01aa9e70b26e079fcb6b42bdc12cba849a21ad9028d243deeff67c49 |
| pretraining mix | crawler pages 59%, record pages 6%, study notes written by a general-purpose LLM 27% and Q&A pairs generated by a general-purpose LLM 8% of the 2,339,400,113 tokens trained (see Pretraining data) |
| unique training tokens | 535,128,781: crawler pages 329,274,568, record pages 22,705,121, study notes 142,834,045, generated Q&A pairs 40,315,047 (each document counted once; +6,387,763 held out) |
| tokens trained | 2,339,373,056 = one pass over the prepared stream (4,462 steps; each document repeated by its planned passes, see Pretraining data), 5.65 tokens per non-embedding param |
| tokenizer | BPE, 16384 tokens, Queen v4's, reused unchanged (trained on its crawler pages and record pages only) |
| validation loss | 0.8304 bits/byte on the pages of 169 held-out sites, never trained on (pretraining; earlier versions held out other pages, so their numbers do not compare) |
| chat tuning (SFT) | synthetic+openbook: 2,390,706 question/answer pairs generated by a general-purpose LLM from 198,882 of 381,992 page chunks (52%; crawler pages and record pages): 792,707 written for Queen v4's chat tuning and 1,597,999 from the FAQ and Q&A of the study notes (see Pretraining data), plus the pairs cut from the pages. The generator was told to use only facts on each page, and a pair whose key words are not on its page was dropped, but its wording is in this model's chat behaviour. headings: 683,478 question/answer pairs cut out of the crawler pages themselves (FAQ questions found on pages, page headings, page titles; answers are verbatim page text). No external model was used. The only human-written words are six fixed question templates such as "Tell me about {heading}." Open book (RAFT-style): 211,975 conversations built from the generated pairs, with 1-6 passages in front of the question (its own page, unless left out, and pages the site's search finds for it); the reply copies a sentence from a passage, then answers (Quote: "<sentence>" / Answer: <answer>); in 15% none of the passages holds the answer and the reply is "The pages don't say."; in 3% one passage is a copy of the source page with its key fact changed, and the reply follows the trustworthy page or says "The pages disagree." The chat tuning trained on 2,342,880 generated pairs, 211,975 open-book conversations and 683,478 headings pairs, 1 epoch. |
| compute | 4x H100 SXM (rented), 1.272 h wall, 5.09 GPU-hours |
| weights | this folder |
Evaluation
Crypto knowledge (plain-frame log-likelihood multiple choice, acc_norm %; Queen rows are the chat-tuned releases, the other models their base checkpoints):
| model | cryptomc | cryptoterms | projectid | DMind (2,874 q) |
|---|---|---|---|---|
| queen-v5 d20 (this model, 645M) | 89.3 | 73.3 | 68.0 | 41.2 |
| queen-v4 (186M) | 61.3 | 43.3 | 43.5 | 32.3 |
| Qwen3.5-2B-Base | 46.0 | 66.0 | 31.0 | 31.0 |
| SmolLM3-3B-Base | 35.3 | 56.7 | 31.5 | 28.9 |
| Qwen3-1.7B-Base | 43.3 | 61.3 | 30.0 | 29.4 |
| Qwen3.5-0.8B-Base | 34.0 | 48.0 | 30.5 | 26.9 |
Every model is scored the same way: the question in a plain frame, each option's log-likelihood normalised by its length (acc_norm); 95% bootstrap intervals are about ±7 points on cryptomc/cryptoterms/projectid and ±1.7 on DMind. queen-v5 d20 vs queen-v4: better on all four, McNemar p < 0.001 each. cryptomc (150), cryptoterms (150) and projectid (200) come from crawled pages and were checked against the training data (matching documents left out); DMind (DMindAI/DMind_Benchmark objective set, 2,874 questions after removing ambiguous ones) was not part of that check, and on DMind picking the longest option scores 47.2, so treat it as a rough signal.
Reading pages from sites it never saw (400 questions about pages from 169 held-out sites (no page of those sites was trained on); the page and one page from another held-out site are put in front of the question in the model's own open-book format; % of answers holding every content word of the reference answer, and word F1):
| model | answer hit % | F1 |
|---|---|---|
| queen-v5 d20 (quote + answer, up to 6 passages) | 68.75 | 0.35 |
| queen-v4 (2-page book) | 61.25 | 0.24 |
queen-v5 d20 quoted a sentence in 91% of its answers (89% verbatim in the pages) and said "The pages don't say." in 9%. Without the pages it scores 17.8% (queen-v4: 11.3%).
LLM-graded answers (the 100 fixed crypto questions of queen_train/judge_questions.json, one seeded answer each (temperature 0.7, top-k 50), graded by a general-purpose LLM: mean correctness 0-2, share fully correct, share repetitive):
| model | correct (0-2) | fully correct % | repetitive % |
|---|---|---|---|
| queen-v5 d20, with retrieved pages (closed book when the pages don't answer) | 1.44 | 50.00 | 4.00 |
| queen-v5 d20, no pages | 1.53 | 55.00 | 5.00 |
| queen-v4, with retrieved pages | 1.06 | 23.00 | 8.00 |
| queen-v4, no pages | 0.86 | 15.00 | 33.00 |
With pages: the site's search over the crawl, as served. When queen-v5's open-book reply is "The pages don't say." (45 of 100 here: the search returned no page answering a general question) the question is asked again without pages (Queen v5's serving code does the same). Lower is better for repetitive.
Pretraining data
| part | what | unique tokens | tokens trained | share |
|---|---|---|---|---|
| crawler pages | 273,395 chunks of the crawled pages, 2-8 passes by a quality label (64,583 chunks it rated junk were not trained) | 329,274,568 | 1,380,738,658 | 59.0% |
| record pages | 45,627 record pages written from public records (8 passes), 172,974 rewordings of them (4 passes), their fields as Q&A at the end of pretraining; templates, no LLM | 22,705,121 | 142,032,958 | 6.1% |
| study notes | an explanation, an FAQ, a list of facts and entities and a Q&A list for 113,078 pages, written by a general-purpose LLM from that page; explanation, FAQ and facts 4, Q&A 6 passes | 142,834,045 | 627,077,310 | 26.8% |
| generated Q&A pairs | 964,943 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, in Q&A documents: 2 passes, then regrouped in other Q&A formats at the end of pretraining | 40,315,047 | 189,551,187 | 8.1% |
| total | 535,128,781 | 2,339,400,113 | 100% |
Unique tokens: every document trained at least once, counted once (with its BOS token and source line). Tokens trained: every pass of it, plus the Q&A documents regrouped for the end of pretraining. The run trained 4,462 steps of 524,288 tokens = 2,339,373,056 of the stream's 2,339,400,113.
Contributors
Queen v5 was trained on pages found by these crawlers. Owners spawned theirs by burning $CRAWLNET; the wallets are the ones that made the burn, public on-chain. Since 9 Oct 2026 every crawl.md NFT is a crawler of its own: its pages are credited to the wallet that holds the NFT (its credits name when one is set).
| crawler | owner | pages | tokens |
|---|---|---|---|
| e02 | AcUm…vLRa (burn) |
1,057 | 2,824,892 |
| crawlzilla | 3dmq…dLBC (burn) |
1,180 | 2,502,120 |
| ch0pper | H8iW…VE3N (burn) |
1,107 | 2,426,241 |
| spiderman | 5WKN…Meic (burn) |
1,149 | 2,424,488 |
| spiderman | 5WKN…Meic (burn) |
1,117 | 2,377,666 |
| sparsity | GZUQ…F59x (burn) |
1,136 | 2,339,308 |
| ghero6 | FcEE…Cin4 (burn) |
999 | 2,335,066 |
| ghero5 | FcEE…Cin4 (burn) |
1,072 | 2,272,166 |
| turtletheory | ipMd…U4xb (burn) |
1,059 | 2,268,498 |
| ghero10 | FcEE…Cin4 (burn) |
1,091 | 2,260,444 |
| spiderman | 5WKN…Meic (burn) |
1,121 | 2,260,069 |
| jdubs | 3v2K…GTgA (burn) |
995 | 2,252,996 |
| needmorecrawls | 4rif…jxnZ (burn) |
879 | 2,247,308 |
| orb-weaver | 3Zgo…wfpF (burn) |
952 | 2,241,142 |
| aze2 | BX6f…WpEf (burn) |
1,123 | 2,237,784 |
| spiderman | 5WKN…Meic (burn) |
1,030 | 2,233,061 |
| pp | 3JVm…Fsm4 (burn) |
1,061 | 2,232,237 |
| ghero1 | FcEE…Cin4 (burn) |
1,073 | 2,219,824 |
| redacted003 | BvVr…URcY (burn) |
1,132 | 2,213,214 |
| ch0pper2 | H8iW…VE3N (burn) |
1,027 | 2,202,950 |
| dash | H2Lt…LbYp (burn) |
1,048 | 2,187,760 |
| deez | CDYb…2Xxv (burn) |
1,113 | 2,186,914 |
| near-weaver | Br2o…hHFG (burn) |
1,065 | 2,170,368 |
| spiderman | 5WKN…Meic (burn) |
1,124 | 2,160,363 |
| blackandwhitedogs | 2eek…pBky (burn) |
1,164 | 2,156,238 |
| fern | 2eek…pBky (burn) |
1,100 | 2,114,093 |
| 41n | EXTV…831R (burn) |
955 | 2,112,008 |
| mrnakamoto | 2eek…pBky (burn) |
1,087 | 2,101,178 |
| 1gjb | AyXY…HcDw (burn) |
991 | 2,093,570 |
| smokie | 8kwY…JY7G (burn) |
1,164 | 2,087,849 |
| test | zero…PPUT (burn) |
1,175 | 2,079,097 |
| wilbur | 2eek…pBky (burn) |
1,034 | 2,073,301 |
| ghero4 | FcEE…Cin4 (burn) |
1,130 | 2,055,853 |
| pp | 3JVm…Fsm4 (burn) |
1,015 | 2,042,977 |
| werkle-weaver | 3api…NA9T (burn) |
1,005 | 2,041,740 |
| onaizahparisnajd | 4KZT…HerD (burn) |
785 | 2,040,932 |
| manylegs | ETgB…wfpa (burn) |
1,019 | 2,035,744 |
| redacted002 | BvVr…URcY (burn) |
1,032 | 2,003,393 |
| v4 | GZUQ…F59x (burn) |
1,098 | 1,983,034 |
| charlotte | 2eek…pBky (burn) |
1,054 | 1,980,296 |
| sneaky | 6yNt…iikJ (burn) |
971 | 1,977,240 |
| e04 | AcUm…vLRa (burn) |
940 | 1,972,625 |
| lmaocry | H6GW…Hmdr (burn) |
993 | 1,970,881 |
| crawlzilla-2 | 3dmq…dLBC (burn) |
929 | 1,966,773 |
| ghero2 | FcEE…Cin4 (burn) |
999 | 1,945,696 |
| 007 | ipMd…U4xb (burn) |
917 | 1,940,709 |
| littledog | 2eek…pBky (burn) |
1,122 | 1,937,464 |
| weaver | F5Vd…3hVn (burn) |
1,000 | 1,923,618 |
| snoopdog | DRmh…ow3V (burn) |
956 | 1,917,627 |
| benny | 36ny…e6gG (burn) |
1,007 | 1,915,803 |
| e05 | AcUm…vLRa (burn) |
937 | 1,906,596 |
| web-reaper | Krrc…fSaT (burn) |
1,019 | 1,896,889 |
| aze | BX6f…WpEf (burn) |
1,081 | 1,893,069 |
| yancy | DPXE…XNqJ (burn) |
1,043 | 1,892,999 |
| ruso | 4rif…jxnZ (burn) |
1,027 | 1,884,765 |
| onepellegrino | 2eek…pBky (burn) |
1,005 | 1,883,858 |
| addu | 4gff…ksBn (burn) |
842 | 1,879,223 |
| weenis | 36ny…e6gG (burn) |
995 | 1,872,181 |
| onaizahparisnajd | 4KZT…HerD (burn) |
818 | 1,870,452 |
| ghero9 | FcEE…Cin4 (burn) |
1,017 | 1,865,507 |
| redacted | BvVr…URcY (burn) |
782 | 1,861,941 |
| ghero8 | FcEE…Cin4 (burn) |
1,005 | 1,859,698 |
| canis | FUy5…cjoG (burn) |
884 | 1,857,197 |
| page-pounder | Extb…VrFG (burn) |
961 | 1,848,770 |
| octangone01 | G64q…5PiU (burn) |
949 | 1,840,311 |
| e01 | AcUm…vLRa (burn) |
1,042 | 1,839,267 |
| aze5 | BX6f…WpEf (burn) |
905 | 1,835,067 |
| denn2 | CQFL…PBVr (burn) |
728 | 1,832,708 |
| redacted | BvVr…URcY (burn) |
706 | 1,829,885 |
| swolecrawls | F3QF…Dfrs (burn) |
1,065 | 1,823,712 |
| seig | 4gtD…rJMz (burn) |
966 | 1,813,995 |
| redacted | BvVr…URcY (burn) |
744 | 1,801,767 |
| aspen | 41CL…Q9fU (burn) |
933 | 1,795,027 |
| aze3 | BX6f…WpEf (burn) |
980 | 1,788,191 |
| e03 | AcUm…vLRa (burn) |
917 | 1,787,026 |
| kk143 | 8PRb…2Ufe (burn) |
905 | 1,773,791 |
| ghero3 | FcEE…Cin4 (burn) |
1,031 | 1,772,759 |
| jack | Diu8…atMh (burn) |
994 | 1,761,698 |
| elevatd | 3MRp…31K7 (burn) |
978 | 1,760,970 |
| smokiev2 | 8kwY…JY7G (burn) |
1,055 | 1,757,409 |
| gummy | 8zqX…Gv6M (burn) |
784 | 1,746,051 |
| callme | H5y3…BR5q (burn) |
739 | 1,739,292 |
| mr-tenacious2 | CQoH…u668 (burn) |
514 | 1,737,508 |
| warm | 6TAx…SE4b (burn) |
1,052 | 1,735,482 |
| gg | 4gff…ksBn (burn) |
778 | 1,732,887 |
| aze4 | BX6f…WpEf (burn) |
924 | 1,729,407 |
| morelegs | ETgB…wfpa (burn) |
1,075 | 1,727,131 |
| redacted001 | BvVr…URcY (burn) |
978 | 1,716,202 |
| spiderman | 5WKN…Meic (burn) |
978 | 1,713,503 |
| wei001 | 9ptc…3LbM (burn) |
679 | 1,710,350 |
| ruso-2 | 4rif…jxnZ (burn) |
820 | 1,708,623 |
| redacted | BvVr…URcY (burn) |
766 | 1,703,106 |
| onaizahparisnajd | 4KZT…HerD (burn) |
721 | 1,699,491 |
| chungus-spawn | H7oG…CsLj (burn) |
939 | 1,692,495 |
| 41n | EXTV…831R (burn) |
949 | 1,682,085 |
| peterparker | AbMD…52T4 (burn) |
1,014 | 1,674,818 |
| bigdog | 2eek…pBky (burn) |
1,068 | 1,667,649 |
| spiderman | FGyn…UuQt (burn) |
1,005 | 1,667,474 |
| babushka | Cj9h…Cf4C (burn) |
993 | 1,645,439 |
| spy | 8zQb…eogQ (burn) |
1,000 | 1,628,313 |
| onaizahparisnajd | 4KZT…HerD (burn) |
797 | 1,628,065 |
| onaizahs-kids | 4KZT…HerD (burn) |
780 | 1,627,030 |
| vinland | 3cSP…oYeU (burn) |
991 | 1,622,828 |
| jeff | 81Y8…GXSt (burn) |
1,052 | 1,622,506 |
| grand-cou | FGcD…k1Dc (burn) |
906 | 1,607,653 |
| onaizahparisnajd | 4KZT…HerD (burn) |
817 | 1,604,063 |
| spiderman | H8iW…VE3N (burn) |
1,010 | 1,602,702 |
| wei004 | 9ptc…3LbM (burn) |
686 | 1,599,798 |
| cash | H2Lt…LbYp (burn) |
892 | 1,590,369 |
| smokiev3 | 8kwY…JY7G (burn) |
935 | 1,587,650 |
| onaizah | 4KZT…HerD (burn) |
853 | 1,587,405 |
| onaizahparisnajd | 4KZT…HerD (burn) |
788 | 1,583,635 |
| onaizahparisnajd | 4KZT…HerD (burn) |
741 | 1,583,240 |
| onaizahparisnajd | 4KZT…HerD (burn) |
817 | 1,580,815 |
| redacted | BvVr…URcY (burn) |
765 | 1,580,252 |
| oreocakester | 36ny…e6gG (burn) |
983 | 1,566,133 |
| redacted | BvVr…URcY (burn) |
705 | 1,564,377 |
| ghero7 | FcEE…Cin4 (burn) |
971 | 1,562,955 |
| wakeme | H5y3…BR5q (burn) |
814 | 1,562,776 |
| onaizahparisnajd | 4KZT…HerD (burn) |
777 | 1,541,225 |
| onaizahparisnajd | 4KZT…HerD (burn) |
736 | 1,540,893 |
| onaizahparisnajd | 4KZT…HerD (burn) |
813 | 1,528,690 |
| son-of-chungus | H7oG…CsLj (burn) |
854 | 1,524,460 |
| wei009 | 9ptc…3LbM (burn) |
673 | 1,519,110 |
| aze6 | BX6f…WpEf (burn) |
868 | 1,506,774 |
| gigachadfuru | AbMD…52T4 (burn) |
948 | 1,506,707 |
| redacted | BvVr…URcY (burn) |
782 | 1,502,893 |
| goatishere2 | 58pZ…waHQ (burn) |
793 | 1,499,222 |
| 9ballshooter | APM9…89Nm (burn) |
766 | 1,495,765 |
| matrix-multiplier | 6s1H…36Jg (burn) |
752 | 1,489,270 |
| onaizahparisnajd | 4KZT…HerD (burn) |
748 | 1,486,068 |
| wei005 | 9ptc…3LbM (burn) |
774 | 1,483,716 |
| wei001 | 9ptc…3LbM (burn) |
739 | 1,475,995 |
| redacted | BvVr…URcY (burn) |
785 | 1,475,548 |
| redacted | BvVr…URcY (burn) |
805 | 1,465,311 |
| redacted | BvVr…URcY (burn) |
759 | 1,460,153 |
| gg | 4gff…ksBn (burn) |
787 | 1,457,708 |
| onaizahparisnajd | 4KZT…HerD (burn) |
727 | 1,448,371 |
| creepycrawly | 3v2K…GTgA (burn) |
756 | 1,435,258 |
| tarantula | BkPa…YiMW (burn) |
888 | 1,417,504 |
| 9ballshooter | APM9…89Nm (burn) |
723 | 1,416,477 |
| onaizahparisnajd | 4KZT…HerD (burn) |
776 | 1,411,053 |
| lost-again | sZWt…NVmN (burn) |
564 | 1,403,424 |
| redacted | BvVr…URcY (burn) |
834 | 1,401,158 |
| onaizahparisnajd | 4KZT…HerD (burn) |
731 | 1,384,550 |
| onaizahparisnajd | 4KZT…HerD (burn) |
795 | 1,384,504 |
| onaizahparisnajd | 4KZT…HerD (burn) |
704 | 1,374,786 |
| redacted | BvVr…URcY (burn) |
800 | 1,365,001 |
| history | G8zB…gepY (burn) |
703 | 1,362,113 |
| wei008 | 9ptc…3LbM (burn) |
638 | 1,361,015 |
| neet2 | 8X78…cmmd (burn) |
645 | 1,346,875 |
| funnelweb1 | DmsF…uvrF (burn) |
846 | 1,343,823 |
| onaizahparisnajd | 4KZT…HerD (burn) |
755 | 1,324,508 |
| onaizahparisnajd | 4KZT…HerD (burn) |
742 | 1,321,255 |
| redacted | BvVr…URcY (burn) |
800 | 1,299,059 |
| gg | 4gff…ksBn (burn) |
750 | 1,291,363 |
| wei010 | 9ptc…3LbM (burn) |
709 | 1,288,932 |
| redacted | BvVr…URcY (burn) |
765 | 1,285,682 |
| laquisha | 3v2K…GTgA (burn) |
682 | 1,263,745 |
| widowmaker | GRjD…spY8 (burn) |
739 | 1,257,835 |
| wei007 | 9ptc…3LbM (burn) |
643 | 1,257,336 |
| charl0tte | Ftgb…9DWW (burn) |
787 | 1,253,428 |
| goatishere | 58pZ…waHQ (burn) |
696 | 1,252,472 |
| onaizahparisnajd | 4KZT…HerD (burn) |
693 | 1,249,205 |
| onaizahskdis | 4KZT…HerD (burn) |
706 | 1,248,173 |
| redacted | BvVr…URcY (burn) |
756 | 1,240,953 |
| denn | CQFL…PBVr (burn) |
803 | 1,240,510 |
| bimboboy | 58pZ…waHQ (burn) |
707 | 1,224,330 |
| kaiyo | 4gff…ksBn (burn) |
781 | 1,211,809 |
| onaizahparisnajd | 4KZT…HerD (burn) |
704 | 1,201,130 |
| snaiduwa | 4KZT…HerD (burn) |
542 | 1,187,704 |
| wei006 | 9ptc…3LbM (burn) |
644 | 1,185,000 |
| neet1 | 8X78…cmmd (burn) |
682 | 1,178,465 |
| scorpion | BkPa…YiMW (burn) |
474 | 1,162,983 |
| wei002 | 9ptc…3LbM (burn) |
601 | 1,161,626 |
| redacted | BvVr…URcY (burn) |
648 | 1,119,157 |
| jeff2 | 81Y8…GXSt (burn) |
599 | 1,069,870 |
| redacted | BvVr…URcY (burn) |
663 | 996,979 |
| venom | Auqo…eajz (burn) |
432 | 993,794 |
| mr-tenacious | CQoH…u668 (burn) |
517 | 964,400 |
| cabbage-merchant | Chhk…W6uw (burn) |
455 | 919,322 |
| snaiduwa | 4KZT…HerD (burn) |
526 | 907,950 |
| amazing-grace | Chhk…W6uw (burn) |
459 | 814,668 |
| lost | sZWt…NVmN (burn) |
491 | 779,511 |
| crypto-scammer | Chhk…W6uw (burn) |
270 | 616,187 |
| crawl.md #0187, #0190, #0194, #0209, #0214, #0226 +74 | 2PfV…gnb6 |
19,040 | 31,310,652 |
| crawl.md #0003, #0010, #0038, #0042, #0048, #0052 +14 | BvVr…URcY |
4,066 | 7,005,545 |
| crawl.md #0011, #0014, #0015, #0016, #0076, #0091 +4 | ghero (FcEE…Cin4) |
2,279 | 3,869,467 |
| crawl.md #0066, #0245, #0284, #0289, #0411, #0423 +3 | Cj9h…Cf4C |
2,124 | 3,469,362 |
| crawl.md #0022, #0127, #0144, #0166, #0833, #0893 +1 | smokies (9JrE…K9du) |
1,548 | 2,685,369 |
| crawl.md #0189, #0213, #0355, #0365, #0528, #0639 +4 | GyJV…UZVs |
1,609 | 2,607,514 |
| crawl.md #0142, #0299, #0434, #0459, #0553, #0769 +2 | E5tZ…suX7 |
1,195 | 2,048,113 |
| crawl.md #0329, #0375, #0852, #0882, #0892 | HkWW…DEki |
977 | 1,796,065 |
| crawl.md #0149, #0463, #0574 | zXzM…q2zW |
698 | 1,130,015 |
| crawl.md #0030, #0156, #0915 | GhiG…3YTf |
646 | 1,102,978 |
| crawl.md #0480, #0506, #0598 | DruL…sxJZ |
722 | 1,098,516 |
| crawl.md #0130, #0781, #0904 | Dyr8…49zo |
643 | 1,094,426 |
| crawl.md #0001, #0065, #0106 | 58pZ…waHQ |
562 | 1,074,763 |
| crawl.md #0286, #0768, #0965 | 8VfP…iRdZ |
621 | 960,925 |
| crawl.md #0006, #0723 | FGyn…UuQt |
460 | 958,774 |
| crawl.md #0019, #0023, #0024 | benny (36ny…e6gG) |
636 | 933,393 |
| crawl.md #0036, #0040, #0087, #0102, #0124 | 9ptc…3LbM |
484 | 923,705 |
| crawl.md #0175, #0177 | freedl21 (Chhk…W6uw) |
447 | 919,985 |
| crawl.md #0018, #0085 | H5y3…BR5q |
521 | 910,492 |
| crawl.md #0139, #0155 | ETgB…wfpa |
549 | 907,858 |
| crawl.md #0230, #0730 | BePg…UsPk |
487 | 874,383 |
| crawl.md #0056, #0715, #0887 | H3cP…CLfE |
440 | 867,350 |
| crawl.md #0060, #0163, #0940 | GZUQ…F59x |
496 | 826,439 |
| crawl.md #0012, #0020 | APM9…89Nm |
456 | 805,250 |
| crawl.md #0692, #0731, #0805 | 8NTR…JHtx |
441 | 785,698 |
| crawl.md #0350, #0669 | 5iBb…WpG7 |
455 | 774,496 |
| crawl.md #0429, #0828, #0912 | B41i…Dywj |
461 | 688,084 |
| crawl.md #0044, #0559 | Krrc…fSaT |
426 | 663,372 |
| crawl.md #0533, #0563 | EaY6…QWJd |
352 | 619,915 |
| crawl.md #0271 | 9ujd…YDRN |
259 | 613,100 |
| crawl.md #0082, #0885 | 9vXj…wCUM |
377 | 608,057 |
| crawl.md #0158, #0800 | venom (Auqo…eajz) |
420 | 605,296 |
| crawl.md #0061, #0063, #0135, #0159, #0178 | 4gff…ksBn |
407 | 566,291 |
| crawl.md #0842 | BFhn…8pBp |
269 | 541,136 |
| crawl.md #0673, #0742 | Gvug…yd5J |
349 | 514,105 |
| crawl.md #0197, #0861 | 8e4j…sz8m |
312 | 507,439 |
| crawl.md #0028 | widowmaker (GRjD…spY8) |
250 | 500,266 |
| crawl.md #0035 | 0x1nine84 (3Zgo…wfpF) |
222 | 441,121 |
| crawl.md #0101 | AbMD…52T4 |
202 | 415,711 |
| crawl.md #0582 | 5xxd…4Hyn |
248 | 391,377 |
| crawl.md #0763 | CMST…9M1L |
223 | 372,262 |
| crawl.md #0138 | scorpionking (Ftgb…9DWW) |
251 | 364,169 |
| crawl.md #0169 | beiji (DPXE…XNqJ) |
183 | 346,225 |
| crawl.md #0683 | themonarch (3kvL…Hi2e) |
152 | 319,538 |
| crawl.md #0039 | grand_cou (FGcD…k1Dc) |
206 | 296,729 |
| crawl.md #0057 | tar0ntuluh (BkPa…YiMW) |
223 | 296,140 |
| crawl.md #0655 | auto (3Rur…3BRe) |
127 | 295,537 |
| crawl.md #0844 | bk201 (GBwm…k3Bf) |
186 | 291,433 |
| crawl.md #0025 | G64q…5PiU |
220 | 288,207 |
| crawl.md #0354 | 8c27…rhKt |
165 | 285,692 |
| crawl.md #0662 | 3Hua…a2hf |
242 | 281,724 |
| crawl.md #0059 | 9S69…NND6 |
204 | 277,621 |
| crawl.md #0520 | 51BY…1vrb |
199 | 275,198 |
| crawl.md #0916 | 8PRb…2Ufe |
156 | 266,918 |
| crawl.md #0064 | 5TYk…wwp7 |
178 | 263,630 |
| crawl.md #0345 | fatboy (FpSm…dhqH) |
150 | 257,441 |
| crawl.md #0029 | denn (CQFL…PBVr) |
133 | 242,823 |
| crawl.md #0239 | GKw3…x9vJ |
136 | 226,392 |
| crawl.md #0999 | F6TX…3sAZ |
142 | 211,022 |
| crawl.md #0458, #0575 | 5CX3…i9Mk |
133 | 210,889 |
| crawl.md #0323 | crawly (Extb…VrFG) |
178 | 202,508 |
| crawl.md #0383 | 8p9k…bmN2 |
121 | 200,063 |
| crawl.md #0906 | 95zk…FyGF |
120 | 197,448 |
| crawl.md #0802 | param (CnnY…tB9H) |
119 | 166,630 |
| crawl.md #0009, #0046 | EPDb…Jrxz |
73 | 141,034 |
| crawl.md #0602 | Gmfc…s5pq |
94 | 130,920 |
| crawl.md #0405 | 4P9z…cvQS |
70 | 95,224 |
| crawl.md #0321 | GY7a…9TNT |
57 | 67,577 |
| the queen's own crawlers (10) | — | 10,685 | 20,444,973 |
Pages and tokens count what made it into this version's dataset after cleaning and dedupe. Machine-readable: contributors.json. The 45,627 record pages (see below) were not found by a crawler and are not in this table.
What went in besides crawler pages
- The nanochat training code, architecture and hyperparameter rules (no weights, no data).
- Six hand-written question templates used to turn page headings into questions (see SFT above).
- Record pages: 45,627 pages, about 10,839,462 tokens, of public market, funding and on-chain records (projects and their funding rounds, investors, token unlocks and token facts, DeFi protocols, chains and stablecoins, DEX and on-chain token statistics), written out as plain text from public data. Only factual fields were kept: no third-party labels, ratings or risk scores, and no social-media posts. They are in the pretraining data, the tokenizer and the generated chat pairs; the pairs cut from page headings use crawler pages only.
- Study notes: a general-purpose LLM was given 113,950 crawled pages and wrote study notes from each: an explanation, an FAQ, a list of facts and entities and a Q&A list. A fact checker dropped every item with a number, address, URL or ticker that is not on its page (15,567 items); no page of a held-out site was given to it, and 5,050 notes that matched a benchmark question were left out. The notes of 113,078 pages are in the pretraining data and their FAQ and Q&A pairs in the chat tuning; their wording, and any mistake the checker could not catch, is in this model.
- Generated Q&A pairs in pretraining: 964,943 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, written as Q&A documents. 39,964 more (a fixed 2% of the questions and the earlier validation split) and 303 that matched a benchmark question never entered pretraining.
- Chat-tuning question/answer pairs generated by a general-purpose LLM from crawler pages and record pages, with the study notes' FAQ and Q&A pairs and open-book conversations built from them (see SFT above).
- The open-book frame of the chat tuning: two fixed replies, "The pages don't say." and "The pages disagree.", and the
Quote:/Answer:reply lines. - Nothing else. nanochat's default SFT mixture (SmolTalk, MMLU, GSM8K) and its CORE eval were switched off.
What it says
Fixed prompts, seeded sampling (temperature 0.7, top-k 50), unedited:
What is Bitcoin?
Bitcoin is a decentralized, peer-to-peer digital currency that operates on a public network without central authorities. It is the first and most widely adopted cryptocurrency, created in 2009 by an anonymous person or group under the pseudonym Satoshi Nakamoto.
Who created Bitcoin?
Bitcoin was created by an individual or group known as Satoshi Nakamoto, who published the Bitcoin whitepaper in 2008 and released the software in 2009.
What is a blockchain?
A blockchain is a distributed ledger where transactions are recorded in chronological blocks, maintained by a network of validators who use consensus algorithms to agree on and record transactions.
Explain proof of work.
Proof of work (pronounced "woot"[1]) is a consensus algorithm for permissionless blockchains that uses mining to decide on the ledger’s state. The mining process involves performing complex calculations that require significant computational power and energy. It is designed to be difficult to produce a hash value that meets the network’s difficulty target, allowing miners to be rewarded with block rewards. The concept of Proof of Work (PoW) was first formally introduced in 1993 for the first cryptocurrency, Bitcoin.
Explain proof of stake.
The proof of stake (PoS) consensus mechanism has been around since 2012, when Sunny King and Scott Nadal created the first major cryptocurrency as a response to the energy consumption issue that PoW has had. PoS is a consensus mechanism whereby participants of the network are required to stake (lock up) or lock up (stake) coins in order to get the ability to validate transactions. Validators are chosen based on the size of their stake. The more you stake, the higher your chances of being selected as a validator.
What is Ethereum?
Ethereum is the second-largest cryptocurrency by market capitalization and the most popular blockchain platform for creating decentralized applications.
What is a smart contract?
A smart contract is a computer protocol intended for the issuance and transfer of tokens, and it is a type of blockchain application.
What is gas on Ethereum?
Gas is the unit measuring the computational work needed to execute a transaction or smart contract on Ethereum, paid for in Ether (ETH).
What is Solana?
Solana is a high-performance blockchain platform designed to support scalable applications and decentralized finance (DeFi) and other blockchain-based services.
Why is Solana fast?
Solana is fast because it is a high-throughput Layer 1 built for frequent transactions. In its typical architecture, validators process transactions in parallel and reach consensus quickly. This model allows the network to handle thousands of operations per second at low cost. The speed is further supported by Proof of History, a cryptographic clock that orders events before they reach consensus. This approach enables the network to process thousands of transactions per second without sacrificing security.
Use it
chat.py in this folder needs only karpathy/nanochat (MIT) at the commit above:
pip install torch tiktoken safetensors
git clone https://github.com/karpathy/nanochat && git -C nanochat checkout 92d63d4e8bb4
PYTHONPATH=nanochat python chat.py "What is staking?"
Or load it yourself: config.json has the nanochat GPTConfig; build nanochat.gpt.GPT, load model.safetensors; the tokenizer is tokenizer.tiktoken (tiktoken rank file) + the pattern and special tokens in tokenizer.json. Chat format: <|bos|><|user_start|>...<|user_end|><|assistant_start|>.
Limitations
Tiny model, narrow data. It makes things up, including prices, dates, addresses and links. It is not financial advice and must not be used as a source of facts.
- Downloads last month
- 40