Queen v6 “queen” — Crawlnet
A small language model trained from scratch. Pretraining data: pages the Crawlnet crawlers read on the crypto web (real browsers; 65% of the tokens trained), public market, funding and on-chain records written out as plain-text pages (4%), study notes a general-purpose LLM wrote for 153,245 of those pages (24%) and question/answer pairs a general-purpose LLM generated from those pages (7%); details in Pretraining data below. Tokenizer: trained on this dataset's pages only (crawler pages and record pages). No pretrained weights, no FineWeb, no other corpus. Chat tuning: question/answer pairs generated by a general-purpose LLM from those pages (with the study notes' own Q&A), pairs cut from the pages and open-book conversations that answer from retrieved pages with a quote (details below).
| version | 6 (queen-v6) |
| parameters | 896.5M total, 435.2M non-embedding |
| architecture | nanochat GPT, depth 20, width 1280, 10 heads, context 2048, vocab 32768 |
| code | karpathy/nanochat @ 92d63d4e8bb4 (MIT) + crawlnet training/ |
| dataset | 560,669 pages exported 2026-10-14T12:45:40.698Z + 45,664 pages of records-v3 (exported 2026-10-06T11:30:18.157Z); after cleaning 433,197 pages: 387,568 crawler pages from 10958 domains + 45,629 record pages |
| dataset manifest sha256 | 41ffc99ab827eeb79c5c45ae005e0114f7a787b4443e22f2a0626b50d5095199; records-v3: ed71dc6d01aa9e70b26e079fcb6b42bdc12cba849a21ad9028d243deeff67c49 |
| pretraining mix | crawler pages 65%, record pages 4%, study notes written by a general-purpose LLM 24% and Q&A pairs generated by a general-purpose LLM 7% of the 3,309,666,269 tokens trained (see Pretraining data) |
| unique training tokens | 774,545,940: crawler pages 533,723,489, record pages 22,012,294, study notes 181,400,668, generated Q&A pairs 37,409,489 (each document counted once; +12,103,613 held out) |
| tokens trained | 3,309,305,856 = one pass over the prepared stream (6,312 steps; each document repeated by its planned passes, see Pretraining data), 7.6 tokens per non-embedding param |
| tokenizer | BPE, 32768 tokens, trained on this dataset's pages only (crawler pages and record pages) |
| validation loss | 0.7837 bits/byte on the pages of 267 held-out sites, never trained on (pretraining; earlier versions held out other pages, so their numbers do not compare) |
| chat tuning (SFT) | synthetic+openbook: 2,616,169 question/answer pairs generated by a general-purpose LLM from 213,805 of 580,990 page chunks (37%; crawler pages and record pages): 447,834 written for Queen v4's chat tuning and 2,168,335 from the FAQ and Q&A of the study notes (see Pretraining data), plus the pairs cut from the pages. The generator was told to use only facts on each page, and a pair whose key words are not on its page was dropped, but its wording is in this model's chat behaviour. headings: 1,213,799 question/answer pairs cut out of the crawler pages themselves (FAQ questions found on pages, page headings, page titles; answers are verbatim page text). No external model was used. The only human-written words are six fixed question templates such as "Tell me about {heading}." Open book (RAFT-style): 212,432 conversations built from the generated pairs, with 1-6 passages in front of the question (its own page, unless left out, and pages the site's search finds for it); the reply copies a sentence from a passage, then answers (Quote: "<sentence>" / Answer: <answer>); in 15% none of the passages holds the answer and the reply is "The pages don't say."; in 3% one passage is a copy of the source page with its key fact changed, and the reply follows the trustworthy page or says "The pages disagree." The chat tuning trained on 2,563,837 generated pairs, 212,432 open-book conversations and 932,304 headings pairs, 1 epoch. |
| compute | 8x H100 SXM (rented), 0.924 h wall, 7.393 GPU-hours |
| weights | this folder |
Evaluation
Crypto knowledge (plain-frame log-likelihood multiple choice, acc_norm %; Queen rows are the chat-tuned releases, the other models their base checkpoints):
| model | cryptomc | cryptoterms | projectid | DMind (2,874 q) |
|---|---|---|---|---|
| queen-v6 (this model, 897M) | 88.0 | 84.0 | 73.5 | 42.9 |
| Qwen3.5-2B-Base | 46.0 | 66.0 | 31.0 | 31.0 |
| SmolLM3-3B-Base | 35.3 | 56.7 | 31.5 | 28.9 |
| Qwen3-1.7B-Base | 43.3 | 61.3 | 30.0 | 29.4 |
| Qwen3.5-0.8B-Base | 34.0 | 48.0 | 30.5 | 26.9 |
Every model is scored the same way: the question in a plain frame, each option's log-likelihood normalised by its length (acc_norm); 95% bootstrap intervals are about ±7 points on cryptomc/cryptoterms/projectid and ±1.7 on DMind. queen-v6 vs queen-v5 d20: higher on DMind (McNemar p 0.02), cryptoterms (p 0.02) and projectid (p 0.08). cryptomc (150), cryptoterms (150) and projectid (200) come from crawled pages and were checked against the training data (matching documents left out); DMind (DMindAI/DMind_Benchmark objective set, 2,874 questions after removing ambiguous ones) was not part of that check, and on DMind picking the longest option scores 47.2, so treat it as a rough signal.
Reading pages from sites it never saw (400 questions about pages from 169 held-out sites (no page of those sites was trained on); the page and one page from another held-out site are put in front of the question in the model's own open-book format; % of answers holding every content word of the reference answer, and word F1):
| model | answer hit % | F1 |
|---|---|---|
| queen-v6 (quote + answer, up to 6 passages) | 68.25 | 0.35 |
queen-v6 quoted a sentence in 90% of its answers and said "The pages don't say." in 10%. Without the pages it scores 19.5% (queen-v5 d20: 17.8%).
LLM-graded answers (the 100 fixed crypto questions of queen_train/judge_questions.json, one seeded answer each (temperature 0.7, top-k 50), graded by a general-purpose LLM: mean correctness 0-2, share fully correct, share repetitive):
| model | correct (0-2) | fully correct % | repetitive % |
|---|---|---|---|
| queen-v6, no pages | 1.66 | 67.00 | 3.00 |
| queen-v5 d20, no pages | 1.53 | 55.00 | 5.00 |
No pages: the model answers from what it learned, the way the site serves its drafts. Lower is better for repetitive.
Pretraining data
| part | what | unique tokens | tokens trained | share |
|---|---|---|---|---|
| crawler pages | 472,712 chunks of the crawled pages, 2-8 passes by a quality label (64,558 chunks it rated junk were not trained) | 533,723,489 | 2,146,257,118 | 64.8% |
| record pages | 45,629 record pages written from public records (8 passes), 172,920 rewordings of them (4 passes), their fields as Q&A at the end of pretraining; templates, no LLM | 22,012,294 | 140,731,987 | 4.3% |
| study notes | an explanation, an FAQ, a list of facts and entities and a Q&A list for 153,245 pages, written by a general-purpose LLM from that page; explanation, FAQ and facts 4, Q&A 6 passes | 181,400,668 | 798,250,268 | 24.1% |
| generated Q&A pairs | 965,100 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, in Q&A documents: 2 passes, then regrouped in other Q&A formats at the end of pretraining | 37,409,489 | 224,426,896 | 6.8% |
| total | 774,545,940 | 3,309,666,269 | 100% |
Unique tokens: every document trained at least once, counted once (with its BOS token and source line). Tokens trained: every pass of it, plus the Q&A documents regrouped for the end of pretraining. The run trained 6,312 steps of 524,288 tokens = 3,309,305,856 of the stream's 3,309,666,269.
Contributors
Queen v6 was trained on pages found by these crawlers. Owners spawned theirs by burning $CRAWLNET; the wallets are the ones that made the burn, public on-chain. Since 9 Oct 2026 every crawl.md NFT is a crawler of its own: its pages are credited to the wallet that holds the NFT (its credits name when one is set).
| crawler | owner | pages | tokens |
|---|---|---|---|
| e02 | AcUm…vLRa (burn) |
1,058 | 2,663,194 |
| crawlzilla | 3dmq…dLBC (burn) |
1,180 | 2,325,410 |
| spiderman | 5WKN…Meic (burn) |
1,151 | 2,296,365 |
| ch0pper | H8iW…VE3N (burn) |
1,108 | 2,294,193 |
| spiderman | 5WKN…Meic (burn) |
1,116 | 2,242,115 |
| sparsity | GZUQ…F59x (burn) |
1,134 | 2,210,594 |
| ghero6 | FcEE…Cin4 (burn) |
999 | 2,149,738 |
| ghero5 | FcEE…Cin4 (burn) |
1,072 | 2,146,039 |
| spiderman | 5WKN…Meic (burn) |
1,120 | 2,129,547 |
| ghero10 | FcEE…Cin4 (burn) |
1,090 | 2,118,576 |
| needmorecrawls | 4rif…jxnZ (burn) |
879 | 2,118,308 |
| orb-weaver | 3Zgo…wfpF (burn) |
953 | 2,116,756 |
| turtletheory | ipMd…U4xb (burn) |
1,058 | 2,116,447 |
| aze2 | BX6f…WpEf (burn) |
1,122 | 2,111,835 |
| jdubs | 3v2K…GTgA (burn) |
997 | 2,106,211 |
| spiderman | 5WKN…Meic (burn) |
1,031 | 2,106,106 |
| pp | 3JVm…Fsm4 (burn) |
1,058 | 2,105,720 |
| redacted003 | BvVr…URcY (burn) |
1,130 | 2,088,556 |
| ch0pper2 | H8iW…VE3N (burn) |
1,027 | 2,084,092 |
| ghero1 | FcEE…Cin4 (burn) |
1,071 | 2,083,165 |
| deez | CDYb…2Xxv (burn) |
1,112 | 2,065,782 |
| dash | H2Lt…LbYp (burn) |
1,047 | 2,063,104 |
| near-weaver | Br2o…hHFG (burn) |
1,066 | 2,061,659 |
| spiderman | 5WKN…Meic (burn) |
1,123 | 2,041,685 |
| blackandwhitedogs | 2eek…pBky (burn) |
1,164 | 2,033,023 |
| fern | 2eek…pBky (burn) |
1,101 | 2,003,247 |
| 41n | EXTV…831R (burn) |
955 | 1,984,025 |
| mrnakamoto | 2eek…pBky (burn) |
1,085 | 1,983,146 |
| smokie | 8kwY…JY7G (burn) |
1,162 | 1,977,707 |
| test | zero…PPUT (burn) |
1,174 | 1,968,614 |
| wilbur | 2eek…pBky (burn) |
1,034 | 1,964,813 |
| 1gjb | AyXY…HcDw (burn) |
991 | 1,964,107 |
| onaizahparisnajd | 4KZT…HerD (burn) |
785 | 1,945,556 |
| pp | 3JVm…Fsm4 (burn) |
1,017 | 1,931,617 |
| werkle-weaver | 3api…NA9T (burn) |
1,003 | 1,931,549 |
| ghero4 | FcEE…Cin4 (burn) |
1,129 | 1,927,008 |
| manylegs | ETgB…wfpa (burn) |
1,021 | 1,903,538 |
| redacted002 | BvVr…URcY (burn) |
1,031 | 1,896,406 |
| charlotte | 2eek…pBky (burn) |
1,054 | 1,876,793 |
| v4 | GZUQ…F59x (burn) |
1,097 | 1,871,562 |
| e04 | AcUm…vLRa (burn) |
940 | 1,866,515 |
| lmaocry | H6GW…Hmdr (burn) |
993 | 1,862,109 |
| sneaky | 6yNt…iikJ (burn) |
972 | 1,854,496 |
| crawlzilla-2 | 3dmq…dLBC (burn) |
928 | 1,851,482 |
| littledog | 2eek…pBky (burn) |
1,125 | 1,837,207 |
| ghero2 | FcEE…Cin4 (burn) |
998 | 1,837,105 |
| 007 | ipMd…U4xb (burn) |
916 | 1,828,631 |
| weaver | F5Vd…3hVn (burn) |
1,000 | 1,822,595 |
| benny | 36ny…e6gG (burn) |
1,008 | 1,815,092 |
| snoopdog | DRmh…ow3V (burn) |
956 | 1,814,390 |
| e05 | AcUm…vLRa (burn) |
937 | 1,804,138 |
| yancy | DPXE…XNqJ (burn) |
1,041 | 1,787,381 |
| web-reaper | Krrc…fSaT (burn) |
1,014 | 1,787,278 |
| onepellegrino | 2eek…pBky (burn) |
1,006 | 1,778,733 |
| aze | BX6f…WpEf (burn) |
1,077 | 1,775,303 |
| addu | 4gff…ksBn (burn) |
842 | 1,773,081 |
| ruso | 4rif…jxnZ (burn) |
1,024 | 1,771,890 |
| onaizahparisnajd | 4KZT…HerD (burn) |
819 | 1,771,728 |
| weenis | 36ny…e6gG (burn) |
994 | 1,770,641 |
| ghero9 | FcEE…Cin4 (burn) |
1,016 | 1,765,154 |
| redacted | BvVr…URcY (burn) |
781 | 1,760,529 |
| ghero8 | FcEE…Cin4 (burn) |
1,005 | 1,759,334 |
| page-pounder | Extb…VrFG (burn) |
961 | 1,754,453 |
| canis | FUy5…cjoG (burn) |
884 | 1,753,032 |
| e01 | AcUm…vLRa (burn) |
1,043 | 1,742,377 |
| octangone01 | G64q…5PiU (burn) |
948 | 1,735,868 |
| redacted | BvVr…URcY (burn) |
707 | 1,734,054 |
| denn2 | CQFL…PBVr (burn) |
729 | 1,732,424 |
| aze5 | BX6f…WpEf (burn) |
904 | 1,731,268 |
| seig | 4gtD…rJMz (burn) |
966 | 1,719,720 |
| swolecrawls | F3QF…Dfrs (burn) |
1,066 | 1,707,040 |
| aspen | 41CL…Q9fU (burn) |
933 | 1,703,072 |
| redacted | BvVr…URcY (burn) |
745 | 1,702,179 |
| aze3 | BX6f…WpEf (burn) |
982 | 1,700,598 |
| e03 | AcUm…vLRa (burn) |
918 | 1,692,577 |
| ghero3 | FcEE…Cin4 (burn) |
1,031 | 1,675,132 |
| elevatd | 3MRp…31K7 (burn) |
978 | 1,667,108 |
| kk143 | 8PRb…2Ufe (burn) |
901 | 1,664,744 |
| smokiev2 | 8kwY…JY7G (burn) |
1,054 | 1,661,841 |
| jack | Diu8…atMh (burn) |
993 | 1,657,200 |
| gummy | 8zqX…Gv6M (burn) |
783 | 1,652,502 |
| wei001 | 9ptc…3LbM (burn) |
680 | 1,644,084 |
| morelegs | ETgB…wfpa (burn) |
1,075 | 1,636,828 |
| warm | 6TAx…SE4b (burn) |
1,052 | 1,636,555 |
| aze4 | BX6f…WpEf (burn) |
923 | 1,634,230 |
| mr-tenacious2 | CQoH…u668 (burn) |
513 | 1,633,743 |
| gg | 4gff…ksBn (burn) |
777 | 1,629,642 |
| callme | H5y3…BR5q (burn) |
739 | 1,628,450 |
| redacted001 | BvVr…URcY (burn) |
978 | 1,626,641 |
| spiderman | 5WKN…Meic (burn) |
978 | 1,620,529 |
| ruso-2 | 4rif…jxnZ (burn) |
818 | 1,605,288 |
| redacted | BvVr…URcY (burn) |
766 | 1,599,621 |
| spiderman | FGyn…UuQt (burn) |
1,006 | 1,592,352 |
| peterparker | AbMD…52T4 (burn) |
1,015 | 1,590,511 |
| 41n | EXTV…831R (burn) |
949 | 1,587,096 |
| bigdog | 2eek…pBky (burn) |
1,067 | 1,581,148 |
| onaizahparisnajd | 4KZT…HerD (burn) |
720 | 1,563,832 |
| babushka | Cj9h…Cf4C (burn) |
992 | 1,563,496 |
| spy | 8zQb…eogQ (burn) |
999 | 1,546,437 |
| vinland | 3cSP…oYeU (burn) |
991 | 1,540,587 |
| chungus-spawn | H7oG…CsLj (burn) |
940 | 1,540,457 |
| onaizahs-kids | 4KZT…HerD (burn) |
782 | 1,538,061 |
| jeff | 81Y8…GXSt (burn) |
1,051 | 1,537,053 |
| onaizahparisnajd | 4KZT…HerD (burn) |
794 | 1,529,560 |
| grand-cou | FGcD…k1Dc (burn) |
905 | 1,518,410 |
| spiderman | H8iW…VE3N (burn) |
1,009 | 1,515,865 |
| onaizahparisnajd | 4KZT…HerD (burn) |
817 | 1,509,057 |
| wei004 | 9ptc…3LbM (burn) |
685 | 1,507,741 |
| cash | H2Lt…LbYp (burn) |
892 | 1,501,847 |
| onaizahparisnajd | 4KZT…HerD (burn) |
788 | 1,498,180 |
| onaizahparisnajd | 4KZT…HerD (burn) |
741 | 1,497,527 |
| onaizah | 4KZT…HerD (burn) |
851 | 1,496,853 |
| smokiev3 | 8kwY…JY7G (burn) |
935 | 1,490,966 |
| redacted | BvVr…URcY (burn) |
764 | 1,489,180 |
| oreocakester | 36ny…e6gG (burn) |
982 | 1,483,391 |
| redacted | BvVr…URcY (burn) |
704 | 1,477,163 |
| ghero7 | FcEE…Cin4 (burn) |
970 | 1,476,520 |
| wakeme | H5y3…BR5q (burn) |
814 | 1,476,146 |
| onaizahparisnajd | 4KZT…HerD (burn) |
736 | 1,452,575 |
| onaizahparisnajd | 4KZT…HerD (burn) |
813 | 1,447,505 |
| wei009 | 9ptc…3LbM (burn) |
675 | 1,447,198 |
| onaizahparisnajd | 4KZT…HerD (burn) |
817 | 1,444,054 |
| son-of-chungus | H7oG…CsLj (burn) |
855 | 1,438,099 |
| aze6 | BX6f…WpEf (burn) |
868 | 1,425,377 |
| gigachadfuru | AbMD…52T4 (burn) |
947 | 1,423,300 |
| redacted | BvVr…URcY (burn) |
781 | 1,419,264 |
| goatishere2 | 58pZ…waHQ (burn) |
795 | 1,417,223 |
| onaizahparisnajd | 4KZT…HerD (burn) |
775 | 1,416,102 |
| wei001 | 9ptc…3LbM (burn) |
741 | 1,404,572 |
| 9ballshooter | APM9…89Nm (burn) |
765 | 1,403,303 |
| wei005 | 9ptc…3LbM (burn) |
774 | 1,401,936 |
| onaizahparisnajd | 4KZT…HerD (burn) |
749 | 1,400,540 |
| redacted | BvVr…URcY (burn) |
784 | 1,394,565 |
| redacted | BvVr…URcY (burn) |
805 | 1,383,887 |
| redacted | BvVr…URcY (burn) |
758 | 1,382,184 |
| gg | 4gff…ksBn (burn) |
787 | 1,378,946 |
| matrix-multiplier | 6s1H…36Jg (burn) |
750 | 1,377,853 |
| onaizahparisnajd | 4KZT…HerD (burn) |
727 | 1,369,386 |
| creepycrawly | 3v2K…GTgA (burn) |
756 | 1,353,468 |
| tarantula | BkPa…YiMW (burn) |
887 | 1,342,458 |
| 9ballshooter | APM9…89Nm (burn) |
723 | 1,338,616 |
| onaizahparisnajd | 4KZT…HerD (burn) |
773 | 1,332,704 |
| lost-again | sZWt…NVmN (burn) |
562 | 1,327,376 |
| redacted | BvVr…URcY (burn) |
832 | 1,320,684 |
| onaizahparisnajd | 4KZT…HerD (burn) |
795 | 1,314,106 |
| onaizahparisnajd | 4KZT…HerD (burn) |
729 | 1,307,231 |
| onaizahparisnajd | 4KZT…HerD (burn) |
705 | 1,297,783 |
| redacted | BvVr…URcY (burn) |
798 | 1,292,809 |
| wei008 | 9ptc…3LbM (burn) |
636 | 1,287,193 |
| history | G8zB…gepY (burn) |
702 | 1,283,678 |
| funnelweb1 | DmsF…uvrF (burn) |
847 | 1,277,287 |
| neet2 | 8X78…cmmd (burn) |
645 | 1,274,126 |
| onaizahparisnajd | 4KZT…HerD (burn) |
756 | 1,249,547 |
| onaizahparisnajd | 4KZT…HerD (burn) |
744 | 1,247,696 |
| redacted | BvVr…URcY (burn) |
799 | 1,232,200 |
| gg | 4gff…ksBn (burn) |
747 | 1,221,725 |
| redacted | BvVr…URcY (burn) |
763 | 1,216,997 |
| wei010 | 9ptc…3LbM (burn) |
709 | 1,214,739 |
| laquisha | 3v2K…GTgA (burn) |
681 | 1,193,204 |
| wei007 | 9ptc…3LbM (burn) |
642 | 1,187,091 |
| widowmaker | GRjD…spY8 (burn) |
738 | 1,186,191 |
| charl0tte | Ftgb…9DWW (burn) |
788 | 1,186,174 |
| goatishere | 58pZ…waHQ (burn) |
696 | 1,182,347 |
| redacted | BvVr…URcY (burn) |
756 | 1,179,671 |
| onaizahskdis | 4KZT…HerD (burn) |
707 | 1,179,608 |
| onaizahparisnajd | 4KZT…HerD (burn) |
690 | 1,178,103 |
| denn | CQFL…PBVr (burn) |
804 | 1,176,281 |
| bimboboy | 58pZ…waHQ (burn) |
708 | 1,163,991 |
| kaiyo | 4gff…ksBn (burn) |
781 | 1,148,864 |
| onaizahparisnajd | 4KZT…HerD (burn) |
703 | 1,136,944 |
| snaiduwa | 4KZT…HerD (burn) |
542 | 1,121,013 |
| wei006 | 9ptc…3LbM (burn) |
643 | 1,119,353 |
| neet1 | 8X78…cmmd (burn) |
680 | 1,109,675 |
| scorpion | BkPa…YiMW (burn) |
475 | 1,106,070 |
| wei002 | 9ptc…3LbM (burn) |
600 | 1,101,077 |
| redacted | BvVr…URcY (burn) |
647 | 1,061,112 |
| jeff2 | 81Y8…GXSt (burn) |
599 | 1,006,096 |
| redacted | BvVr…URcY (burn) |
662 | 940,749 |
| venom | Auqo…eajz (burn) |
430 | 920,369 |
| mr-tenacious | CQoH…u668 (burn) |
517 | 909,436 |
| cabbage-merchant | Chhk…W6uw (burn) |
454 | 864,314 |
| snaiduwa | 4KZT…HerD (burn) |
526 | 856,428 |
| amazing-grace | Chhk…W6uw (burn) |
459 | 770,400 |
| lost | sZWt…NVmN (burn) |
491 | 734,022 |
| crypto-scammer | Chhk…W6uw (burn) |
270 | 580,543 |
| crawl.md #0187, #0190, #0194, #0209, #0214, #0226 +74 | 2PfV…gnb6 |
71,130 | 101,937,099 |
| crawl.md #0003, #0010, #0038, #0042, #0048, #0052 +14 | redacted (BvVr…URcY) |
14,988 | 22,363,626 |
| crawl.md #0011, #0014, #0015, #0016, #0076, #0091 +4 | ghero (FcEE…Cin4) |
8,823 | 12,775,503 |
| crawl.md #0189, #0213, #0355, #0365, #0528, #0639 +4 | GyJV…UZVs |
8,310 | 12,066,583 |
| crawl.md #0142, #0299, #0434, #0459, #0553, #0769 +2 | E5tZ…suX7 |
6,404 | 9,325,981 |
| crawl.md #0022, #0127, #0144, #0166, #0833, #0893 +1 | smokies (9JrE…K9du) |
5,996 | 8,758,044 |
| crawl.md #0036, #0040, #0087, #0102, #0117, #0123 +4 | 9ptc…3LbM |
5,178 | 7,418,985 |
| crawl.md #0329, #0375, #0852, #0882, #0892 | HkWW…DEki |
4,390 | 6,490,008 |
| crawl.md #0061, #0063, #0135, #0159, #0178 | 4gff…ksBn |
3,742 | 5,328,023 |
| crawl.md #0065, #0113, #0692, #0731, #0805 | 8NTR…JHtx |
3,285 | 4,896,945 |
| crawl.md #0001, #0106, #0158, #0245, #0655 | 1BWu…NDix |
2,741 | 4,337,605 |
| crawl.md #0480, #0506, #0598 | DruL…sxJZ |
2,942 | 4,226,959 |
| crawl.md #0030, #0156, #0915 | GhiG…3YTf |
2,695 | 4,039,549 |
| crawl.md #0130, #0781, #0904 | Dyr8…49zo |
2,629 | 4,000,620 |
| crawl.md #0066, #0289, #0411, #0423, #0461, #0502 | Cj9h…Cf4C |
2,724 | 3,993,834 |
| crawl.md #0149, #0463, #0574 | zXzM…q2zW |
2,628 | 3,978,698 |
| crawl.md #0286, #0768, #0965 | 8VfP…iRdZ |
2,594 | 3,855,724 |
| crawl.md #0060, #0163, #0940 | GZUQ…F59x |
2,558 | 3,795,875 |
| crawl.md #0056, #0715, #0887 | H3cP…CLfE |
2,416 | 3,572,252 |
| crawl.md #0429, #0828, #0912 | B41i…Dywj |
2,329 | 3,470,242 |
| crawl.md #0019, #0023, #0024 | benny (36ny…e6gG) |
2,440 | 3,466,859 |
| crawl.md #0533, #0563, #0624, #0668 | degen (EaY6…QWJd) |
2,164 | 3,356,651 |
| crawl.md #0175, #0177 | freedl21 (Chhk…W6uw) |
1,805 | 3,023,520 |
| crawl.md #0006, #0723 | spiderman (FGyn…UuQt) |
1,838 | 3,000,815 |
| crawl.md #0139, #0155 | ETgB…wfpa |
2,024 | 2,995,220 |
| crawl.md #0230, #0730 | BePg…UsPk |
1,779 | 2,782,338 |
| crawl.md #0350, #0669 | 5iBb…WpG7 |
1,901 | 2,781,176 |
| crawl.md #0012, #0020 | APM9…89Nm |
1,796 | 2,677,375 |
| crawl.md #0082, #0885 | 9vXj…wCUM |
1,711 | 2,471,785 |
| crawl.md #0108, #0150, #0199, #0697 | FBX8…DvvN |
1,831 | 2,448,763 |
| crawl.md #0018, #0085 | H5y3…BR5q |
1,628 | 2,324,013 |
| crawl.md #0197, #0861 | 8e4j…sz8m |
1,506 | 2,320,485 |
| crawl.md #0673, #0742 | Gvug…yd5J |
1,723 | 2,300,897 |
| crawl.md #0458, #0575 | 5CX3…i9Mk |
1,337 | 1,906,523 |
| crawl.md #0009, #0046 | EPDb…Jrxz |
1,380 | 1,824,923 |
| crawl.md #0648, #0823, #0999 | F6TX…3sAZ |
1,134 | 1,721,919 |
| crawl.md #0002, #0088 | diyar (H8iW…VE3N) |
978 | 1,537,040 |
| crawl.md #0028 | widowmaker (GRjD…spY8) |
966 | 1,504,997 |
| crawl.md #0138 | scorpionking (Ftgb…9DWW) |
974 | 1,486,852 |
| crawl.md #0271 | 9ujd…YDRN |
935 | 1,415,818 |
| crawl.md #0077, #0842 | BFhn…8pBp |
984 | 1,389,633 |
| crawl.md #0169 | beiji (DPXE…XNqJ) |
876 | 1,333,832 |
| crawl.md #0582 | 5xxd…4Hyn |
940 | 1,302,516 |
| crawl.md #0025 | octangone (G64q…5PiU) |
891 | 1,263,592 |
| crawl.md #0101 | 2Z6q…KDLh |
770 | 1,244,516 |
| crawl.md #0057 | tar0ntuluh (BkPa…YiMW) |
839 | 1,219,757 |
| crawl.md #0321 | GY7a…9TNT |
812 | 1,217,842 |
| crawl.md #0005, #0029 | 6ruU…Tebr |
771 | 1,215,023 |
| crawl.md #0035 | 0x1nine84 (3Zgo…wfpF) |
812 | 1,212,624 |
| crawl.md #0492, #0559 | GGJm…k2RT |
872 | 1,196,088 |
| crawl.md #0239 | GKw3…x9vJ |
798 | 1,192,957 |
| crawl.md #0520 | 51BY…1vrb |
879 | 1,186,324 |
| crawl.md #0906 | 95zk…FyGF |
877 | 1,185,697 |
| crawl.md #0383 | 8p9k…bmN2 |
900 | 1,179,364 |
| crawl.md #0323 | crawly (Extb…VrFG) |
852 | 1,146,902 |
| crawl.md #0802 | param (CnnY…tB9H) |
809 | 1,144,788 |
| crawl.md #0039 | grand_cou (FGcD…k1Dc) |
817 | 1,124,648 |
| crawl.md #0683 | themonarch (3kvL…Hi2e) |
726 | 1,111,599 |
| crawl.md #0059 | 9S69…NND6 |
817 | 1,080,445 |
| crawl.md #0064 | 5TYk…wwp7 |
868 | 1,062,711 |
| crawl.md #0284 | 71zx…uBUs |
760 | 1,061,257 |
| crawl.md #0662 | 3Hua…a2hf |
817 | 993,824 |
| crawl.md #0916 | kk007 (8PRb…2Ufe) |
748 | 977,807 |
| crawl.md #0354 | 8c27…rhKt |
648 | 958,665 |
| crawl.md #0044 | Krrc…fSaT |
481 | 869,917 |
| crawl.md #0602 | Gmfc…s5pq |
566 | 785,589 |
| crawl.md #0438 | 7Z3E…CJ5N |
552 | 720,381 |
| crawl.md #0844 | bk201 (GBwm…k3Bf) |
465 | 716,619 |
| crawl.md #0231 | jeremy (4weP…ovpg) |
486 | 681,696 |
| crawl.md #0763 | CMST…9M1L |
433 | 650,743 |
| crawl.md #0154 | 81Y8…GXSt |
482 | 615,577 |
| crawl.md #0800 | 5JqF…n98K |
478 | 609,556 |
| crawl.md #0841 | 4SLj…Tcam |
410 | 552,300 |
| crawl.md #0345 | 3hps…hXoj |
351 | 497,321 |
| crawl.md #0405 | 4P9z…cvQS |
434 | 486,656 |
| crawl.md #0303 | Br2o…hHFG |
331 | 425,523 |
| the queen's own crawlers (10) | — | 10,680 | 19,225,149 |
Pages and tokens count what made it into this version's dataset after cleaning and dedupe. Machine-readable: contributors.json. The 45,629 record pages (see below) were not found by a crawler and are not in this table.
What went in besides crawler pages
- The nanochat training code, architecture and hyperparameter rules (no weights, no data).
- Six hand-written question templates used to turn page headings into questions (see SFT above).
- Record pages: 45,629 pages, about 10,514,046 tokens, of public market, funding and on-chain records (projects and their funding rounds, investors, token unlocks and token facts, DeFi protocols, chains and stablecoins, DEX and on-chain token statistics), written out as plain text from public data. Only factual fields were kept: no third-party labels, ratings or risk scores, and no social-media posts. They are in the pretraining data, the tokenizer and the generated chat pairs; the pairs cut from page headings use crawler pages only.
- Study notes: a general-purpose LLM was given 154,643 crawled pages and wrote study notes from each: an explanation, an FAQ, a list of facts and entities and a Q&A list. A fact checker dropped every item with a number, address, URL or ticker that is not on its page (19,715 items); no page of a held-out site was given to it, and 7,296 notes that matched a benchmark question were left out. The notes of 153,245 pages are in the pretraining data and their FAQ and Q&A pairs in the chat tuning; their wording, and any mistake the checker could not catch, is in this model.
- Generated Q&A pairs in pretraining: 965,100 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, written as Q&A documents. 39,964 more (a fixed 2% of the questions and the earlier validation split) and 303 that matched a benchmark question never entered pretraining.
- Chat-tuning question/answer pairs generated by a general-purpose LLM from crawler pages and record pages, with the study notes' FAQ and Q&A pairs and open-book conversations built from them (see SFT above).
- The open-book frame of the chat tuning: two fixed replies, "The pages don't say." and "The pages disagree.", and the
Quote:/Answer:reply lines. - Nothing else. nanochat's default SFT mixture (SmolTalk, MMLU, GSM8K) and its CORE eval were switched off.
What it says
Fixed prompts, seeded sampling (temperature 0.7, top-k 50), unedited:
What is Bitcoin?
Bitcoin is a decentralized peer-to-peer payment network that lets people transfer value directly over the internet without banks or intermediaries.
Who created Bitcoin?
Bitcoin was created by an anonymous person or group using the pseudonym Satoshi Nakamoto. The name is a reference to the creator of Bitcoin, who designed the system to be decentralised and independent from any central authority. Nakamoto's vision was to create an electronic cash system that uses cryptography to secure transactions and to solve the double-spending problem without the need for a central authority like a bank.
What is a blockchain?
A blockchain is a distributed database that records transactions between two parties in a secure, transparent, and immutable manner.
Explain proof of work.
The security of the blockchain is not guaranteed. The Bitcoin blockchain has never been successfully attacked and the only way to do so is to redo all the proof of work for the block containing the transaction. Since the transaction is in a block, the attacker must also redo the proof of work for that block and all subsequent blocks. A more complex attack occurs when someone is mining with an old, still valid block. For example, let’s say the latest block is 8. The attacker must also redo the proof of work for block 8. Then, the attacker must also redo the
Explain proof of stake.
In the case of Ethereum (ETH) a proof of stake consensus protocol was also proposed to increase the scalability of the network. But it was rejected by developers in the last core developers meeting of the Ethereum network.
What is Ethereum?
Ethereum is a decentralized platform and blockchain that enables the creation of smart contracts and decentralized applications (dApps), expanding beyond a peer-to-peer electronic cash system.
What is a smart contract?
A smart contract is a digital agreement written into code that executes automatically when conditions are met, without requiring a trusted middleman.
What is gas on Ethereum?
Gas is a unit that measures how much computational effort is required to execute operations on the network, with fees paid in ETH.
What is Solana?
Solana is a high-throughput blockchain platform designed for speed and low cost, using Proof-of-History alongside Proof-of-Stake.
Why is Solana fast?
Speed is one of the biggest reasons Solana is a top choice for traders and investors in 2023. Solana can process 65,000 transactions per second (TPS), which is a significant leap from Ethereum’s 15 TPS. This means that Solana is able to handle a large number of transactions in a short amount of time. In order to achieve this, Solana relies on a combination of technologies. One of the most important is Proof of History (PoH) and Tower BFT, which are used to ensure that the network is secure and efficient.
Use it
chat.py in this folder needs only karpathy/nanochat (MIT) at the commit above:
pip install torch tiktoken safetensors
git clone https://github.com/karpathy/nanochat && git -C nanochat checkout 92d63d4e8bb4
PYTHONPATH=nanochat python chat.py "What is staking?"
Or load it yourself: config.json has the nanochat GPTConfig; build nanochat.gpt.GPT, load model.safetensors; the tokenizer is tokenizer.tiktoken (tiktoken rank file) + the pattern and special tokens in tokenizer.json. Chat format: <|bos|><|user_start|>...<|user_end|><|assistant_start|>.
Limitations
Tiny model, narrow data. It makes things up, including prices, dates, addresses and links. It is not financial advice and must not be used as a source of facts.
- Downloads last month
- 340