FineVideo: behind the scenes
•
22
Thomas Wolf PRO
thomwolf
AI & ML interests
NLP and open-source :-)
Articles
Organizations
Posts
7
Post
3701
Parents in the 1990: Teach the kids to code
Parents now: Teach the kids to fix the code when it starts walking around 🤖✨
Parents now: Teach the kids to fix the code when it starts walking around 🤖✨
Post
4354
[New crazy blog post alert] We are releasing an extensive blog post on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens 🍷FineWeb release
Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.
And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA
We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)
HuggingFaceFW/blogpost-fineweb-v1
Inspired by the distill.pub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on so prepare for a 45-mmin read with interactive graphics and all.
And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA
We also make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)
HuggingFaceFW/blogpost-fineweb-v1
spaces
10
models
6
datasets
17
thomwolf/blue_sort
Updated
•
202
thomwolf/data_test
Updated
•
49
thomwolf/cameras_conditions
Viewer
•
Updated
•
214
•
76
thomwolf/lerobot-sort2
Preview
•
Updated
•
126
•
1
thomwolf/lerobot-sort3
Preview
•
Updated
•
106
thomwolf/act-raw-sort3
Updated
•
127
thomwolf/act-raw-sort2
Updated
•
99
thomwolf/test-1.1
Updated
•
6
thomwolf/test_queries
Updated
•
9
thomwolf/very-good-dataset
Viewer
•
Updated
•
6
•
15