[ACL'24] Analysing the Impact of Sequence Composition on Language Model Pre-Training. https://github.com/yuzhaouoe/pretraining-data-packing
Yu Zhao
yuzhaouoe
AI & ML interests
NLP/ML
Organizations
Collections
1
models
10
yuzhaouoe/Llama2-7b-SAE
Updated
yuzhaouoe/IntraDoc-2048
Text Generation
•
Updated
•
1.84k
yuzhaouoe/BM25Chunk-2048
Text Generation
•
Updated
•
1.86k
yuzhaouoe/MixChunk-2048
Text Generation
•
Updated
•
1.85k
yuzhaouoe/UniChunk-2048
Text Generation
•
Updated
•
1.84k
yuzhaouoe/MixChunk
Text Generation
•
Updated
•
1.17k
yuzhaouoe/UniChunk
Text Generation
•
Updated
•
1.19k
yuzhaouoe/IntraDoc
Text Generation
•
Updated
•
425
yuzhaouoe/BM25Chunk
Text Generation
•
Updated
•
424
yuzhaouoe/eval_data
Updated
datasets
None public yet