Don yang Lin

dylin7

https://github.com/DefengXie/Edit_Everything

Defengxie

AI & ML interests

Large language models

Recent Activity

commented on an article about 1 month ago

4D masks support in Transformers

commented on an article about 1 month ago

Improving Hugging Face Training Efficiency Through Packing with Flash Attention

liked a Space 4 months ago

akhaliq/anycoder

View all activity

Organizations

dylin7's activity

commented on 4D masks support in Transformers about 1 month ago

@poedator About Sequences packing in SFT (supervised finetuning) training, do you have any example script? If you have it, could you provide it? Thank you very much.

commented on Improving Hugging Face Training Efficiency Through Packing with Flash Attention about 1 month ago

data_collator = DataCollatorWithFlattening() This method cannot inherently prevent attention isolation. But it can have position isolation. Do you realize this problem? @RQlee @ArthurZ

liked a Space 4 months ago

2.09k

Anycoder

🏢

Select and view code snippets for different providers