m-ric (Aymeric Roucher)

Posts 9

Post

2159

𝐏𝐚𝐩𝐞𝐫 𝐑𝐞𝐯𝐢𝐞𝐰: 𝐑𝐡𝐨-𝟏 - 𝐃𝐨 𝐧𝐨𝐭 𝐮𝐬𝐞 𝐚𝐥𝐥 𝐭𝐨𝐤𝐞𝐧𝐬 𝐞𝐪𝐮𝐚𝐥𝐥𝐲 𝐢𝐧 𝐲𝐨𝐮𝐫 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠! ⚖️⛔️

A new paper topping Daily papers questions a hidden assumption in LLM training:

🤔 𝙎𝙝𝙤𝙪𝙡𝙙 𝙬𝙚 𝙧𝙚𝙖𝙡𝙡𝙮 𝙪𝙨𝙚 𝙖𝙡𝙡 𝙩𝙤𝙠𝙚𝙣𝙨 𝙚𝙦𝙪𝙖𝙡𝙡𝙮 𝙞𝙣 𝙤𝙪𝙧 𝙇𝙇𝙈'𝙨 𝙩𝙧𝙖𝙞𝙣𝙞𝙣𝙜 ?

Some tokens are more relevant than others, and some are mostly noise (just look up the history of 𝘚𝘰𝘭𝘪𝘥𝘎𝘰𝘭𝘥𝘔𝘢𝘨𝘪𝘬𝘢𝘳𝘱).

So this paper introduces 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝘃𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴, which is actually really simple:
➡️ A specific metric measures the relevance of each token. Then during training, only the top k% tokens for this relevance metric count in the loss calculation.

Authors test this method by training models on the difficult MATH dataset (only competition mathematics problems).

➡️ Their technique seems like a new must-do in LLM training: Training is much faster and reaches an impressive performance!

𝐑𝐞𝐬𝐮𝐥𝐭𝐬:
◆ ⏱️ Training is x5 to x10 faster to reach equivalent performance compared to standard language modeling.
◆ 💪 Their 1B model achieves close to GPT4 Chain-of-Thought performance on MATH!
◆ 🚀 Their 7B model match performance of the state-of-the-art DeepSeek for the same size, while trained on only 3% of tokens

𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡
◆ Datasets used for pre-training, even after pre-filtering, still contain a large proportion of noisy tokens 😖
◆ Authors show that when you reduce loss on noisy tokens, you actually reduce accuracy (Figure 7). So Selective Language Modeling seems fundamental! ✅

Find great reads in @akhaliq 's Daily Papers 👉 https://huggingface.co/papers
Paper added to my collection 👉 m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7

Post

2037

𝗡𝗲𝘄 𝗦𝗽𝗮𝗰𝗲: 𝘼𝙄 𝙏𝙧𝙖𝙫𝙚𝙡 𝙥𝙡𝙖𝙣𝙣𝙚𝙧 🗺️🏕️ Plan your next vacation in a few minutes!

I wanted to try out if a powerful LLM like Mixtral-8x7b had geographical reasoning capabilities.
So I built a small space that prompts the LLM to provide a JSON list of places based on a user input.

And the result was impressive! 🤯

⇒ 𝗜𝘁 𝘀𝗲𝗲𝗺𝘀 𝗹𝗶𝗸𝗲 𝗠𝗶𝘅𝘁𝗿𝗮𝗹 𝗵𝗮𝘀 𝗮 𝗴𝗿𝗮𝘀𝗽 𝗼𝗳 𝗴𝗲𝗼𝗴𝗿𝗮𝗽𝗵𝗶𝗰𝗮𝗹 𝗰𝗼𝗻𝗰𝗲𝗽𝘁𝘀 𝗹𝗶𝗸𝗲 𝗡𝗼𝗿𝘁𝗵 - 𝗦𝗼𝘂𝘁𝗵, 𝗼𝗿 𝘀𝗽𝗮𝘁𝗶𝗮𝗹 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁.🧭 Not just describing these concepts, but really applying them in practice, for instance to successfully answer "give me 4 European cities that are aligned on the map". This is a 𝗻𝗶𝗰𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮𝗻 𝗲𝗺𝗲𝗿𝗴𝗲𝗻𝘁 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆, since nothing in the LLM's training data should prepare it for this specific task.

Anyway, I added API calls and a nice visualization on top of the LLM, streaming output, caching for the answers and locations... and ta-da! ✨ I got the 𝗔𝗜 𝗧𝗿𝗮𝘃𝗲𝗹 𝗣𝗹𝗮𝗻𝗻𝗲𝗿.

𝙔𝙤𝙪 𝙘𝙖𝙣 𝙙𝙚𝙨𝙘𝙧𝙞𝙗𝙚 𝙞𝙩 𝙮𝙤𝙪𝙧 𝙩𝙧𝙞𝙥, 𝙖𝙣𝙙 𝙞𝙩 𝙬𝙞𝙡𝙡 𝙘𝙤𝙢𝙚 𝙪𝙥 𝙬𝙞𝙩𝙝 𝙣𝙞𝙘𝙚 𝙖𝙣𝙙 𝙘𝙤𝙣𝙫𝙚𝙣𝙞𝙚𝙣𝙩 𝙡𝙤𝙘𝙖𝙩𝙞𝙤𝙣𝙨!

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉 m-ric/ai-travel-planner

Thank you @freddyaboulton for the 𝚐𝚛𝚊𝚍𝚒𝚘_𝚏𝚘𝚕𝚒𝚞𝚖 component, and @clem , @pngwn , @abidlabs for your ideas and support!

View all posts