Stephen Genusa PRO

StephenGenusa

AI & ML interests

LFM, LLM, Quantization, Vision, RAG/Hybrid/Graph, Multimodality, NLP (will take us further down the road with existing LLM tech)

Recent Activity

upvoted a paper about 2 months ago

SliceGPT: Compress Large Language Models by Deleting Rows and Columns

Reacted to m-ric's post with 🚀 about 2 months ago

𝗔𝗱𝗱 𝘀𝗼𝘂𝗿𝗰𝗲 𝗵𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝗶𝗻𝗴 𝘁𝗼 𝘆𝗼𝘂𝗿 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺! 📄💡 RAG systems are supposed to make your LLM's answer more trustworthy, by inserting in the prompt some supporting documents from a knowledge base : we say that we're "adding some context". 👎 But if you don't know which part of the answer has been generated based on which input tokens, it's hard to tell wether it was effectively grounded in the context knowledge or not! 🤔 I've been working on the question: is it possible to add notes to the answer linking to which part of the context they're generated from? And I've found a great solution: a great technique called Layer-wise Relevance Propagation (LRP), showcased in a paper at ICML `24 by Reduan Achtibat et al allows, allows to precisely score how important each input token was in generating your output! They've made it into a library called LXT. 📊 For each generated output token, LXT gives you attribution scores for each input token. ⚙️ So I've worked a bit more on aggregating these scores into meaningful spans between successive input and output tokens, and I finally obtained my desired result: RAG with source highlighting! Try the demo here 👉 https://huggingface.co/spaces/m-ric/rag_highlights Caveats: - It slows down generation (for now quite a lot, could hopefully be reduced a lot) - For now it supports only specific models: Llama models and Mixtral If there's enough interest in this solution, I can improve it further and spin it off into a specific library for RAG! 🚀

upvoted a paper about 2 months ago

MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling

View all activity

Organizations

StephenGenusa's activity

upvoted 2 papers about 2 months ago

SliceGPT: Compress Large Language Models by Deleting Rows and Columns

Paper • 2401.15024 • Published Jan 26 • 69

MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling

Paper • 2409.16160 • Published Sep 24 • 32