Detect hallucinations in answers based on context and questions using ModernBERT with 8192-token context support!
### Model Details - **Model Name**: [lettucedect-large-modernbert-en-v1](KRLabsOrg/lettucedect-large-modernbert-en-v1) - **Organization**: [KRLabsOrg](https://huggingface.co/KRLabsOrg) - **Github**: [https://github.com/KRLabsOrg/LettuceDetect](https://github.com/KRLabsOrg/LettuceDetect) - **Architecture**: ModernBERT (Large) with extended context support up to 8192 tokens - **Task**: Token Classification / Hallucination Detection - **Training Dataset**: [RagTruth](wandb/RAGTruth-processed) - **Language**: English - **Capabilities**: Detects hallucinated spans in answers, provides confidence scores, and calculates average confidence across detected spans.
LettuceDetect excels at processing long documents to determine if an answer aligns with the provided context, making it a powerful tool for ensuring factual accuracy.
Hugging Face just launched the AI Agents Course – a free journey from beginner to expert in AI agents!
- Learn AI Agent fundamentals, use cases and frameworks - Use top libraries like LangChain & LlamaIndex - Compete in challenges & earn a certificate - Hands-on projects & real-world applications
🙋🏻♂️Hey there folks , Open LLM Europe just released Lucie 7B-Instruct model , a billingual instruct model trained on open data ! You can check out my unofficial demo here while we wait for the official inference api from the group : Tonic/Lucie-7B hope you like it 🚀
Published a new blogpost 📖 In this blogpost I have gone through the transformers' architecture emphasizing how shapes propagate throughout each layer. 🔗 https://huggingface.co/blog/not-lain/tensor-dims some interesting takeaways :
(P1, P2): P2 has a shorter survival time and a higher risk score → Concordant ✅ (P1, P3): P3 has a longer survival time and a lower risk score → Concordant ✅ (P2, P3): P3 has a longer survival time and a lower risk score → Concordant ✅ Total pairs = 3 Total concordant pairs = 3
C-index for Group A = Concordant pairs/Total pairs= 3/3 = 1.0
Step 2: Calculate C-index for All Groups Repeat the process for all groups. For now we can assume:
Group A: C-index = 1.0 Group B: C-index = 0.8 Group C: C-index = 0.6 Step 3: Stratified Concordance Index The Stratified Concordance Index combines the C-index scores of all groups and focusing on the following:
Average performance across groups (mean of C-indices). Consistency across groups (low standard deviation of C-indices). Formula: Stratified C-index = Mean(C-index scores) - Standard Deviation(C-index scores)
Calculate the mean: Mean=1.0 + 0.8 + 0.6/3 = 0.8
Calculate the standard deviation: Standard Deviation= sqrt((1.0-0.8)^2 + (0.8-0.8)^2 + (0.6-0.8)^/3) = 0.16