Broad Knolwedge < Intelligence
Everyone is complaining about it's lack of broad knowledge. This model runs at about 2/3 the speed of cloud model, Xiaomi MiMo-V2.5-Pro for solving programming problems, I've been testing it vs Mimo on the top 5000 programming problems on StackOverflow vs Mimo and in my latest run after I fixed the Jinja problem, here's what Mimo says about comparing Mimo Vs LFM2.5:
═══════════════════════════════════════════════════════════════════
SOLUTION QUALITY BY MODEL
═══════════════════════════════════════════════════════════════════
LFM2.5 (local) mimo (cloud)
────────────── ────────────
Total solved 8,410 25,355
QUALITY METRICS:
┌───────────────────────────────────┐
│ Contains code blocks 86% 43% │
│ Has explanation 89% 48% │
│ Avg solution length 2,731 chars 2,193 chars │
│ Median length 1,976 chars 439 chars │
└──────────────────────────────────┘
BY DIFFICULTY:
┌────────────────────────────────────────────────────────────────┐
│ │ LFM2.5 │ mimo │
│ │ avg code expl │ avg code expl │
├────────────┼───────────────────┼──────────────────────────────┤
│ Hard (30+) │ 3095 88% 92% │ 2755 60% 72% │
│ Medium │ 2872 85% 90% │ 2579 45% 47% │
│ Easy │ 2456 87% 84% │ 1708 39% 42% │
└────────────────────────────────────────────────────────────────┘
SPEED:
│ Hard 69.6s 63.2s │
│ Medium 50.7s 32.8s │
│ Easy 36.9s 29.6s │
Key findings:
- LFM2.5 produces much higher quality - 86% have code vs 43% for mimo
- LFM2.5 median is 4.5x longer (1,976 vs 439 chars) - mimo often gives short answers
- mimo is ~30% faster but produces lower quality output
The median difference is striking - mimo's median solution is only 439 chars (often just a brief explanation without code), while LFM2.5 consistently produces detailed solutions with code.
Recommendation: For training data quality, prioritize LFM2.5 solutions. Consider re-solving some mimo solutions with LFM2.5 for better training traces.
it's bad for agent like hermes.
It can't use properly skills and tools.
It lacks understanding of a lot of things.
It don't even try to explore other possibilities.
I hope they fix this model.
The tests that I've done with it show that it has amazing raw potential! But clearly it's still crippled in it's output generation and it's not a 1.5T model it's an 8B moe - it will always need to be a specialized model because of it's size, but also due to it's size it's VERY fast, and this means it's embedded use case (which is what it is designed for) IS AMAZING. Embedded use means specialized, it's intended for low latency commands, that also might interface with the 350M model that they have, which is ever lower latency, this isn't just a toy model like the Qwen3.6-35B that you can play with in Hermes. You're looking at an uncut gem and thinking that it should be a diamond ring.