Is it really good?
According to my tests, hermes 2 pro mistral 7b still wins. Tested on https://www.lsac.org/lsat/taking-lsat/test-format/logical-reasoning/logical-reasoning-sample-questions . And some other questions that i usually ask. I may be wrong, but that's what i got...
you can't really compare some extremely finetuned model like hermes 2 pro to a basic instruct finetune. I'm sure NousResearch will do their own instruct finetune of llama 3 8b and that is what you would compare. otherwise you could compare this llm to mistral instruct that's directly from mistral
It's better than any of the models I tested before. And I tested A LOT. I believe Hermes etc. are trained to pass those tests. This model is actually the smartest one I've ever tested.
you can't really compare some extremely finetuned model like hermes 2 pro to a basic instruct finetune. I'm sure NousResearch will do their own instruct finetune of llama 3 8b and that is what you would compare. otherwise you could compare this llm to mistral instruct that's directly from mistral
You are right, but I'm just a simple user, whichever model performs best, preferably lower in size, i use it and love it :)
It is good. I put on a colab notebook and challenged Llama3-8B-instructioin model to extract entities from resume and also a chat-with-your resume Langchain chain.
It worked flawlessly!
Mistral-7B-instruct gave me hard time with entity extraction.
T