Hugging Face's logo
Morgan McGuire's picture
1 12

Morgan McGuire

morgan
Hesham-wandb's profile picture osanseviero's profile picture realtamimuddin's profile picture
·
https://www.ntentional.com

AI & ML interests

None yet

Organizations

Weights and Biases's profile picture fastai X Hugging Face Group 2022's profile picture CompVis Community's profile picture Webhooks Explorers (BETA)'s profile picture Flax Community's profile picture AblateIt's profile picture Social Post Explorers's profile picture Hugging Face Discord Community's profile picture

Posts 3

view post
Post
1195
Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard

Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19

The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.

Sammple of 1 of the eval calls here: https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1

Quick auto-logging tracing for openai-compatible clients and many more here: https://wandb.github.io/weave/quickstart/

view post
Post
Fine-tuning LLMs is rad, but how do you manage all your checkpoints and evals in a production setting?

We partnered with @hamel to ship an Enterprise Model Management course packed full of learnings for those training, evaluating and deploying models at work.

Topics include:
- What webhooks are & how to use them to create integrations with different tools
- How to automate train -> eval runs
- Improving model governance and documentation
- Comparing candidate and baseline models
- Design patterns & recipes
- Lots more...

Would love to hear what you think!

👉 https://www.wandb.courses/courses/enterprise-model-management
View all posts