Post
1702
Please... feed this Llama some Sauerkraut! 🍲
Said and done. Here it is. Our Sauerkraut Version of the strong Llama3-8b by Meta. Released from HANNOVER MESSE, just in front of meta booth.
VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
According to benchmarks (LM-Evaluation-Harness 0.4.2), our #SauerkrautLM Dataset and fine-tuning pipeline improved the Model noticeably (AVG = 74,57), especially Reasoning and Common Sense capabilities.
Again we provide some more detail on the whole process:
✅ Original model: Llama-3-8b-Instruct
✅ Training Duration: 12 hours
✅ Training procedure: 2-staged DPO
✅ Trained data: 70k (first stage) and 20k (second stage)
✅ GPU: 4x RTX6000 ADA
✅ New model: Llama-3-SauerkrautLM-8b-Instruct
✅ Total training costs: 54,72 Dollar 💴 - RunPod FTW (excluding synthesizing data, curating data, benchmarks, error handling, testing)
See our model card on Hugging Face for more details: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
There will be more details on benchmarks during the next days.
Said and done. Here it is. Our Sauerkraut Version of the strong Llama3-8b by Meta. Released from HANNOVER MESSE, just in front of meta booth.
VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
According to benchmarks (LM-Evaluation-Harness 0.4.2), our #SauerkrautLM Dataset and fine-tuning pipeline improved the Model noticeably (AVG = 74,57), especially Reasoning and Common Sense capabilities.
Again we provide some more detail on the whole process:
✅ Original model: Llama-3-8b-Instruct
✅ Training Duration: 12 hours
✅ Training procedure: 2-staged DPO
✅ Trained data: 70k (first stage) and 20k (second stage)
✅ GPU: 4x RTX6000 ADA
✅ New model: Llama-3-SauerkrautLM-8b-Instruct
✅ Total training costs: 54,72 Dollar 💴 - RunPod FTW (excluding synthesizing data, curating data, benchmarks, error handling, testing)
See our model card on Hugging Face for more details: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
There will be more details on benchmarks during the next days.