Running
on
CPU Upgrade
7
๐ง
Hugging Face Discord Community
community
AI & ML interests
Collaborating towards Good ML!
Recent Activity
View all activity
discord-community's activity
Not able to see any channel even after completing the Discord verification
3
#2 opened 4 days ago
by
hyraxdata
attempt at preventing some of that endless clutter
1
#4 opened about 1 month ago
by
nroggendorff
Verify Your Email
3
#30 opened 4 days ago
by
vsanchezn
Not able to see any channel even after completing the Discord verification
3
#2 opened 4 days ago
by
hyraxdata
Verify Your Email
3
#30 opened 4 days ago
by
vsanchezn
Not able to see any channel even after completing the Discord verification
3
#2 opened 4 days ago
by
hyraxdata
Verify Your Email
3
#30 opened 4 days ago
by
vsanchezn
not-lainย
updated
a
Space
4 days ago
I am not redirected after login
9
#29 opened 10 days ago
by
eduardoworrel
Post
1533
๐๐๐ฐ ๐๐ญ๐๐ฅ๐ข๐๐ง ๐๐ฆ๐๐ฅ๐ฅ ๐๐๐ง๐ ๐ฎ๐๐ ๐ ๐๐จ๐๐๐ฅ๐ฌ: ๐๐๐ฆ๐ฆ๐ ๐๐๐จ๐ ๐๐ง๐๐ฌ๐ข๐ฌ ๐๐จ๐ฅ๐ฅ๐๐๐ญ๐ข๐จ๐ง ๐๐๐ฎ๐น
I am happy to release two new language models for the Italian Language!
๐ช Gemma 2 9B Neogenesis ITA
anakin87/gemma-2-9b-neogenesis-ita
Building on the impressive work by VAGO Solutions, I applied Direct Preference Optimization with a mix of Italian and English data.
Using Spectrum, I trained 20% of model layers.
๐ Evaluated on the Open ITA LLM leaderboard ( mii-llm/open_ita_llm_leaderboard), this model achieves strong performance.
To beat it on this benchmark, you'd need a 27B model ๐
๐ค Gemma 2 2B Neogenesis ITA
anakin87/gemma-2-2b-neogenesis-ita
This smaller variant is fine-tuned from the original Gemma 2 2B it by Google.
Through a combination of Supervised Fine-Tuning and Direct Preference Optimization, I trained 25% of the layers using Spectrum.
๐ Compared to the original model, it shows improved Italian proficiency, good for its small size.
Both models were developed during the recent #gemma competition on Kaggle.
๐ Training code: https://www.kaggle.com/code/anakin87/post-training-gemma-for-italian-and-beyond
๐ Thanks @FinancialSupport and mii-llm for the help during evaluation.
I am happy to release two new language models for the Italian Language!
๐ช Gemma 2 9B Neogenesis ITA
anakin87/gemma-2-9b-neogenesis-ita
Building on the impressive work by VAGO Solutions, I applied Direct Preference Optimization with a mix of Italian and English data.
Using Spectrum, I trained 20% of model layers.
๐ Evaluated on the Open ITA LLM leaderboard ( mii-llm/open_ita_llm_leaderboard), this model achieves strong performance.
To beat it on this benchmark, you'd need a 27B model ๐
๐ค Gemma 2 2B Neogenesis ITA
anakin87/gemma-2-2b-neogenesis-ita
This smaller variant is fine-tuned from the original Gemma 2 2B it by Google.
Through a combination of Supervised Fine-Tuning and Direct Preference Optimization, I trained 25% of the layers using Spectrum.
๐ Compared to the original model, it shows improved Italian proficiency, good for its small size.
Both models were developed during the recent #gemma competition on Kaggle.
๐ Training code: https://www.kaggle.com/code/anakin87/post-training-gemma-for-italian-and-beyond
๐ Thanks @FinancialSupport and mii-llm for the help during evaluation.
- 3 replies
I am not redirected after login
9
#29 opened 10 days ago
by
eduardoworrel
I am not redirected after login
9
#29 opened 10 days ago
by
eduardoworrel
I am not redirected after login
9
#29 opened 10 days ago
by
eduardoworrel
Post
520
Hey, it has been a while... I was busy participating in ๐ ๐๐๐ฆ๐ฆ๐ ๐๐จ๐ฆ๐ฉ๐๐ญ๐ข๐ญ๐ข๐จ๐ง!
Here's the idea: Gemma open models have a large vocabulary size (256K), so improving them for a specific language or cultural context should be pretty affordable - no need for continued pre-training.
My submission: ๐๐๐ฎ๐น ๐๐๐จ๐ ๐๐ง๐๐ฌ๐ข๐ฌ - ๐๐จ๐ฌ๐ญ-๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐๐ฆ๐ฆ๐ ๐๐จ๐ซ ๐๐ญ๐๐ฅ๐ข๐๐ง ๐๐ง๐ ๐๐๐ฒ๐จ๐ง๐
๐ Kaggle notebook: https://www.kaggle.com/code/anakin87/post-training-gemma-for-italian-and-beyond
In this notebook, I show how I improve the performance of Gemma 2 2B on Italian via Post-Training.
I believe this method is adaptable to other languages and model sizes.
๐๐ฆ๐บ ๐๐ต๐ฆ๐ฑ๐ด
๐ Choose reference metrics
๐งโ๐ฌ Data curation for Instruction Fine Tuning: identify existing datasets + generate synthetic data
๐๏ธโโ๏ธ Efficient Instruction Fine Tuning with Spectrum
๐งโ๐ฌ Data curation for Preference Tuning: identify existing datasets + generate synthetic data
๐๐ Efficient Direct Preference Optimization with Spectrum
๐ Evaluation
๐ค Hugging Face collection (with models and datasets): anakin87/gemma-neogenesis-67824b7bf13ac9cfe091fe2e
I'm also planning a ๐ Gemma Giveaway (on LinkedIn - https://www.linkedin.com/in/stefano-fiorucci) in the next few days - sharing techniques, datasets, and models I used for my project... so stay tuned! ๐ป
Here's the idea: Gemma open models have a large vocabulary size (256K), so improving them for a specific language or cultural context should be pretty affordable - no need for continued pre-training.
My submission: ๐๐๐ฎ๐น ๐๐๐จ๐ ๐๐ง๐๐ฌ๐ข๐ฌ - ๐๐จ๐ฌ๐ญ-๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐๐ฆ๐ฆ๐ ๐๐จ๐ซ ๐๐ญ๐๐ฅ๐ข๐๐ง ๐๐ง๐ ๐๐๐ฒ๐จ๐ง๐
๐ Kaggle notebook: https://www.kaggle.com/code/anakin87/post-training-gemma-for-italian-and-beyond
In this notebook, I show how I improve the performance of Gemma 2 2B on Italian via Post-Training.
I believe this method is adaptable to other languages and model sizes.
๐๐ฆ๐บ ๐๐ต๐ฆ๐ฑ๐ด
๐ Choose reference metrics
๐งโ๐ฌ Data curation for Instruction Fine Tuning: identify existing datasets + generate synthetic data
๐๏ธโโ๏ธ Efficient Instruction Fine Tuning with Spectrum
๐งโ๐ฌ Data curation for Preference Tuning: identify existing datasets + generate synthetic data
๐๐ Efficient Direct Preference Optimization with Spectrum
๐ Evaluation
๐ค Hugging Face collection (with models and datasets): anakin87/gemma-neogenesis-67824b7bf13ac9cfe091fe2e
I'm also planning a ๐ Gemma Giveaway (on LinkedIn - https://www.linkedin.com/in/stefano-fiorucci) in the next few days - sharing techniques, datasets, and models I used for my project... so stay tuned! ๐ป
"Talking with Documents" with LoRA
2
#28 opened 16 days ago
by
Luli1409
๐ฉ Report: Not working
1
#27 opened 20 days ago
by
wrengy
How to evaluate zero-shot Llama3.1 !!
7
#1 opened 26 days ago
by
adeem6