meditsolutions/Llama-3.2-SUN-HDIC-1B-Instruct
Text Generation
β’
Updated
β’
272
MedIT Solutions
company
Verified
AI & ML interests
None defined yet.
Recent Activity
View all activity
meditsolutions's activity
mkurmanΒ
updated
a
model
24 days ago
Post
294
How Do I Contribute (HDIC)
Exciting times to come? We are working on a layer self-esteem technique to score their contribution to the final prediction. For now, it unlocks a lot of knowledge already stored in weights we couldn't force the model to extract by further fine-tuning!
Exciting times to come? We are working on a layer self-esteem technique to score their contribution to the final prediction. For now, it unlocks a lot of knowledge already stored in weights we couldn't force the model to extract by further fine-tuning!
Post
434
What AI-enhanced research tools would you recommend for searching and analyzing scientific papers?
-
- 5 replies
Adding Evaluation Results
#1 opened about 1 month ago
by
leaderboard-pr-bot
Post
1178
We built a new small language model SmolLM2-MedIT-Upscale-2B, based on SmolLM2-1.7B-Instruct from Hugging Face. The premise was simple - increasing the vector in attention layers would positively impact the model's capabilities.
What did we prove?
In total, not much really, since we don't have the original trained under the same conditions as our upscale. However...
1. We scaled up the model without losing its quality
2. We confirmed that the method we devised works
3. After extremely short fine-tuning, the model achieved much better results in IFEval compared to the original (53.68 vs 64.29) and a higher overall average score in Open LLM Leaderboard (14.75 vs 15.17)
I consider this a big success π, since surpassing the original in metrics is often very time-consuming, generates high costs, and doesn't always work out.
Meanwhile, we're moving forward, training SmolLM2 400M Instruct as an upscale of 136M.
We're curious about how increasing the base and intermediate vectors will affect the model's quality. We'll compare it to the original and the 360M Instruct version released by Hugging Face.
License: Apache 2.0ββββββββββββββββ
meditsolutions/SmolLM2-MedIT-Upscale-2B
What did we prove?
In total, not much really, since we don't have the original trained under the same conditions as our upscale. However...
1. We scaled up the model without losing its quality
2. We confirmed that the method we devised works
3. After extremely short fine-tuning, the model achieved much better results in IFEval compared to the original (53.68 vs 64.29) and a higher overall average score in Open LLM Leaderboard (14.75 vs 15.17)
I consider this a big success π, since surpassing the original in metrics is often very time-consuming, generates high costs, and doesn't always work out.
Meanwhile, we're moving forward, training SmolLM2 400M Instruct as an upscale of 136M.
We're curious about how increasing the base and intermediate vectors will affect the model's quality. We'll compare it to the original and the 360M Instruct version released by Hugging Face.
License: Apache 2.0ββββββββββββββββ
meditsolutions/SmolLM2-MedIT-Upscale-2B
mkurmanΒ
updated
a
model
about 1 month ago
Adding Evaluation Results
#1 opened about 1 month ago
by
mkurman
mkurmanΒ
updated
a
model
about 1 month ago
mkurmanΒ
updated
a
collection
about 1 month ago
mkurmanΒ
updated
a
collection
about 2 months ago
mkurmanΒ
updated
2
models
about 2 months ago
Adding Evaluation Results
#1 opened about 2 months ago
by
leaderboard-pr-bot
mkurmanΒ
updated
a
model
about 2 months ago
Post
715
We are happy to introduce MedIT SUN 1B, a downscaled version of the MedIT SUN 2.5B Llama 3.2 variant.
Give it a try!
meditsolutions/Llama-3.2-SUN-1B-chat
Give it a try!
meditsolutions/Llama-3.2-SUN-1B-chat
mkurmanΒ
updated
a
collection
2 months ago
mkurmanΒ
updated
4
models
2 months ago
meditsolutions/Llama-3.2-SUN-2.5B-chat
Text Generation
β’
Updated
β’
355
β’
1
meditsolutions/MedIT-Mesh-3B-Instruct
Updated
β’
23
meditsolutions/MSH-v1-Bielik-v2.3-Instruct-MedIT-merge-GGUF
Text Generation
β’
Updated
β’
7
β’
1
meditsolutions/MSH-v1-Bielik-v2.3-Instruct-MedIT-merge
Text Generation
β’
Updated
β’
1.64k
β’
1