giannisan/penny5-dolphin-einstein-llama3-dare-ties-chatml

Model Card for Model ID

This is a fine tune of a merged model using the DARE TIES merge method using cognitivecomputations/dolphin-2.9-llama3-8b as a base. The following models were included in the merge:

• Weyaxi/Einstein-v6.1-Llama3-8B

This model should be mostly uncensored out of the box. I personally add a system prompt with the chatml template to guide the model.

Model Details

Quant Q8_0 GGUF

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub.

• Developed by: Gianni Sanrochman
• Funded by: [Merildo Sanrochman]
• Model type: LLaMA-3
• Language(s) (NLP): [English]
• License: llama3
• Finetuned from model: giannisan/dolphin-einstein-llama3-dare-ties using the PENNY dataset

Training Details

Training Data

[More Information Needed]

Training Procedure

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Metric	Value
Avg.	66.72
ARC (25-shot)	61.01
HellaSwag (10-shot)	82.50
MMLU (5-shot)	64.48
TruthfulQA (0-shot)	50.73
Winogrande (5-shot)	74.11
GSM8K (5-shot)	67.48

full results here

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [Nvidia RTX A100]
• Hours used: [2]
• Cloud Provider: [RunPod]
• Compute Region: [Europe]
• Carbon Emitted: [More Information Needed]

Model Card Authors

[Gianni Sanrochman]