README.md

metadata

base_model: h2oai/h2o-danube3-500m-base
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft

Uploaded model

base model pretrained on 2.5 epochs on no-prompt-oasst

hf (pretrained=appvoid/palmer-004-turbo-v1.2), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
arc_challenge	1	none	0	acc	↑	0.3097	±	0.0135
		none	0	acc_norm	↑	0.3464	±	0.0139
hellaswag	1	none	0	acc	↑	0.4660	±	0.0050
		none	0	acc_norm	↑	0.6130	±	0.0049
mmlu	1	none		acc	↑	0.2728	±	0.0037
- humanities	1	none		acc	↑	0.2593	±	0.0064
- formal_logic	0	none	0	acc	↑	0.3175	±	0.0416
- high_school_european_history	0	none	0	acc	↑	0.2970	±	0.0357
- high_school_us_history	0	none	0	acc	↑	0.2941	±	0.0320
- high_school_world_history	0	none	0	acc	↑	0.2489	±	0.0281
- international_law	0	none	0	acc	↑	0.3306	±	0.0429
- jurisprudence	0	none	0	acc	↑	0.2685	±	0.0428
- logical_fallacies	0	none	0	acc	↑	0.2515	±	0.0341
- moral_disputes	0	none	0	acc	↑	0.2486	±	0.0233
- moral_scenarios	0	none	0	acc	↑	0.2380	±	0.0142
- philosophy	0	none	0	acc	↑	0.2797	±	0.0255
- prehistory	0	none	0	acc	↑	0.2593	±	0.0244
- professional_law	0	none	0	acc	↑	0.2555	±	0.0111
- world_religions	0	none	0	acc	↑	0.2339	±	0.0325
- other	1	none		acc	↑	0.2813	±	0.0080
- business_ethics	0	none	0	acc	↑	0.1700	±	0.0378
- clinical_knowledge	0	none	0	acc	↑	0.3434	±	0.0292
- college_medicine	0	none	0	acc	↑	0.3006	±	0.0350
- global_facts	0	none	0	acc	↑	0.3600	±	0.0482
- human_aging	0	none	0	acc	↑	0.1659	±	0.0250
- management	0	none	0	acc	↑	0.3883	±	0.0483
- marketing	0	none	0	acc	↑	0.2137	±	0.0269
- medical_genetics	0	none	0	acc	↑	0.2500	±	0.0435
- miscellaneous	0	none	0	acc	↑	0.2784	±	0.0160
- nutrition	0	none	0	acc	↑	0.2386	±	0.0244
- professional_accounting	0	none	0	acc	↑	0.2376	±	0.0254
- professional_medicine	0	none	0	acc	↑	0.4265	±	0.0300
- virology	0	none	0	acc	↑	0.3133	±	0.0361
- social sciences	1	none		acc	↑	0.2847	±	0.0081
- econometrics	0	none	0	acc	↑	0.2719	±	0.0419
- high_school_geography	0	none	0	acc	↑	0.3788	±	0.0346
- high_school_government_and_politics	0	none	0	acc	↑	0.2850	±	0.0326
- high_school_macroeconomics	0	none	0	acc	↑	0.3359	±	0.0239
- high_school_microeconomics	0	none	0	acc	↑	0.3361	±	0.0307
- high_school_psychology	0	none	0	acc	↑	0.3064	±	0.0198
- human_sexuality	0	none	0	acc	↑	0.2519	±	0.0381
- professional_psychology	0	none	0	acc	↑	0.2190	±	0.0167
- public_relations	0	none	0	acc	↑	0.3091	±	0.0443
- security_studies	0	none	0	acc	↑	0.2449	±	0.0275
- sociology	0	none	0	acc	↑	0.2388	±	0.0301
- us_foreign_policy	0	none	0	acc	↑	0.2800	±	0.0451
- stem	1	none		acc	↑	0.2731	±	0.0079
- abstract_algebra	0	none	0	acc	↑	0.1900	±	0.0394
- anatomy	0	none	0	acc	↑	0.1926	±	0.0341
- astronomy	0	none	0	acc	↑	0.2829	±	0.0367
- college_biology	0	none	0	acc	↑	0.2083	±	0.0340
- college_chemistry	0	none	0	acc	↑	0.3400	±	0.0476
- college_computer_science	0	none	0	acc	↑	0.3000	±	0.0461
- college_mathematics	0	none	0	acc	↑	0.3100	±	0.0465
- college_physics	0	none	0	acc	↑	0.2941	±	0.0453
- computer_security	0	none	0	acc	↑	0.2400	±	0.0429
- conceptual_physics	0	none	0	acc	↑	0.2553	±	0.0285
- electrical_engineering	0	none	0	acc	↑	0.2828	±	0.0375
- elementary_mathematics	0	none	0	acc	↑	0.2513	±	0.0223
- high_school_biology	0	none	0	acc	↑	0.2935	±	0.0259
- high_school_chemistry	0	none	0	acc	↑	0.2808	±	0.0316
- high_school_computer_science	0	none	0	acc	↑	0.2400	±	0.0429
- high_school_mathematics	0	none	0	acc	↑	0.2741	±	0.0272
- high_school_physics	0	none	0	acc	↑	0.3775	±	0.0396
- high_school_statistics	0	none	0	acc	↑	0.3194	±	0.0318
- machine_learning	0	none	0	acc	↑	0.2321	±	0.0401
piqa	1	none	0	acc	↑	0.7269	±	0.0104
		none	0	acc_norm	↑	0.7242	±	0.0104
winogrande	1	none	0	acc	↑	0.5848	±	0.0138

Groups	Version	Filter	n-shot	Metric		Value		Stderr
mmlu	1	none		acc	↑	0.2728	±	0.0037
- humanities	1	none		acc	↑	0.2593	±	0.0064
- other	1	none		acc	↑	0.2813	±	0.0080
- social sciences	1	none		acc	↑	0.2847	±	0.0081
- stem	1	none		acc	↑	0.2731	±	0.0079