RichardErkhov/udkai_-_Turdus-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Turdus - GGUF

• Model creator: https://huggingface.co/udkai/
• Original model: https://huggingface.co/udkai/Turdus/

Name	Quant method	Size
Turdus.Q2_K.gguf	Q2_K	2.53GB
Turdus.IQ3_XS.gguf	IQ3_XS	2.81GB
Turdus.IQ3_S.gguf	IQ3_S	0.65GB
Turdus.Q3_K_S.gguf	Q3_K_S	2.95GB
Turdus.IQ3_M.gguf	IQ3_M	3.06GB
Turdus.Q3_K.gguf	Q3_K	3.28GB
Turdus.Q3_K_M.gguf	Q3_K_M	3.28GB
Turdus.Q3_K_L.gguf	Q3_K_L	3.56GB
Turdus.IQ4_XS.gguf	IQ4_XS	3.67GB
Turdus.Q4_0.gguf	Q4_0	3.83GB
Turdus.IQ4_NL.gguf	IQ4_NL	3.87GB
Turdus.Q4_K_S.gguf	Q4_K_S	3.86GB
Turdus.Q4_K.gguf	Q4_K	4.07GB
Turdus.Q4_K_M.gguf	Q4_K_M	4.07GB
Turdus.Q4_1.gguf	Q4_1	4.24GB
Turdus.Q5_0.gguf	Q5_0	4.65GB
Turdus.Q5_K_S.gguf	Q5_K_S	4.65GB
Turdus.Q5_K.gguf	Q5_K	4.78GB
Turdus.Q5_K_M.gguf	Q5_K_M	4.78GB
Turdus.Q5_1.gguf	Q5_1	5.07GB
Turdus.Q6_K.gguf	Q6_K	5.53GB
Turdus.Q8_0.gguf	Q8_0	7.17GB

Original model description:

base_model: mlabonne/NeuralMarcoro14-7B license: cc-by-nc-4.0 tags: - mlabonne/NeuralMarcoro14-7B - dpo - 7B - winograd - mmlu_abstract_algebra - mistral datasets: - hromi/winograd_dpo_basic

udkai_Turdus

A less contaminated version of udkai/Garrulus and the second model to be discussed in the paper Subtle DPO-Contamination with modified Winogrande increases TruthfulQA, Hellaswag & ARC.

Contrary to Garrulus which was obtained after 2 epochs, this model was obtained after one single epoch of "direct preference optimization" of NeuralMarcoro14-7B with [https://huggingface.co/datasets/hromi/winograd_dpo ] .

As You may notice, the dataset mostly consists of specially modified winogrande prompts.

But before flagging this (or recommending this to be flagged), consider this:

Subtle DPO-Contamination with modified Winogrande causes the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) to be 0.2% higher than the underlying model.

Model	ARC	HellaSwag	MMLU	Truthful QA	GSM8K	Average
mlabonne/NeuralMarcoro14-7B	71.42	87.59	64.84	65.64	70.74	72.046
udkai/Turdus	73.38	88.56	64.52	67.11	67.7	72,254

Yes, as strange as it may sound, one can indeed increase ARC from 71.42% to 73.38 % with one single epoch of cca 1200 repetitive winograd schematas...

BibTex

Should this model - or quasi-methodology which lead to it - be of certain pratical or theoretical interest for You, would be honored if You would refer to it in Your work:

@misc {udk_dot_ai_turdus,
    author       = { {UDK dot AI, Daniel Devatman Hromada} },
    title        = { Turdus (Revision 923c305) },
    year         = 2024,
    url          = { https://huggingface.co/udkai/Turdus },
    doi          = { 10.57967/hf/1611 },
    publisher    = { Hugging Face }
}