RichardErkhov/curiositytech_-_MARS-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

MARS - GGUF

• Model creator: https://huggingface.co/curiositytech/
• Original model: https://huggingface.co/curiositytech/MARS/

Name	Quant method	Size
MARS.Q2_K.gguf	Q2_K	2.96GB
MARS.Q3_K_S.gguf	Q3_K_S	3.41GB
MARS.Q3_K.gguf	Q3_K	3.74GB
MARS.Q3_K_M.gguf	Q3_K_M	3.74GB
MARS.Q3_K_L.gguf	Q3_K_L	4.03GB
MARS.IQ4_XS.gguf	IQ4_XS	4.18GB
MARS.Q4_0.gguf	Q4_0	4.34GB
MARS.IQ4_NL.gguf	IQ4_NL	4.38GB
MARS.Q4_K_S.gguf	Q4_K_S	4.37GB
MARS.Q4_K.gguf	Q4_K	4.58GB
MARS.Q4_K_M.gguf	Q4_K_M	4.58GB
MARS.Q4_1.gguf	Q4_1	4.78GB
MARS.Q5_0.gguf	Q5_0	5.21GB
MARS.Q5_K_S.gguf	Q5_K_S	5.21GB
MARS.Q5_K.gguf	Q5_K	5.34GB
MARS.Q5_K_M.gguf	Q5_K_M	5.34GB
MARS.Q5_1.gguf	Q5_1	5.65GB
MARS.Q6_K.gguf	Q6_K	6.14GB
MARS.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: llama3 language: - tr - en base_model: meta-llama/Meta-Llama-3-8B-Instruct model-index: - name: MARS results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge TR v0.2 type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc value: 46.08 name: accuracy - task: type: text-generation name: Text Generation dataset: name: MMLU TR v0.2 type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 47.02 name: accuracy - task: type: text-generation name: Text Generation dataset: name: TruthfulQA TR v0.2 type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: acc name: accuracy value: 49.38 - task: type: text-generation name: Text Generation dataset: name: Winogrande TR v0.2 type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 53.71 name: accuracy - task: type: text-generation name: Text Generation dataset: name: GSM8k TR v0.2 type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 53.08 name: accuracy pipeline_tag: text-generation

MARS

by Curiosity Technology

MARS is the first iteration of Curiosity Technology models, based on Llama 3 8B.

We have trained MARS on in-house Turkish dataset, as well as several open-source datasets and their Turkish translations. It is our intention to release Turkish translations in near future for community to have their go on them.

MARS have been trained for 3 days on 4xA100.

Model Details

• Base Model: Meta Llama 3 8B Instruct
• Training Dataset: In-house & Translated Open Source Turkish Datasets
• Training Method: LoRA Fine Tuning

How to use

You can run conversational inference using the Transformers pipeline abstraction, or by leveraging the Auto classes with the generate() function. Let's see examples of both.

Transformers pipeline

import transformers
import torch

model_id = "curiositytech/MARS"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "Sen korsan gibi konuşan bir korsan chatbotsun!"},
    {"role": "user", "content": "Sen kimsin?"},
]

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    messages,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][-1])

Transformers AutoModelForCausalLM

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "curiositytech/MARS"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "Sen korsan gibi konuşan bir korsan chatbotsun!"},
    {"role": "user", "content": "Sen kimsin?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))