Edit model card
Configuration Parsing Warning: In config.json: "quantization_config.bits" must be an integer

image/webp

The Faro chat model focuses on practicality and long-context modeling. It handles various downstream tasks with higher quality, delivering stable and reliable results even when inputs contain lengthy documents or complex instructions. Faro seamlessly works in both English and Chinese.

Faro-Yi-9B

Faro-Yi-9B is an improved Yi-9B-200K with extensive instruction tuning on Fusang-V1. Compared to Yi-9B-200K, Faro-Yi-9B has gained greater capability in various downstream tasks and long-context modeling thanks to the large-scale synthetic data in Fusang-V1.

How to Use

Faro-Yi-9B uses chatml template. This make it easy to set up system prompt and multi-turn conversations.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="cuda"
)
tokenizer = AutoTokenizer.from_pretrained(model_path)

messages = [
    {"role": "system", "content": "You are a helpful assistant. Always answer with a short response."},
    {"role": "user", "content": "Tell me what is Pythagorean theorem like you are a pirate."}
]
input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(input_ids, max_new_tokens=512, temperature=0.5)
response = tokenizer.decode(generated_ids[0], skip_special_tokens=True)

# Aye, matey! The Pythagorean theorem is a nautical rule that helps us find the length of the third side of a triangle. It's like this: if you have a triangle with two sides, you can find the length of the third side by squaring the two sides and then adding them together. The square root of that sum will give you the length of the third side! It's useful for sailing and navigating, so you always know how far you've traveled. Remember, it's all about the sum of squares, me hearties!

Performance

Faro-Yi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially in long-range modeling and bilingual (English, Chinese) understanding. Faro is competitive among all open-sourced models at around 9B parameters.

Benchmark Results

Fact-based Evaluation (Open LLM Leaderboard)

Metric MMLU GSM8K HellaSwag TruthfulQA Arc Winogrande
Yi-9B-200K 65.73 50.49 56.72 33.80 69.25 71.67
Faro-Yi-9B 68.80 63.08 57.28 40.86 72.58 71.11

Long-context Modeling (LongBench)

Name Average_zh Average_en Code Completion
Yi-9B-200K 30.288 36.7071 72.2
Faro-Yi-9B 41.092 40.9536 46.0
Score breakdown
Name Few-shot Learning_en Synthetic Tasks_en Single-Doc QA_en Multi-Doc QA_en Summarization_en Few-shot Learning_zh Synthetic Tasks_zh Single-Doc QA_zh Multi-Doc QA_zh Summarization_zh
Yi-9B-200K 60.6 22.8 30.9 38.9 25.8 46.5 28.0 49.6 17.7 9.7
Faro-Yi-9B 63.8 40.2 36.2 38.0 26.3 30.0 75.1 55.6 30.7 14.1

Bilingual Ability (CMMLU & MMLU)

Name MMLU CMMLU
Yi-9B-200K 65.73 71.97
Faro-Yi-9B 68.80 73.28
Downloads last month
5
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Datasets used to train blockblockblock/Faro-Yi-9B-200K-bpw5.5