README.md

---
license: apache-2.0
language:
- en
- zh
library_name: transformers
pipeline_tag: text-generation
tags:
- llm
- nanbeige
- custom_code
extra_gated_prompt: "访问此模型需要阅读并同意以下协议[这里](https://github.com/Nanbeige/Nanbeige/blob/main/%E5%8D%97%E5%8C%97%E9%98%81%E5%A4%A7%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E8%AE%B8%E5%8F%AF%E5%8D%8F%E8%AE%AE.pdf)\nAccess to this model requires reading and agreeing to the following agreement [here](https://github.com/Nanbeige/Nanbeige/blob/main/License_Agreement_for_Large_Language_Models_Nanbeige.pdf)"
extra_gated_fields:
 Name: text
 Country: text
 Affiliation: text
 Email: text
 I agree to the license terms described in the above agreement: checkbox
---

<!-- markdownlint-disable first-line-h1 -->

<!-- markdownlint-disable html -->

<div align="center">
<h1>
  Nanbeige2-8B-Chat
</h1>
</div>


## <span id="Introduction">Introduction</span>

The Nanbeige2-8B-Chat is the latest 8B model developed by the Nanbeige Lab, which utilized 4.5T tokens of high-quality training data during the training phase. 
During the alignment phase, we initially trained our model using 1 million samples through Supervised Fine-Tuning (SFT). We then engaged in curriculum learning with 400,000 high-quality samples that presented a greater level of difficulty. Subsequently, we incorporated human feedback through the Direct Preference Optimization (DPO), culminating in the development of Nanbeige2-8B-Chat. Nanbeige2-8B-Chat has achieved superior performance across various authoritative benchmark datasets.


## <span id="Evaluation">Evaluation</span>

We have evaluated Nanbeige2-8B-Chat's general question-answering capabilities and human preference alignments on several popular benchmark datasets. The model has achieved notable results in single-turn English QA ([AlpacaEval 2.0](https://tatsu-lab.github.io/alpaca_eval/)), single-turn Chinese QA ([AlignBench](https://github.com/THUDM/AlignBench)), and multi-turn English QA ([MT-Bench](https://arxiv.org/abs/2306.05685)).

| AlpacaEval 2.0 | AlignBench | MT-Bench |
| -------------- | ---------- | -------- |
| 25.2%/39.4%    | 7.03       | 7.82     |

Additionally, we have published the performance of our model on [FlagEval](https://flageval.baai.ac.cn/#/trending)(SFT model leaderboard) and [OpenCompass Leaderboard](https://rank.opencompass.org.cn/leaderboard-llm). It is highly competitive among models with less than 10B parameters.

## <span id="Inference">Inference</span>

```
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
  'Nanbeige/Nanbeige2-8B-Chat',
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  'Nanbeige/Nanbeige2-8B-Chat',
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)
messages = [
  {'role': 'user', 'content': 'Hello'}
]
prompt = tokenizer.apply_chat_template(
  messages,
  add_generation_prompt=True,
  tokenize=False
)
input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids
output_ids = model.generate(input_ids.to('cuda'))
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)
```

## <span id="Limitations">Limitations</span>

While we place great emphasis on the safety of the model during the training process, striving to ensure that its outputs align with ethical and legal requirements, it may not completely avoid generating unexpected outputs due to the model's size and probabilistic nature. These outputs may include harmful content such as bias or discrimination. Please don't propagate such content. We do not assume any responsibility for the consequences resulting from the dissemination of inappropriate information.

## <span id="License">License</span>

When using the Nanbeige models, you must comply with the Apache 2.0 License and the [License Agreement for Large Language Models Nanbeige](https://huggingface.co/Nanbeige/Nanbeige-16B-Base-32k/resolve/main/License_Agreement_for_Large_Language_Models_Nanbeige.pdf). If you intend to use the Nanbeige Models or its derivatives for commercial purposes, please submit application materials to meet the requirements of the Nanbeige Models Community License Agreement by contacting nanbeige@126.com. After review, We will grant you a non-exclusive, worldwide, non-transferable, non-sublicensable and revocable commercial copyright license.