--- license: apache-2.0 language: - en - zh metrics: - accuracy library_name: transformers pipeline_tag: question-answering tags: - llm - nanbeige --- **Exllamav2** quant (**exl2** / **3.5 bpw**) made with ExLlamaV2 v0.0.21 Other EXL2 quants: | **Quant** | **Model Size** | **lm_head** | | ----- | ---------- | ------- | |
**[2.2](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-2_2bpw_exl2)**
|
4803 MB
|
6
| |
**[2.5](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-2_5bpw_exl2)**
|
5344 MB
|
6
| |
**[3.0](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-3_0bpw_exl2)**
|
6253 MB
|
6
| |
**[3.5](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-3_5bpw_exl2)**
|
7161 MB
|
6
| |
**[3.75](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-3_75bpw_exl2)**
|
7616 MB
|
6
| |
**[4.0](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-4_0bpw_exl2)**
|
8068 MB
|
6
| |
**[4.25](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-4_25bpw_exl2)**
|
8522 MB
|
6
| |
**[5.0](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-5_0bpw_exl2)**
|
9883 MB
|
6
| |
**[6.0](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-6_0bpw_exl2)**
|
11726 MB
|
8
| |
**[6.5](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-6_5bpw_exl2)**
|
12649 MB
|
8
| |
**[8.0](https://huggingface.co/Zoyd/Nanbeige_Nanbeige2-16B-Chat-8_0bpw_exl2)**
|
14397 MB
|
8
|

Nanbeige2-16B-Chat

## 模型介绍（Introduction） Nanbeige2-16B-Chat是南北阁实验室最新研发的160亿参数模型，在预训练中使用4.5T Tokens高质量语料。 在对齐阶段，我们首先使用了100万条样本进行SFT训练，然后用40万高质量且难度较高的样本进行课程学习，再通过人类反馈DPO，得到Nanbeige2-16B-Chat。Nanbeige2-16B-Chat在各个权威测评数据集上都取得了较优的效果。 The Nanbeige2-16B-Chat is the latest 16B model developed by the Nanbeige Lab, which utilized 4.5T tokens of high-quality training data during the training phase. During the alignment phase, we initially trained our model using 1 million samples through Supervised Fine-Tuning (SFT). We then engaged in curriculum learning with 400,000 high-quality samples that presented a greater level of difficulty. Subsequently, we incorporated human feedback through the Dynamic Policy Optimization (DPO), culminating in the development of Nanbeige2-16B-Chat. Nanbeige2-16B-Chat has achieved superior performance across various authoritative benchmark datasets. ## 模型推理（Inference） ``` from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained( 'Nanbeige/Nanbeige2-16B-Chat', use_fast=False, trust_remote_code=True ) model = AutoModelForCausalLM.from_pretrained( 'Nanbeige/Nanbeige2-16B-Chat', torch_dtype='auto', device_map='auto', trust_remote_code=True ) messages = [ {'role': 'user', 'content': 'Hello'} ] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=False ) input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids output_ids = model.generate(input_ids.to('cuda')) resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True) print(resp) ``` ## 局限性（Limitations） 虽然我们在训练过程中非常注重模型的安全性，力求确保其输出符合伦理和法律要求的文本，但由于模型大小和概率生成范式的限制，无法完全避免产生各种不符合预期的输出情况。这些输出可能包含偏见、歧视等有害内容，请勿传播这些内容。我们不承担因传播不良信息而导致的任何后果。 While we place great emphasis on the safety of the model during the training process, striving to ensure that its outputs align with ethical and legal requirements, it may not completely avoid generating unexpected outputs due to the model's size and probabilistic nature. These outputs may include harmful content such as bias or discrimination. Please don't propagate such content. We do not assume any responsibility for the consequences resulting from the dissemination of inappropriate information. ## 协议（License） 使用 Nanbeige 模型时，您必须遵守 Apache 2.0 许可证和[《南北阁大语言模型许可协议》](https://huggingface.co/Nanbeige/Nanbeige-16B-Base-32k/resolve/main/南北阁大语言模型许可协议.pdf)。如果您打算将 Nanbeige 模型或其衍生产品用于商业目的，请通过以下联系邮箱 nanbeige@126.com 提交申请材料，以满足《南北阁大语言模型许可协议》的要求。经过审核后，我们将授予您非排他性、全球范围内、不可转让、不可再许可、可撤销的商业版权许可。 When using the Nanbeige models, you must comply with the Apache 2.0 License and the [License Agreement for Large Language Models Nanbeige](https://huggingface.co/Nanbeige/Nanbeige-16B-Base-32k/resolve/main/License_Agreement_for_Large_Language_Models_Nanbeige.pdf). If you intend to use the Nanbeige Models or its derivatives for commercial purposes, please submit application materials to meet the requirements of the Nanbeige Models Community License Agreement by contacting nanbeige@126.com. After review, We will grant you a non-exclusive, worldwide, non-transferable, non-sublicensable and revocable commercial copyright license.