--- language: - en - zh license: other library_name: transformers tags: - llama - qwen license_name: qwen license_link: https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT pipeline_tag: text-generation inference: false model-index: - name: Qwen-14B-Chat-LLaMAfied results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 57.51 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 82.11 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 65.57 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 51.99 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 72.93 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 39.5 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=hiyouga/Qwen-14B-Chat-LLaMAfied name: Open LLM Leaderboard --- This is the LLaMAfied version of [Qwen-14B-Chat](https://huggingface.co/Qwen/Qwen-14B-Chat) model by Alibaba Cloud. This model is converted with https://github.com/hiyouga/LLaMA-Factory/blob/main/tests/llamafy_qwen.py The tokenizer is borrowed from https://huggingface.co/CausalLM/72B-preview-llamafied-qwen-llamafy You may use this model for fine-tuning in downstream tasks, we recommend using our efficient fine-tuning toolkit. https://github.com/hiyouga/LLaMA-Factory - **Developed by:** Alibaba Cloud. - **Language(s) (NLP):** Chinese/English - **License:** [Tongyi Qianwen License](https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT) Usage: ```python from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer tokenizer = AutoTokenizer.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied") model = AutoModelForCausalLM.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied", torch_dtype="auto", device_map="auto") streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True) messages = [ {"role": "user", "content": "Who are you?"} ] inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt") inputs = inputs.to("cuda") generate_ids = model.generate(inputs, streamer=streamer) ``` You could also alternatively launch a CLI demo by using the script in [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) ```bash python src/cli_demo.py --template qwen --model_name_or_path hiyouga/Qwen-14B-Chat-LLaMAfied ``` # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_hiyouga__Qwen-14B-Chat-LLaMAfied) | Metric |Value| |---------------------------------|----:| |Avg. |61.60| |AI2 Reasoning Challenge (25-Shot)|57.51| |HellaSwag (10-Shot) |82.11| |MMLU (5-Shot) |65.57| |TruthfulQA (0-shot) |51.99| |Winogrande (5-shot) |72.93| |GSM8k (5-shot) |39.50|