--- language: - en license: apache-2.0 base_model: Locutusque/TinyMistral-248M datasets: - HuggingFaceH4/ultrachat_200k - Felladrin/ChatML-ultrachat_200k - Open-Orca/OpenOrca - Felladrin/ChatML-OpenOrca - hkust-nlp/deita-10k-v0 - Felladrin/ChatML-deita-10k-v0 - LDJnr/Capybara - Felladrin/ChatML-Capybara - databricks/databricks-dolly-15k - Felladrin/ChatML-databricks-dolly-15k - euclaise/reddit-instruct-curated - Felladrin/ChatML-reddit-instruct-curated - CohereForAI/aya_dataset - Felladrin/ChatML-aya_dataset pipeline_tag: text-generation widget: - messages: - role: system content: You are a highly knowledgeable and friendly assistant. Your goal is to understand and respond to user inquiries with clarity. Your interactions are always respectful, helpful, and focused on delivering the most accurate information to the user. - role: user content: Hey! Got a question for you! - role: assistant content: Sure! What's it? - role: user content: What are some potential applications for quantum computing? - messages: - role: user content: Heya! - role: assistant content: Hi! How may I help you? - role: user content: I'm interested in developing a career in software engineering. What would you recommend me to do? - messages: - role: user content: Morning! - role: assistant content: Good morning! How can I help you today? - role: user content: Could you give me some tips for becoming a healthier person? - messages: - role: system content: You are a very creative assistant. User will give you a task, which you should complete with all your knowledge. - role: user content: Hello! Can you please elaborate a background story of an RPG game about wizards and dragons in a sci-fi world? inference: parameters: max_new_tokens: 250 penalty_alpha: 0.5 top_k: 5 model-index: - name: TinyMistral-248M-Chat-v2 results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 23.29 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 27.39 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 23.52 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 41.32 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 49.01 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 0.0 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/TinyMistral-248M-Chat-v2 name: Open LLM Leaderboard --- # Locutusque's TinyMistral-248M trained on chat datasets - Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M) with two additional special tokens (`<|im_start|>` and `<|im_end|>`) - Datasets: - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-ultrachat_200k)] [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-OpenOrca)] [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-deita-10k-v0)] [hkust-nlp/deita-10k-v0](https://huggingface.co/datasets/hkust-nlp/deita-10k-v0) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-Capybara)] [LDJnr/Capybara](https://huggingface.co/datasets/LDJnr/Capybara) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-databricks-dolly-15k)] [databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-reddit-instruct-curated)] [euclaise/reddit-instruct-curated](https://huggingface.co/datasets/euclaise/reddit-instruct-curated) - [[ChatML](https://huggingface.co/datasets/Felladrin/ChatML-aya_dataset)] [CohereForAI/aya_dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset) - License: [Apache License 2.0](https://huggingface.co/Felladrin/TinyMistral-248M-Chat-v2/resolve/main/license.txt) - Availability in other ML formats: - GGUF: [Felladrin/gguf-TinyMistral-248M-Chat-v2](https://huggingface.co/Felladrin/gguf-TinyMistral-248M-Chat-v2) - ONNX: [Felladrin/onnx-TinyMistral-248M-Chat-v2](https://huggingface.co/Felladrin/onnx-TinyMistral-248M-Chat-v2) ## Recommended Prompt Format ``` <|im_start|>system {system_message}<|im_end|> <|im_start|>user {user_message}<|im_end|> <|im_start|>assistant ``` ## Recommended Inference Parameters ```yml penalty_alpha: 0.5 top_k: 5 ``` ## Usage Example ```python from transformers import pipeline generate = pipeline("text-generation", "Felladrin/TinyMistral-248M-Chat-v2") messages = [ { "role": "system", "content": "You are a highly knowledgeable and friendly assistant. Your goal is to understand and respond to user inquiries with clarity. Your interactions are always respectful, helpful, and focused on delivering the most accurate information to the user.", }, { "role": "user", "content": "Hey! Got a question for you!", }, { "role": "assistant", "content": "Sure! What's it?", }, { "role": "user", "content": "What are some potential applications for quantum computing?", }, ] prompt = generate.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) output = generate( prompt, max_new_tokens=256, penalty_alpha=0.5, top_k=5, ) print(output[0]["generated_text"]) ``` ## How it was trained This model was trained with [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) using the following settings: | Hyperparameter | Value | | :--------------------- | :-------------------------------------------- | | Learning rate | 2e-5 | | Total train batch size | 32 | | Max. sequence length | 2048 | | Weight decay | 0.01 | | Warmup ratio | 0.1 | | NEFTune Noise Alpha | 5 | | Optimizer | Adam with betas=(0.9,0.999) and epsilon=1e-08 | | Scheduler | cosine | | Seed | 42 | ## [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Felladrin__TinyMistral-248M-Chat-v2) | Metric |Value| |---------------------------------|----:| |Avg. |27.42| |AI2 Reasoning Challenge (25-Shot)|23.29| |HellaSwag (10-Shot) |27.39| |MMLU (5-Shot) |23.52| |TruthfulQA (0-shot) |41.32| |Winogrande (5-shot) |49.01| |GSM8k (5-shot) | 0.00|