--- language: - en datasets: - natural_instructions - the_pile - cot - Muennighoff/P3 tags: - gpt pipeline_tag: text-generation inference: parameters: temperature: 0.1 widget: - text: "Is this review positive or negative? Review: Best cast iron skillet you will ever buy. Answer:" example_title: "Sentiment analysis" - text: "Where is Zurich? Ans:" example_title: "Question Answering" --- # Quick Start ```python from transformers import pipeline pipe = pipeline(model='togethercomputer/GPT-JT-6B-v0') pipe("Where is Zurich? Ans:") ``` # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_togethercomputer__GPT-JT-6B-v0) | Metric | Value | |-----------------------|---------------------------| | Avg. | 38.37 | | ARC (25-shot) | 42.06 | | HellaSwag (10-shot) | 67.96 | | MMLU (5-shot) | 49.34 | | TruthfulQA (0-shot) | 38.89 | | Winogrande (5-shot) | 64.8 | | GSM8K (5-shot) | 1.21 | | DROP (3-shot) | 4.31 |