File size: 2,848 Bytes
1f54b88 4946369 1f54b88 3122c6c 51a604c 3122c6c 1f54b88 deefc28 ec8bb94 1f54b88 4946369 da8efc4 255becb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 |
---
pipeline_tag: text-generation
inference: true
widget:
- text: 'def factorial(n):'
example_title: Factorial
group: Python
- text: 'def recur_fibo(n):'
example_title: Recursive Fibonacci
group: Python
license: llama2
library_name: transformers
tags:
- text-generation
- code
language:
- en
---
# lemur-70b-v1
<p align="center">
<img src="https://huggingface.co/datasets/OpenLemur/assets/resolve/main/lemur_icon.png" width="300" height="300" alt="Lemur">
</p>
<div align="center">
<img src="https://huggingface.co/datasets/OpenLemur/assets/resolve/main/lemur_base_radar.png">
</div>
📄Paper: https://arxiv.org/abs/2310.06830
👩💻Code: https://github.com/OpenLemur/Lemur
## Use
### Setup
First, we have to install all the libraries listed in `requirements.txt` in [GitHub](https://github.com/OpenLemur/lemur-v1):
```bash
pip install -r requirements.txt
```
### Intended use
Since it is not trained on instruction following corpus, it won't respond well to questions like "What is the Python code to do quick sort?".
### Generation
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("OpenLemur/lemur-70b-v1")
model = AutoModelForCausalLM.from_pretrained("OpenLemur/lemur-70b-v1", device_map="auto", load_in_8bit=True)
# Text Generation Example
prompt = "The world is "
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=50, num_return_sequences=1)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)
# Code Generation Example
prompt = """
def factorial(n):
if n == 0:
return 1
"""
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=200, num_return_sequences=1)
generated_code = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_code)
```
# License
The model is licensed under the Llama-2 community license agreement.
# Acknowledgements
The Lemur project is an open collaborative research effort between [XLang Lab](https://www.xlang.ai/) and Salesforce Research. We thank Salesforce, Google Research and Amazon AWS for their gift support.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_OpenLemur__lemur-70b-v1)
| Metric | Value |
|-----------------------|---------------------------|
| Avg. | 54.03 |
| ARC (25-shot) | 64.33 |
| HellaSwag (10-shot) | 85.72 |
| MMLU (5-shot) | 65.85 |
| TruthfulQA (0-shot) | 44.78 |
| Winogrande (5-shot) | 83.03 |
| GSM8K (5-shot) | 28.73 |
| DROP (3-shot) | 5.74 |
|