File size: 1,514 Bytes
314538e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/model-cards
{}
---
# Model Card for Phi 1.5 SlimOrca
<!-- Provide a quick summary of what the model is/does. -->
Phi 1.5 finetuned on SlimOrca-Dedup. This model was trained with the goal of giving Phi 1.5 the ablity to generate the EOS token together with being capable of doing beam search.
## Model Details
## How to Get Started with the Model
```python
import torch
import transformers
model = transformers.AutoModelForCausalLM.from_pretrained(
"miguelcarv/phi-1_5-slimorca",
trust_remote_code=True
)
tokenizer = transformers.AutoTokenizer.from_pretrained("microsoft/phi-1_5")
SYSTEM_PROMPT = "You are an AI assistant. You will be given a task. You must generate a detailed and long answer."
input_text = f"""{SYSTEM_PROMPT}
Instruction: Give me the first 5 prime numbers and explain what prime numbers are.
Output:"""
with torch.no_grad():
outputs = model.generate(
tokenizer(input_text, return_tensors="pt")['input_ids'],
max_length=256,
num_beams = 3,
eos_token_id = tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
## Training Details
- Trained for one epoch on SlimOrca-Dedup
- Learning rate: 1e-5
- Optimizer: AdamW
- Effective batch size: 64
- Gradient accumulation steps (mini batch size): 16 (4)
- Trained with FP32
|