Model Card for Model ID
Fine-tuned version of Phi-3-mini-4k-instruct on a subset of the hendrycks/ethics dataset
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoModel
model = AutoModel.from_pretrained("fc91/phi3-mini-instruct-full_ethics-lora")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
Training Details
Training Data
The following subsets of the above dataset were leveraged:
-commonsense (10k random samples)
-deontology (10k random samples)
-justice (10k random samples)
-utilitarianism (10k random samples)
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
per_device_train_batch_size=16
per_device_eval_batch_size=32
gradient_accumulation_steps=2
gradient_checkpointing=True
warmup_steps=100
num_train_epochs=1
learning_rate=0.00005
weight_decay=0.01
optim="adamw_hf"
fp16=True
Speeds, Sizes, Times [optional]
The overall training took 3 hours and 23 minutes.
Evaluation
Training Loss = 0.181700
Validation Loss = 0.119734
Testing Data, Factors & Metrics
Testing Data
The following subsets of the above dataset were leveraged:
-commonsense (2.5k random samples)
-deontology (2.5k random samples)
-justice (2.5k random samples)
-utilitarianism (2.5k random samples)
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
NVIDIA A100-SXM4-40GB
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
Unable to determine this model’s pipeline type. Check the
docs
.