llama-3-8b-chat-patent-small
This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the english translation of a small dataset of 16,000 Korean patents.
Model description
This model is provided for testing purposes only.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 3.0
- mixed_precision_training: Native AMP
Framework versions
- Transformers 4.39.3
- Pytorch 2.1.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 38
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for kimhyeongjun/llama-3-8b-patent-small-dataset
Base model
meta-llama/Meta-Llama-3-8B-Instruct
Finetuned
this model