metadata
license: bigscience-bloom-rail-1.0
tags:
- generated_from_trainer
model-index:
- name: bloom-560m-finetuned-cdn_law
results: []
Canadian Appellate Judgement Model
This model is a fine-tuned version of bigscience/bloom-560m on Canadian appellate decisions (Ontario Court of Appeal and the British Columbia Court of Appeal) found in the Pile of Law dataset. It achieves the following results on the evaluation set:
- Loss: 2.0135
Intended uses & limitations
This model is intended to facilitate research into large language models and legal reasoning. It is not intended for use in any legal domain or to support legal work .
Training procedure
This model was trained using the methodology set out in this notebook.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.1285 | 1.0 | 8298 | 2.0347 |
| 1.7999 | 2.0 | 16596 | 1.9876 |
| 1.6069 | 3.0 | 24894 | 2.0135 |
Framework versions
- Transformers 4.23.1
- Pytorch 1.11.0
- Datasets 2.5.2
- Tokenizers 0.13.1