Canadian Appellate Judgement Model

This model is a fine-tuned version of bigscience/bloom-560m on Canadian appellate decisions (Ontario Court of Appeal and the British Columbia Court of Appeal) found in the Pile of Law dataset. It achieves the following results on the evaluation set:

• Loss: 2.0135

Intended uses & limitations

This model is intended to facilitate research into large language models and legal reasoning. It is not intended for use in any legal domain or to support legal work .

Training procedure

This model was trained using the methodology set out in this notebook.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.23.1
• Pytorch 1.11.0
• Datasets 2.5.2
• Tokenizers 0.13.1