Edit model card

Canadian Appellate Judgement Model

This model is a fine-tuned version of bigscience/bloom-560m on Canadian appellate decisions (Ontario Court of Appeal and the British Columbia Court of Appeal) found in the Pile of Law dataset. It achieves the following results on the evaluation set:

  • Loss: 2.0135

Intended uses & limitations

This model is intended to facilitate research into large language models and legal reasoning. It is not intended for use in any legal domain or to support legal work .

Training procedure

This model was trained using the methodology set out in this notebook.

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3.0

Training results

Training Loss Epoch Step Validation Loss
2.1285 1.0 8298 2.0347
1.7999 2.0 16596 1.9876
1.6069 3.0 24894 2.0135

Framework versions

  • Transformers 4.23.1
  • Pytorch 1.11.0
  • Datasets 2.5.2
  • Tokenizers 0.13.1
Downloads last month
Hosted inference API
Text Generation
This model can be loaded on the Inference API on-demand.

Space using simonosgoode/bloom-560m-finetuned-cdn_law 1