Hugging Face's logo

p1atdev
/
mt5-base-xlsum-ja

Summarization
Transformers
PyTorch
Safetensors
Japanese
mt5
text2text-generation
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
1

mt5-base-xlsum-ja

This model is a fine-tuned version of google/mt5-base on the xlsum dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 90
  • num_epochs: 10

Training results

Framework versions

  • Transformers 4.34.0
  • Pytorch 2.0.0+cu118
  • Datasets 2.14.5
  • Tokenizers 0.14.0

Related Models

Downloads last month
66
Safetensors
Model size
582M params
Tensor type
BF16
·
Inference Examples
Summarization
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for p1atdev/mt5-base-xlsum-ja

Base model

google/mt5-base
Finetuned
(161)
this model

Dataset used to train p1atdev/mt5-base-xlsum-ja

Collection including p1atdev/mt5-base-xlsum-ja

Evaluation results

View on Papers With Code