---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: masked-sentence-generation-t5-base
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# masked-sentence-generation-t5-base

This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.7392

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.9984        | 0.05  | 80   | 2.7041          |
| 2.8752        | 0.1   | 160  | 2.7021          |
| 2.9314        | 0.15  | 240  | 2.6966          |
| 2.8541        | 0.2   | 320  | 2.6968          |
| 2.8674        | 0.25  | 400  | 2.6900          |
| 2.8706        | 0.3   | 480  | 2.6886          |
| 2.7718        | 0.34  | 560  | 2.6908          |
| 2.8503        | 0.39  | 640  | 2.6877          |
| 2.8195        | 0.44  | 720  | 2.6902          |
| 2.8569        | 0.49  | 800  | 2.6893          |
| 2.8372        | 0.54  | 880  | 2.6859          |
| 2.8915        | 0.59  | 960  | 2.6898          |
| 2.9687        | 0.64  | 1040 | 2.6909          |
| 2.832         | 0.69  | 1120 | 2.6841          |
| 2.8425        | 0.74  | 1200 | 2.6842          |
| 2.8114        | 0.79  | 1280 | 2.6766          |
| 2.8101        | 0.84  | 1360 | 2.6783          |
| 2.8837        | 0.89  | 1440 | 2.6781          |
| 2.894         | 0.94  | 1520 | 2.6754          |
| 2.9183        | 0.99  | 1600 | 2.6762          |
| 2.6916        | 1.03  | 1680 | 2.6889          |
| 2.5812        | 1.08  | 1760 | 2.6896          |
| 2.5522        | 1.13  | 1840 | 2.6943          |
| 2.5368        | 1.18  | 1920 | 2.6928          |
| 2.5987        | 1.23  | 2000 | 2.6927          |
| 2.5625        | 1.28  | 2080 | 2.6899          |
| 2.4946        | 1.33  | 2160 | 2.6942          |
| 2.5902        | 1.38  | 2240 | 2.6900          |
| 2.5415        | 1.43  | 2320 | 2.6897          |
| 2.5767        | 1.48  | 2400 | 2.6858          |
| 2.6262        | 1.53  | 2480 | 2.6825          |
| 2.6066        | 1.58  | 2560 | 2.6818          |
| 2.5387        | 1.63  | 2640 | 2.6840          |
| 2.5795        | 1.67  | 2720 | 2.6828          |
| 2.5521        | 1.72  | 2800 | 2.6871          |
| 2.5477        | 1.77  | 2880 | 2.6836          |
| 2.587         | 1.82  | 2960 | 2.6824          |
| 2.529         | 1.87  | 3040 | 2.6871          |
| 2.6221        | 1.92  | 3120 | 2.6838          |
| 2.6353        | 1.97  | 3200 | 2.6803          |
| 2.5419        | 2.02  | 3280 | 2.6879          |
| 2.4521        | 2.07  | 3360 | 2.7027          |
| 2.3415        | 2.12  | 3440 | 2.7105          |
| 2.3483        | 2.17  | 3520 | 2.7140          |
| 2.3493        | 2.22  | 3600 | 2.7144          |
| 2.3967        | 2.27  | 3680 | 2.7134          |
| 2.3544        | 2.32  | 3760 | 2.7122          |
| 2.3192        | 2.36  | 3840 | 2.7175          |
| 2.3381        | 2.41  | 3920 | 2.7166          |
| 2.3667        | 2.46  | 4000 | 2.7165          |
| 2.3997        | 2.51  | 4080 | 2.7106          |
| 2.3178        | 2.56  | 4160 | 2.7154          |
| 2.4036        | 2.61  | 4240 | 2.7144          |
| 2.3797        | 2.66  | 4320 | 2.7129          |
| 2.3354        | 2.71  | 4400 | 2.7136          |
| 2.4109        | 2.76  | 4480 | 2.7118          |
| 2.387         | 2.81  | 4560 | 2.7097          |
| 2.3934        | 2.86  | 4640 | 2.7103          |
| 2.3956        | 2.91  | 4720 | 2.7103          |
| 2.4086        | 2.96  | 4800 | 2.7111          |
| 2.4083        | 3.0   | 4880 | 2.7110          |
| 2.3121        | 3.05  | 4960 | 2.7230          |
| 2.263         | 3.1   | 5040 | 2.7252          |
| 2.2722        | 3.15  | 5120 | 2.7296          |
| 2.2053        | 3.2   | 5200 | 2.7309          |
| 2.1969        | 3.25  | 5280 | 2.7363          |
| 2.2684        | 3.3   | 5360 | 2.7396          |
| 2.2789        | 3.35  | 5440 | 2.7376          |
| 2.2227        | 3.4   | 5520 | 2.7384          |
| 2.2886        | 3.45  | 5600 | 2.7390          |
| 2.2182        | 3.5   | 5680 | 2.7376          |
| 2.2738        | 3.55  | 5760 | 2.7394          |
| 2.1687        | 3.6   | 5840 | 2.7386          |
| 2.2548        | 3.65  | 5920 | 2.7371          |
| 2.2391        | 3.69  | 6000 | 2.7372          |
| 2.2031        | 3.74  | 6080 | 2.7391          |
| 2.1885        | 3.79  | 6160 | 2.7400          |
| 2.216         | 3.84  | 6240 | 2.7406          |
| 2.272         | 3.89  | 6320 | 2.7401          |
| 2.3455        | 3.94  | 6400 | 2.7395          |
| 2.2889        | 3.99  | 6480 | 2.7392          |


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0
- Datasets 2.12.0
- Tokenizers 0.11.0