---
tags:
- generated_from_trainer
model-index:
- name: my_awesome_pokemon_party_mlm_model
  results: []
widget:
- text: "グライオン ハピナス クレセリア クレベース モロバレル [MASK]"
  example_title: "example1"
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# my_awesome_pokemon_party_mlm_model

ポケモンのパーティの並びで学習した BERT(Masked Language Model) です。
学習に使ったデータは、自分自身で収集したパーティの並びをシャッフルなどして増やしたものを使用しています。

This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4446

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 3.786         | 1.0   | 1995  | 3.7066          |
| 3.5106        | 2.0   | 3990  | 3.5062          |
| 3.3389        | 3.0   | 5985  | 3.2973          |
| 3.2422        | 4.0   | 7980  | 3.3000          |
| 3.173         | 5.0   | 9975  | nan             |
| 3.0516        | 6.0   | 11970 | 3.0902          |
| 2.9928        | 7.0   | 13965 | 3.1138          |
| 2.9509        | 8.0   | 15960 | 3.0007          |
| 2.8988        | 9.0   | 17955 | 3.0047          |
| 2.8105        | 10.0  | 19950 | 2.9341          |
| 2.8212        | 11.0  | 21945 | 2.8955          |
| 2.6472        | 12.0  | 23940 | 2.7615          |
| 2.6196        | 13.0  | 25935 | 2.7013          |
| 2.6267        | 14.0  | 27930 | 2.7081          |
| 2.5083        | 15.0  | 29925 | 2.4976          |
| 2.447         | 16.0  | 31920 | 2.5197          |
| 2.3858        | 17.0  | 33915 | 2.4245          |
| 2.3841        | 18.0  | 35910 | 2.3988          |
| 2.3517        | 19.0  | 37905 | 2.3718          |
| 2.2163        | 20.0  | 39900 | 2.3823          |
| 2.1698        | 21.0  | 41895 | 2.2830          |
| 2.1829        | 22.0  | 43890 | 2.1554          |
| 2.0978        | 23.0  | 45885 | 2.2174          |
| 2.0231        | 24.0  | 47880 | 2.2168          |
| 1.9973        | 25.0  | 49875 | 2.2039          |
| 1.9273        | 26.0  | 51870 | 2.1422          |
| 1.9079        | 27.0  | 53865 | 2.0993          |
| 1.8985        | 28.0  | 55860 | 1.9250          |
| 1.8285        | 29.0  | 57855 | 1.9467          |
| 1.8864        | 30.0  | 59850 | nan             |
| 1.7704        | 31.0  | 61845 | 1.8719          |
| 1.7629        | 32.0  | 63840 | 1.8193          |
| 1.6854        | 33.0  | 65835 | 1.8562          |
| 1.6579        | 34.0  | 67830 | 1.7616          |
| 1.5921        | 35.0  | 69825 | 1.7197          |
| 1.637         | 36.0  | 71820 | nan             |
| 1.6207        | 37.0  | 73815 | nan             |
| 1.6019        | 38.0  | 75810 | 1.6114          |
| 1.5648        | 39.0  | 77805 | nan             |
| 1.5385        | 40.0  | 79800 | nan             |
| 1.5333        | 41.0  | 81795 | nan             |
| 1.5206        | 42.0  | 83790 | 1.6189          |
| 1.4768        | 43.0  | 85785 | 1.5352          |
| 1.4768        | 44.0  | 87780 | 1.5310          |
| 1.5099        | 45.0  | 89775 | 1.5464          |
| 1.455         | 46.0  | 91770 | 1.5714          |
| 1.4361        | 47.0  | 93765 | nan             |
| 1.4291        | 48.0  | 95760 | 1.5281          |
| 1.443         | 49.0  | 97755 | 1.5089          |
| 1.4544        | 50.0  | 99750 | nan             |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0