---
license: apache-2.0
base_model: facebook/vit-mae-base
tags:
- masked-auto-encoding
- generated_from_trainer
datasets:
- imagefolder
model-index:
- name: vit-pretraining-2024_04_02
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-pretraining-2024_04_02

This model is a fine-tuned version of [facebook/vit-mae-base](https://huggingface.co/facebook/vit-mae-base) on the imagefolder dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0132

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.6875e-06
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 400.0

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| 0.0282        | 1.0   | 3583    | 0.0275          |
| 0.0271        | 2.0   | 7166    | 0.0234          |
| 0.0203        | 3.0   | 10749   | 0.0218          |
| 0.0218        | 4.0   | 14332   | 0.0203          |
| 0.0195        | 5.0   | 17915   | 0.0191          |
| 0.0174        | 6.0   | 21498   | 0.0186          |
| 0.0173        | 7.0   | 25081   | 0.0181          |
| 0.0163        | 8.0   | 28664   | 0.0177          |
| 0.0172        | 9.0   | 32247   | 0.0173          |
| 0.0165        | 10.0  | 35830   | 0.0173          |
| 0.0173        | 11.0  | 39413   | 0.0169          |
| 0.0156        | 12.0  | 42996   | 0.0166          |
| 0.0168        | 13.0  | 46579   | 0.0165          |
| 0.0156        | 14.0  | 50162   | 0.0163          |
| 0.0152        | 15.0  | 53745   | 0.0161          |
| 0.0153        | 16.0  | 57328   | 0.0159          |
| 0.0163        | 17.0  | 60911   | 0.0158          |
| 0.0148        | 18.0  | 64494   | 0.0160          |
| 0.0144        | 19.0  | 68077   | 0.0159          |
| 0.0148        | 20.0  | 71660   | 0.0155          |
| 0.0169        | 21.0  | 75243   | 0.0156          |
| 0.0154        | 22.0  | 78826   | 0.0155          |
| 0.0151        | 23.0  | 82409   | 0.0154          |
| 0.0135        | 24.0  | 85992   | 0.0153          |
| 0.0157        | 25.0  | 89575   | 0.0153          |
| 0.0165        | 26.0  | 93158   | 0.0152          |
| 0.0146        | 27.0  | 96741   | 0.0151          |
| 0.0171        | 28.0  | 100324  | 0.0150          |
| 0.015         | 29.0  | 103907  | 0.0151          |
| 0.0165        | 30.0  | 107490  | 0.0152          |
| 0.0136        | 31.0  | 111073  | 0.0150          |
| 0.0147        | 32.0  | 114656  | 0.0147          |
| 0.0153        | 33.0  | 118239  | 0.0149          |
| 0.0135        | 34.0  | 121822  | 0.0147          |
| 0.0144        | 35.0  | 125405  | 0.0147          |
| 0.0155        | 36.0  | 128988  | 0.0148          |
| 0.0172        | 37.0  | 132571  | 0.0147          |
| 0.0141        | 38.0  | 136154  | 0.0149          |
| 0.0152        | 39.0  | 139737  | 0.0146          |
| 0.014         | 40.0  | 143320  | 0.0148          |
| 0.0158        | 41.0  | 146903  | 0.0147          |
| 0.0151        | 42.0  | 150486  | 0.0145          |
| 0.0155        | 43.0  | 154069  | 0.0147          |
| 0.0152        | 44.0  | 157652  | 0.0148          |
| 0.014         | 45.0  | 161235  | 0.0147          |
| 0.0142        | 46.0  | 164818  | 0.0145          |
| 0.014         | 47.0  | 168401  | 0.0147          |
| 0.0156        | 48.0  | 171984  | 0.0147          |
| 0.0138        | 49.0  | 175567  | 0.0145          |
| 0.0142        | 50.0  | 179150  | 0.0147          |
| 0.0142        | 51.0  | 182733  | 0.0143          |
| 0.014         | 52.0  | 186316  | 0.0143          |
| 0.0139        | 53.0  | 189899  | 0.0145          |
| 0.0133        | 54.0  | 193482  | 0.0146          |
| 0.0135        | 55.0  | 197065  | 0.0144          |
| 0.0137        | 56.0  | 200648  | 0.0145          |
| 0.0138        | 57.0  | 204231  | 0.0144          |
| 0.014         | 58.0  | 207814  | 0.0144          |
| 0.0139        | 59.0  | 211397  | 0.0144          |
| 0.0163        | 60.0  | 214980  | 0.0144          |
| 0.0163        | 61.0  | 218563  | 0.0142          |
| 0.016         | 62.0  | 222146  | 0.0144          |
| 0.0136        | 63.0  | 225729  | 0.0141          |
| 0.0152        | 64.0  | 229312  | 0.0142          |
| 0.0134        | 65.0  | 232895  | 0.0143          |
| 0.0146        | 66.0  | 236478  | 0.0143          |
| 0.0149        | 67.0  | 240061  | 0.0141          |
| 0.0148        | 68.0  | 243644  | 0.0144          |
| 0.0141        | 69.0  | 247227  | 0.0143          |
| 0.0145        | 70.0  | 250810  | 0.0142          |
| 0.0137        | 71.0  | 254393  | 0.0141          |
| 0.0137        | 72.0  | 257976  | 0.0141          |
| 0.0134        | 73.0  | 261559  | 0.0143          |
| 0.0186        | 74.0  | 265142  | 0.0141          |
| 0.0143        | 75.0  | 268725  | 0.0141          |
| 0.0155        | 76.0  | 272308  | 0.0142          |
| 0.0164        | 77.0  | 275891  | 0.0142          |
| 0.0137        | 78.0  | 279474  | 0.0141          |
| 0.0148        | 79.0  | 283057  | 0.0141          |
| 0.0136        | 80.0  | 286640  | 0.0142          |
| 0.0151        | 81.0  | 290223  | 0.0142          |
| 0.0137        | 82.0  | 293806  | 0.0141          |
| 0.0149        | 83.0  | 297389  | 0.0142          |
| 0.0145        | 84.0  | 300972  | 0.0141          |
| 0.0129        | 85.0  | 304555  | 0.0140          |
| 0.0144        | 86.0  | 308138  | 0.0142          |
| 0.0145        | 87.0  | 311721  | 0.0141          |
| 0.0125        | 88.0  | 315304  | 0.0140          |
| 0.0136        | 89.0  | 318887  | 0.0139          |
| 0.0138        | 90.0  | 322470  | 0.0141          |
| 0.0138        | 91.0  | 326053  | 0.0139          |
| 0.0144        | 92.0  | 329636  | 0.0139          |
| 0.0145        | 93.0  | 333219  | 0.0140          |
| 0.0136        | 94.0  | 336802  | 0.0140          |
| 0.0126        | 95.0  | 340385  | 0.0139          |
| 0.0154        | 96.0  | 343968  | 0.0140          |
| 0.0139        | 97.0  | 347551  | 0.0141          |
| 0.0135        | 98.0  | 351134  | 0.0138          |
| 0.0131        | 99.0  | 354717  | 0.0138          |
| 0.0123        | 100.0 | 358300  | 0.0139          |
| 0.014         | 101.0 | 361883  | 0.0140          |
| 0.0118        | 102.0 | 365466  | 0.0138          |
| 0.0151        | 103.0 | 369049  | 0.0140          |
| 0.0138        | 104.0 | 372632  | 0.0140          |
| 0.0156        | 105.0 | 376215  | 0.0139          |
| 0.0137        | 106.0 | 379798  | 0.0139          |
| 0.0136        | 107.0 | 383381  | 0.0140          |
| 0.0137        | 108.0 | 386964  | 0.0139          |
| 0.0151        | 109.0 | 390547  | 0.0138          |
| 0.0154        | 110.0 | 394130  | 0.0137          |
| 0.0142        | 111.0 | 397713  | 0.0140          |
| 0.0121        | 112.0 | 401296  | 0.0139          |
| 0.0131        | 113.0 | 404879  | 0.0139          |
| 0.0127        | 114.0 | 408462  | 0.0139          |
| 0.0131        | 115.0 | 412045  | 0.0140          |
| 0.014         | 116.0 | 415628  | 0.0137          |
| 0.0126        | 117.0 | 419211  | 0.0140          |
| 0.0135        | 118.0 | 422794  | 0.0140          |
| 0.0131        | 119.0 | 426377  | 0.0138          |
| 0.013         | 120.0 | 429960  | 0.0138          |
| 0.0145        | 121.0 | 433543  | 0.0139          |
| 0.0141        | 122.0 | 437126  | 0.0139          |
| 0.0134        | 123.0 | 440709  | 0.0140          |
| 0.0134        | 124.0 | 444292  | 0.0139          |
| 0.0138        | 125.0 | 447875  | 0.0139          |
| 0.0143        | 126.0 | 451458  | 0.0139          |
| 0.0127        | 127.0 | 455041  | 0.0139          |
| 0.0126        | 128.0 | 458624  | 0.0139          |
| 0.0141        | 129.0 | 462207  | 0.0139          |
| 0.0129        | 130.0 | 465790  | 0.0138          |
| 0.0138        | 131.0 | 469373  | 0.0139          |
| 0.0137        | 132.0 | 472956  | 0.0140          |
| 0.0143        | 133.0 | 476539  | 0.0138          |
| 0.0136        | 134.0 | 480122  | 0.0138          |
| 0.0142        | 135.0 | 483705  | 0.0138          |
| 0.0133        | 136.0 | 487288  | 0.0138          |
| 0.0162        | 137.0 | 490871  | 0.0136          |
| 0.0128        | 138.0 | 494454  | 0.0140          |
| 0.0137        | 139.0 | 498037  | 0.0139          |
| 0.0131        | 140.0 | 501620  | 0.0139          |
| 0.0137        | 141.0 | 505203  | 0.0138          |
| 0.0122        | 142.0 | 508786  | 0.0138          |
| 0.0132        | 143.0 | 512369  | 0.0137          |
| 0.0131        | 144.0 | 515952  | 0.0136          |
| 0.0139        | 145.0 | 519535  | 0.0138          |
| 0.0141        | 146.0 | 523118  | 0.0139          |
| 0.0148        | 147.0 | 526701  | 0.0137          |
| 0.0138        | 148.0 | 530284  | 0.0136          |
| 0.0139        | 149.0 | 533867  | 0.0139          |
| 0.0136        | 150.0 | 537450  | 0.0137          |
| 0.0121        | 151.0 | 541033  | 0.0139          |
| 0.014         | 152.0 | 544616  | 0.0138          |
| 0.013         | 153.0 | 548199  | 0.0136          |
| 0.0146        | 154.0 | 551782  | 0.0136          |
| 0.0138        | 155.0 | 555365  | 0.0136          |
| 0.0144        | 156.0 | 558948  | 0.0137          |
| 0.0141        | 157.0 | 562531  | 0.0138          |
| 0.0141        | 158.0 | 566114  | 0.0136          |
| 0.0116        | 159.0 | 569697  | 0.0137          |
| 0.012         | 160.0 | 573280  | 0.0136          |
| 0.0131        | 161.0 | 576863  | 0.0138          |
| 0.0131        | 162.0 | 580446  | 0.0136          |
| 0.0125        | 163.0 | 584029  | 0.0137          |
| 0.0145        | 164.0 | 587612  | 0.0137          |
| 0.0135        | 165.0 | 591195  | 0.0139          |
| 0.012         | 166.0 | 594778  | 0.0138          |
| 0.0144        | 167.0 | 598361  | 0.0137          |
| 0.0136        | 168.0 | 601944  | 0.0137          |
| 0.0123        | 169.0 | 605527  | 0.0136          |
| 0.0134        | 170.0 | 609110  | 0.0136          |
| 0.0128        | 171.0 | 612693  | 0.0134          |
| 0.0134        | 172.0 | 616276  | 0.0137          |
| 0.0129        | 173.0 | 619859  | 0.0137          |
| 0.014         | 174.0 | 623442  | 0.0136          |
| 0.0152        | 175.0 | 627025  | 0.0138          |
| 0.0139        | 176.0 | 630608  | 0.0136          |
| 0.0125        | 177.0 | 634191  | 0.0138          |
| 0.0142        | 178.0 | 637774  | 0.0136          |
| 0.0137        | 179.0 | 641357  | 0.0136          |
| 0.0168        | 180.0 | 644940  | 0.0134          |
| 0.0143        | 181.0 | 648523  | 0.0135          |
| 0.0118        | 182.0 | 652106  | 0.0137          |
| 0.0144        | 183.0 | 655689  | 0.0136          |
| 0.0125        | 184.0 | 659272  | 0.0136          |
| 0.014         | 185.0 | 662855  | 0.0136          |
| 0.0132        | 186.0 | 666438  | 0.0136          |
| 0.013         | 187.0 | 670021  | 0.0137          |
| 0.0128        | 188.0 | 673604  | 0.0136          |
| 0.0134        | 189.0 | 677187  | 0.0136          |
| 0.0138        | 190.0 | 680770  | 0.0137          |
| 0.013         | 191.0 | 684353  | 0.0135          |
| 0.0137        | 192.0 | 687936  | 0.0136          |
| 0.0124        | 193.0 | 691519  | 0.0135          |
| 0.0133        | 194.0 | 695102  | 0.0135          |
| 0.0135        | 195.0 | 698685  | 0.0136          |
| 0.0142        | 196.0 | 702268  | 0.0136          |
| 0.0122        | 197.0 | 705851  | 0.0136          |
| 0.0124        | 198.0 | 709434  | 0.0137          |
| 0.0154        | 199.0 | 713017  | 0.0136          |
| 0.0137        | 200.0 | 716600  | 0.0135          |
| 0.0128        | 201.0 | 720183  | 0.0135          |
| 0.0133        | 202.0 | 723766  | 0.0136          |
| 0.0139        | 203.0 | 727349  | 0.0136          |
| 0.0136        | 204.0 | 730932  | 0.0136          |
| 0.0143        | 205.0 | 734515  | 0.0134          |
| 0.0129        | 206.0 | 738098  | 0.0134          |
| 0.0142        | 207.0 | 741681  | 0.0136          |
| 0.0135        | 208.0 | 745264  | 0.0136          |
| 0.014         | 209.0 | 748847  | 0.0136          |
| 0.0132        | 210.0 | 752430  | 0.0135          |
| 0.0142        | 211.0 | 756013  | 0.0135          |
| 0.0136        | 212.0 | 759596  | 0.0134          |
| 0.0131        | 213.0 | 763179  | 0.0135          |
| 0.014         | 214.0 | 766762  | 0.0135          |
| 0.0131        | 215.0 | 770345  | 0.0135          |
| 0.0122        | 216.0 | 773928  | 0.0135          |
| 0.0128        | 217.0 | 777511  | 0.0135          |
| 0.0135        | 218.0 | 781094  | 0.0136          |
| 0.0136        | 219.0 | 784677  | 0.0135          |
| 0.0151        | 220.0 | 788260  | 0.0135          |
| 0.013         | 221.0 | 791843  | 0.0135          |
| 0.0123        | 222.0 | 795426  | 0.0135          |
| 0.013         | 223.0 | 799009  | 0.0134          |
| 0.0146        | 224.0 | 802592  | 0.0134          |
| 0.0144        | 225.0 | 806175  | 0.0135          |
| 0.0137        | 226.0 | 809758  | 0.0135          |
| 0.0115        | 227.0 | 813341  | 0.0134          |
| 0.0132        | 228.0 | 816924  | 0.0135          |
| 0.0141        | 229.0 | 820507  | 0.0135          |
| 0.013         | 230.0 | 824090  | 0.0136          |
| 0.0128        | 231.0 | 827673  | 0.0134          |
| 0.0138        | 232.0 | 831256  | 0.0135          |
| 0.0136        | 233.0 | 834839  | 0.0135          |
| 0.013         | 234.0 | 838422  | 0.0135          |
| 0.0137        | 235.0 | 842005  | 0.0134          |
| 0.0147        | 236.0 | 845588  | 0.0135          |
| 0.0134        | 237.0 | 849171  | 0.0135          |
| 0.013         | 238.0 | 852754  | 0.0133          |
| 0.0146        | 239.0 | 856337  | 0.0135          |
| 0.0145        | 240.0 | 859920  | 0.0136          |
| 0.0149        | 241.0 | 863503  | 0.0136          |
| 0.0131        | 242.0 | 867086  | 0.0134          |
| 0.0128        | 243.0 | 870669  | 0.0134          |
| 0.0139        | 244.0 | 874252  | 0.0133          |
| 0.0128        | 245.0 | 877835  | 0.0132          |
| 0.0138        | 246.0 | 881418  | 0.0135          |
| 0.0126        | 247.0 | 885001  | 0.0133          |
| 0.0144        | 248.0 | 888584  | 0.0134          |
| 0.0138        | 249.0 | 892167  | 0.0134          |
| 0.0137        | 250.0 | 895750  | 0.0134          |
| 0.0146        | 251.0 | 899333  | 0.0136          |
| 0.0135        | 252.0 | 902916  | 0.0134          |
| 0.0124        | 253.0 | 906499  | 0.0134          |
| 0.013         | 254.0 | 910082  | 0.0135          |
| 0.0136        | 255.0 | 913665  | 0.0134          |
| 0.0146        | 256.0 | 917248  | 0.0133          |
| 0.0129        | 257.0 | 920831  | 0.0136          |
| 0.013         | 258.0 | 924414  | 0.0134          |
| 0.0135        | 259.0 | 927997  | 0.0134          |
| 0.0134        | 260.0 | 931580  | 0.0131          |
| 0.0151        | 261.0 | 935163  | 0.0133          |
| 0.0126        | 262.0 | 938746  | 0.0133          |
| 0.0124        | 263.0 | 942329  | 0.0134          |
| 0.013         | 264.0 | 945912  | 0.0134          |
| 0.0139        | 265.0 | 949495  | 0.0133          |
| 0.0142        | 266.0 | 953078  | 0.0133          |
| 0.0129        | 267.0 | 956661  | 0.0134          |
| 0.0128        | 268.0 | 960244  | 0.0134          |
| 0.0177        | 269.0 | 963827  | 0.0133          |
| 0.0124        | 270.0 | 967410  | 0.0134          |
| 0.0165        | 271.0 | 970993  | 0.0133          |
| 0.0131        | 272.0 | 974576  | 0.0135          |
| 0.0119        | 273.0 | 978159  | 0.0133          |
| 0.0125        | 274.0 | 981742  | 0.0133          |
| 0.0127        | 275.0 | 985325  | 0.0133          |
| 0.013         | 276.0 | 988908  | 0.0134          |
| 0.0131        | 277.0 | 992491  | 0.0136          |
| 0.0131        | 278.0 | 996074  | 0.0134          |
| 0.0128        | 279.0 | 999657  | 0.0133          |
| 0.0146        | 280.0 | 1003240 | 0.0133          |
| 0.0138        | 281.0 | 1006823 | 0.0134          |
| 0.0125        | 282.0 | 1010406 | 0.0135          |
| 0.0121        | 283.0 | 1013989 | 0.0133          |
| 0.0137        | 284.0 | 1017572 | 0.0135          |
| 0.0144        | 285.0 | 1021155 | 0.0135          |
| 0.0131        | 286.0 | 1024738 | 0.0134          |
| 0.0133        | 287.0 | 1028321 | 0.0134          |
| 0.013         | 288.0 | 1031904 | 0.0133          |
| 0.013         | 289.0 | 1035487 | 0.0132          |
| 0.0133        | 290.0 | 1039070 | 0.0134          |
| 0.0127        | 291.0 | 1042653 | 0.0133          |
| 0.0121        | 292.0 | 1046236 | 0.0133          |
| 0.013         | 293.0 | 1049819 | 0.0132          |
| 0.0116        | 294.0 | 1053402 | 0.0131          |
| 0.0151        | 295.0 | 1056985 | 0.0133          |
| 0.013         | 296.0 | 1060568 | 0.0133          |
| 0.0121        | 297.0 | 1064151 | 0.0133          |
| 0.0127        | 298.0 | 1067734 | 0.0132          |
| 0.0125        | 299.0 | 1071317 | 0.0134          |
| 0.0132        | 300.0 | 1074900 | 0.0131          |
| 0.0121        | 301.0 | 1078483 | 0.0134          |
| 0.0147        | 302.0 | 1082066 | 0.0134          |
| 0.0127        | 303.0 | 1085649 | 0.0133          |
| 0.0131        | 304.0 | 1089232 | 0.0132          |
| 0.0144        | 305.0 | 1092815 | 0.0133          |
| 0.0145        | 306.0 | 1096398 | 0.0135          |
| 0.014         | 307.0 | 1099981 | 0.0132          |
| 0.0162        | 308.0 | 1103564 | 0.0132          |
| 0.0125        | 309.0 | 1107147 | 0.0133          |
| 0.0146        | 310.0 | 1110730 | 0.0134          |
| 0.0145        | 311.0 | 1114313 | 0.0132          |
| 0.0133        | 312.0 | 1117896 | 0.0133          |
| 0.0146        | 313.0 | 1121479 | 0.0133          |
| 0.0137        | 314.0 | 1125062 | 0.0133          |
| 0.0131        | 315.0 | 1128645 | 0.0132          |
| 0.014         | 316.0 | 1132228 | 0.0131          |
| 0.0134        | 317.0 | 1135811 | 0.0132          |
| 0.0138        | 318.0 | 1139394 | 0.0133          |
| 0.0133        | 319.0 | 1142977 | 0.0131          |
| 0.0131        | 320.0 | 1146560 | 0.0132          |
| 0.0127        | 321.0 | 1150143 | 0.0132          |
| 0.0137        | 322.0 | 1153726 | 0.0133          |
| 0.0129        | 323.0 | 1157309 | 0.0133          |
| 0.0123        | 324.0 | 1160892 | 0.0132          |
| 0.0135        | 325.0 | 1164475 | 0.0133          |
| 0.0132        | 326.0 | 1168058 | 0.0132          |
| 0.0125        | 327.0 | 1171641 | 0.0132          |
| 0.0135        | 328.0 | 1175224 | 0.0132          |
| 0.0135        | 329.0 | 1178807 | 0.0135          |
| 0.0139        | 330.0 | 1182390 | 0.0133          |
| 0.0133        | 331.0 | 1185973 | 0.0130          |
| 0.0123        | 332.0 | 1189556 | 0.0134          |
| 0.0141        | 333.0 | 1193139 | 0.0131          |
| 0.013         | 334.0 | 1196722 | 0.0130          |
| 0.015         | 335.0 | 1200305 | 0.0132          |
| 0.0139        | 336.0 | 1203888 | 0.0132          |
| 0.0125        | 337.0 | 1207471 | 0.0133          |
| 0.0133        | 338.0 | 1211054 | 0.0132          |
| 0.0131        | 339.0 | 1214637 | 0.0133          |
| 0.0136        | 340.0 | 1218220 | 0.0131          |
| 0.0114        | 341.0 | 1221803 | 0.0132          |
| 0.0146        | 342.0 | 1225386 | 0.0132          |
| 0.014         | 343.0 | 1228969 | 0.0132          |
| 0.0138        | 344.0 | 1232552 | 0.0131          |
| 0.0123        | 345.0 | 1236135 | 0.0133          |
| 0.0129        | 346.0 | 1239718 | 0.0132          |
| 0.0135        | 347.0 | 1243301 | 0.0133          |
| 0.0142        | 348.0 | 1246884 | 0.0132          |
| 0.0134        | 349.0 | 1250467 | 0.0132          |
| 0.0121        | 350.0 | 1254050 | 0.0132          |
| 0.0124        | 351.0 | 1257633 | 0.0132          |
| 0.0152        | 352.0 | 1261216 | 0.0133          |
| 0.0126        | 353.0 | 1264799 | 0.0134          |
| 0.012         | 354.0 | 1268382 | 0.0132          |
| 0.0138        | 355.0 | 1271965 | 0.0133          |
| 0.0136        | 356.0 | 1275548 | 0.0132          |
| 0.012         | 357.0 | 1279131 | 0.0132          |
| 0.0123        | 358.0 | 1282714 | 0.0134          |
| 0.0131        | 359.0 | 1286297 | 0.0131          |
| 0.0127        | 360.0 | 1289880 | 0.0133          |
| 0.0132        | 361.0 | 1293463 | 0.0131          |
| 0.0135        | 362.0 | 1297046 | 0.0131          |
| 0.0119        | 363.0 | 1300629 | 0.0132          |
| 0.0132        | 364.0 | 1304212 | 0.0132          |
| 0.013         | 365.0 | 1307795 | 0.0131          |
| 0.0124        | 366.0 | 1311378 | 0.0132          |
| 0.0122        | 367.0 | 1314961 | 0.0132          |
| 0.0146        | 368.0 | 1318544 | 0.0132          |
| 0.0121        | 369.0 | 1322127 | 0.0134          |
| 0.0119        | 370.0 | 1325710 | 0.0131          |
| 0.0138        | 371.0 | 1329293 | 0.0134          |
| 0.0132        | 372.0 | 1332876 | 0.0134          |
| 0.0146        | 373.0 | 1336459 | 0.0133          |
| 0.0126        | 374.0 | 1340042 | 0.0132          |
| 0.0125        | 375.0 | 1343625 | 0.0133          |
| 0.0129        | 376.0 | 1347208 | 0.0132          |
| 0.0117        | 377.0 | 1350791 | 0.0133          |
| 0.0114        | 378.0 | 1354374 | 0.0132          |
| 0.0123        | 379.0 | 1357957 | 0.0132          |
| 0.0119        | 380.0 | 1361540 | 0.0132          |
| 0.0131        | 381.0 | 1365123 | 0.0133          |
| 0.0135        | 382.0 | 1368706 | 0.0132          |
| 0.0154        | 383.0 | 1372289 | 0.0131          |
| 0.0151        | 384.0 | 1375872 | 0.0133          |
| 0.0125        | 385.0 | 1379455 | 0.0132          |
| 0.0137        | 386.0 | 1383038 | 0.0132          |
| 0.0125        | 387.0 | 1386621 | 0.0132          |
| 0.012         | 388.0 | 1390204 | 0.0132          |
| 0.0155        | 389.0 | 1393787 | 0.0132          |
| 0.013         | 390.0 | 1397370 | 0.0133          |
| 0.0131        | 391.0 | 1400953 | 0.0131          |
| 0.013         | 392.0 | 1404536 | 0.0132          |
| 0.0141        | 393.0 | 1408119 | 0.0132          |
| 0.0123        | 394.0 | 1411702 | 0.0132          |
| 0.0126        | 395.0 | 1415285 | 0.0132          |
| 0.0122        | 396.0 | 1418868 | 0.0132          |
| 0.0126        | 397.0 | 1422451 | 0.0131          |
| 0.0135        | 398.0 | 1426034 | 0.0131          |
| 0.0126        | 399.0 | 1429617 | 0.0133          |
| 0.0136        | 400.0 | 1433200 | 0.0131          |


### Framework versions

- Transformers 4.39.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2