---
license: mit
tags:
- generated_from_keras_callback
datasets:
- wikitext
mask_token: '[MASK]'
widget:
- text: Goal of my life is to [MASK].
base_model: roberta-base
model-index:
- name: tf-tpu/roberta-base-epochs-500-no-wd
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# tf-tpu/roberta-base-epochs-500-no-wd

This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.7074
- Train Accuracy: 0.1221
- Validation Loss: 0.7739
- Validation Accuracy: 0.1213
- Epoch: 499

## Model description

The model was trained on the [WikiText dataset](https://huggingface.co/datasets/wikitext) (v1). Training details can be found [here](https://github.com/huggingface/transformers/tree/examples/main/examples/tensorflow/tpu/language-modeling).

## Intended uses & limitations

More information needed

## Training and evaluation data

[WikiText (v1)](https://huggingface.co/datasets/wikitext)

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 0.0001, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 278825, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 14675, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.001}
- training_precision: mixed_bfloat16

### Training results

| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch |
|:----------:|:--------------:|:---------------:|:-------------------:|:-----:|
| 8.3284     | 0.0211         | 7.1523          | 0.0266              | 0     |
| 6.3670     | 0.0318         | 5.7812          | 0.0342              | 1     |
| 5.6051     | 0.0380         | 5.4414          | 0.0420              | 2     |
| 5.3602     | 0.0433         | 5.2734          | 0.0432              | 3     |
| 5.2285     | 0.0444         | 5.1562          | 0.0442              | 4     |
| 5.1371     | 0.0446         | 5.1133          | 0.0436              | 5     |
| 5.0673     | 0.0446         | 5.0703          | 0.0442              | 6     |
| 5.0132     | 0.0447         | 4.9883          | 0.0442              | 7     |
| 4.9642     | 0.0448         | 4.9219          | 0.0441              | 8     |
| 4.9217     | 0.0448         | 4.9258          | 0.0440              | 9     |
| 4.8871     | 0.0448         | 4.8867          | 0.0439              | 10    |
| 4.8548     | 0.0449         | 4.8672          | 0.0439              | 11    |
| 4.8277     | 0.0449         | 4.8047          | 0.0445              | 12    |
| 4.8033     | 0.0449         | 4.8477          | 0.0437              | 13    |
| 4.7807     | 0.0449         | 4.7617          | 0.0439              | 14    |
| 4.7592     | 0.0449         | 4.7773          | 0.0437              | 15    |
| 4.7388     | 0.0449         | 4.7539          | 0.0441              | 16    |
| 4.7225     | 0.0449         | 4.7266          | 0.0439              | 17    |
| 4.7052     | 0.0449         | 4.6914          | 0.0450              | 18    |
| 4.6917     | 0.0449         | 4.7188          | 0.0444              | 19    |
| 4.6789     | 0.0449         | 4.6914          | 0.0444              | 20    |
| 4.6689     | 0.0449         | 4.7031          | 0.0439              | 21    |
| 4.6570     | 0.0449         | 4.7031          | 0.0437              | 22    |
| 4.6486     | 0.0450         | 4.6758          | 0.0446              | 23    |
| 4.6393     | 0.0449         | 4.6914          | 0.0441              | 24    |
| 4.5898     | 0.0449         | 4.4688          | 0.0452              | 25    |
| 4.3024     | 0.0472         | 3.8730          | 0.0551              | 26    |
| 3.1689     | 0.0693         | 2.4375          | 0.0835              | 27    |
| 2.3780     | 0.0844         | 2.0498          | 0.0922              | 28    |
| 2.0789     | 0.0907         | 1.8604          | 0.0958              | 29    |
| 1.9204     | 0.0940         | 1.7549          | 0.0982              | 30    |
| 1.8162     | 0.0961         | 1.6836          | 0.0983              | 31    |
| 1.7370     | 0.0978         | 1.5869          | 0.1014              | 32    |
| 1.6723     | 0.0991         | 1.5381          | 0.1029              | 33    |
| 1.6215     | 0.1002         | 1.5283          | 0.1015              | 34    |
| 1.5753     | 0.1012         | 1.4736          | 0.1037              | 35    |
| 1.5295     | 0.1022         | 1.4238          | 0.1052              | 36    |
| 1.4944     | 0.1030         | 1.4141          | 0.1059              | 37    |
| 1.4631     | 0.1037         | 1.3721          | 0.1053              | 38    |
| 1.4363     | 0.1043         | 1.3467          | 0.1060              | 39    |
| 1.4098     | 0.1049         | 1.3213          | 0.1076              | 40    |
| 1.3867     | 0.1054         | 1.3018          | 0.1071              | 41    |
| 1.3658     | 0.1058         | 1.2832          | 0.1083              | 42    |
| 1.3469     | 0.1063         | 1.2637          | 0.1081              | 43    |
| 1.3288     | 0.1067         | 1.2598          | 0.1082              | 44    |
| 1.3111     | 0.1071         | 1.2334          | 0.1096              | 45    |
| 1.2962     | 0.1075         | 1.2490          | 0.1084              | 46    |
| 1.2816     | 0.1078         | 1.2168          | 0.1093              | 47    |
| 1.2672     | 0.1081         | 1.2070          | 0.1090              | 48    |
| 1.2537     | 0.1084         | 1.1680          | 0.1106              | 49    |
| 1.2411     | 0.1087         | 1.1904          | 0.1094              | 50    |
| 1.2285     | 0.1090         | 1.1709          | 0.1103              | 51    |
| 1.2180     | 0.1093         | 1.1602          | 0.1122              | 52    |
| 1.2075     | 0.1095         | 1.1396          | 0.1117              | 53    |
| 1.1973     | 0.1098         | 1.1191          | 0.1124              | 54    |
| 1.1876     | 0.1100         | 1.1260          | 0.1123              | 55    |
| 1.1782     | 0.1102         | 1.1289          | 0.1111              | 56    |
| 1.1698     | 0.1104         | 1.1211          | 0.1117              | 57    |
| 1.1596     | 0.1106         | 1.0977          | 0.1125              | 58    |
| 1.1530     | 0.1108         | 1.1172          | 0.1118              | 59    |
| 1.1462     | 0.1110         | 1.0703          | 0.1126              | 60    |
| 1.1370     | 0.1112         | 1.0830          | 0.1140              | 61    |
| 1.1309     | 0.1113         | 1.0762          | 0.1119              | 62    |
| 1.1234     | 0.1115         | 1.0625          | 0.1137              | 63    |
| 1.1162     | 0.1117         | 1.0781          | 0.1127              | 64    |
| 1.1114     | 0.1118         | 1.0474          | 0.1138              | 65    |
| 1.1036     | 0.1120         | 1.0703          | 0.1134              | 66    |
| 1.0984     | 0.1121         | 1.0366          | 0.1139              | 67    |
| 1.0931     | 0.1122         | 1.0513          | 0.1134              | 68    |
| 1.0860     | 0.1124         | 1.0264          | 0.1137              | 69    |
| 1.0807     | 0.1126         | 1.0215          | 0.1148              | 70    |
| 1.0758     | 0.1127         | 1.0269          | 0.1143              | 71    |
| 1.0704     | 0.1129         | 1.0356          | 0.1141              | 72    |
| 1.0656     | 0.1129         | 1.0195          | 0.1144              | 73    |
| 1.0607     | 0.1131         | 1.0093          | 0.1146              | 74    |
| 1.0559     | 0.1132         | 0.9956          | 0.1155              | 75    |
| 1.0517     | 0.1133         | 0.9995          | 0.1139              | 76    |
| 1.0462     | 0.1134         | 0.9839          | 0.1151              | 77    |
| 1.0422     | 0.1135         | 0.9868          | 0.1153              | 78    |
| 1.0372     | 0.1137         | 0.9995          | 0.1151              | 79    |
| 1.0340     | 0.1137         | 1.0059          | 0.1153              | 80    |
| 1.0296     | 0.1138         | 0.9961          | 0.1152              | 81    |
| 1.0272     | 0.1138         | 1.0132          | 0.1138              | 82    |
| 1.0211     | 0.1140         | 0.9575          | 0.1150              | 83    |
| 1.0182     | 0.1141         | 0.9868          | 0.1150              | 84    |
| 1.0146     | 0.1142         | 0.9678          | 0.1164              | 85    |
| 1.0111     | 0.1143         | 0.9839          | 0.1161              | 86    |
| 1.0083     | 0.1144         | 0.9722          | 0.1162              | 87    |
| 1.0039     | 0.1144         | 0.9619          | 0.1167              | 88    |
| 1.0017     | 0.1145         | 0.9575          | 0.1151              | 89    |
| 0.9973     | 0.1146         | 0.9624          | 0.1149              | 90    |
| 0.9947     | 0.1147         | 0.9570          | 0.1157              | 91    |
| 0.9921     | 0.1148         | 0.9360          | 0.1166              | 92    |
| 0.9884     | 0.1149         | 0.9546          | 0.1156              | 93    |
| 0.9851     | 0.1149         | 0.9536          | 0.1149              | 94    |
| 0.9829     | 0.1150         | 0.9575          | 0.1163              | 95    |
| 0.9795     | 0.1151         | 0.9561          | 0.1156              | 96    |
| 0.9773     | 0.1151         | 0.9438          | 0.1163              | 97    |
| 0.9740     | 0.1152         | 0.9512          | 0.1169              | 98    |
| 0.9712     | 0.1153         | 0.9375          | 0.1159              | 99    |
| 0.9678     | 0.1154         | 0.9453          | 0.1166              | 100   |
| 0.9660     | 0.1154         | 0.9507          | 0.1169              | 101   |
| 0.9636     | 0.1155         | 0.9507          | 0.1161              | 102   |
| 0.9609     | 0.1155         | 0.9727          | 0.1164              | 103   |
| 0.9589     | 0.1156         | 0.9395          | 0.1176              | 104   |
| 0.9561     | 0.1157         | 0.9346          | 0.1173              | 105   |
| 0.9537     | 0.1157         | 0.9331          | 0.1168              | 106   |
| 0.9515     | 0.1158         | 0.9434          | 0.1161              | 107   |
| 0.9488     | 0.1158         | 0.9131          | 0.1176              | 108   |
| 0.9471     | 0.1159         | 0.9360          | 0.1174              | 109   |
| 0.9449     | 0.1159         | 0.9175          | 0.1164              | 110   |
| 0.9422     | 0.1160         | 0.9121          | 0.1167              | 111   |
| 0.9412     | 0.1160         | 0.8970          | 0.1165              | 112   |
| 0.9379     | 0.1161         | 0.9111          | 0.1175              | 113   |
| 0.9362     | 0.1161         | 0.9048          | 0.1176              | 114   |
| 0.9345     | 0.1162         | 0.9082          | 0.1169              | 115   |
| 0.9317     | 0.1163         | 0.9277          | 0.1169              | 116   |
| 0.9295     | 0.1164         | 0.9292          | 0.1169              | 117   |
| 0.9287     | 0.1163         | 0.9243          | 0.1169              | 118   |
| 0.9266     | 0.1163         | 0.8892          | 0.1170              | 119   |
| 0.9233     | 0.1165         | 0.9058          | 0.1174              | 120   |
| 0.9221     | 0.1165         | 0.9106          | 0.1175              | 121   |
| 0.9205     | 0.1166         | 0.8979          | 0.1173              | 122   |
| 0.9181     | 0.1167         | 0.8989          | 0.1174              | 123   |
| 0.9180     | 0.1166         | 0.9053          | 0.1172              | 124   |
| 0.9158     | 0.1167         | 0.8877          | 0.1176              | 125   |
| 0.9135     | 0.1168         | 0.9160          | 0.1169              | 126   |
| 0.9116     | 0.1167         | 0.8940          | 0.1180              | 127   |
| 0.9095     | 0.1168         | 0.8945          | 0.1173              | 128   |
| 0.9081     | 0.1168         | 0.9126          | 0.1166              | 129   |
| 0.9064     | 0.1169         | 0.8872          | 0.1177              | 130   |
| 0.9053     | 0.1169         | 0.9175          | 0.1172              | 131   |
| 0.9035     | 0.1170         | 0.8989          | 0.1180              | 132   |
| 0.9023     | 0.1170         | 0.8965          | 0.1179              | 133   |
| 0.8999     | 0.1170         | 0.8979          | 0.1181              | 134   |
| 0.8981     | 0.1171         | 0.8799          | 0.1186              | 135   |
| 0.8976     | 0.1171         | 0.8984          | 0.1174              | 136   |
| 0.8957     | 0.1172         | 0.8857          | 0.1181              | 137   |
| 0.8948     | 0.1172         | 0.9019          | 0.1172              | 138   |
| 0.8929     | 0.1172         | 0.8804          | 0.1180              | 139   |
| 0.8915     | 0.1173         | 0.8848          | 0.1183              | 140   |
| 0.8898     | 0.1173         | 0.8911          | 0.1177              | 141   |
| 0.8894     | 0.1173         | 0.9033          | 0.1173              | 142   |
| 0.8869     | 0.1174         | 0.8853          | 0.1184              | 143   |
| 0.8863     | 0.1174         | 0.8921          | 0.1184              | 144   |
| 0.8848     | 0.1175         | 0.8848          | 0.1177              | 145   |
| 0.8838     | 0.1175         | 0.8896          | 0.1177              | 146   |
| 0.8822     | 0.1175         | 0.8945          | 0.1181              | 147   |
| 0.8804     | 0.1176         | 0.8843          | 0.1177              | 148   |
| 0.8794     | 0.1175         | 0.8774          | 0.1181              | 149   |
| 0.8780     | 0.1176         | 0.875           | 0.1178              | 150   |
| 0.8756     | 0.1176         | 0.8862          | 0.1170              | 151   |
| 0.8747     | 0.1177         | 0.8730          | 0.1178              | 152   |
| 0.8737     | 0.1177         | 0.8696          | 0.1195              | 153   |
| 0.8736     | 0.1177         | 0.8726          | 0.1184              | 154   |
| 0.8716     | 0.1178         | 0.8647          | 0.1186              | 155   |
| 0.8705     | 0.1178         | 0.8804          | 0.1179              | 156   |
| 0.8695     | 0.1178         | 0.8652          | 0.1190              | 157   |
| 0.8675     | 0.1179         | 0.8804          | 0.1197              | 158   |
| 0.8670     | 0.1179         | 0.8462          | 0.1192              | 159   |
| 0.8656     | 0.1180         | 0.8594          | 0.1188              | 160   |
| 0.8649     | 0.1180         | 0.8535          | 0.1188              | 161   |
| 0.8633     | 0.1181         | 0.8555          | 0.1185              | 162   |
| 0.8622     | 0.1180         | 0.8633          | 0.1173              | 163   |
| 0.8603     | 0.1181         | 0.8667          | 0.1177              | 164   |
| 0.8598     | 0.1181         | 0.8813          | 0.1185              | 165   |
| 0.8591     | 0.1181         | 0.8862          | 0.1176              | 166   |
| 0.8580     | 0.1181         | 0.8853          | 0.1177              | 167   |
| 0.8573     | 0.1181         | 0.8691          | 0.1181              | 168   |
| 0.8558     | 0.1182         | 0.8481          | 0.1176              | 169   |
| 0.8541     | 0.1182         | 0.8652          | 0.1187              | 170   |
| 0.8541     | 0.1183         | 0.8477          | 0.1198              | 171   |
| 0.8522     | 0.1183         | 0.8721          | 0.1190              | 172   |
| 0.8516     | 0.1183         | 0.8965          | 0.1173              | 173   |
| 0.8506     | 0.1183         | 0.8574          | 0.1173              | 174   |
| 0.8496     | 0.1183         | 0.8452          | 0.1188              | 175   |
| 0.8487     | 0.1184         | 0.8545          | 0.1183              | 176   |
| 0.8478     | 0.1184         | 0.8594          | 0.1191              | 177   |
| 0.8466     | 0.1184         | 0.8608          | 0.1187              | 178   |
| 0.8456     | 0.1184         | 0.8472          | 0.1186              | 179   |
| 0.8451     | 0.1185         | 0.8672          | 0.1178              | 180   |
| 0.8429     | 0.1185         | 0.8364          | 0.1196              | 181   |
| 0.8420     | 0.1185         | 0.8525          | 0.1187              | 182   |
| 0.8419     | 0.1186         | 0.8525          | 0.1196              | 183   |
| 0.8406     | 0.1186         | 0.8521          | 0.1193              | 184   |
| 0.8391     | 0.1186         | 0.8560          | 0.1188              | 185   |
| 0.8396     | 0.1186         | 0.8413          | 0.1188              | 186   |
| 0.8378     | 0.1186         | 0.8628          | 0.1185              | 187   |
| 0.8374     | 0.1186         | 0.8374          | 0.1195              | 188   |
| 0.8364     | 0.1187         | 0.8691          | 0.1189              | 189   |
| 0.8348     | 0.1187         | 0.8457          | 0.1196              | 190   |
| 0.8354     | 0.1187         | 0.8286          | 0.1191              | 191   |
| 0.8334     | 0.1187         | 0.8486          | 0.1187              | 192   |
| 0.8325     | 0.1188         | 0.8535          | 0.1182              | 193   |
| 0.8322     | 0.1188         | 0.8574          | 0.1199              | 194   |
| 0.8314     | 0.1188         | 0.8472          | 0.1202              | 195   |
| 0.8307     | 0.1188         | 0.8584          | 0.1186              | 196   |
| 0.8294     | 0.1189         | 0.8345          | 0.1197              | 197   |
| 0.8285     | 0.1189         | 0.8491          | 0.1181              | 198   |
| 0.8275     | 0.1189         | 0.8472          | 0.1193              | 199   |
| 0.8265     | 0.1189         | 0.8521          | 0.1185              | 200   |
| 0.8262     | 0.1190         | 0.8501          | 0.1195              | 201   |
| 0.8247     | 0.1190         | 0.8491          | 0.1194              | 202   |
| 0.8245     | 0.1190         | 0.8389          | 0.1191              | 203   |
| 0.8237     | 0.1190         | 0.8491          | 0.1184              | 204   |
| 0.8229     | 0.1190         | 0.8525          | 0.1193              | 205   |
| 0.8215     | 0.1190         | 0.8345          | 0.1199              | 206   |
| 0.8213     | 0.1190         | 0.8511          | 0.1206              | 207   |
| 0.8204     | 0.1191         | 0.8296          | 0.1195              | 208   |
| 0.8193     | 0.1192         | 0.8516          | 0.1183              | 209   |
| 0.8195     | 0.1191         | 0.8672          | 0.1181              | 210   |
| 0.8188     | 0.1191         | 0.8267          | 0.1197              | 211   |
| 0.8177     | 0.1192         | 0.8408          | 0.1185              | 212   |
| 0.8167     | 0.1192         | 0.8447          | 0.1191              | 213   |
| 0.8153     | 0.1192         | 0.8374          | 0.1191              | 214   |
| 0.8158     | 0.1192         | 0.8438          | 0.1198              | 215   |
| 0.8149     | 0.1192         | 0.8286          | 0.1191              | 216   |
| 0.8141     | 0.1193         | 0.8389          | 0.1202              | 217   |
| 0.8133     | 0.1192         | 0.8491          | 0.1202              | 218   |
| 0.8127     | 0.1193         | 0.8730          | 0.1185              | 219   |
| 0.8118     | 0.1193         | 0.8198          | 0.1183              | 220   |
| 0.8115     | 0.1193         | 0.8164          | 0.1200              | 221   |
| 0.8095     | 0.1194         | 0.8340          | 0.1195              | 222   |
| 0.8090     | 0.1194         | 0.8071          | 0.1208              | 223   |
| 0.8089     | 0.1194         | 0.8101          | 0.1195              | 224   |
| 0.8081     | 0.1194         | 0.8311          | 0.1184              | 225   |
| 0.8081     | 0.1194         | 0.8413          | 0.1198              | 226   |
| 0.8065     | 0.1195         | 0.8379          | 0.1202              | 227   |
| 0.8064     | 0.1194         | 0.8398          | 0.1196              | 228   |
| 0.8045     | 0.1195         | 0.8159          | 0.1199              | 229   |
| 0.8045     | 0.1195         | 0.8350          | 0.1187              | 230   |
| 0.8049     | 0.1195         | 0.8369          | 0.1191              | 231   |
| 0.8037     | 0.1195         | 0.8159          | 0.1201              | 232   |
| 0.8024     | 0.1196         | 0.8213          | 0.1186              | 233   |
| 0.8023     | 0.1196         | 0.8384          | 0.1187              | 234   |
| 0.8011     | 0.1196         | 0.8262          | 0.1201              | 235   |
| 0.8006     | 0.1196         | 0.8252          | 0.1195              | 236   |
| 0.8005     | 0.1196         | 0.8267          | 0.1196              | 237   |
| 0.7989     | 0.1196         | 0.8389          | 0.1199              | 238   |
| 0.7989     | 0.1196         | 0.8394          | 0.1185              | 239   |
| 0.7983     | 0.1197         | 0.8110          | 0.1208              | 240   |
| 0.7978     | 0.1197         | 0.8066          | 0.1208              | 241   |
| 0.7969     | 0.1197         | 0.8257          | 0.1185              | 242   |
| 0.7954     | 0.1197         | 0.8242          | 0.1189              | 243   |
| 0.7962     | 0.1197         | 0.8291          | 0.1197              | 244   |
| 0.7951     | 0.1197         | 0.8320          | 0.1187              | 245   |
| 0.7944     | 0.1198         | 0.8389          | 0.1184              | 246   |
| 0.7927     | 0.1198         | 0.8184          | 0.1187              | 247   |
| 0.7933     | 0.1198         | 0.8242          | 0.1199              | 248   |
| 0.7935     | 0.1198         | 0.8369          | 0.1192              | 249   |
| 0.7916     | 0.1199         | 0.8242          | 0.1202              | 250   |
| 0.7913     | 0.1198         | 0.8223          | 0.1182              | 251   |
| 0.7902     | 0.1199         | 0.8232          | 0.1192              | 252   |
| 0.7915     | 0.1199         | 0.8159          | 0.1206              | 253   |
| 0.7897     | 0.1198         | 0.8281          | 0.1195              | 254   |
| 0.7894     | 0.1199         | 0.8140          | 0.1193              | 255   |
| 0.7884     | 0.1200         | 0.8379          | 0.1204              | 256   |
| 0.7882     | 0.1199         | 0.8271          | 0.1194              | 257   |
| 0.7872     | 0.1199         | 0.8188          | 0.1198              | 258   |
| 0.7866     | 0.1200         | 0.8174          | 0.1198              | 259   |
| 0.7857     | 0.1200         | 0.8379          | 0.1198              | 260   |
| 0.7859     | 0.1200         | 0.8174          | 0.1204              | 261   |
| 0.7859     | 0.1200         | 0.8228          | 0.1199              | 262   |
| 0.7844     | 0.1200         | 0.8237          | 0.1201              | 263   |
| 0.7844     | 0.1200         | 0.8311          | 0.1185              | 264   |
| 0.7834     | 0.1201         | 0.8193          | 0.1193              | 265   |
| 0.7834     | 0.1201         | 0.8276          | 0.1191              | 266   |
| 0.7833     | 0.1200         | 0.8291          | 0.1194              | 267   |
| 0.7821     | 0.1201         | 0.8335          | 0.1195              | 268   |
| 0.7818     | 0.1201         | 0.8350          | 0.1199              | 269   |
| 0.7812     | 0.1201         | 0.8223          | 0.1184              | 270   |
| 0.7809     | 0.1201         | 0.8330          | 0.1202              | 271   |
| 0.7794     | 0.1202         | 0.8193          | 0.1196              | 272   |
| 0.7793     | 0.1201         | 0.8237          | 0.1201              | 273   |
| 0.7787     | 0.1202         | 0.8389          | 0.1206              | 274   |
| 0.7786     | 0.1202         | 0.8286          | 0.1208              | 275   |
| 0.7788     | 0.1202         | 0.8325          | 0.1202              | 276   |
| 0.7777     | 0.1202         | 0.8301          | 0.1194              | 277   |
| 0.7771     | 0.1202         | 0.8164          | 0.1207              | 278   |
| 0.7762     | 0.1202         | 0.8154          | 0.1194              | 279   |
| 0.7757     | 0.1202         | 0.8242          | 0.1196              | 280   |
| 0.7751     | 0.1203         | 0.8140          | 0.1215              | 281   |
| 0.7751     | 0.1203         | 0.8193          | 0.1197              | 282   |
| 0.7746     | 0.1203         | 0.8008          | 0.1186              | 283   |
| 0.7746     | 0.1203         | 0.8105          | 0.1193              | 284   |
| 0.7733     | 0.1203         | 0.8223          | 0.1206              | 285   |
| 0.7733     | 0.1204         | 0.8125          | 0.1199              | 286   |
| 0.7720     | 0.1204         | 0.8228          | 0.1201              | 287   |
| 0.7721     | 0.1204         | 0.8164          | 0.1203              | 288   |
| 0.7719     | 0.1203         | 0.8359          | 0.1205              | 289   |
| 0.7713     | 0.1203         | 0.8145          | 0.1204              | 290   |
| 0.7703     | 0.1204         | 0.8057          | 0.1202              | 291   |
| 0.7698     | 0.1204         | 0.8174          | 0.1204              | 292   |
| 0.7697     | 0.1204         | 0.8091          | 0.1210              | 293   |
| 0.7686     | 0.1204         | 0.8154          | 0.1195              | 294   |
| 0.7690     | 0.1204         | 0.8242          | 0.1204              | 295   |
| 0.7679     | 0.1205         | 0.7979          | 0.1208              | 296   |
| 0.7680     | 0.1205         | 0.8105          | 0.1194              | 297   |
| 0.7673     | 0.1205         | 0.8003          | 0.1215              | 298   |
| 0.7672     | 0.1205         | 0.7925          | 0.1212              | 299   |
| 0.7661     | 0.1205         | 0.8115          | 0.1191              | 300   |
| 0.7654     | 0.1205         | 0.8188          | 0.1206              | 301   |
| 0.7657     | 0.1205         | 0.8140          | 0.1202              | 302   |
| 0.7644     | 0.1206         | 0.8228          | 0.1199              | 303   |
| 0.7651     | 0.1205         | 0.7954          | 0.1213              | 304   |
| 0.7640     | 0.1206         | 0.7861          | 0.1206              | 305   |
| 0.7633     | 0.1206         | 0.8223          | 0.1194              | 306   |
| 0.7632     | 0.1206         | 0.8037          | 0.1201              | 307   |
| 0.7628     | 0.1206         | 0.8120          | 0.1196              | 308   |
| 0.7633     | 0.1206         | 0.8101          | 0.1198              | 309   |
| 0.7612     | 0.1206         | 0.8296          | 0.1203              | 310   |
| 0.7613     | 0.1206         | 0.8105          | 0.1195              | 311   |
| 0.7614     | 0.1206         | 0.8203          | 0.1201              | 312   |
| 0.7606     | 0.1207         | 0.7900          | 0.1201              | 313   |
| 0.7597     | 0.1207         | 0.8057          | 0.1201              | 314   |
| 0.7600     | 0.1207         | 0.8237          | 0.1189              | 315   |
| 0.7584     | 0.1207         | 0.8315          | 0.1198              | 316   |
| 0.7592     | 0.1207         | 0.8228          | 0.1198              | 317   |
| 0.7678     | 0.1205         | 0.8008          | 0.1205              | 318   |
| 0.7598     | 0.1207         | 0.8091          | 0.1216              | 319   |
| 0.7579     | 0.1208         | 0.8174          | 0.1202              | 320   |
| 0.7572     | 0.1207         | 0.8232          | 0.1196              | 321   |
| 0.7565     | 0.1207         | 0.8018          | 0.1192              | 322   |
| 0.7556     | 0.1208         | 0.7949          | 0.1207              | 323   |
| 0.7555     | 0.1208         | 0.8105          | 0.1200              | 324   |
| 0.7555     | 0.1208         | 0.7925          | 0.1208              | 325   |
| 0.7553     | 0.1208         | 0.7847          | 0.1201              | 326   |
| 0.7544     | 0.1208         | 0.8022          | 0.1208              | 327   |
| 0.7542     | 0.1208         | 0.8096          | 0.1203              | 328   |
| 0.7540     | 0.1208         | 0.7949          | 0.1209              | 329   |
| 0.7536     | 0.1209         | 0.8184          | 0.1205              | 330   |
| 0.7536     | 0.1208         | 0.8013          | 0.1209              | 331   |
| 0.7531     | 0.1209         | 0.8149          | 0.1197              | 332   |
| 0.7523     | 0.1209         | 0.8110          | 0.1197              | 333   |
| 0.7521     | 0.1209         | 0.7998          | 0.1208              | 334   |
| 0.7519     | 0.1209         | 0.7798          | 0.1211              | 335   |
| 0.7505     | 0.1209         | 0.8076          | 0.1202              | 336   |
| 0.7504     | 0.1210         | 0.7974          | 0.1217              | 337   |
| 0.7506     | 0.1210         | 0.7910          | 0.1206              | 338   |
| 0.7493     | 0.1209         | 0.7969          | 0.1209              | 339   |
| 0.7498     | 0.1209         | 0.8105          | 0.1205              | 340   |
| 0.7493     | 0.1209         | 0.8145          | 0.1204              | 341   |
| 0.7491     | 0.1210         | 0.8062          | 0.1209              | 342   |
| 0.7485     | 0.1210         | 0.8091          | 0.1199              | 343   |
| 0.7480     | 0.1210         | 0.8101          | 0.1201              | 344   |
| 0.7482     | 0.1209         | 0.7993          | 0.1203              | 345   |
| 0.7468     | 0.1210         | 0.7939          | 0.1213              | 346   |
| 0.7473     | 0.1210         | 0.8140          | 0.1201              | 347   |
| 0.7468     | 0.1210         | 0.8066          | 0.1201              | 348   |
| 0.7460     | 0.1211         | 0.7964          | 0.1208              | 349   |
| 0.7460     | 0.1210         | 0.8184          | 0.1206              | 350   |
| 0.7446     | 0.1211         | 0.8047          | 0.1199              | 351   |
| 0.7453     | 0.1211         | 0.8091          | 0.1197              | 352   |
| 0.7449     | 0.1211         | 0.7969          | 0.1201              | 353   |
| 0.7441     | 0.1211         | 0.7905          | 0.1210              | 354   |
| 0.7437     | 0.1211         | 0.8018          | 0.1207              | 355   |
| 0.7439     | 0.1211         | 0.8013          | 0.1203              | 356   |
| 0.7437     | 0.1211         | 0.8130          | 0.1204              | 357   |
| 0.7426     | 0.1211         | 0.8013          | 0.1205              | 358   |
| 0.7419     | 0.1211         | 0.8003          | 0.1199              | 359   |
| 0.7421     | 0.1212         | 0.8081          | 0.1200              | 360   |
| 0.7417     | 0.1212         | 0.7964          | 0.1199              | 361   |
| 0.7408     | 0.1212         | 0.8027          | 0.1203              | 362   |
| 0.7404     | 0.1212         | 0.8052          | 0.1207              | 363   |
| 0.7402     | 0.1212         | 0.7993          | 0.1204              | 364   |
| 0.7412     | 0.1212         | 0.7896          | 0.1207              | 365   |
| 0.7404     | 0.1212         | 0.8071          | 0.1208              | 366   |
| 0.7398     | 0.1212         | 0.8037          | 0.1196              | 367   |
| 0.7389     | 0.1212         | 0.7949          | 0.1194              | 368   |
| 0.7399     | 0.1212         | 0.8125          | 0.1211              | 369   |
| 0.7389     | 0.1212         | 0.8101          | 0.1201              | 370   |
| 0.7380     | 0.1212         | 0.7983          | 0.1207              | 371   |
| 0.7380     | 0.1213         | 0.7969          | 0.1210              | 372   |
| 0.7373     | 0.1212         | 0.7822          | 0.1204              | 373   |
| 0.7367     | 0.1213         | 0.8164          | 0.1204              | 374   |
| 0.7370     | 0.1213         | 0.7920          | 0.1205              | 375   |
| 0.7366     | 0.1213         | 0.7842          | 0.1205              | 376   |
| 0.7362     | 0.1213         | 0.7905          | 0.1205              | 377   |
| 0.7359     | 0.1213         | 0.8105          | 0.1200              | 378   |
| 0.7360     | 0.1213         | 0.8037          | 0.1203              | 379   |
| 0.7352     | 0.1213         | 0.7974          | 0.1203              | 380   |
| 0.7350     | 0.1213         | 0.8140          | 0.1203              | 381   |
| 0.7341     | 0.1213         | 0.7891          | 0.1217              | 382   |
| 0.7349     | 0.1214         | 0.7891          | 0.1208              | 383   |
| 0.7340     | 0.1214         | 0.7739          | 0.1208              | 384   |
| 0.7339     | 0.1214         | 0.7871          | 0.1210              | 385   |
| 0.7334     | 0.1214         | 0.7856          | 0.1205              | 386   |
| 0.7337     | 0.1214         | 0.7856          | 0.1201              | 387   |
| 0.7330     | 0.1214         | 0.7817          | 0.1203              | 388   |
| 0.7334     | 0.1214         | 0.8193          | 0.1215              | 389   |
| 0.7319     | 0.1214         | 0.7788          | 0.1208              | 390   |
| 0.7319     | 0.1214         | 0.8042          | 0.1203              | 391   |
| 0.7315     | 0.1214         | 0.7935          | 0.1211              | 392   |
| 0.7312     | 0.1214         | 0.7959          | 0.1198              | 393   |
| 0.7310     | 0.1215         | 0.7993          | 0.1207              | 394   |
| 0.7300     | 0.1214         | 0.8057          | 0.1208              | 395   |
| 0.7302     | 0.1215         | 0.8008          | 0.1202              | 396   |
| 0.7306     | 0.1214         | 0.7817          | 0.1212              | 397   |
| 0.7293     | 0.1215         | 0.7827          | 0.1207              | 398   |
| 0.7288     | 0.1215         | 0.8115          | 0.1202              | 399   |
| 0.7296     | 0.1215         | 0.7998          | 0.1206              | 400   |
| 0.7290     | 0.1215         | 0.7983          | 0.1208              | 401   |
| 0.7284     | 0.1215         | 0.7842          | 0.1219              | 402   |
| 0.7280     | 0.1215         | 0.7896          | 0.1221              | 403   |
| 0.7282     | 0.1215         | 0.7935          | 0.1199              | 404   |
| 0.7266     | 0.1215         | 0.7891          | 0.1208              | 405   |
| 0.7276     | 0.1216         | 0.7808          | 0.1209              | 406   |
| 0.7275     | 0.1215         | 0.7842          | 0.1204              | 407   |
| 0.7266     | 0.1216         | 0.7930          | 0.1210              | 408   |
| 0.7262     | 0.1215         | 0.8042          | 0.1204              | 409   |
| 0.7258     | 0.1216         | 0.8071          | 0.1217              | 410   |
| 0.7253     | 0.1216         | 0.7920          | 0.1198              | 411   |
| 0.7258     | 0.1216         | 0.7979          | 0.1211              | 412   |
| 0.7256     | 0.1215         | 0.8066          | 0.1200              | 413   |
| 0.7246     | 0.1216         | 0.7749          | 0.1213              | 414   |
| 0.7246     | 0.1216         | 0.7861          | 0.1214              | 415   |
| 0.7238     | 0.1216         | 0.8101          | 0.1204              | 416   |
| 0.7244     | 0.1216         | 0.7939          | 0.1213              | 417   |
| 0.7243     | 0.1216         | 0.7896          | 0.1219              | 418   |
| 0.7233     | 0.1216         | 0.7891          | 0.1216              | 419   |
| 0.7238     | 0.1217         | 0.7930          | 0.1216              | 420   |
| 0.7231     | 0.1216         | 0.7935          | 0.1210              | 421   |
| 0.7235     | 0.1216         | 0.7949          | 0.1191              | 422   |
| 0.7226     | 0.1216         | 0.7925          | 0.1203              | 423   |
| 0.7222     | 0.1217         | 0.7910          | 0.1204              | 424   |
| 0.7220     | 0.1217         | 0.7720          | 0.1211              | 425   |
| 0.7218     | 0.1216         | 0.7979          | 0.1207              | 426   |
| 0.7205     | 0.1217         | 0.7798          | 0.1205              | 427   |
| 0.7215     | 0.1217         | 0.7954          | 0.1218              | 428   |
| 0.7210     | 0.1217         | 0.7817          | 0.1208              | 429   |
| 0.7195     | 0.1217         | 0.7871          | 0.1215              | 430   |
| 0.7206     | 0.1217         | 0.7778          | 0.1211              | 431   |
| 0.7209     | 0.1217         | 0.7715          | 0.1212              | 432   |
| 0.7195     | 0.1218         | 0.7974          | 0.1214              | 433   |
| 0.7191     | 0.1218         | 0.7954          | 0.1202              | 434   |
| 0.7185     | 0.1218         | 0.7866          | 0.1211              | 435   |
| 0.7185     | 0.1218         | 0.7881          | 0.1220              | 436   |
| 0.7187     | 0.1218         | 0.7910          | 0.1214              | 437   |
| 0.7180     | 0.1218         | 0.7949          | 0.1201              | 438   |
| 0.7183     | 0.1218         | 0.7847          | 0.1210              | 439   |
| 0.7177     | 0.1218         | 0.7744          | 0.1214              | 440   |
| 0.7176     | 0.1218         | 0.7754          | 0.1209              | 441   |
| 0.7176     | 0.1218         | 0.7764          | 0.1213              | 442   |
| 0.7170     | 0.1218         | 0.7812          | 0.1203              | 443   |
| 0.7170     | 0.1218         | 0.7935          | 0.1206              | 444   |
| 0.7171     | 0.1218         | 0.7959          | 0.1204              | 445   |
| 0.7165     | 0.1218         | 0.7979          | 0.1208              | 446   |
| 0.7164     | 0.1218         | 0.7930          | 0.1215              | 447   |
| 0.7164     | 0.1219         | 0.8003          | 0.1210              | 448   |
| 0.7157     | 0.1219         | 0.7764          | 0.1203              | 449   |
| 0.7154     | 0.1219         | 0.7935          | 0.1208              | 450   |
| 0.7150     | 0.1219         | 0.8047          | 0.1212              | 451   |
| 0.7147     | 0.1219         | 0.7847          | 0.1208              | 452   |
| 0.7153     | 0.1218         | 0.7817          | 0.1199              | 453   |
| 0.7146     | 0.1219         | 0.7886          | 0.1210              | 454   |
| 0.7150     | 0.1219         | 0.7920          | 0.1218              | 455   |
| 0.7144     | 0.1219         | 0.7793          | 0.1211              | 456   |
| 0.7143     | 0.1219         | 0.7676          | 0.1209              | 457   |
| 0.7140     | 0.1219         | 0.7920          | 0.1210              | 458   |
| 0.7143     | 0.1219         | 0.7925          | 0.1203              | 459   |
| 0.7137     | 0.1219         | 0.7886          | 0.1227              | 460   |
| 0.7135     | 0.1219         | 0.7964          | 0.1206              | 461   |
| 0.7128     | 0.1219         | 0.7969          | 0.1207              | 462   |
| 0.7125     | 0.1219         | 0.7837          | 0.1208              | 463   |
| 0.7134     | 0.1219         | 0.7788          | 0.1219              | 464   |
| 0.7125     | 0.1219         | 0.7759          | 0.1210              | 465   |
| 0.7127     | 0.1219         | 0.8013          | 0.1207              | 466   |
| 0.7129     | 0.1219         | 0.7812          | 0.1214              | 467   |
| 0.7118     | 0.1219         | 0.8052          | 0.1217              | 468   |
| 0.7114     | 0.1220         | 0.7847          | 0.1208              | 469   |
| 0.7107     | 0.1220         | 0.7646          | 0.1219              | 470   |
| 0.7111     | 0.1220         | 0.7939          | 0.1204              | 471   |
| 0.7115     | 0.1219         | 0.7861          | 0.1214              | 472   |
| 0.7111     | 0.1220         | 0.7744          | 0.1215              | 473   |
| 0.7106     | 0.1220         | 0.7695          | 0.1209              | 474   |
| 0.7109     | 0.1220         | 0.7573          | 0.1208              | 475   |
| 0.7099     | 0.1220         | 0.8003          | 0.1201              | 476   |
| 0.7107     | 0.1220         | 0.7725          | 0.1222              | 477   |
| 0.7101     | 0.1220         | 0.7881          | 0.1206              | 478   |
| 0.7096     | 0.1220         | 0.8027          | 0.1201              | 479   |
| 0.7094     | 0.1221         | 0.7861          | 0.1204              | 480   |
| 0.7094     | 0.1221         | 0.7798          | 0.1214              | 481   |
| 0.7097     | 0.1221         | 0.7837          | 0.1205              | 482   |
| 0.7096     | 0.1220         | 0.7793          | 0.1210              | 483   |
| 0.7082     | 0.1220         | 0.7627          | 0.1217              | 484   |
| 0.7092     | 0.1220         | 0.7954          | 0.1219              | 485   |
| 0.7086     | 0.1221         | 0.7837          | 0.1206              | 486   |
| 0.7087     | 0.1221         | 0.7856          | 0.1213              | 487   |
| 0.7079     | 0.1221         | 0.7876          | 0.1206              | 488   |
| 0.7082     | 0.1221         | 0.7778          | 0.1210              | 489   |
| 0.7083     | 0.1221         | 0.7905          | 0.1205              | 490   |
| 0.7084     | 0.1221         | 0.7842          | 0.1212              | 491   |
| 0.7075     | 0.1221         | 0.7793          | 0.1210              | 492   |
| 0.7074     | 0.1221         | 0.7749          | 0.1215              | 493   |
| 0.7075     | 0.1221         | 0.7764          | 0.1201              | 494   |
| 0.7078     | 0.1220         | 0.7842          | 0.1216              | 495   |
| 0.7079     | 0.1221         | 0.7900          | 0.1211              | 496   |
| 0.7085     | 0.1221         | 0.7744          | 0.1212              | 497   |
| 0.7075     | 0.1221         | 0.7725          | 0.1213              | 498   |
| 0.7074     | 0.1221         | 0.7739          | 0.1213              | 499   |


### Framework versions

- Transformers 4.27.0.dev0
- TensorFlow 2.9.1
- Tokenizers 0.13.2