|
--- |
|
license: mit |
|
tags: |
|
- generated_from_keras_callback |
|
model-index: |
|
- name: tf-tpu/roberta-base-epochs-500-no-wd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information Keras had access to. You should |
|
probably proofread and complete it, then remove this comment. --> |
|
|
|
# tf-tpu/roberta-base-epochs-500-no-wd |
|
|
|
This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Train Loss: 0.8045 |
|
- Train Accuracy: 0.1195 |
|
- Validation Loss: 0.8350 |
|
- Validation Accuracy: 0.1187 |
|
- Epoch: 230 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 0.0001, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 278825, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 14675, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.001} |
|
- training_precision: mixed_bfloat16 |
|
|
|
### Training results |
|
|
|
| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch | |
|
|:----------:|:--------------:|:---------------:|:-------------------:|:-----:| |
|
| 8.3284 | 0.0211 | 7.1523 | 0.0266 | 0 | |
|
| 6.3670 | 0.0318 | 5.7812 | 0.0342 | 1 | |
|
| 5.6051 | 0.0380 | 5.4414 | 0.0420 | 2 | |
|
| 5.3602 | 0.0433 | 5.2734 | 0.0432 | 3 | |
|
| 5.2285 | 0.0444 | 5.1562 | 0.0442 | 4 | |
|
| 5.1371 | 0.0446 | 5.1133 | 0.0436 | 5 | |
|
| 5.0673 | 0.0446 | 5.0703 | 0.0442 | 6 | |
|
| 5.0132 | 0.0447 | 4.9883 | 0.0442 | 7 | |
|
| 4.9642 | 0.0448 | 4.9219 | 0.0441 | 8 | |
|
| 4.9217 | 0.0448 | 4.9258 | 0.0440 | 9 | |
|
| 4.8871 | 0.0448 | 4.8867 | 0.0439 | 10 | |
|
| 4.8548 | 0.0449 | 4.8672 | 0.0439 | 11 | |
|
| 4.8277 | 0.0449 | 4.8047 | 0.0445 | 12 | |
|
| 4.8033 | 0.0449 | 4.8477 | 0.0437 | 13 | |
|
| 4.7807 | 0.0449 | 4.7617 | 0.0439 | 14 | |
|
| 4.7592 | 0.0449 | 4.7773 | 0.0437 | 15 | |
|
| 4.7388 | 0.0449 | 4.7539 | 0.0441 | 16 | |
|
| 4.7225 | 0.0449 | 4.7266 | 0.0439 | 17 | |
|
| 4.7052 | 0.0449 | 4.6914 | 0.0450 | 18 | |
|
| 4.6917 | 0.0449 | 4.7188 | 0.0444 | 19 | |
|
| 4.6789 | 0.0449 | 4.6914 | 0.0444 | 20 | |
|
| 4.6689 | 0.0449 | 4.7031 | 0.0439 | 21 | |
|
| 4.6570 | 0.0449 | 4.7031 | 0.0437 | 22 | |
|
| 4.6486 | 0.0450 | 4.6758 | 0.0446 | 23 | |
|
| 4.6393 | 0.0449 | 4.6914 | 0.0441 | 24 | |
|
| 4.5898 | 0.0449 | 4.4688 | 0.0452 | 25 | |
|
| 4.3024 | 0.0472 | 3.8730 | 0.0551 | 26 | |
|
| 3.1689 | 0.0693 | 2.4375 | 0.0835 | 27 | |
|
| 2.3780 | 0.0844 | 2.0498 | 0.0922 | 28 | |
|
| 2.0789 | 0.0907 | 1.8604 | 0.0958 | 29 | |
|
| 1.9204 | 0.0940 | 1.7549 | 0.0982 | 30 | |
|
| 1.8162 | 0.0961 | 1.6836 | 0.0983 | 31 | |
|
| 1.7370 | 0.0978 | 1.5869 | 0.1014 | 32 | |
|
| 1.6723 | 0.0991 | 1.5381 | 0.1029 | 33 | |
|
| 1.6215 | 0.1002 | 1.5283 | 0.1015 | 34 | |
|
| 1.5753 | 0.1012 | 1.4736 | 0.1037 | 35 | |
|
| 1.5295 | 0.1022 | 1.4238 | 0.1052 | 36 | |
|
| 1.4944 | 0.1030 | 1.4141 | 0.1059 | 37 | |
|
| 1.4631 | 0.1037 | 1.3721 | 0.1053 | 38 | |
|
| 1.4363 | 0.1043 | 1.3467 | 0.1060 | 39 | |
|
| 1.4098 | 0.1049 | 1.3213 | 0.1076 | 40 | |
|
| 1.3867 | 0.1054 | 1.3018 | 0.1071 | 41 | |
|
| 1.3658 | 0.1058 | 1.2832 | 0.1083 | 42 | |
|
| 1.3469 | 0.1063 | 1.2637 | 0.1081 | 43 | |
|
| 1.3288 | 0.1067 | 1.2598 | 0.1082 | 44 | |
|
| 1.3111 | 0.1071 | 1.2334 | 0.1096 | 45 | |
|
| 1.2962 | 0.1075 | 1.2490 | 0.1084 | 46 | |
|
| 1.2816 | 0.1078 | 1.2168 | 0.1093 | 47 | |
|
| 1.2672 | 0.1081 | 1.2070 | 0.1090 | 48 | |
|
| 1.2537 | 0.1084 | 1.1680 | 0.1106 | 49 | |
|
| 1.2411 | 0.1087 | 1.1904 | 0.1094 | 50 | |
|
| 1.2285 | 0.1090 | 1.1709 | 0.1103 | 51 | |
|
| 1.2180 | 0.1093 | 1.1602 | 0.1122 | 52 | |
|
| 1.2075 | 0.1095 | 1.1396 | 0.1117 | 53 | |
|
| 1.1973 | 0.1098 | 1.1191 | 0.1124 | 54 | |
|
| 1.1876 | 0.1100 | 1.1260 | 0.1123 | 55 | |
|
| 1.1782 | 0.1102 | 1.1289 | 0.1111 | 56 | |
|
| 1.1698 | 0.1104 | 1.1211 | 0.1117 | 57 | |
|
| 1.1596 | 0.1106 | 1.0977 | 0.1125 | 58 | |
|
| 1.1530 | 0.1108 | 1.1172 | 0.1118 | 59 | |
|
| 1.1462 | 0.1110 | 1.0703 | 0.1126 | 60 | |
|
| 1.1370 | 0.1112 | 1.0830 | 0.1140 | 61 | |
|
| 1.1309 | 0.1113 | 1.0762 | 0.1119 | 62 | |
|
| 1.1234 | 0.1115 | 1.0625 | 0.1137 | 63 | |
|
| 1.1162 | 0.1117 | 1.0781 | 0.1127 | 64 | |
|
| 1.1114 | 0.1118 | 1.0474 | 0.1138 | 65 | |
|
| 1.1036 | 0.1120 | 1.0703 | 0.1134 | 66 | |
|
| 1.0984 | 0.1121 | 1.0366 | 0.1139 | 67 | |
|
| 1.0931 | 0.1122 | 1.0513 | 0.1134 | 68 | |
|
| 1.0860 | 0.1124 | 1.0264 | 0.1137 | 69 | |
|
| 1.0807 | 0.1126 | 1.0215 | 0.1148 | 70 | |
|
| 1.0758 | 0.1127 | 1.0269 | 0.1143 | 71 | |
|
| 1.0704 | 0.1129 | 1.0356 | 0.1141 | 72 | |
|
| 1.0656 | 0.1129 | 1.0195 | 0.1144 | 73 | |
|
| 1.0607 | 0.1131 | 1.0093 | 0.1146 | 74 | |
|
| 1.0559 | 0.1132 | 0.9956 | 0.1155 | 75 | |
|
| 1.0517 | 0.1133 | 0.9995 | 0.1139 | 76 | |
|
| 1.0462 | 0.1134 | 0.9839 | 0.1151 | 77 | |
|
| 1.0422 | 0.1135 | 0.9868 | 0.1153 | 78 | |
|
| 1.0372 | 0.1137 | 0.9995 | 0.1151 | 79 | |
|
| 1.0340 | 0.1137 | 1.0059 | 0.1153 | 80 | |
|
| 1.0296 | 0.1138 | 0.9961 | 0.1152 | 81 | |
|
| 1.0272 | 0.1138 | 1.0132 | 0.1138 | 82 | |
|
| 1.0211 | 0.1140 | 0.9575 | 0.1150 | 83 | |
|
| 1.0182 | 0.1141 | 0.9868 | 0.1150 | 84 | |
|
| 1.0146 | 0.1142 | 0.9678 | 0.1164 | 85 | |
|
| 1.0111 | 0.1143 | 0.9839 | 0.1161 | 86 | |
|
| 1.0083 | 0.1144 | 0.9722 | 0.1162 | 87 | |
|
| 1.0039 | 0.1144 | 0.9619 | 0.1167 | 88 | |
|
| 1.0017 | 0.1145 | 0.9575 | 0.1151 | 89 | |
|
| 0.9973 | 0.1146 | 0.9624 | 0.1149 | 90 | |
|
| 0.9947 | 0.1147 | 0.9570 | 0.1157 | 91 | |
|
| 0.9921 | 0.1148 | 0.9360 | 0.1166 | 92 | |
|
| 0.9884 | 0.1149 | 0.9546 | 0.1156 | 93 | |
|
| 0.9851 | 0.1149 | 0.9536 | 0.1149 | 94 | |
|
| 0.9829 | 0.1150 | 0.9575 | 0.1163 | 95 | |
|
| 0.9795 | 0.1151 | 0.9561 | 0.1156 | 96 | |
|
| 0.9773 | 0.1151 | 0.9438 | 0.1163 | 97 | |
|
| 0.9740 | 0.1152 | 0.9512 | 0.1169 | 98 | |
|
| 0.9712 | 0.1153 | 0.9375 | 0.1159 | 99 | |
|
| 0.9678 | 0.1154 | 0.9453 | 0.1166 | 100 | |
|
| 0.9660 | 0.1154 | 0.9507 | 0.1169 | 101 | |
|
| 0.9636 | 0.1155 | 0.9507 | 0.1161 | 102 | |
|
| 0.9609 | 0.1155 | 0.9727 | 0.1164 | 103 | |
|
| 0.9589 | 0.1156 | 0.9395 | 0.1176 | 104 | |
|
| 0.9561 | 0.1157 | 0.9346 | 0.1173 | 105 | |
|
| 0.9537 | 0.1157 | 0.9331 | 0.1168 | 106 | |
|
| 0.9515 | 0.1158 | 0.9434 | 0.1161 | 107 | |
|
| 0.9488 | 0.1158 | 0.9131 | 0.1176 | 108 | |
|
| 0.9471 | 0.1159 | 0.9360 | 0.1174 | 109 | |
|
| 0.9449 | 0.1159 | 0.9175 | 0.1164 | 110 | |
|
| 0.9422 | 0.1160 | 0.9121 | 0.1167 | 111 | |
|
| 0.9412 | 0.1160 | 0.8970 | 0.1165 | 112 | |
|
| 0.9379 | 0.1161 | 0.9111 | 0.1175 | 113 | |
|
| 0.9362 | 0.1161 | 0.9048 | 0.1176 | 114 | |
|
| 0.9345 | 0.1162 | 0.9082 | 0.1169 | 115 | |
|
| 0.9317 | 0.1163 | 0.9277 | 0.1169 | 116 | |
|
| 0.9295 | 0.1164 | 0.9292 | 0.1169 | 117 | |
|
| 0.9287 | 0.1163 | 0.9243 | 0.1169 | 118 | |
|
| 0.9266 | 0.1163 | 0.8892 | 0.1170 | 119 | |
|
| 0.9233 | 0.1165 | 0.9058 | 0.1174 | 120 | |
|
| 0.9221 | 0.1165 | 0.9106 | 0.1175 | 121 | |
|
| 0.9205 | 0.1166 | 0.8979 | 0.1173 | 122 | |
|
| 0.9181 | 0.1167 | 0.8989 | 0.1174 | 123 | |
|
| 0.9180 | 0.1166 | 0.9053 | 0.1172 | 124 | |
|
| 0.9158 | 0.1167 | 0.8877 | 0.1176 | 125 | |
|
| 0.9135 | 0.1168 | 0.9160 | 0.1169 | 126 | |
|
| 0.9116 | 0.1167 | 0.8940 | 0.1180 | 127 | |
|
| 0.9095 | 0.1168 | 0.8945 | 0.1173 | 128 | |
|
| 0.9081 | 0.1168 | 0.9126 | 0.1166 | 129 | |
|
| 0.9064 | 0.1169 | 0.8872 | 0.1177 | 130 | |
|
| 0.9053 | 0.1169 | 0.9175 | 0.1172 | 131 | |
|
| 0.9035 | 0.1170 | 0.8989 | 0.1180 | 132 | |
|
| 0.9023 | 0.1170 | 0.8965 | 0.1179 | 133 | |
|
| 0.8999 | 0.1170 | 0.8979 | 0.1181 | 134 | |
|
| 0.8981 | 0.1171 | 0.8799 | 0.1186 | 135 | |
|
| 0.8976 | 0.1171 | 0.8984 | 0.1174 | 136 | |
|
| 0.8957 | 0.1172 | 0.8857 | 0.1181 | 137 | |
|
| 0.8948 | 0.1172 | 0.9019 | 0.1172 | 138 | |
|
| 0.8929 | 0.1172 | 0.8804 | 0.1180 | 139 | |
|
| 0.8915 | 0.1173 | 0.8848 | 0.1183 | 140 | |
|
| 0.8898 | 0.1173 | 0.8911 | 0.1177 | 141 | |
|
| 0.8894 | 0.1173 | 0.9033 | 0.1173 | 142 | |
|
| 0.8869 | 0.1174 | 0.8853 | 0.1184 | 143 | |
|
| 0.8863 | 0.1174 | 0.8921 | 0.1184 | 144 | |
|
| 0.8848 | 0.1175 | 0.8848 | 0.1177 | 145 | |
|
| 0.8838 | 0.1175 | 0.8896 | 0.1177 | 146 | |
|
| 0.8822 | 0.1175 | 0.8945 | 0.1181 | 147 | |
|
| 0.8804 | 0.1176 | 0.8843 | 0.1177 | 148 | |
|
| 0.8794 | 0.1175 | 0.8774 | 0.1181 | 149 | |
|
| 0.8780 | 0.1176 | 0.875 | 0.1178 | 150 | |
|
| 0.8756 | 0.1176 | 0.8862 | 0.1170 | 151 | |
|
| 0.8747 | 0.1177 | 0.8730 | 0.1178 | 152 | |
|
| 0.8737 | 0.1177 | 0.8696 | 0.1195 | 153 | |
|
| 0.8736 | 0.1177 | 0.8726 | 0.1184 | 154 | |
|
| 0.8716 | 0.1178 | 0.8647 | 0.1186 | 155 | |
|
| 0.8705 | 0.1178 | 0.8804 | 0.1179 | 156 | |
|
| 0.8695 | 0.1178 | 0.8652 | 0.1190 | 157 | |
|
| 0.8675 | 0.1179 | 0.8804 | 0.1197 | 158 | |
|
| 0.8670 | 0.1179 | 0.8462 | 0.1192 | 159 | |
|
| 0.8656 | 0.1180 | 0.8594 | 0.1188 | 160 | |
|
| 0.8649 | 0.1180 | 0.8535 | 0.1188 | 161 | |
|
| 0.8633 | 0.1181 | 0.8555 | 0.1185 | 162 | |
|
| 0.8622 | 0.1180 | 0.8633 | 0.1173 | 163 | |
|
| 0.8603 | 0.1181 | 0.8667 | 0.1177 | 164 | |
|
| 0.8598 | 0.1181 | 0.8813 | 0.1185 | 165 | |
|
| 0.8591 | 0.1181 | 0.8862 | 0.1176 | 166 | |
|
| 0.8580 | 0.1181 | 0.8853 | 0.1177 | 167 | |
|
| 0.8573 | 0.1181 | 0.8691 | 0.1181 | 168 | |
|
| 0.8558 | 0.1182 | 0.8481 | 0.1176 | 169 | |
|
| 0.8541 | 0.1182 | 0.8652 | 0.1187 | 170 | |
|
| 0.8541 | 0.1183 | 0.8477 | 0.1198 | 171 | |
|
| 0.8522 | 0.1183 | 0.8721 | 0.1190 | 172 | |
|
| 0.8516 | 0.1183 | 0.8965 | 0.1173 | 173 | |
|
| 0.8506 | 0.1183 | 0.8574 | 0.1173 | 174 | |
|
| 0.8496 | 0.1183 | 0.8452 | 0.1188 | 175 | |
|
| 0.8487 | 0.1184 | 0.8545 | 0.1183 | 176 | |
|
| 0.8478 | 0.1184 | 0.8594 | 0.1191 | 177 | |
|
| 0.8466 | 0.1184 | 0.8608 | 0.1187 | 178 | |
|
| 0.8456 | 0.1184 | 0.8472 | 0.1186 | 179 | |
|
| 0.8451 | 0.1185 | 0.8672 | 0.1178 | 180 | |
|
| 0.8429 | 0.1185 | 0.8364 | 0.1196 | 181 | |
|
| 0.8420 | 0.1185 | 0.8525 | 0.1187 | 182 | |
|
| 0.8419 | 0.1186 | 0.8525 | 0.1196 | 183 | |
|
| 0.8406 | 0.1186 | 0.8521 | 0.1193 | 184 | |
|
| 0.8391 | 0.1186 | 0.8560 | 0.1188 | 185 | |
|
| 0.8396 | 0.1186 | 0.8413 | 0.1188 | 186 | |
|
| 0.8378 | 0.1186 | 0.8628 | 0.1185 | 187 | |
|
| 0.8374 | 0.1186 | 0.8374 | 0.1195 | 188 | |
|
| 0.8364 | 0.1187 | 0.8691 | 0.1189 | 189 | |
|
| 0.8348 | 0.1187 | 0.8457 | 0.1196 | 190 | |
|
| 0.8354 | 0.1187 | 0.8286 | 0.1191 | 191 | |
|
| 0.8334 | 0.1187 | 0.8486 | 0.1187 | 192 | |
|
| 0.8325 | 0.1188 | 0.8535 | 0.1182 | 193 | |
|
| 0.8322 | 0.1188 | 0.8574 | 0.1199 | 194 | |
|
| 0.8314 | 0.1188 | 0.8472 | 0.1202 | 195 | |
|
| 0.8307 | 0.1188 | 0.8584 | 0.1186 | 196 | |
|
| 0.8294 | 0.1189 | 0.8345 | 0.1197 | 197 | |
|
| 0.8285 | 0.1189 | 0.8491 | 0.1181 | 198 | |
|
| 0.8275 | 0.1189 | 0.8472 | 0.1193 | 199 | |
|
| 0.8265 | 0.1189 | 0.8521 | 0.1185 | 200 | |
|
| 0.8262 | 0.1190 | 0.8501 | 0.1195 | 201 | |
|
| 0.8247 | 0.1190 | 0.8491 | 0.1194 | 202 | |
|
| 0.8245 | 0.1190 | 0.8389 | 0.1191 | 203 | |
|
| 0.8237 | 0.1190 | 0.8491 | 0.1184 | 204 | |
|
| 0.8229 | 0.1190 | 0.8525 | 0.1193 | 205 | |
|
| 0.8215 | 0.1190 | 0.8345 | 0.1199 | 206 | |
|
| 0.8213 | 0.1190 | 0.8511 | 0.1206 | 207 | |
|
| 0.8204 | 0.1191 | 0.8296 | 0.1195 | 208 | |
|
| 0.8193 | 0.1192 | 0.8516 | 0.1183 | 209 | |
|
| 0.8195 | 0.1191 | 0.8672 | 0.1181 | 210 | |
|
| 0.8188 | 0.1191 | 0.8267 | 0.1197 | 211 | |
|
| 0.8177 | 0.1192 | 0.8408 | 0.1185 | 212 | |
|
| 0.8167 | 0.1192 | 0.8447 | 0.1191 | 213 | |
|
| 0.8153 | 0.1192 | 0.8374 | 0.1191 | 214 | |
|
| 0.8158 | 0.1192 | 0.8438 | 0.1198 | 215 | |
|
| 0.8149 | 0.1192 | 0.8286 | 0.1191 | 216 | |
|
| 0.8141 | 0.1193 | 0.8389 | 0.1202 | 217 | |
|
| 0.8133 | 0.1192 | 0.8491 | 0.1202 | 218 | |
|
| 0.8127 | 0.1193 | 0.8730 | 0.1185 | 219 | |
|
| 0.8118 | 0.1193 | 0.8198 | 0.1183 | 220 | |
|
| 0.8115 | 0.1193 | 0.8164 | 0.1200 | 221 | |
|
| 0.8095 | 0.1194 | 0.8340 | 0.1195 | 222 | |
|
| 0.8090 | 0.1194 | 0.8071 | 0.1208 | 223 | |
|
| 0.8089 | 0.1194 | 0.8101 | 0.1195 | 224 | |
|
| 0.8081 | 0.1194 | 0.8311 | 0.1184 | 225 | |
|
| 0.8081 | 0.1194 | 0.8413 | 0.1198 | 226 | |
|
| 0.8065 | 0.1195 | 0.8379 | 0.1202 | 227 | |
|
| 0.8064 | 0.1194 | 0.8398 | 0.1196 | 228 | |
|
| 0.8045 | 0.1195 | 0.8159 | 0.1199 | 229 | |
|
| 0.8045 | 0.1195 | 0.8350 | 0.1187 | 230 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.27.0.dev0 |
|
- TensorFlow 2.9.1 |
|
- Tokenizers 0.13.2 |
|
|
