File size: 7,182 Bytes
2bc6870 15312f9 2bc6870 53ad6d9 d6eba47 8a82e17 bff874d 6b6ec78 ea35c7a 695c18a 81555c7 1debcdd 2f03e73 47eda48 22aea34 72ee4d4 8cd99b6 414d598 b416c35 deacb04 608d4dd fa377d2 37ebf72 6304609 41c2c48 e947221 7e96d8d 6202a72 26fc5e4 da6c6da e213a21 b437ee2 4184a67 8a14282 7087cca e8601da 2c71f1c 3a57baa 40cdc38 dbc244a 24449a8 8a6e38f 44af771 f47d714 7a2570a f21c61a 4e29aef cddc347 92e6692 39890a5 ef178e0 2f1725c 0a9a238 9a4231c f6fb138 46118f3 0970377 3e4f7d9 81b73d1 140ca33 ec86b04 c0ae2ee b773d47 f381da1 c5e3681 ed81298 a184e90 0493f13 8c19c14 fea11bf 15312f9 2bc6870 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 |
---
license: mit
tags:
- generated_from_keras_callback
model-index:
- name: tf-tpu/roberta-base-epochs-500-no-wd
results: []
---
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->
# tf-tpu/roberta-base-epochs-500-no-wd
This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 1.0984
- Train Accuracy: 0.1121
- Validation Loss: 1.0366
- Validation Accuracy: 0.1139
- Epoch: 67
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 0.0001, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 278825, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 14675, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.001}
- training_precision: mixed_bfloat16
### Training results
| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch |
|:----------:|:--------------:|:---------------:|:-------------------:|:-----:|
| 8.3284 | 0.0211 | 7.1523 | 0.0266 | 0 |
| 6.3670 | 0.0318 | 5.7812 | 0.0342 | 1 |
| 5.6051 | 0.0380 | 5.4414 | 0.0420 | 2 |
| 5.3602 | 0.0433 | 5.2734 | 0.0432 | 3 |
| 5.2285 | 0.0444 | 5.1562 | 0.0442 | 4 |
| 5.1371 | 0.0446 | 5.1133 | 0.0436 | 5 |
| 5.0673 | 0.0446 | 5.0703 | 0.0442 | 6 |
| 5.0132 | 0.0447 | 4.9883 | 0.0442 | 7 |
| 4.9642 | 0.0448 | 4.9219 | 0.0441 | 8 |
| 4.9217 | 0.0448 | 4.9258 | 0.0440 | 9 |
| 4.8871 | 0.0448 | 4.8867 | 0.0439 | 10 |
| 4.8548 | 0.0449 | 4.8672 | 0.0439 | 11 |
| 4.8277 | 0.0449 | 4.8047 | 0.0445 | 12 |
| 4.8033 | 0.0449 | 4.8477 | 0.0437 | 13 |
| 4.7807 | 0.0449 | 4.7617 | 0.0439 | 14 |
| 4.7592 | 0.0449 | 4.7773 | 0.0437 | 15 |
| 4.7388 | 0.0449 | 4.7539 | 0.0441 | 16 |
| 4.7225 | 0.0449 | 4.7266 | 0.0439 | 17 |
| 4.7052 | 0.0449 | 4.6914 | 0.0450 | 18 |
| 4.6917 | 0.0449 | 4.7188 | 0.0444 | 19 |
| 4.6789 | 0.0449 | 4.6914 | 0.0444 | 20 |
| 4.6689 | 0.0449 | 4.7031 | 0.0439 | 21 |
| 4.6570 | 0.0449 | 4.7031 | 0.0437 | 22 |
| 4.6486 | 0.0450 | 4.6758 | 0.0446 | 23 |
| 4.6393 | 0.0449 | 4.6914 | 0.0441 | 24 |
| 4.5898 | 0.0449 | 4.4688 | 0.0452 | 25 |
| 4.3024 | 0.0472 | 3.8730 | 0.0551 | 26 |
| 3.1689 | 0.0693 | 2.4375 | 0.0835 | 27 |
| 2.3780 | 0.0844 | 2.0498 | 0.0922 | 28 |
| 2.0789 | 0.0907 | 1.8604 | 0.0958 | 29 |
| 1.9204 | 0.0940 | 1.7549 | 0.0982 | 30 |
| 1.8162 | 0.0961 | 1.6836 | 0.0983 | 31 |
| 1.7370 | 0.0978 | 1.5869 | 0.1014 | 32 |
| 1.6723 | 0.0991 | 1.5381 | 0.1029 | 33 |
| 1.6215 | 0.1002 | 1.5283 | 0.1015 | 34 |
| 1.5753 | 0.1012 | 1.4736 | 0.1037 | 35 |
| 1.5295 | 0.1022 | 1.4238 | 0.1052 | 36 |
| 1.4944 | 0.1030 | 1.4141 | 0.1059 | 37 |
| 1.4631 | 0.1037 | 1.3721 | 0.1053 | 38 |
| 1.4363 | 0.1043 | 1.3467 | 0.1060 | 39 |
| 1.4098 | 0.1049 | 1.3213 | 0.1076 | 40 |
| 1.3867 | 0.1054 | 1.3018 | 0.1071 | 41 |
| 1.3658 | 0.1058 | 1.2832 | 0.1083 | 42 |
| 1.3469 | 0.1063 | 1.2637 | 0.1081 | 43 |
| 1.3288 | 0.1067 | 1.2598 | 0.1082 | 44 |
| 1.3111 | 0.1071 | 1.2334 | 0.1096 | 45 |
| 1.2962 | 0.1075 | 1.2490 | 0.1084 | 46 |
| 1.2816 | 0.1078 | 1.2168 | 0.1093 | 47 |
| 1.2672 | 0.1081 | 1.2070 | 0.1090 | 48 |
| 1.2537 | 0.1084 | 1.1680 | 0.1106 | 49 |
| 1.2411 | 0.1087 | 1.1904 | 0.1094 | 50 |
| 1.2285 | 0.1090 | 1.1709 | 0.1103 | 51 |
| 1.2180 | 0.1093 | 1.1602 | 0.1122 | 52 |
| 1.2075 | 0.1095 | 1.1396 | 0.1117 | 53 |
| 1.1973 | 0.1098 | 1.1191 | 0.1124 | 54 |
| 1.1876 | 0.1100 | 1.1260 | 0.1123 | 55 |
| 1.1782 | 0.1102 | 1.1289 | 0.1111 | 56 |
| 1.1698 | 0.1104 | 1.1211 | 0.1117 | 57 |
| 1.1596 | 0.1106 | 1.0977 | 0.1125 | 58 |
| 1.1530 | 0.1108 | 1.1172 | 0.1118 | 59 |
| 1.1462 | 0.1110 | 1.0703 | 0.1126 | 60 |
| 1.1370 | 0.1112 | 1.0830 | 0.1140 | 61 |
| 1.1309 | 0.1113 | 1.0762 | 0.1119 | 62 |
| 1.1234 | 0.1115 | 1.0625 | 0.1137 | 63 |
| 1.1162 | 0.1117 | 1.0781 | 0.1127 | 64 |
| 1.1114 | 0.1118 | 1.0474 | 0.1138 | 65 |
| 1.1036 | 0.1120 | 1.0703 | 0.1134 | 66 |
| 1.0984 | 0.1121 | 1.0366 | 0.1139 | 67 |
### Framework versions
- Transformers 4.27.0.dev0
- TensorFlow 2.9.1
- Tokenizers 0.13.2
|