--- license: apache-2.0 base_model: distilbert-base-uncased tags: - generated_from_keras_callback model-index: - name: edyfjm07/distilbert-base-uncased-v2-finetuned-squad-es results: [] --- # edyfjm07/distilbert-base-uncased-v2-finetuned-squad-es This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset. It achieves the following results on the evaluation set: - Train Loss: 0.0170 - Train End Logits Accuracy: 0.9975 - Train Start Logits Accuracy: 0.9950 - Validation Loss: 0.6848 - Validation End Logits Accuracy: 0.8922 - Validation Start Logits Accuracy: 0.8848 - Epoch: 49 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 5000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False} - training_precision: float32 ### Training results | Train Loss | Train End Logits Accuracy | Train Start Logits Accuracy | Validation Loss | Validation End Logits Accuracy | Validation Start Logits Accuracy | Epoch | |:----------:|:-------------------------:|:---------------------------:|:---------------:|:------------------------------:|:--------------------------------:|:-----:| | 2.4937 | 0.4238 | 0.3650 | 1.6575 | 0.4535 | 0.5688 | 0 | | 1.1993 | 0.625 | 0.6425 | 0.8766 | 0.6952 | 0.6840 | 1 | | 0.7478 | 0.7262 | 0.7462 | 0.7438 | 0.7323 | 0.7286 | 2 | | 0.6099 | 0.7700 | 0.7763 | 0.6805 | 0.7361 | 0.7286 | 3 | | 0.4741 | 0.8163 | 0.8263 | 0.5590 | 0.8104 | 0.7658 | 4 | | 0.4413 | 0.8263 | 0.8138 | 0.6294 | 0.7955 | 0.7918 | 5 | | 0.4165 | 0.8450 | 0.8388 | 0.5712 | 0.8030 | 0.7918 | 6 | | 0.3614 | 0.8625 | 0.8525 | 0.5701 | 0.8141 | 0.7695 | 7 | | 0.3260 | 0.8737 | 0.8788 | 0.6174 | 0.8216 | 0.7807 | 8 | | 0.3187 | 0.875 | 0.8687 | 0.5824 | 0.8216 | 0.7955 | 9 | | 0.2739 | 0.9050 | 0.8825 | 0.5829 | 0.8216 | 0.8067 | 10 | | 0.2465 | 0.9087 | 0.9087 | 0.5796 | 0.8216 | 0.8104 | 11 | | 0.2507 | 0.8950 | 0.8913 | 0.6048 | 0.8587 | 0.7881 | 12 | | 0.2102 | 0.9225 | 0.9075 | 0.5560 | 0.8662 | 0.8253 | 13 | | 0.2129 | 0.9187 | 0.9137 | 0.5616 | 0.8439 | 0.8439 | 14 | | 0.1939 | 0.9237 | 0.9225 | 0.5186 | 0.8587 | 0.8439 | 15 | | 0.1621 | 0.9400 | 0.9413 | 0.5331 | 0.8587 | 0.8476 | 16 | | 0.1620 | 0.9463 | 0.9463 | 0.5752 | 0.8550 | 0.8513 | 17 | | 0.1450 | 0.9463 | 0.9362 | 0.5934 | 0.8699 | 0.8476 | 18 | | 0.1374 | 0.9400 | 0.9525 | 0.5648 | 0.8699 | 0.8625 | 19 | | 0.1234 | 0.9438 | 0.9488 | 0.6096 | 0.8848 | 0.8327 | 20 | | 0.1300 | 0.9525 | 0.9613 | 0.5854 | 0.8699 | 0.8625 | 21 | | 0.1095 | 0.9600 | 0.9513 | 0.5962 | 0.8662 | 0.8587 | 22 | | 0.1168 | 0.9588 | 0.9588 | 0.6229 | 0.8736 | 0.8513 | 23 | | 0.0919 | 0.9650 | 0.9638 | 0.6139 | 0.8773 | 0.8699 | 24 | | 0.0880 | 0.9725 | 0.9700 | 0.6668 | 0.8699 | 0.8401 | 25 | | 0.0828 | 0.9725 | 0.9600 | 0.6261 | 0.8699 | 0.8550 | 26 | | 0.0846 | 0.9675 | 0.9725 | 0.7065 | 0.8662 | 0.8662 | 27 | | 0.0833 | 0.9725 | 0.9638 | 0.6470 | 0.8699 | 0.8662 | 28 | | 0.0772 | 0.9787 | 0.9688 | 0.6112 | 0.8810 | 0.8922 | 29 | | 0.0465 | 0.9837 | 0.9837 | 0.6582 | 0.8699 | 0.8736 | 30 | | 0.0619 | 0.9700 | 0.9800 | 0.6287 | 0.8810 | 0.8736 | 31 | | 0.0589 | 0.9800 | 0.9775 | 0.6796 | 0.8736 | 0.8625 | 32 | | 0.0446 | 0.9862 | 0.9825 | 0.6717 | 0.8848 | 0.8699 | 33 | | 0.0401 | 0.9862 | 0.9837 | 0.6632 | 0.8848 | 0.8848 | 34 | | 0.0432 | 0.9800 | 0.9887 | 0.6478 | 0.8773 | 0.8736 | 35 | | 0.0406 | 0.9837 | 0.9862 | 0.6627 | 0.8773 | 0.8810 | 36 | | 0.0392 | 0.9837 | 0.9875 | 0.6827 | 0.8848 | 0.8699 | 37 | | 0.0351 | 0.9825 | 0.9912 | 0.6693 | 0.8810 | 0.8699 | 38 | | 0.0308 | 0.9912 | 0.9900 | 0.6689 | 0.8810 | 0.8810 | 39 | | 0.0303 | 0.9850 | 0.9912 | 0.7091 | 0.8922 | 0.8699 | 40 | | 0.0334 | 0.9937 | 0.9850 | 0.6542 | 0.8885 | 0.8810 | 41 | | 0.0346 | 0.9912 | 0.9850 | 0.6472 | 0.8885 | 0.8736 | 42 | | 0.0264 | 0.9912 | 0.9925 | 0.6369 | 0.8885 | 0.8848 | 43 | | 0.0261 | 0.9937 | 0.9912 | 0.6484 | 0.8885 | 0.8810 | 44 | | 0.0255 | 0.9912 | 0.9937 | 0.6768 | 0.8885 | 0.8773 | 45 | | 0.0223 | 0.9912 | 0.9925 | 0.6858 | 0.8922 | 0.8848 | 46 | | 0.0254 | 0.9937 | 0.9925 | 0.6755 | 0.8922 | 0.8885 | 47 | | 0.0208 | 0.9962 | 0.9900 | 0.6838 | 0.8922 | 0.8848 | 48 | | 0.0170 | 0.9975 | 0.9950 | 0.6848 | 0.8922 | 0.8848 | 49 | ### Framework versions - Transformers 4.40.2 - TensorFlow 2.15.0 - Datasets 2.19.1 - Tokenizers 0.19.1