Training complete

README.md

@@ -1,6 +1,7 @@
 ---
 library_name: transformers
-base_model: allenai/biomed_roberta_base
+license: mit
+base_model: emilyalsentzer/Bio_ClinicalBERT
 tags:
 - generated_from_trainer
 metrics:
@@ -18,13 +19,13 @@ should probably proofread and complete it, then remove this comment. -->
 
 # BioMedRoBERTa-finetuned-ner-pablo-just-classifier
 
-This model is a fine-tuned version of [allenai/biomed_roberta_base](https://huggingface.co/allenai/biomed_roberta_base) on the None dataset.
+This model is a fine-tuned version of [emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1276
-- Precision: 0.6818
-- Recall: 0.7031
-- F1: 0.6923
-- Accuracy: 0.9672
+- Loss: 0.1150
+- Precision: 0.6869
+- Recall: 0.7076
+- F1: 0.6971
+- Accuracy: 0.9677
 
 ## Model description
 
@@ -44,24 +45,31 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.1
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 128
+- eval_batch_size: 128
 - seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
+- lr_scheduler_type: cosine_with_restarts
 - lr_scheduler_warmup_ratio: 0.05
-- num_epochs: 5
+- num_epochs: 10
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
-|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
-| 0.7191        | 1.0   | 2509  | 0.5248          | 0.5127    | 0.6334 | 0.5667 | 0.9486   |
-| 0.5382        | 2.0   | 5018  | 0.4280          | 0.5378    | 0.6500 | 0.5886 | 0.9556   |
-| 0.3968        | 3.0   | 7527  | 0.3095          | 0.4997    | 0.6714 | 0.5730 | 0.9531   |
-| 0.2528        | 4.0   | 10036 | 0.1872          | 0.5631    | 0.6850 | 0.6181 | 0.9599   |
-| 0.1541        | 5.0   | 12545 | 0.1276          | 0.6818    | 0.7031 | 0.6923 | 0.9672   |
+| Training Loss | Epoch  | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
+| No log        | 0.9655 | 14   | 0.3729          | 0.4205    | 0.6119 | 0.4985 | 0.9430   |
+| No log        | 2.0    | 29   | 0.2544          | 0.5272    | 0.6683 | 0.5894 | 0.9574   |
+| No log        | 2.9655 | 43   | 0.2117          | 0.5702    | 0.6884 | 0.6238 | 0.9604   |
+| No log        | 4.0    | 58   | 0.1747          | 0.5934    | 0.7001 | 0.6424 | 0.9628   |
+| No log        | 4.9655 | 72   | 0.1420          | 0.6280    | 0.6827 | 0.6542 | 0.9642   |
+| No log        | 6.0    | 87   | 0.1287          | 0.6639    | 0.7033 | 0.6830 | 0.9667   |
+| No log        | 6.9655 | 101  | 0.1309          | 0.6471    | 0.7009 | 0.6729 | 0.9654   |
+| No log        | 8.0    | 116  | 0.1260          | 0.6349    | 0.7199 | 0.6748 | 0.9652   |
+| No log        | 8.9655 | 130  | 0.1159          | 0.6621    | 0.7118 | 0.6860 | 0.9670   |
+| No log        | 9.6552 | 140  | 0.1150          | 0.6869    | 0.7076 | 0.6971 | 0.9677   |
 
 
 ### Framework versions