Training completed!

README.md

@@ -1,4 +1,6 @@
 ---
+license: mit
+base_model: microsoft/deberta-v3-base
 tags:
 - generated_from_trainer
 metrics:
@@ -14,12 +16,12 @@ should probably proofread and complete it, then remove this comment. -->
 
 # deberta-pii-finetuned
 
-This model was trained from scratch on the None dataset.
+This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0046
-- F Beta: 0.7376
-- Precision: 0.9894
-- Recall: 0.7302
+- Loss: 0.0018
+- F Beta: 0.8127
+- Precision: 0.9818
+- Recall: 0.8071
 
 ## Model description
 
@@ -38,29 +40,29 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-06
+- learning_rate: 2e-05
 - train_batch_size: 8
 - eval_batch_size: 16
 - seed: 42
-- gradient_accumulation_steps: 3
-- total_train_batch_size: 24
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.01
-- num_epochs: 2
+- lr_scheduler_warmup_ratio: 0.05
+- num_epochs: 3
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | F Beta | Precision | Recall |
 |:-------------:|:-----:|:----:|:---------------:|:------:|:---------:|:------:|
-| 0.0001        | 0.27  | 70   | 0.0119          | 0.3242 | 0.9727    | 0.3158 |
-| 0.0019        | 0.54  | 140  | 0.0056          | 0.6786 | 0.9907    | 0.6701 |
-| 0.0002        | 0.82  | 210  | 0.0027          | 0.8106 | 0.9878    | 0.8048 |
-| 0.0009        | 1.09  | 280  | 0.0047          | 0.6636 | 0.9890    | 0.6550 |
-| 0.0005        | 1.36  | 350  | 0.0031          | 0.7765 | 0.9893    | 0.7698 |
-| 0.0011        | 1.63  | 420  | 0.0049          | 0.7335 | 0.9893    | 0.7260 |
-| 0.0006        | 1.91  | 490  | 0.0046          | 0.7376 | 0.9894    | 0.7302 |
+| 0.0074        | 0.41  | 250  | 0.0022          | 0.9594 | 0.9851    | 0.9584 |
+| 0.0031        | 0.82  | 500  | 0.0011          | 0.9541 | 0.9879    | 0.9528 |
+| 0.0035        | 1.24  | 750  | 0.0015          | 0.8814 | 0.9869    | 0.8776 |
+| 0.0029        | 1.65  | 1000 | 0.0024          | 0.7401 | 0.9849    | 0.7328 |
+| 0.0016        | 2.06  | 1250 | 0.0015          | 0.8240 | 0.9810    | 0.8188 |
+| 0.0012        | 2.47  | 1500 | 0.0020          | 0.7848 | 0.9812    | 0.7786 |
+| 0.003         | 2.88  | 1750 | 0.0018          | 0.8127 | 0.9818    | 0.8071 |
 
 
 ### Framework versions

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "/kaggle/input/pretrained-1-0-450/Pretrained 1.0 - 450",
+  "_name_or_path": "microsoft/deberta-v3-base",
   "architectures": [
     "DebertaForTokenClassification"
   ],

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f29560228f0eab8645beb39208663facf0bf7c24b6b9d950ce7a7f523250c0d0
+oid sha256:d0d468ba03c9545cccbf708480202d8fe642dae11dd071373dc70abb0b51fa2f
 size 737753016

special_tokens_map.json

@@ -1,46 +1,10 @@
 {
-  "bos_token": {
-    "content": "[CLS]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "cls_token": {
-    "content": "[CLS]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "eos_token": {
-    "content": "[SEP]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "mask_token": {
-    "content": "[MASK]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "pad_token": {
-    "content": "[PAD]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "sep_token": {
-    "content": "[SEP]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
   "unk_token": {
     "content": "[UNK]",
     "lstrip": false,

tokenizer_config.json

@@ -47,16 +47,12 @@
   "do_lower_case": false,
   "eos_token": "[SEP]",
   "mask_token": "[MASK]",
-  "max_length": 1024,
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "[PAD]",
   "sep_token": "[SEP]",
   "sp_model_kwargs": {},
   "split_by_punct": false,
-  "stride": 0,
   "tokenizer_class": "DebertaV2Tokenizer",
-  "truncation_side": "right",
-  "truncation_strategy": "longest_first",
   "unk_token": "[UNK]",
   "vocab_type": "spm"
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:546260b6375cebcadbeae4c0d49ee3d4d76f230c7e29a94a1d71e279c13adff1
+oid sha256:36b6d67087c1db7369a859b4cae2d3a1c67587f2f7d2a247fe6aedf03d491883
 size 4219