Initial Commit

README.md

@@ -0,0 +1,78 @@
+---
+license: mit
+base_model: haryoaw/scenario-MDBT-TCR_data-cl-cardiff_cl_only
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+model-index:
+- name: scenario-KD-PO-CDF-EN-FROM-CL-D2_data-en-cardiff_eng_only55
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# scenario-KD-PO-CDF-EN-FROM-CL-D2_data-en-cardiff_eng_only55
+
+This model is a fine-tuned version of [haryoaw/scenario-MDBT-TCR_data-cl-cardiff_cl_only](https://huggingface.co/haryoaw/scenario-MDBT-TCR_data-cl-cardiff_cl_only) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 39.4604
+- Accuracy: 0.4537
+- F1: 0.4426
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 55
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 30
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
+| No log        | 1.72  | 100  | 24.2103         | 0.4616   | 0.4518 |
+| No log        | 3.45  | 200  | 29.3271         | 0.4255   | 0.3714 |
+| No log        | 5.17  | 300  | 29.8979         | 0.4489   | 0.4414 |
+| No log        | 6.9   | 400  | 31.7211         | 0.4669   | 0.4627 |
+| 15.3839       | 8.62  | 500  | 37.0421         | 0.4581   | 0.4492 |
+| 15.3839       | 10.34 | 600  | 32.5884         | 0.4669   | 0.4658 |
+| 15.3839       | 12.07 | 700  | 39.9517         | 0.4493   | 0.4332 |
+| 15.3839       | 13.79 | 800  | 38.5249         | 0.4630   | 0.4470 |
+| 15.3839       | 15.52 | 900  | 38.7918         | 0.4414   | 0.4286 |
+| 2.5437        | 17.24 | 1000 | 40.0345         | 0.4524   | 0.4391 |
+| 2.5437        | 18.97 | 1100 | 38.3918         | 0.4612   | 0.4527 |
+| 2.5437        | 20.69 | 1200 | 41.3974         | 0.4396   | 0.4169 |
+| 2.5437        | 22.41 | 1300 | 38.7372         | 0.4603   | 0.4532 |
+| 2.5437        | 24.14 | 1400 | 40.1541         | 0.4405   | 0.4288 |
+| 1.0429        | 25.86 | 1500 | 40.0459         | 0.4568   | 0.4383 |
+| 1.0429        | 27.59 | 1600 | 39.3779         | 0.4590   | 0.4457 |
+| 1.0429        | 29.31 | 1700 | 39.4604         | 0.4537   | 0.4426 |
+
+
+### Framework versions
+
+- Transformers 4.33.3
+- Pytorch 2.1.1+cu121
+- Datasets 2.14.5
+- Tokenizers 0.13.3

config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "haryoaw/scenario-MDBT-TCR_data-cl-cardiff_cl_only",
+  "architectures": [
+    "DebertaForSequenceClassificationKD"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 768,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.33.3",
+  "type_vocab_size": 0,
+  "vocab_size": 251000
+}

eval_results_cardiff.json

@@ -0,0 +1 @@
+{"arabic": {"f1": 0.47748666077704865, "accuracy": 0.4827586206896552, "confusion_matrix": [[189, 75, 26], [125, 122, 43], [109, 72, 109]]}, "english": {"f1": 0.6062700636332565, "accuracy": 0.6114942528735632, "confusion_matrix": [[227, 47, 16], [103, 134, 53], [55, 64, 171]]}, "french": {"f1": 0.4297181829135792, "accuracy": 0.4574712643678161, "confusion_matrix": [[117, 157, 16], [39, 222, 29], [65, 166, 59]]}, "german": {"f1": 0.5596284626208937, "accuracy": 0.5609195402298851, "confusion_matrix": [[134, 100, 56], [51, 174, 65], [37, 73, 180]]}, "hindi": {"f1": 0.45352147267024295, "accuracy": 0.4574712643678161, "confusion_matrix": [[100, 116, 74], [58, 162, 70], [55, 99, 136]]}, "italian": {"f1": 0.5240459961978949, "accuracy": 0.532183908045977, "confusion_matrix": [[157, 102, 31], [35, 204, 51], [64, 124, 102]]}, "portuguese": {"f1": 0.43449828678836616, "accuracy": 0.43908045977011495, "confusion_matrix": [[97, 148, 45], [70, 175, 45], [44, 136, 110]]}, "spanish": {"f1": 0.4792917115643119, "accuracy": 0.47701149425287354, "confusion_matrix": [[142, 127, 21], [85, 158, 47], [53, 122, 115]]}, "all": {"f1": 0.5009864458749811, "accuracy": 0.5018678160919541, "confusion_matrix": [[1142, 885, 293], [546, 1383, 391], [478, 874, 968]]}}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4ceaf494f40e98e2f87fbbce0e3b027919c795dc93246c326d3d37b683a055b
+size 946740394

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f23fc3d5a009d0b7c9824b432e22adc1cc84ad78b2b336c203fc29793a44f7f3
+size 4600