aishanur/HV_Roberta_Large_2

README.md

@@ -0,0 +1,124 @@
+---
+license: mit
+base_model: FacebookAI/roberta-large
+tags:
+- generated_from_trainer
+model-index:
+- name: roberta_large_hv_3
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# roberta_large_hv_3
+
+This model is a fine-tuned version of [FacebookAI/roberta-large](https://huggingface.co/FacebookAI/roberta-large) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0599
+- F1-macro subtask 1: 0.2635
+- F1-micro subtask 1: 0.3292
+- Roc auc macro subtask 1: 0.6016
+- F1-macro subtask 2: 0.0502
+- F1-micro subtask 2: 0.0513
+- Roc auc macro subtask 2: 0.6128
+- Self-direction: thought1: 0.0588
+- Self-direction: action1: 0.1801
+- Stimulation1: 0.1815
+- Hedonism1: 0.2645
+- Achievement1: 0.2612
+- Power: dominance1: 0.3455
+- Power: resources1: 0.2642
+- Face1: 0.1941
+- Security: personal1: 0.2454
+- Security: societal1: 0.4119
+- Tradition1: 0.4361
+- Conformity: rules1: 0.4578
+- Conformity: interpersonal1: 0.0700
+- Humility1: 0.0833
+- Benevolence: caring1: 0.1117
+- Benevolence: dependability1: 0.2426
+- Universalism: concern1: 0.3619
+- Universalism: nature1: 0.5693
+- Universalism: tolerance1: 0.2667
+- Self-direction: thought attained2: 0.0190
+- Self-direction: thought constrained2: 0.0561
+- Self-direction: action attained2: 0.0525
+- Self-direction: action constrained2: 0.0762
+- Stimulation attained2: 0.0631
+- Stimulation constrained2: 0.0095
+- Hedonism attained2: 0.0143
+- Hedonism constrained2: 0.0071
+- Achievement attained2: 0.1248
+- Achievement constrained2: 0.0840
+- Power: dominance attained2: 0.0701
+- Power: dominance constrained2: 0.0767
+- Power: resources attained2: 0.0806
+- Power: resources constrained2: 0.0807
+- Face attained2: 0.0239
+- Face constrained2: 0.0361
+- Security: personal attained2: 0.0187
+- Security: personal constrained2: 0.0425
+- Security: societal attained2: 0.1038
+- Security: societal constrained2: 0.1802
+- Tradition attained2: 0.0312
+- Tradition constrained2: 0.0338
+- Conformity: rules attained2: 0.1044
+- Conformity: rules constrained2: 0.0914
+- Conformity: interpersonal attained2: 0.0162
+- Conformity: interpersonal constrained2: 0.0294
+- Humility attained2: 0.0053
+- Humility constrained2: 0.0024
+- Benevolence: caring attained2: 0.0438
+- Benevolence: caring constrained2: 0.0137
+- Benevolence: dependability attained2: 0.0362
+- Benevolence: dependability constrained2: 0.016
+- Universalism: concern attained2: 0.0852
+- Universalism: concern constrained2: 0.0583
+- Universalism: nature attained2: 0.0423
+- Universalism: nature constrained2: 0.0516
+- Universalism: tolerance attained2: 0.0142
+- Universalism: tolerance constrained2: 0.0133
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.2
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | F1-macro subtask 1 | F1-micro subtask 1 | Roc auc macro subtask 1 | F1-macro subtask 2 | F1-micro subtask 2 | Roc auc macro subtask 2 | Self-direction: thought1 | Self-direction: action1 | Stimulation1 | Hedonism1 | Achievement1 | Power: dominance1 | Power: resources1 | Face1  | Security: personal1 | Security: societal1 | Tradition1 | Conformity: rules1 | Conformity: interpersonal1 | Humility1 | Benevolence: caring1 | Benevolence: dependability1 | Universalism: concern1 | Universalism: nature1 | Universalism: tolerance1 | Self-direction: thought attained2 | Self-direction: thought constrained2 | Self-direction: action attained2 | Self-direction: action constrained2 | Stimulation attained2 | Stimulation constrained2 | Hedonism attained2 | Hedonism constrained2 | Achievement attained2 | Achievement constrained2 | Power: dominance attained2 | Power: dominance constrained2 | Power: resources attained2 | Power: resources constrained2 | Face attained2 | Face constrained2 | Security: personal attained2 | Security: personal constrained2 | Security: societal attained2 | Security: societal constrained2 | Tradition attained2 | Tradition constrained2 | Conformity: rules attained2 | Conformity: rules constrained2 | Conformity: interpersonal attained2 | Conformity: interpersonal constrained2 | Humility attained2 | Humility constrained2 | Benevolence: caring attained2 | Benevolence: caring constrained2 | Benevolence: dependability attained2 | Benevolence: dependability constrained2 | Universalism: concern attained2 | Universalism: concern constrained2 | Universalism: nature attained2 | Universalism: nature constrained2 | Universalism: tolerance attained2 | Universalism: tolerance constrained2 |
+|:-------------:|:-----:|:-----:|:---------------:|:------------------:|:------------------:|:-----------------------:|:------------------:|:------------------:|:-----------------------:|:------------------------:|:-----------------------:|:------------:|:---------:|:------------:|:-----------------:|:-----------------:|:------:|:-------------------:|:-------------------:|:----------:|:------------------:|:--------------------------:|:---------:|:--------------------:|:---------------------------:|:----------------------:|:---------------------:|:------------------------:|:---------------------------------:|:------------------------------------:|:--------------------------------:|:-----------------------------------:|:---------------------:|:------------------------:|:------------------:|:---------------------:|:---------------------:|:------------------------:|:--------------------------:|:-----------------------------:|:--------------------------:|:-----------------------------:|:--------------:|:-----------------:|:----------------------------:|:-------------------------------:|:----------------------------:|:-------------------------------:|:-------------------:|:----------------------:|:---------------------------:|:------------------------------:|:-----------------------------------:|:--------------------------------------:|:------------------:|:---------------------:|:-----------------------------:|:--------------------------------:|:------------------------------------:|:---------------------------------------:|:-------------------------------:|:----------------------------------:|:------------------------------:|:---------------------------------:|:---------------------------------:|:------------------------------------:|
+| 0.0603        | 1.0   | 7183  | 0.0604          | 0.1393             | 0.2079             | 0.5512                  | 0.0485             | 0.0509             | 0.6252                  | 0.0                      | 0.0431                  | 0.0          | 0.0       | 0.2512       | 0.1886            | 0.0844            | 0.2393 | 0.0                 | 0.2646              | 0.2359     | 0.3725             | 0.1083                     | 0.0       | 0.0277               | 0.0385                      | 0.0819                 | 0.5374                | 0.1734                   | 0.0188                            | 0.0137                               | 0.0550                           | 0.0418                              | 0.0530                | 0.0141                   | 0.0119             | 0.0110                | 0.1262                | 0.0807                   | 0.0659                     | 0.0600                        | 0.0806                     | 0.0833                        | 0.0201         | 0.0576            | 0.0161                       | 0.0506                          | 0.1231                       | 0.1471                          | 0.0345              | 0.0188                 | 0.1007                      | 0.0982                         | 0.0157                              | 0.0351                                 | 0.0069             | 0.0018                | 0.0476                        | 0.0151                           | 0.0480                               | 0.0142                                  | 0.0836                          | 0.0540                             | 0.0380                         | 0.0727                            | 0.0120                            | 0.0166                               |
+| 0.0411        | 2.0   | 14366 | 0.0607          | 0.2394             | 0.3018             | 0.5948                  | 0.0491             | 0.0516             | 0.6270                  | 0.0664                   | 0.1223                  | 0.3371       | 0.3165    | 0.3128       | 0.1460            | 0.1394            | 0.2103 | 0.1218              | 0.4200              | 0.4190     | 0.4399             | 0.0972                     | 0.1013    | 0.0874               | 0.1508                      | 0.3057                 | 0.5369                | 0.2182                   | 0.0179                            | 0.0375                               | 0.0526                           | 0.0773                              | 0.0589                | 0.0104                   | 0.0170             | 0.0059                | 0.1271                | 0.0824                   | 0.0694                     | 0.0645                        | 0.0841                     | 0.0744                        | 0.0250         | 0.0356            | 0.0180                       | 0.0406                          | 0.1057                       | 0.1696                          | 0.0329              | 0.0286                 | 0.1114                      | 0.0839                         | 0.0166                              | 0.0295                                 | 0.0054             | 0.0027                | 0.0439                        | 0.0208                           | 0.0375                               | 0.0203                                  | 0.0896                          | 0.0542                             | 0.0477                         | 0.0405                            | 0.0119                            | 0.0158                               |
+| 0.0282        | 3.0   | 21549 | 0.0599          | 0.2635             | 0.3292             | 0.6016                  | 0.0502             | 0.0513             | 0.6128                  | 0.0588                   | 0.1801                  | 0.1815       | 0.2645    | 0.2612       | 0.3455            | 0.2642            | 0.1941 | 0.2454              | 0.4119              | 0.4361     | 0.4578             | 0.0700                     | 0.0833    | 0.1117               | 0.2426                      | 0.3619                 | 0.5693                | 0.2667                   | 0.0190                            | 0.0561                               | 0.0525                           | 0.0762                              | 0.0631                | 0.0095                   | 0.0143             | 0.0071                | 0.1248                | 0.0840                   | 0.0701                     | 0.0767                        | 0.0806                     | 0.0807                        | 0.0239         | 0.0361            | 0.0187                       | 0.0425                          | 0.1038                       | 0.1802                          | 0.0312              | 0.0338                 | 0.1044                      | 0.0914                         | 0.0162                              | 0.0294                                 | 0.0053             | 0.0024                | 0.0438                        | 0.0137                           | 0.0362                               | 0.016                                   | 0.0852                          | 0.0583                             | 0.0423                         | 0.0516                            | 0.0142                            | 0.0133                               |
+| 0.0272        | 4.0   | 28732 | 0.0630          | 0.2883             | 0.3504             | 0.6141                  | 0.0492             | 0.0515             | 0.6185                  | 0.0935                   | 0.2240                  | 0.2606       | 0.2857    | 0.3633       | 0.3567            | 0.2972            | 0.2133 | 0.3140              | 0.4242              | 0.4379     | 0.4547             | 0.0662                     | 0.0385    | 0.2308               | 0.2571                      | 0.3798                 | 0.5229                | 0.2577                   | 0.0190                            | 0.0619                               | 0.0525                           | 0.0757                              | 0.0651                | 0.0137                   | 0.0158             | 0.0073                | 0.1216                | 0.0894                   | 0.0744                     | 0.035                         | 0.0804                     | 0.0828                        | 0.0240         | 0.0343            | 0.0205                       | 0.0392                          | 0.1079                       | 0.1682                          | 0.0329              | 0.0220                 | 0.1066                      | 0.0931                         | 0.0159                              | 0.0265                                 | 0.0049             | 0.0023                | 0.0449                        | 0.0135                           | 0.0355                               | 0.0200                                  | 0.0830                          | 0.0601                             | 0.0465                         | 0.0437                            | 0.0126                            | 0.0152                               |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.3.0
+- Datasets 2.19.0
+- Tokenizers 0.15.1

config.json

@@ -0,0 +1,108 @@
+{
+  "_name_or_path": "FacebookAI/roberta-large",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "Self-direction: thought attained",
+    "1": "Self-direction: thought constrained",
+    "2": "Self-direction: action attained",
+    "3": "Self-direction: action constrained",
+    "4": "Stimulation attained",
+    "5": "Stimulation constrained",
+    "6": "Hedonism attained",
+    "7": "Hedonism constrained",
+    "8": "Achievement attained",
+    "9": "Achievement constrained",
+    "10": "Power: dominance attained",
+    "11": "Power: dominance constrained",
+    "12": "Power: resources attained",
+    "13": "Power: resources constrained",
+    "14": "Face attained",
+    "15": "Face constrained",
+    "16": "Security: personal attained",
+    "17": "Security: personal constrained",
+    "18": "Security: societal attained",
+    "19": "Security: societal constrained",
+    "20": "Tradition attained",
+    "21": "Tradition constrained",
+    "22": "Conformity: rules attained",
+    "23": "Conformity: rules constrained",
+    "24": "Conformity: interpersonal attained",
+    "25": "Conformity: interpersonal constrained",
+    "26": "Humility attained",
+    "27": "Humility constrained",
+    "28": "Benevolence: caring attained",
+    "29": "Benevolence: caring constrained",
+    "30": "Benevolence: dependability attained",
+    "31": "Benevolence: dependability constrained",
+    "32": "Universalism: concern attained",
+    "33": "Universalism: concern constrained",
+    "34": "Universalism: nature attained",
+    "35": "Universalism: nature constrained",
+    "36": "Universalism: tolerance attained",
+    "37": "Universalism: tolerance constrained"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "Achievement attained": 8,
+    "Achievement constrained": 9,
+    "Benevolence: caring attained": 28,
+    "Benevolence: caring constrained": 29,
+    "Benevolence: dependability attained": 30,
+    "Benevolence: dependability constrained": 31,
+    "Conformity: interpersonal attained": 24,
+    "Conformity: interpersonal constrained": 25,
+    "Conformity: rules attained": 22,
+    "Conformity: rules constrained": 23,
+    "Face attained": 14,
+    "Face constrained": 15,
+    "Hedonism attained": 6,
+    "Hedonism constrained": 7,
+    "Humility attained": 26,
+    "Humility constrained": 27,
+    "Power: dominance attained": 10,
+    "Power: dominance constrained": 11,
+    "Power: resources attained": 12,
+    "Power: resources constrained": 13,
+    "Security: personal attained": 16,
+    "Security: personal constrained": 17,
+    "Security: societal attained": 18,
+    "Security: societal constrained": 19,
+    "Self-direction: action attained": 2,
+    "Self-direction: action constrained": 3,
+    "Self-direction: thought attained": 0,
+    "Self-direction: thought constrained": 1,
+    "Stimulation attained": 4,
+    "Stimulation constrained": 5,
+    "Tradition attained": 20,
+    "Tradition constrained": 21,
+    "Universalism: concern attained": 32,
+    "Universalism: concern constrained": 33,
+    "Universalism: nature attained": 34,
+    "Universalism: nature constrained": 35,
+    "Universalism: tolerance attained": 36,
+    "Universalism: tolerance constrained": 37
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "multi_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.37.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d6043bbc95fa388e400cfa930e333fa8436bdc911978acdbc27b122cf99014b
+size 1421643016

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,57 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5a366126646727a930dbe015ab53e6516163404cdd90b05937c25c724bfa3ff
+size 4728