Saving best model of baseline_BERT_10K_steps to hub

README.md

@@ -0,0 +1,111 @@
+---
+license: apache-2.0
+base_model: bert-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- arxiv_dataset
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+model-index:
+- name: baseline_BERT_10K_steps
+  results:
+  - task:
+      name: Text Classification
+      type: text-classification
+    dataset:
+      name: arxiv_dataset
+      type: arxiv_dataset
+      config: default
+      split: train
+      args: default
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9905994544286106
+    - name: Precision
+      type: precision
+      value: 0.7827298050139275
+    - name: Recall
+      type: recall
+      value: 0.05172572480441785
+    - name: F1
+      type: f1
+      value: 0.09703876370543037
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# baseline_BERT_10K_steps
+
+This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the arxiv_dataset dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0356
+- Accuracy: 0.9906
+- Precision: 0.7827
+- Recall: 0.0517
+- F1: 0.0970
+- Hamming: 0.0094
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 6
+- eval_batch_size: 6
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+- training_steps: 10000
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision | Recall | F1     | Hamming |
+|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:------:|:-------:|
+| No log        | 0.0   | 500   | 0.1602          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.0   | 1000  | 0.0573          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.0   | 1500  | 0.0504          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 2000  | 0.0492          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 2500  | 0.0488          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 3000  | 0.0485          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 3500  | 0.0477          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 4000  | 0.0467          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 4500  | 0.0455          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.01  | 5000  | 0.0442          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.02  | 5500  | 0.0422          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.02  | 6000  | 0.0408          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.02  | 6500  | 0.0394          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
+| No log        | 0.02  | 7000  | 0.0385          | 0.9902   | 1.0       | 0.0011 | 0.0022 | 0.0098  |
+| No log        | 0.02  | 7500  | 0.0376          | 0.9903   | 0.7949    | 0.0057 | 0.0113 | 0.0097  |
+| No log        | 0.02  | 8000  | 0.0368          | 0.9903   | 0.8071    | 0.0146 | 0.0287 | 0.0097  |
+| No log        | 0.03  | 8500  | 0.0363          | 0.9905   | 0.7372    | 0.0465 | 0.0874 | 0.0095  |
+| No log        | 0.03  | 9000  | 0.0359          | 0.9905   | 0.7811    | 0.0381 | 0.0727 | 0.0095  |
+| No log        | 0.03  | 9500  | 0.0357          | 0.9906   | 0.8029    | 0.0562 | 0.1051 | 0.0094  |
+| 0.0665        | 0.03  | 10000 | 0.0356          | 0.9906   | 0.7827    | 0.0517 | 0.0970 | 0.0094  |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 1.12.1+cu113
+- Datasets 2.16.1
+- Tokenizers 0.15.1

all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 0.03,
+    "train_loss": 0.066533935546875,
+    "train_runtime": 6679.0776,
+    "train_samples_per_second": 8.983,
+    "train_steps_per_second": 1.497
+}

config.json

@@ -0,0 +1,361 @@
+{
+  "_name_or_path": "bert-base-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "adap-org",
+    "1": "alg-geom",
+    "2": "astro-ph",
+    "3": "astro-ph.CO",
+    "4": "astro-ph.EP",
+    "5": "astro-ph.GA",
+    "6": "astro-ph.HE",
+    "7": "astro-ph.IM",
+    "8": "astro-ph.SR",
+    "9": "atom-ph",
+    "10": "chao-dyn",
+    "11": "chem-ph",
+    "12": "cmp-lg",
+    "13": "comp-gas",
+    "14": "cond-mat",
+    "15": "cond-mat.dis-nn",
+    "16": "cond-mat.mes-hall",
+    "17": "cond-mat.mtrl-sci",
+    "18": "cond-mat.other",
+    "19": "cond-mat.quant-gas",
+    "20": "cond-mat.soft",
+    "21": "cond-mat.stat-mech",
+    "22": "cond-mat.str-el",
+    "23": "cond-mat.supr-con",
+    "24": "cs.AI",
+    "25": "cs.AR",
+    "26": "cs.CC",
+    "27": "cs.CE",
+    "28": "cs.CG",
+    "29": "cs.CL",
+    "30": "cs.CR",
+    "31": "cs.CV",
+    "32": "cs.CY",
+    "33": "cs.DB",
+    "34": "cs.DC",
+    "35": "cs.DL",
+    "36": "cs.DM",
+    "37": "cs.DS",
+    "38": "cs.ET",
+    "39": "cs.FL",
+    "40": "cs.GL",
+    "41": "cs.GR",
+    "42": "cs.GT",
+    "43": "cs.HC",
+    "44": "cs.IR",
+    "45": "cs.LG",
+    "46": "cs.LO",
+    "47": "cs.MA",
+    "48": "cs.MM",
+    "49": "cs.MS",
+    "50": "cs.NA",
+    "51": "cs.NE",
+    "52": "cs.NI",
+    "53": "cs.OH",
+    "54": "cs.OS",
+    "55": "cs.PF",
+    "56": "cs.PL",
+    "57": "cs.RO",
+    "58": "cs.SC",
+    "59": "cs.SD",
+    "60": "cs.SE",
+    "61": "cs.SI",
+    "62": "cs.SY",
+    "63": "dg-ga",
+    "64": "econ.EM",
+    "65": "econ.GN",
+    "66": "econ.TH",
+    "67": "eess.AS",
+    "68": "eess.IV",
+    "69": "eess.SP",
+    "70": "eess.SY",
+    "71": "funct-an",
+    "72": "gr-qc",
+    "73": "hep-ex",
+    "74": "hep-lat",
+    "75": "hep-ph",
+    "76": "hep-th",
+    "77": "math.AC",
+    "78": "math.AG",
+    "79": "math.AP",
+    "80": "math.AT",
+    "81": "math.CA",
+    "82": "math.CO",
+    "83": "math.CT",
+    "84": "math.CV",
+    "85": "math.DG",
+    "86": "math.DS",
+    "87": "math.FA",
+    "88": "math.GM",
+    "89": "math.GN",
+    "90": "math.GR",
+    "91": "math.GT",
+    "92": "math.HO",
+    "93": "math.KT",
+    "94": "math.LO",
+    "95": "math.MG",
+    "96": "math.NA",
+    "97": "math.NT",
+    "98": "math.OA",
+    "99": "math.OC",
+    "100": "math.PR",
+    "101": "math.QA",
+    "102": "math.RA",
+    "103": "math.RT",
+    "104": "math.SG",
+    "105": "math.SP",
+    "106": "mtrl-th",
+    "107": "nlin.AO",
+    "108": "nlin.CD",
+    "109": "nlin.CG",
+    "110": "nlin.PS",
+    "111": "nlin.SI",
+    "112": "nucl-ex",
+    "113": "nucl-th",
+    "114": "patt-sol",
+    "115": "physics.acc-ph",
+    "116": "physics.ao-ph",
+    "117": "physics.app-ph",
+    "118": "physics.atm-clus",
+    "119": "physics.atom-ph",
+    "120": "physics.bio-ph",
+    "121": "physics.chem-ph",
+    "122": "physics.class-ph",
+    "123": "physics.comp-ph",
+    "124": "physics.data-an",
+    "125": "physics.ed-ph",
+    "126": "physics.flu-dyn",
+    "127": "physics.gen-ph",
+    "128": "physics.geo-ph",
+    "129": "physics.hist-ph",
+    "130": "physics.ins-det",
+    "131": "physics.med-ph",
+    "132": "physics.optics",
+    "133": "physics.plasm-ph",
+    "134": "physics.pop-ph",
+    "135": "physics.soc-ph",
+    "136": "physics.space-ph",
+    "137": "q-alg",
+    "138": "q-bio.BM",
+    "139": "q-bio.CB",
+    "140": "q-bio.GN",
+    "141": "q-bio.MN",
+    "142": "q-bio.NC",
+    "143": "q-bio.OT",
+    "144": "q-bio.PE",
+    "145": "q-bio.QM",
+    "146": "q-bio.SC",
+    "147": "q-bio.TO",
+    "148": "q-fin.CP",
+    "149": "q-fin.EC",
+    "150": "q-fin.GN",
+    "151": "q-fin.MF",
+    "152": "q-fin.PM",
+    "153": "q-fin.PR",
+    "154": "q-fin.RM",
+    "155": "q-fin.ST",
+    "156": "q-fin.TR",
+    "157": "quant-ph",
+    "158": "solv-int",
+    "159": "stat.AP",
+    "160": "stat.CO",
+    "161": "stat.ME",
+    "162": "stat.ML",
+    "163": "stat.OT",
+    "164": "supr-con"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "adap-org": 0,
+    "alg-geom": 1,
+    "astro-ph": 2,
+    "astro-ph.CO": 3,
+    "astro-ph.EP": 4,
+    "astro-ph.GA": 5,
+    "astro-ph.HE": 6,
+    "astro-ph.IM": 7,
+    "astro-ph.SR": 8,
+    "atom-ph": 9,
+    "chao-dyn": 10,
+    "chem-ph": 11,
+    "cmp-lg": 12,
+    "comp-gas": 13,
+    "cond-mat": 14,
+    "cond-mat.dis-nn": 15,
+    "cond-mat.mes-hall": 16,
+    "cond-mat.mtrl-sci": 17,
+    "cond-mat.other": 18,
+    "cond-mat.quant-gas": 19,
+    "cond-mat.soft": 20,
+    "cond-mat.stat-mech": 21,
+    "cond-mat.str-el": 22,
+    "cond-mat.supr-con": 23,
+    "cs.AI": 24,
+    "cs.AR": 25,
+    "cs.CC": 26,
+    "cs.CE": 27,
+    "cs.CG": 28,
+    "cs.CL": 29,
+    "cs.CR": 30,
+    "cs.CV": 31,
+    "cs.CY": 32,
+    "cs.DB": 33,
+    "cs.DC": 34,
+    "cs.DL": 35,
+    "cs.DM": 36,
+    "cs.DS": 37,
+    "cs.ET": 38,
+    "cs.FL": 39,
+    "cs.GL": 40,
+    "cs.GR": 41,
+    "cs.GT": 42,
+    "cs.HC": 43,
+    "cs.IR": 44,
+    "cs.LG": 45,
+    "cs.LO": 46,
+    "cs.MA": 47,
+    "cs.MM": 48,
+    "cs.MS": 49,
+    "cs.NA": 50,
+    "cs.NE": 51,
+    "cs.NI": 52,
+    "cs.OH": 53,
+    "cs.OS": 54,
+    "cs.PF": 55,
+    "cs.PL": 56,
+    "cs.RO": 57,
+    "cs.SC": 58,
+    "cs.SD": 59,
+    "cs.SE": 60,
+    "cs.SI": 61,
+    "cs.SY": 62,
+    "dg-ga": 63,
+    "econ.EM": 64,
+    "econ.GN": 65,
+    "econ.TH": 66,
+    "eess.AS": 67,
+    "eess.IV": 68,
+    "eess.SP": 69,
+    "eess.SY": 70,
+    "funct-an": 71,
+    "gr-qc": 72,
+    "hep-ex": 73,
+    "hep-lat": 74,
+    "hep-ph": 75,
+    "hep-th": 76,
+    "math.AC": 77,
+    "math.AG": 78,
+    "math.AP": 79,
+    "math.AT": 80,
+    "math.CA": 81,
+    "math.CO": 82,
+    "math.CT": 83,
+    "math.CV": 84,
+    "math.DG": 85,
+    "math.DS": 86,
+    "math.FA": 87,
+    "math.GM": 88,
+    "math.GN": 89,
+    "math.GR": 90,
+    "math.GT": 91,
+    "math.HO": 92,
+    "math.KT": 93,
+    "math.LO": 94,
+    "math.MG": 95,
+    "math.NA": 96,
+    "math.NT": 97,
+    "math.OA": 98,
+    "math.OC": 99,
+    "math.PR": 100,
+    "math.QA": 101,
+    "math.RA": 102,
+    "math.RT": 103,
+    "math.SG": 104,
+    "math.SP": 105,
+    "mtrl-th": 106,
+    "nlin.AO": 107,
+    "nlin.CD": 108,
+    "nlin.CG": 109,
+    "nlin.PS": 110,
+    "nlin.SI": 111,
+    "nucl-ex": 112,
+    "nucl-th": 113,
+    "patt-sol": 114,
+    "physics.acc-ph": 115,
+    "physics.ao-ph": 116,
+    "physics.app-ph": 117,
+    "physics.atm-clus": 118,
+    "physics.atom-ph": 119,
+    "physics.bio-ph": 120,
+    "physics.chem-ph": 121,
+    "physics.class-ph": 122,
+    "physics.comp-ph": 123,
+    "physics.data-an": 124,
+    "physics.ed-ph": 125,
+    "physics.flu-dyn": 126,
+    "physics.gen-ph": 127,
+    "physics.geo-ph": 128,
+    "physics.hist-ph": 129,
+    "physics.ins-det": 130,
+    "physics.med-ph": 131,
+    "physics.optics": 132,
+    "physics.plasm-ph": 133,
+    "physics.pop-ph": 134,
+    "physics.soc-ph": 135,
+    "physics.space-ph": 136,
+    "q-alg": 137,
+    "q-bio.BM": 138,
+    "q-bio.CB": 139,
+    "q-bio.GN": 140,
+    "q-bio.MN": 141,
+    "q-bio.NC": 142,
+    "q-bio.OT": 143,
+    "q-bio.PE": 144,
+    "q-bio.QM": 145,
+    "q-bio.SC": 146,
+    "q-bio.TO": 147,
+    "q-fin.CP": 148,
+    "q-fin.EC": 149,
+    "q-fin.GN": 150,
+    "q-fin.MF": 151,
+    "q-fin.PM": 152,
+    "q-fin.PR": 153,
+    "q-fin.RM": 154,
+    "q-fin.ST": 155,
+    "q-fin.TR": 156,
+    "quant-ph": 157,
+    "solv-int": 158,
+    "stat.AP": 159,
+    "stat.CO": 160,
+    "stat.ME": 161,
+    "stat.ML": 162,
+    "stat.OT": 163,
+    "supr-con": 164
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "multi_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.37.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6937e53c080b6d84783f58bb137239df2be17cbed6c78e71b339d6e61c3415a5
+size 438460036

runs/Feb04_21-00-25_jordy-OMEN/events.out.tfevents.1707076830.jordy-OMEN.136277.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4a518ec0354847f8971e4cc9a02d0a500b685b3f2b5b4f37a773ae1c720f843
+size 11595

runs/Feb04_21-08-02_jordy-OMEN/events.out.tfevents.1707077284.jordy-OMEN.138698.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e7c424a4a356ec4855f67f2a4cf4c5cbe93d30cedf0541add9e6d6bfaad7ea3
+size 11593

runs/Feb04_21-14-31_jordy-OMEN/events.out.tfevents.1707077673.jordy-OMEN.140624.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e0e25943c995eb353e510de6df2013d5063ecc6e7a31d13991ba88f1a46d968
+size 22564

runs/Feb04_21-14-31_jordy-OMEN/events.out.tfevents.1707084580.jordy-OMEN.140624.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be1385c1fc6647f69a2a4aecd1331b361e9bd1dc59dfb2c721127c6a1f8399ed
+size 563

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

train_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 0.03,
+    "train_loss": 0.066533935546875,
+    "train_runtime": 6679.0776,
+    "train_samples_per_second": 8.983,
+    "train_steps_per_second": 1.497
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d9a73efccf9afb73ed537e06f72678538b1b78e0c80fbdcdf5cbba862c89ba9
+size 4271