clone repository

README.md

@@ -1,3 +1,47 @@
 ---
-license: other
+language: 
+  - da
+tags:
+- summarization
+widget:
+- text: "Det nye studie Cognitive Science på Aarhus Universitet, som i år havde Østjyllands højeste adgangskrav på 11,7 i karaktergennemsnit, udklækker det første hold bachelorer til sommer.
+
+Men når de skal læse videre på kandidaten må de til udlandet, hvis ikke de vil skifte til et andet fag. Aarhus Universitet kan nemlig ikke nå at oprette en kandidat i Cognitive Science til næste sommer, hvor det første hold bachelorer er færdige.
+
+Det rammer blandt andre Julie Sohn, der startede på uddannelsen i sommeren 2015, og derfor kun mangler et år, før hun er bachelor.
+
+- Jeg synes, at det er ærgerligt, at vi som nye studerende på et populært studie ikke kan tage en kandidat i Danmark, siger hun.
+
+Bacheloruddannelsen i Cognitive Science blev oprettet af Aarhus Universitet i 2015, og uddannelsen kombinerer viden om menneskelig adfærd med avanceret statistik. Da der endnu ikke er oprettet en kandidatuddannelse indenfor dette område, har Julie Sohn i stedet mulighed for at læse en kandidatgrad i for eksempel informationsvidenskab.
+
+Hun vil dog hellere fortsætte på Cognitive Science, og derfor overvejer hun nu at læse videre i udlandet.
+
+- Det ser ud til, at det er den eneste mulighed, hvis man gerne vil læse videre på noget, der faktisk passer ind til vores studie, siger hun.
+
+Nye regler giver forsinkelse
+På Aarhus Universitet havde man håbet på at have kandidatuddannelsen klar, når det første hold bachelorer bliver færdige til sommer. Arbejdet er dog blevet forsinket, fordi der er kommet nye regler for, hvornår man må oprette en uddannelse, fortæller Niels Lehmann, prodekan på fakultetet Arts, som Cognitive Science hører under.
+
+Det er nogle meget dygtige studerende, der kommer ind på uddannelsen, og det er klart, at de i et vist omfang vil orientere sig mod udlandet, hvor man så kan forestille sig, at de bider sig fast.
+NIELS LEHMANN, PRODEKAN, AARHUS UNIVERSITET
+Tidligere skulle Danmarks Akkrediteringsinstitution se alle nye uddannelser efter i sømmene for at sikre, at kvaliteten var i orden. Nu skal uddannelsesinstitutionerne selv stå for det kvalitetstjek.
+
+Men det tjek har Aarhus Universitet endnu ikke fået grønt lys til selv at udføre, fortæller prodekanen.
+
+- Vi ville meget gerne have kunnet nå at få et udbud på kandidaten i gang i 2018, men så længe man er under institutionsakkreditering, så kan man ikke ansøge om nye uddannelser, siger han.
+
+Det er endnu usikkert, hvornår Aarhus Universitet kan oprette kandidaten i Cognitive Science. Hvis de får alle de nødvendige godkendelser, kan den tidligst være klar i 2019.
+
+Prodekan Niels Lehmann frygter, at Danmark kommer til at miste nogle af landets skarpeste studerende, hvis de er nødt til at rejse til udlandet for at gøre deres uddannelse færdig.
+
+- Det er nogle meget, meget dygtige studerende, der kommer ind på denne uddannelse, og det er klart, at de i et vist omfang vil orientere sig mod udlandet, hvor man så kan forestille sig, at de bider sig fast, siger han.
+
+Hos Danmarks Akkrediteringsinstitution forstår man godt, at universitets ansatte og studenrede ærgrer sig.
+
+- Jeg kan godt forstå, at Aarhus Universitet ærgrer sig over, at det trækker ud, og at der går noget tid, før man får mulighed for at oprette nye uddannelser, og at man ikke har fået den genvej til at oprette nye uddannelser, som ville være fuldt med, hvis man havde opnået en positiv institutionsakkreditering, siger kommunikationsansvarlig Daniel Sebastian Larsen.
+
+I år var Cognitive Science i Aarhus den uddannelse i Danmark, der havde det fjerde højeste karakterkrav - det højeste var 'AP Graduate in Marketing Management' på Erhvervsakademi Sjælland med et krav på 12,3."
+  example_title: "Summarization"
+  
 ---
+
+This repository contains a model for Danish abstractive summarisation of news articles. The summariser is based on a language-specific mT5-base, where the vocabulary is condensed to include tokens used in Danish and English. The model is fine-tuned using an abstractive subset of the DaNewsroom dataset (Varab & Schluter, 2020), according to the binned density categories employed in Newsroom (Grusky et al., 2019).

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "sarakolding/daT5-base",
+  "architectures": [
+    "MT5ForConditionalGeneration"
+  ],
+  "d_ff": 2048,
+  "d_kv": 64,
+  "d_model": 768,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.01,
+  "early_stopping": true,
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "length_penalty": 5,
+  "max_length": 128,
+  "min_length": 15,
+  "model_type": "mt5",
+  "no_repeat_ngram_size": 3,
+  "num_beams": 4,
+  "num_decoder_layers": 12,
+  "num_heads": 12,
+  "num_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "tie_word_embeddings": false,
+  "tokenizer_class": "T5Tokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.20.0.dev0",
+  "use_cache": true,
+  "vocab_size": 30000
+}

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc91e2fa415ded95ce0b768fcf744fde086c771a2610643122ac4a93e015b460
+size 1954640153

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a46dc9d540973454dc66d8964580d2bacf8336960c1d019241dbf2f5ac7dfa1d
+size 977332173

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0362545844db8f9d3ba81e0d323c72d39cf5ff6a92b9f21b560faeaf428e6e6e
+size 14503

scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb514f3549501e7fbfb775146d40e9d3027ea20fa05036f069600337153c15ff
+size 559

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5343e0993f724df030d39b5a8bed28c5148bbf4ad76c9e3fb3569506e763e049
+size 623

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dabe934d94d2182b6b15a66f895d7de8197a2c5e478c5a9e16746c51005e8b22
+size 766659

tokenizer_config.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0, "additional_special_tokens": null, "sp_model_kwargs": {}, "special_tokens_map_file": "/home/ucloud/.cache/huggingface/transformers/90e4e02f17b0b14f97d96c94bd8e5652250a216e5950bb43c79afe24011e808c.294ebaa4cd17bb284635004c92d2c4d522ec488c828dcce0c2471b6f28e3fe82", "name_or_path": "sarakolding/daT5-base", "tokenizer_class": "T5Tokenizer"}

trainer_state.json

@@ -0,0 +1,516 @@
+{
+  "best_metric": 2.132361888885498,
+  "best_model_checkpoint": "./26-125356_megasuperkanin/checkpoint-100000",
+  "epoch": 0.9769822970807769,
+  "global_step": 100000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.02,
+      "learning_rate": 5e-05,
+      "loss": 2.6761,
+      "step": 2500
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 5e-05,
+      "loss": 2.551,
+      "step": 5000
+    },
+    {
+      "epoch": 0.05,
+      "eval_gen_len": 28.4674,
+      "eval_loss": 2.423037052154541,
+      "eval_rouge1": 0.214,
+      "eval_rouge2": 0.0668,
+      "eval_rougeL": 0.1717,
+      "eval_rougeLsum": 0.1777,
+      "eval_runtime": 1015.6418,
+      "eval_samples_per_second": 2.265,
+      "eval_steps_per_second": 0.284,
+      "step": 5000
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 5e-05,
+      "loss": 2.5186,
+      "step": 7500
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 5e-05,
+      "loss": 2.4717,
+      "step": 10000
+    },
+    {
+      "epoch": 0.1,
+      "eval_gen_len": 25.6604,
+      "eval_loss": 2.3709843158721924,
+      "eval_rouge1": 0.2071,
+      "eval_rouge2": 0.0634,
+      "eval_rougeL": 0.1686,
+      "eval_rougeLsum": 0.1745,
+      "eval_runtime": 951.1096,
+      "eval_samples_per_second": 2.418,
+      "eval_steps_per_second": 0.303,
+      "step": 10000
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 5e-05,
+      "loss": 2.4593,
+      "step": 12500
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 5e-05,
+      "loss": 2.4281,
+      "step": 15000
+    },
+    {
+      "epoch": 0.15,
+      "eval_gen_len": 28.8296,
+      "eval_loss": 2.3228819370269775,
+      "eval_rouge1": 0.2137,
+      "eval_rouge2": 0.0662,
+      "eval_rougeL": 0.1711,
+      "eval_rougeLsum": 0.1768,
+      "eval_runtime": 1022.9494,
+      "eval_samples_per_second": 2.248,
+      "eval_steps_per_second": 0.282,
+      "step": 15000
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 5e-05,
+      "loss": 2.4049,
+      "step": 17500
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 5e-05,
+      "loss": 2.3735,
+      "step": 20000
+    },
+    {
+      "epoch": 0.2,
+      "eval_gen_len": 29.9183,
+      "eval_loss": 2.2881429195404053,
+      "eval_rouge1": 0.2164,
+      "eval_rouge2": 0.0668,
+      "eval_rougeL": 0.1735,
+      "eval_rougeLsum": 0.1808,
+      "eval_runtime": 1036.2984,
+      "eval_samples_per_second": 2.219,
+      "eval_steps_per_second": 0.278,
+      "step": 20000
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 5e-05,
+      "loss": 2.3732,
+      "step": 22500
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 5e-05,
+      "loss": 2.377,
+      "step": 25000
+    },
+    {
+      "epoch": 0.24,
+      "eval_gen_len": 29.5183,
+      "eval_loss": 2.2759358882904053,
+      "eval_rouge1": 0.2209,
+      "eval_rouge2": 0.0694,
+      "eval_rougeL": 0.1782,
+      "eval_rougeLsum": 0.1851,
+      "eval_runtime": 1036.1071,
+      "eval_samples_per_second": 2.22,
+      "eval_steps_per_second": 0.278,
+      "step": 25000
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 5e-05,
+      "loss": 2.3513,
+      "step": 27500
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 5e-05,
+      "loss": 2.3444,
+      "step": 30000
+    },
+    {
+      "epoch": 0.29,
+      "eval_gen_len": 29.3183,
+      "eval_loss": 2.2552034854888916,
+      "eval_rouge1": 0.2194,
+      "eval_rouge2": 0.0679,
+      "eval_rougeL": 0.1757,
+      "eval_rougeLsum": 0.1829,
+      "eval_runtime": 1037.4604,
+      "eval_samples_per_second": 2.217,
+      "eval_steps_per_second": 0.278,
+      "step": 30000
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 5e-05,
+      "loss": 2.3504,
+      "step": 32500
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 5e-05,
+      "loss": 2.3203,
+      "step": 35000
+    },
+    {
+      "epoch": 0.34,
+      "eval_gen_len": 32.2061,
+      "eval_loss": 2.235518455505371,
+      "eval_rouge1": 0.2284,
+      "eval_rouge2": 0.0722,
+      "eval_rougeL": 0.1819,
+      "eval_rougeLsum": 0.1892,
+      "eval_runtime": 1121.1561,
+      "eval_samples_per_second": 2.051,
+      "eval_steps_per_second": 0.257,
+      "step": 35000
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 5e-05,
+      "loss": 2.3087,
+      "step": 37500
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 5e-05,
+      "loss": 2.3132,
+      "step": 40000
+    },
+    {
+      "epoch": 0.39,
+      "eval_gen_len": 29.5452,
+      "eval_loss": 2.2289836406707764,
+      "eval_rouge1": 0.2183,
+      "eval_rouge2": 0.0673,
+      "eval_rougeL": 0.1759,
+      "eval_rougeLsum": 0.1827,
+      "eval_runtime": 1055.2895,
+      "eval_samples_per_second": 2.179,
+      "eval_steps_per_second": 0.273,
+      "step": 40000
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 5e-05,
+      "loss": 2.3063,
+      "step": 42500
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 5e-05,
+      "loss": 2.3116,
+      "step": 45000
+    },
+    {
+      "epoch": 0.44,
+      "eval_gen_len": 30.2935,
+      "eval_loss": 2.218207359313965,
+      "eval_rouge1": 0.2239,
+      "eval_rouge2": 0.07,
+      "eval_rougeL": 0.1798,
+      "eval_rougeLsum": 0.1879,
+      "eval_runtime": 1063.5185,
+      "eval_samples_per_second": 2.163,
+      "eval_steps_per_second": 0.271,
+      "step": 45000
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 5e-05,
+      "loss": 2.3014,
+      "step": 47500
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 5e-05,
+      "loss": 2.2852,
+      "step": 50000
+    },
+    {
+      "epoch": 0.49,
+      "eval_gen_len": 28.6443,
+      "eval_loss": 2.2090706825256348,
+      "eval_rouge1": 0.2251,
+      "eval_rouge2": 0.0703,
+      "eval_rougeL": 0.1812,
+      "eval_rougeLsum": 0.1887,
+      "eval_runtime": 1045.7282,
+      "eval_samples_per_second": 2.199,
+      "eval_steps_per_second": 0.275,
+      "step": 50000
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 5e-05,
+      "loss": 2.2963,
+      "step": 52500
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 5e-05,
+      "loss": 2.2683,
+      "step": 55000
+    },
+    {
+      "epoch": 0.54,
+      "eval_gen_len": 29.9661,
+      "eval_loss": 2.1879115104675293,
+      "eval_rouge1": 0.2257,
+      "eval_rouge2": 0.0716,
+      "eval_rougeL": 0.1806,
+      "eval_rougeLsum": 0.1876,
+      "eval_runtime": 1061.3075,
+      "eval_samples_per_second": 2.167,
+      "eval_steps_per_second": 0.271,
+      "step": 55000
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 5e-05,
+      "loss": 2.2735,
+      "step": 57500
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 5e-05,
+      "loss": 2.2614,
+      "step": 60000
+    },
+    {
+      "epoch": 0.59,
+      "eval_gen_len": 30.4435,
+      "eval_loss": 2.1871089935302734,
+      "eval_rouge1": 0.2316,
+      "eval_rouge2": 0.075,
+      "eval_rougeL": 0.1863,
+      "eval_rougeLsum": 0.1936,
+      "eval_runtime": 1083.7377,
+      "eval_samples_per_second": 2.122,
+      "eval_steps_per_second": 0.266,
+      "step": 60000
+    },
+    {
+      "epoch": 0.61,
+      "learning_rate": 5e-05,
+      "loss": 2.2735,
+      "step": 62500
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 5e-05,
+      "loss": 2.252,
+      "step": 65000
+    },
+    {
+      "epoch": 0.64,
+      "eval_gen_len": 30.6239,
+      "eval_loss": 2.175469160079956,
+      "eval_rouge1": 0.226,
+      "eval_rouge2": 0.0729,
+      "eval_rougeL": 0.1834,
+      "eval_rougeLsum": 0.1914,
+      "eval_runtime": 1080.4009,
+      "eval_samples_per_second": 2.129,
+      "eval_steps_per_second": 0.267,
+      "step": 65000
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 5e-05,
+      "loss": 2.2509,
+      "step": 67500
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 5e-05,
+      "loss": 2.262,
+      "step": 70000
+    },
+    {
+      "epoch": 0.68,
+      "eval_gen_len": 30.9983,
+      "eval_loss": 2.16789174079895,
+      "eval_rouge1": 0.2256,
+      "eval_rouge2": 0.0716,
+      "eval_rougeL": 0.1815,
+      "eval_rougeLsum": 0.1889,
+      "eval_runtime": 1104.0224,
+      "eval_samples_per_second": 2.083,
+      "eval_steps_per_second": 0.261,
+      "step": 70000
+    },
+    {
+      "epoch": 0.71,
+      "learning_rate": 5e-05,
+      "loss": 2.2398,
+      "step": 72500
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 5e-05,
+      "loss": 2.228,
+      "step": 75000
+    },
+    {
+      "epoch": 0.73,
+      "eval_gen_len": 29.9704,
+      "eval_loss": 2.1669178009033203,
+      "eval_rouge1": 0.2253,
+      "eval_rouge2": 0.0725,
+      "eval_rougeL": 0.1822,
+      "eval_rougeLsum": 0.1894,
+      "eval_runtime": 1052.7669,
+      "eval_samples_per_second": 2.185,
+      "eval_steps_per_second": 0.274,
+      "step": 75000
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 5e-05,
+      "loss": 2.25,
+      "step": 77500
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 5e-05,
+      "loss": 2.234,
+      "step": 80000
+    },
+    {
+      "epoch": 0.78,
+      "eval_gen_len": 29.4826,
+      "eval_loss": 2.1604671478271484,
+      "eval_rouge1": 0.2283,
+      "eval_rouge2": 0.0747,
+      "eval_rougeL": 0.1855,
+      "eval_rougeLsum": 0.1937,
+      "eval_runtime": 1075.8159,
+      "eval_samples_per_second": 2.138,
+      "eval_steps_per_second": 0.268,
+      "step": 80000
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 5e-05,
+      "loss": 2.236,
+      "step": 82500
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 5e-05,
+      "loss": 2.2289,
+      "step": 85000
+    },
+    {
+      "epoch": 0.83,
+      "eval_gen_len": 30.0213,
+      "eval_loss": 2.1517326831817627,
+      "eval_rouge1": 0.2226,
+      "eval_rouge2": 0.0705,
+      "eval_rougeL": 0.1801,
+      "eval_rougeLsum": 0.1873,
+      "eval_runtime": 1072.8178,
+      "eval_samples_per_second": 2.144,
+      "eval_steps_per_second": 0.268,
+      "step": 85000
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 5e-05,
+      "loss": 2.2214,
+      "step": 87500
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 5e-05,
+      "loss": 2.2043,
+      "step": 90000
+    },
+    {
+      "epoch": 0.88,
+      "eval_gen_len": 29.5361,
+      "eval_loss": 2.1455490589141846,
+      "eval_rouge1": 0.2265,
+      "eval_rouge2": 0.075,
+      "eval_rougeL": 0.1838,
+      "eval_rougeLsum": 0.1908,
+      "eval_runtime": 1058.731,
+      "eval_samples_per_second": 2.172,
+      "eval_steps_per_second": 0.272,
+      "step": 90000
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 5e-05,
+      "loss": 2.2419,
+      "step": 92500
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 5e-05,
+      "loss": 2.2259,
+      "step": 95000
+    },
+    {
+      "epoch": 0.93,
+      "eval_gen_len": 29.6874,
+      "eval_loss": 2.1389129161834717,
+      "eval_rouge1": 0.2287,
+      "eval_rouge2": 0.0713,
+      "eval_rougeL": 0.1844,
+      "eval_rougeLsum": 0.1911,
+      "eval_runtime": 1069.2344,
+      "eval_samples_per_second": 2.151,
+      "eval_steps_per_second": 0.269,
+      "step": 95000
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 5e-05,
+      "loss": 2.2202,
+      "step": 97500
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 5e-05,
+      "loss": 2.2307,
+      "step": 100000
+    },
+    {
+      "epoch": 0.98,
+      "eval_gen_len": 30.7513,
+      "eval_loss": 2.132361888885498,
+      "eval_rouge1": 0.2293,
+      "eval_rouge2": 0.0741,
+      "eval_rougeL": 0.1845,
+      "eval_rougeLsum": 0.1924,
+      "eval_runtime": 1089.9927,
+      "eval_samples_per_second": 2.11,
+      "eval_steps_per_second": 0.264,
+      "step": 100000
+    }
+  ],
+  "max_steps": 102356,
+  "num_train_epochs": 1,
+  "total_flos": 1.8696291573252096e+17,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38a62961f75bc74d8cd1c3b0178206ed678b491c8842aa32921846d2fbbba66c
+size 3375