Gabriel commited on
Commit
4358cae
1 Parent(s): a9201df

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -1
README.md CHANGED
@@ -42,9 +42,45 @@ However, this model can be used to interpret and analyse historical textual mate
42
 
43
  ## Model Dscription
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
 
47
  ## Acknowledgements
 
 
 
 
 
 
 
 
48
 
49
- We gratefully acknowledge the HPC RIVR consortium (https://www.hpc-rivr.si) and EuroHPC JU (https://eurohpc-ju.europa.eu) for funding this research by providing computing resources of the HPC system Vega at the Institute of Information Science (https://www.izum.si).
50
 
 
42
 
43
  ## Model Dscription
44
 
45
+ The following hyperparameters were used during training:
46
+ - learning_rate: 3e-05
47
+ - train_batch_size: 8
48
+ - eval_batch_size: 8
49
+ - seed: 42
50
+ - gradient_accumulation_steps: 0
51
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
+ - lr_scheduler_type: linear
53
+ - num_epochs: 6
54
+ - fp16: False
55
+
56
+ Dataset:
57
+ - Khubist2, which has been cleaned and chunked
58
+
59
+ ## Intended uses & limitations
60
+ This model should primarly be used to fine-tune further on and downstream tasks.
61
+
62
+ Inference for fill-mask with Huggingface Transformers in python:
63
+
64
+ ```python
65
+ from transformers import pipeline
66
+
67
+ summarizer = pipeline("fill-mask", model="Riksarkivet/bert-base-cased-swe-1800")
68
+ historical_text = """Det vore [MASK] häller nödvändigt att bita af tungan än berättat hvad jag varit med om."""
69
+ print(summarizer(historical_text))
70
+ ```
71
+
72
+
73
+
74
 
75
 
76
  ## Acknowledgements
77
+ We gratefully acknowledge EuroHPC (https://eurohpc-ju.europa.eu) for funding this research by providing computing resources of the HPC system Vega at the Institute of Information Science (https://www.izum.si)
78
+ and Språkbanken (Swe-Clarin) for the datasets.
79
+
80
+ ## Citation Information
81
+
82
+ Eva Pettersson and Lars Borin (2022)
83
+ Swedish Diachronic Corpus
84
+ In Darja Fišer & Andreas Witt (eds.), CLARIN. The Infrastructure for Language Resources. Berlin: deGruyter. https://degruyter.com/document/doi/10.1515/9783110767377-022/html
85
 
 
86