Kkordik commited on
Commit
eb0c5b9
1 Parent(s): 80b2034

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -17
README.md CHANGED
@@ -26,34 +26,65 @@ model-index:
26
 
27
  ---
28
 
29
- This markdown file contains the spec for the modelcard metadata regarding evaluation parameters. When present, and only then, 'model-index', 'datasets' and 'license' contents will be verified when git pushing changes to your README.md file.
30
- Valid license identifiers can be found in [our docs](https://huggingface.co/docs/hub/repositories-licenses).
31
-
32
- For the full model card template, see: [modelcard_template.md file](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md).
33
- ---
34
-
35
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
36
- should probably proofread and complete it, then remove this comment. -->
37
-
38
  # longformer_4096_qsi
39
 
40
- This model is a fine-tuned version of [mrm8488/longformer-base-4096-finetuned-squadv2](https://huggingface.co/mrm8488/longformer-base-4096-finetuned-squadv2) on an unknown dataset.
41
  It achieves the following results on the evaluation set:
42
  - Loss: 2.9598
43
 
44
  ## Model description
45
 
46
- More information needed
47
-
48
- ## Intended uses & limitations
49
-
50
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
 
52
  ## Training and evaluation data
53
 
54
- More information needed
 
 
55
 
56
- ## Training procedure
57
 
58
  ### Training hyperparameters
59
 
 
26
 
27
  ---
28
 
 
 
 
 
 
 
 
 
 
29
  # longformer_4096_qsi
30
 
31
+ This model is a fine-tuned version of [mrm8488/longformer-base-4096-finetuned-squadv2](https://huggingface.co/mrm8488/longformer-base-4096-finetuned-squadv2) on a tiny [NovelQSI](https://huggingface.co/datasets/Kkordik/NovelQSI) dataset.
32
  It achieves the following results on the evaluation set:
33
  - Loss: 2.9598
34
 
35
  ## Model description
36
 
37
+ This model is test model for my research project. The idea of the model is to understand which novel character said the requested quote.
38
+ It achieves a bit better results on the ´test´ split of the NovelQSI dataset than base longformer-base-4096-finetuned-squadv2 model on the same dataset split.
39
+
40
+ **Base model results:**
41
+
42
+ ```
43
+ {
44
+ "exact_match": {
45
+ "confidence_interval": [8.754452551305853, 14.718614718614718],
46
+ "score": 12.121212121212121,
47
+ "standard_error": 1.8579217243778676
48
+ },
49
+ "f1": {
50
+ "confidence_interval": [18.469101076147584, 28.28409063313956],
51
+ "score": 22.799422799422796,
52
+ "standard_error": 2.896728175757627
53
+ },
54
+ "latency_in_seconds": 0.7730605573419919,
55
+ "samples_per_second": 1.2935597224598967,
56
+ "total_time_in_seconds": 178.5769887460001
57
+ }
58
+
59
+ ```
60
+
61
+ **Achieved results:**
62
+
63
+ ```
64
+ {
65
+ "exact_match": {
66
+ "confidence_interval": [16.017316017316016, 24.242424242424242],
67
+ "score": 20.346320346320347,
68
+ "standard_error": 2.9434375492784994
69
+ },
70
+ "f1": {
71
+ "confidence_interval": [23.123469058324783, 31.823648733317036],
72
+ "score": 26.580086580086572,
73
+ "standard_error": 2.593030474995015
74
+ },
75
+ "latency_in_seconds": 0.8093855569913422,
76
+ "samples_per_second": 1.235505120349827,
77
+ "total_time_in_seconds": 186.96806366500005
78
+ }
79
+ ```
80
 
81
  ## Training and evaluation data
82
 
83
+ You can find training code in the github repo of my research:
84
+
85
+ https://github.com/Kkordik/NovelQSI/tree/main
86
 
87
+ It was trained and evaluated in notebooks, so it is easy to reproduce.
88
 
89
  ### Training hyperparameters
90