File size: 6,169 Bytes
4389ada 80b2034 2343c6f 984ce8f 80b2034 984ce8f 80b2034 68ad3d6 99408a5 73ca76f 99408a5 156ba8a 99408a5 156ba8a 99408a5 156ba8a 99408a5 80b2034 4389ada eb0c5b9 4389ada 5057a4a eb0c5b9 4389ada 5057a4a 4389ada eb0c5b9 5057a4a 4389ada eb0c5b9 4389ada 984ce8f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 |
---
base_model: mrm8488/longformer-base-4096-finetuned-squadv2
tags:
- generated_from_trainer
- qsi
- quote_speaker_identification
license: apache-2.0
datasets:
- Kkordik/NovelQSI
language:
- en
widget:
- text: >-
Which character said 'You know, I read somewhere that the brightest stars are those that have undergone the most turmoil. Maybe it's the same with us – our struggles make us shine brighter'?
context: >-
Characters:
Alex: Aliases: {'Alex'}. Gender: Male, The character is: major
Bella: Aliases: {'Bella'}. Gender: Female, The character is: major
Charlie: Aliases: {'Charlie'}. Gender: Non-binary, The character is: major
Summary:
In the novel's previous section, Alex, Bella, and Charlie, three friends in Luminara, engage in a deep conversation at The Starry Night café. They debate destiny, with Alex believing in fate, Bella advocating for self-made destiny, and Charlie suggesting a combination of both. Personal reflections emerge, such as Bella's musings on heartbreak and Alex's thoughts on longing. Charlie compares people's struggles to stars, implying that challenges enhance personal growth. The night progresses with their varied, meaningful discussions.
Novel Text:
In the bustling city of Luminara, three friends, Alex, Bella, and Charlie, often met at their favorite café, The Starry Night, to discuss life, love, and the mysteries of the universe. The café, with its warm ambiance and the soft hum of jazz in the background, provided the perfect setting for their deep conversations.
One evening, as the city lights twinkled outside, the trio found themselves engrossed in a discussion about destiny.
Alex, a firm believer in fate, argued passionately, "I truly believe that our paths are predestined. The universe has a plan for each of us, and all our choices lead us to our ultimate destiny."
Bella, a skeptic, laughed softly and countered, "That's a romantic notion, Alex, but I think we make our own destiny. It's our decisions, not some cosmic plan, that shape our lives."
Charlie, always the mediator, added thoughtfully, "Maybe it's a bit of both. Perhaps there's a grand design, but within it, we have the freedom to make choices that influence our journey."
Their conversation drifted to other topics as the evening wore on. At one point, Bella, reflecting on a recent heartbreak, said, "Sometimes, I wonder if the heart ever truly heals from loss, or if it just learns to live with the pain."
Alex, looking out the window at the starry sky, mused, "It's strange how the heart yearns for what it can't have. The unattainable always seems so much more alluring."
Charlie, who had been quiet for a while, suddenly spoke up with a gleam in his eye, "You know, I read somewhere that the brightest stars are those that have undergone the most turmoil. Maybe it's the same with us – our struggles make us shine brighter."
As the night deepened, their conversation meandered through various topics.
model-index:
- name: Kkordik/test_longformer_4096_qsi
results:
- task:
type: question-answering
dataset:
type: Kkordik/NovelQSI
name: NovelQSI
split: test
metrics:
- type: exact_match
value: 20.346
verified: false
- type: f1
value: 26.58
verified: false
---
# longformer_4096_qsi
This model is a fine-tuned version of [mrm8488/longformer-base-4096-finetuned-squadv2](https://huggingface.co/mrm8488/longformer-base-4096-finetuned-squadv2) on a tiny [NovelQSI](https://huggingface.co/datasets/Kkordik/NovelQSI) dataset.
It achieves the following results on the evaluation set:
- Loss: 2.9598
## Model description
This model is a test model for my research project. The idea of the model is to understand which novel character said the requested quote.
It achieves a bit better results on the ´test´ split of the NovelQSI dataset than base longformer-base-4096-finetuned-squadv2 model on the same dataset split.
**Base model results:**
```
{
"exact_match": {
"confidence_interval": [8.754452551305853, 14.718614718614718],
"score": 12.121212121212121,
"standard_error": 1.8579217243778676
},
"f1": {
"confidence_interval": [18.469101076147584, 28.28409063313956],
"score": 22.799422799422796,
"standard_error": 2.896728175757627
},
"latency_in_seconds": 0.7730605573419919,
"samples_per_second": 1.2935597224598967,
"total_time_in_seconds": 178.5769887460001
}
```
**Achieved results:**
```
{
"exact_match": {
"confidence_interval": [16.017316017316016, 24.242424242424242],
"score": 20.346320346320347,
"standard_error": 2.9434375492784994
},
"f1": {
"confidence_interval": [23.123469058324783, 31.823648733317036],
"score": 26.580086580086572,
"standard_error": 2.593030474995015
},
"latency_in_seconds": 0.8093855569913422,
"samples_per_second": 1.235505120349827,
"total_time_in_seconds": 186.96806366500005
}
```
The results have shown, that the technique has its future.
## Training and evaluation data
You can find training code in the github repo of my research:
https://github.com/Kkordik/NovelQSI
It was trained and evaluated in notebooks, so it is easy to reproduce.
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log | 1.0 | 93 | 3.0886 |
| No log | 1.99 | 186 | 3.3755 |
| No log | 2.99 | 279 | 2.9598 |
### Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0 |