Update README.md
Browse files
README.md
CHANGED
@@ -11,10 +11,10 @@ pipeline_tag: sentence-similarity
|
|
11 |
|
12 |
## Evaluation
|
13 |
|
14 |
-
To assess the performance of the reranker, we will utilize the "validation"
|
15 |
-
the paragraph constituting the excerpt that should be ranked Top-1 for an Oracle modeling. What's intriguing is that
|
16 |
-
a corresponding theme that does not match the question forms a hard negative (other excerpts outside the theme are
|
17 |
-
table, with each theme showing the number of excerpts and associated questions:
|
18 |
|
19 |
| Theme name | Context number |
|
20 |
|---------------------------------------------:|:---------------|
|
|
|
11 |
|
12 |
## Evaluation
|
13 |
|
14 |
+
To assess the performance of the reranker, we will utilize the "validation" split of the [SQuAD]()https://huggingface.co/datasets/rajpurkar/squad dataset. We will select
|
15 |
+
the first question from each paragraph, along with the paragraph constituting the excerpt that should be ranked Top-1 for an Oracle modeling. What's intriguing is that
|
16 |
+
the number of themes is limited, and each excerpt from a corresponding theme that does not match the question forms a hard negative (other excerpts outside the theme are
|
17 |
+
simple negatives). Thus, we can construct the following table, with each theme showing the number of excerpts and associated questions:
|
18 |
|
19 |
| Theme name | Context number |
|
20 |
|---------------------------------------------:|:---------------|
|