Spaces:
Running
Running
added explanation for spurious case
Browse files- constants.py +8 -7
constants.py
CHANGED
@@ -27,13 +27,14 @@ Comparing the ground truth and predicted span objects we get the following broad
|
|
27 |
|
28 |
##### Comparison Cases
|
29 |
|
30 |
-
| Category | Explanation
|
31 |
-
| --------------- |
|
32 |
-
| Correct (COR) | both are the same
|
33 |
-
| Incorrect (INC) | the output of a system and the golden annotation don’t match
|
34 |
-
| Partial (PAR) | system and the golden annotation are somewhat “similar” but not the same
|
35 |
-
| Missing (MIS) | a golden annotation is not captured by a system
|
36 |
-
| Spurious (SPU) | system produces a response which doesn’t exist in the golden annotation
|
|
|
37 |
|
38 |
The specifics of this categorization is defined based on our metric of choice. For example, in some cases we might want to consider a partial overlap of offsets correct and in other cases incorrect.
|
39 |
Based on this we have the Partial & Exact span based criterias. These categorization of these schemas are shown below
|
|
|
27 |
|
28 |
##### Comparison Cases
|
29 |
|
30 |
+
| Category | Explanation |
|
31 |
+
| --------------- | --------------------------------------------------------------------------------------- |
|
32 |
+
| Correct (COR) | both are the same |
|
33 |
+
| Incorrect (INC) | the output of a system and the golden annotation don’t match |
|
34 |
+
| Partial (PAR) | system and the golden annotation are somewhat “similar” but not the same |
|
35 |
+
| Missing (MIS) | a golden annotation is not captured by a system |
|
36 |
+
| Spurious (SPU) | system produces a response which doesn’t exist in the golden annotation (Hallucination) |
|
37 |
+
|
38 |
|
39 |
The specifics of this categorization is defined based on our metric of choice. For example, in some cases we might want to consider a partial overlap of offsets correct and in other cases incorrect.
|
40 |
Based on this we have the Partial & Exact span based criterias. These categorization of these schemas are shown below
|