wadood commited on
Commit
0347c0d
1 Parent(s): 77c75da

added explanation for spurious case

Browse files
Files changed (1) hide show
  1. constants.py +8 -7
constants.py CHANGED
@@ -27,13 +27,14 @@ Comparing the ground truth and predicted span objects we get the following broad
27
 
28
  ##### Comparison Cases
29
 
30
- | Category | Explanation |
31
- | --------------- | ------------------------------------------------------------------------ |
32
- | Correct (COR) | both are the same |
33
- | Incorrect (INC) | the output of a system and the golden annotation don’t match |
34
- | Partial (PAR) | system and the golden annotation are somewhat “similar” but not the same |
35
- | Missing (MIS) | a golden annotation is not captured by a system |
36
- | Spurious (SPU) | system produces a response which doesn’t exist in the golden annotation |
 
37
 
38
  The specifics of this categorization is defined based on our metric of choice. For example, in some cases we might want to consider a partial overlap of offsets correct and in other cases incorrect.
39
  Based on this we have the Partial & Exact span based criterias. These categorization of these schemas are shown below
 
27
 
28
  ##### Comparison Cases
29
 
30
+ | Category | Explanation |
31
+ | --------------- | --------------------------------------------------------------------------------------- |
32
+ | Correct (COR) | both are the same |
33
+ | Incorrect (INC) | the output of a system and the golden annotation don’t match |
34
+ | Partial (PAR) | system and the golden annotation are somewhat “similar” but not the same |
35
+ | Missing (MIS) | a golden annotation is not captured by a system |
36
+ | Spurious (SPU) | system produces a response which doesn’t exist in the golden annotation (Hallucination) |
37
+
38
 
39
  The specifics of this categorization is defined based on our metric of choice. For example, in some cases we might want to consider a partial overlap of offsets correct and in other cases incorrect.
40
  Based on this we have the Partial & Exact span based criterias. These categorization of these schemas are shown below