RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Claim: We further compare the runtime against BARTScore, which only requires a single forward-pass through a neural net and can be batched easily, resulting in a 10x speed-up.
Context: Results in Table 1 reveal that the co-reference system is not always improving scores, particularly on the CNN/DailyMail-derived datasets. However, the use of co-reference resolution will significantly increase the processing time, as shown in Table 5 . This is expected, given that there are now more fact tuples due to the tuple expansion ; since the presented scoring method requires the comparison of each fact tuple in the summary against all input text tuples.
Caption: Table 5: Average processing time (in seconds) per instance in QAGS-CNN/DM. SRLScore uses ROUGE similarity. BARTScore is run with a batch size of 4.