jiazheng's picture
Duplicate from ml6team/post-processing-summarization
6d444ff
One of the dependencies that, when found in the summary but not in the article, indicates a possible error is the *"amod"* (adjectival modifier) dependency.
Applied to this summary, we have *"First"* as the entity, and it is the adjectival modifier of the word *"phone"*.
And indeed, this unmatched dependency indicates an actual error here. The sentence is not factual, since the article talks about a **new** type of flagship phone,
and not the **first** flagship phone. This error was found by filtering on this specific kind of dependency. Empirical results showed that unmatched *amod* dependencies often suggest
that the summary sentence contains an error. <br> <br>
Another dependency that we use is the *"pobj"* (object of preposition) dependency.
Furthermore, we only match *pobj* dependencies when the target word is *"in"*, as in this example.
In this case the sentence itself contains a factual error (because the article states *"there's no word on a US release date yet"*).
However, this could have already been found with entity matching (as *January 18* is unmatched), and the unmatched dependency can not be completely blamed for this error here.