The LSDSem'17 shared task is the Story Cloze Test, a new evaluation for story understanding and script learning. This test provides a system with a four-sentence story and two possible endings, and the system must choose the correct ending to the story. Successful narrative understanding (getting closer to human performance of 100%) requires systems to link various levels of semantics to commonsense knowledge. A total of eight systems participated in the shared task, with a variety of approaches including end-to-end neural networks, feature-based regression models, and rule-based methods. The highest performing system achieves an accuracy of 75.2%, a substantial improvement over the previous state-of-the-art.
By contrast, the Story Cloze Test dataset REF evaluates the ability of story understanding and script learning, where the task requires to select or generate a reasonable sentence to complete the story context.
LSDSem 2017 Shared Task: The Story Cloze Test
LSDSem@EACL
Computer Science