In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Q: Also, note that the performance for this task is not expected to achieve a perfect accuracy, as there may be situations where more than one action is reasonable, and also because writers tell a story playing with elements such as surprise or uncertainty.
Question: Why do they think this task is hard? What is the baseline performance?
A:
1. there may be situations where more than one action is reasonable, and also because writers tell a story playing with elements such as surprise or uncertainty.
2. Macro F1 = 14.6 (MLR, length 96 snippet)
Weighted F1 = 31.1 (LSTM, length 128 snippet)