Generating Sports News from Live Commentary: A C hinese Dataset for Sports Game Summarization (2020)

Abstract: Sports game summarization focuses on generating news articles from live commentaries. Unlike traditional summarization tasks, the source documents and the target summaries for sports game summarization tasks are written in quite different writing styles. In addition, live commentaries usually contain many named entities, which makes summarizing sports games precisely very challenging. To deeply study this task, we present SportsSum, a Chinese sports game summarization dataset which contains 5,428 soccer games of live commentaries and the corresponding news articles. Additionally, we propose a two-step summarization model consisting of a selector and a rewriter for SportsSum. To evaluate the correctness of generated sports summaries, we design two novel score metrics: name matching score and event matching score. Experimental results show that our model performs better than other summarization baselines on ROUGE scores as well as the two designed scores.

## introduction there are a large number of sports games playing every day. apparently, manually writing sports news articles to summarize every game is laborintensive and infeasible. how to automatically generate sports summaries, therefore, becomes a popular and demanding task. recently, generating news from live commentaries has gradually attracted attention in the academic community @xcite . at the same time, several trials have been done in the industry such as sports news from toutiao's xiaoming bot foot_1 , sohu ruibao foot_2 and ai football news foot_3 . 71' 1-0 拜仁慕尼黑球员里贝里大禁区左侧尝试右脚射 门,可惜皮球高出球门.给他传球的是拉姆. bayern munich's player ribery tried to shoot with his right foot from the penalty area's left side, but the ball was higher than the crossbar. lahm passed the ball to him. ## dataset we present sportssum, a sports game summarization dataset in chinese. ## sports game summarization the goal of sports game summarization is to generate a sports news report @xmath0, r2 , .., rn } from a given live commentary document @xmath1, s 1 , c 1 ), ..., (t m , s m , c m )}. the generated news report r is expected to cover most of the important events in the games and describe those events correctly. in this paper, we propose a twostep model for sportssum. the proposed model first learns a selector to extract important commentary sentences and then utilizes a rewriter to convert the selected sentences to a news article. sentence mapping. to train the selector and rewriter, we need some labels to indicate the importance of commentary sentences and the corresponding news sentences. to obtain the labels, we consider the timeline information and bertscore @xcite , a metric to measure the sentence similarity, and map each news sentence to a commentary sentence. although we have no explicit timeline information for news sentences, we observe that many news sentences start with "in the n-th minute" and thus we can extract the timeline information for some news sentences. we map sentences by the following steps: 1) for each news sentence r i , we extract the timeline information h i if possible. otherwise, we do not map this news sentence. 2) we consider those commentary sentences c j with t j being close to h i . more specifically, we consider c (i) = {c k , c k+1 , ...c k+l }, where c j is the commentary sentence with timeline information t @xmath2, h i +3] with the above mapping process, we obtain a set of mapped commentary sentences and news sentences @xmath3, r1 ), (c 2 , r2 ), ..., (c s , rs )}, which can be used for training our selector and rewriter. selector. there are many commentary sentences in a live commentary document, but only few of them contain valuable information and should be reported in the news article. therefore, we learn a selector to pick up those important sentences. more specifically, given a commentary document @xmath4, s 1 , c 1 ), ..., (t m , s m , c m )}, the selector outputs a set c @xmath5, c2 , ..., cn } which contains only important commentary sentences. we train a binary classifier as the selector to choose important commentary sentences. when training, for each commentary sentence c i in c, we assign a positive label if c i can be mapped with a news sentence by the aforementioned mapping process. otherwise, we give a negative label. rewriter. the rewriter converts the selected commentary sentences c @xmath6, c2 , ..., cn } to a news report @xmath7, r2 , .., rn }. we focus on the sentence-level rewriter. that is, we convert each selected commentary sentence ci to a news sentence ri . an intuitive way to learn the sentencelevel rewriter is training a sequence-to-sequence (seq2seq) model, such as lstm @xcite and transformer @xcite , on the mapped sentences d. however, as illustrated in to solve the name mismatch problem, we train the rewriter in a template-to-template (tem2tem) way instead of in a seq2seq way. we first build a dictionary of player names from the lineup data (metadata). next, for each (c i , ri ) in d, we replace all the player names in ci and ri with a special token "[player]" so that the new sentence is like a template. if there are multiple player names in a sentence, we append a number to the special token to distinguish them, as shown in after converting ci and ri to the template sentences, we train a seq2seq model on the template sentences. by training models in a tem2tem way, the model focuses more on the relations between players and actions and is less influenced by the high-frequency player names. when predicting, for each commentary sentence ci in c select , we use the aforementioned way to convert ci to a commentary template sentence. then, we generate a news template sentence by the rewriter and replace all the special tokens in the sentence with the original player names. evaluation. we consider rouge scores @xcite , which are standard metrics for summarization tasks. more precisely, we focus on rouge-1, rouge-2, and rouge-l. however, we observe that rouge scores cannot accurately evaluate the correctness of summaries. some summaries may get high rouge scores but contain many incorrect facts. therefore, we design two metrics: name matching score (nms) and event matching score (ems). the name matching score evaluates the closeness of the player names in the ground truth news article r and the generated summaries r. let n g and n p denote the set of the player names appearing in r and r, respectively. we define the name matching score as nms(r, r) = f-score(n g , n p ). similarly, the event matching score evaluates the closeness of the events in r and r. we define an event as a pair (subject, verb) in the sentence. two pairs (subject 1 , verb 1 ) and (subject 2 , verb 2 ) are viewed as equivalent if and only if 1) subject 1 is the same as subject 2 and 2) verb 1 and verb 2 are synonym foot_4 to each other. let e g and e p represent the set of events in r and r, respectively, the event matching score is defined as implementations and models. we consider the convolutional neural network @xcite as the selector. for the rewriter, we consider the following: (1) lstm: a bidirectional lstm with attention mechanism @xcite . (2) transformer. @xcite (3) pgnet: pointergenerator network, an encoder-decoder model with copy mechanism @xcite . for comparison, we consider two extractive summarization baselines: (1) rawsent: the raw sentences selected by the selector without rewriting. in addition, we train a bidirectional lstm with attention mechanism (abs-lstm) and a pointergenerator network (abs-pgnet) on the paired commentaries and news articles as two simple abstractive summarization baselines. more implementation details can be found in appendix c. ## conclusion we present sportssum, a chinese dataset for sports game summarization, as well as a model that consists of a selector and a rewriter. to improve the quality of generated news, we train the model in a tem2tem way. we design two metrics to evaluate the correctness of generated summaries. the experimental results demonstrate that the proposed model performs well on rouge scores and the two designed scores.