Refining Data for Text Generation (2020)

Abstract: Recent work on data-to-text generation has made progress under the neural encoder-decoder architectures. However, the data input size is often enormous, while not all data records are important for text generation and inappropriate input may bring noise into the final output. To solve this problem, we propose a two-step approach which first selects and orders the important data records and then generates text from the noise-reduced data. Here we propose a learning to rank model to rank the importance of each record which is supervised by a relation extractor. With the noise-reduced data as input, we implement a text generator which sequentially models the input data records and emits a summary. Experiments on the ROTOWIRE dataset verifies the effectiveness of our proposed method in both performance and efficiency.

## introduction recently the task of generating text based on structured data has attracted a lot of interest from the natural language processing community. in its early stage, text generation (tg) is mainly accomplished with manually compiled rules or templates, which are inflexible and mainly based on expert knowledge @xcite @xcite . with the development of neural network techniques, especially sequence-to-sequence (seq2seq) models, generating short descriptive texts from structured data has achieved great successes, including generating wikipedia-style biographies @xcite and restaurant introductions @xcite . however, the task of generating long text, such as generating sports news from data, still fails to achieve satisfactory results. the existing models often forge fake context, lose sight of key facts and display inter-sentence incoherence @xcite . for the sports news generation task, one challenging problem is that the input records are both large and noisy. specifically, the inputted box scores, which contains hundreds of data records, belong to 40 different categories, such as fouls, threepointer, starting position and so on. meanwhile, not all of the inputted records are reflected in the sports news, and there exists a serious non-parallelism between data records and texts. according to our statistics for 3000 parallel sports news and its data records which is shown in in addition, sport news is far more complex than short descriptive text in that they need to consider overall coherence @xcite . for example, it would be weird if there is an abrupt topic change between neighboring sentences. if we just pour all the data records with no order into a model, it would be difficult for the summarization model to learn content planning by itself. thus, it is a good practice to order the data records before text generation. as stated above, in this paper, we propose to refine data records for the data-to-text generation task by training a model to select an appropriate subset of data records, which carries the key facts of the mentioned data records in one summary game, and further to plan an appropriate order for the selected records. this is also similar to the action of human writers who usually plan the important information to include before they write their articles. next, one key problem is to label the important records which would be time consuming and expensive. to solve this problem, inspired by @xcite which used an information extraction (ie) system for evaluation and @xcite which used distance learning for relation extraction, we build an ie system based on distant supervision. the ie system extracts relations from gold text, matches them to the corresponding data records and its results can then be used to supervise the process of content selection and planning. then, we design a ranking unit to learn which data records are selected and in what order they appear. here we choose to use the learning-to-rank (l2r) method instead of a classifier, because there exists heavy imbalance between positive and negative instances. we also design a rulebased model to further help select the data records. we rank each data record by an overall score based on the two rankers and rule-based system. finally, we feed the selected and ordered records, which not only the noise and the input size is reduced but also the content is planned, to the generator to obtain the summaries. in this way memory usage could be largely reduced, thus the training process could be accelerated. we evaluate our method on the rotowire dataset @xcite . the results show how our system improves the model's ability of selecting appropriate context and ordering them. while we achieve comparable bleu score, the efficiency of the model is greatly improved. proceedings of the 19th china national conference on computational linguistics, pages 881-891, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 computational linguistics 2 related work data-to-text generation has been an important topic of natural language generation for decades. early approaches mainly use templates and rules to perform content selection and surface realization @xcite @xcite . these models have good interpretability and controllability, but the generated content often have problems in terms of diversity and consistency. recently, neural network techniques have greatly improved the results of generating short descriptive text from data. the e2e dataset @xcite stated the task of generating natural language descriptive text of the restaurants from structured information of the restaurants. the wikibio dataset @xcite gives the infobox of wikipedia as the input data and the first sentence of the corresponding biography as output text. various approaches have achieved good results on these two datasets which considered content selection and planning. sha et al. @xcite proposed a method that models the order of information via link-based attention between different types of data records. perez-beltrachini and lapata (perez-beltrachini and lapata, 2018) introduce a content selection method based on multi-instance learning. generating sport news summaries on the other hand,is more challenging because not only the output text is longer and more complex, but also the input data records are numerous and diversed. wiseman et al. @xcite proposed the rotowire data set and gave baselines model based on end-to-end neural networks with attention and copy mechanism, these models often overlook key facts, repeatedly output the same information and make up irrelevant content. puduppully et al. @xcite designed a system that uses gate mechanism and pointer network to select and plan the content. they only used the ie system to guide content planning, while we let the ie system guide both content selecting and planning. meanwhile our system is lighter and has higher efficiency since we only feed the neural network with a small subset of the large set of data records. our model consists of three modules: information extraction, data refining (record selection and planning) and text generation. figure 2 is a brief flow chart showing the pipeline of our model, which illustrates the data flow and how the models are trained. proceedings of the 19th china national conference on computational linguistics, pages 881-891, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 computational linguistics ## conclusion in this paper we propose a data-to-text generating model which can learn data selecting and ordering from an ie system. different from previous methods, our model learns what to say and how to say from the supervision of an ie system. to achieve our goal, we propose to use a ranking unit to learn selecting and ordering content from the ie system and refine the input of the text generator. experiments on the rotowire dataset verifies the effectiveness of our proposed method. ie (new york knicks, 82, team-pts), (new york knicks, 9, team-wins), (indiana pacers, 31, team-losses), (indiana pacers, 17, team-wins), (indiana pacers, 103, team-pts), (new york knicks, 38, team-losses), (roy hibbert, 18, player-pts), (roy hibbert, 10, player-reb), (carmelo anthony, 7, player-fgm), (carmelo anthony, 16, player-fga), (carmelo anthony, 18, player-pts), (rodney stuckey, 22, player-pts), (rodney stuckey, 13, player-fga), (rodney stuckey, 8, player-fg) refine (knicks, 38, team-losses), (pacers, 31, team-losses), (knicks, 9, team-wins), (pacers, 17, team-wins), (knicks, 42, team-fg pct), (pacers, 53, team-fg pct), (pacers, 33, team-fg3 pct), (knicks, 31, team-fg3 pct), (knicks, 82, team-pts), (pacers, 103, team-pts), (carmelo anthony, 18, player-pts), (carmelo anthony, 16, player-fga), (carmelo anthony, 7, player-fgm), (carmelo anthony, 2, player-fg3m), (carmelo anthony, 4, player-fg3a), (ian mahinmi, 10, player-reb), (carmelo anthony, 25, player-min), (lou amundson, 17, player-pts), (rodney stuckey, 22, player-pts), (jose calderon, 9, player-pts), (jose calderon, 28, player-min), (jose calderon, 4, player-fgm), (jose calderon, 7, player-fga), (carmelo anthony, 2, player-ftm), (carmelo anthony, 2, player-fta), (carmelo anthony, 1, player-reb), (carmelo anthony, 1, player-ast), (roy hibbert, 10, player-reb), (roy hibbert, 18, player-pt) gold the indiana pacers ( 17 -31 ) throttled the new york knicks ( 9 -38 ) 103 -82 thursday at the bankers life fieldhouse . the knicks held a five -point deficit at halftime before having a complete meltdown in the third quarter . the pacers outscored the knicks 33 -12 in the third to all but ice the game . roy hibbert scored 10 of his 18 points in the third and added 10 rebounds in the contest to notch his 12th double -double on the season . new york earned an impressive 100 -92 win over oklahoma city on wednesday but was unable to carry that momentum into thursday 's contest against indiana . the knicks were unable to get much of anything going in the second half . carmelo anthony had n't scored less than 20 points in any game this month , but was held to 18 points on 7 -of -16 shooting against the pacers . the knicks have two days off before hosting the lakers on sunday . indiana was able to rebound nicely after its 104 -91 loss to toronto on tuesday . george hill and c.j miles were both able to play , but were on minutes restrictions and only saw 12 and 13 minutes respectively . rodney stuckey led the team in scoring , putting up 22 points on 8 -of -13 shooting . the pacers will look to secure their second straight win saturday when they host the kings . generated the pacers ( 17 -31 ) held on for a 103 -82 win over the knicks ( 9 -38 ) on friday . carmelo anthony led the way with a game -high 18 points on 7 -of -16 shooting , to go along with one rebound , one assist and one block in 25 minutes . jose calderon was the only other starter to score in double figures , scoring 10 points on 4 -of -7 shooting in 28 minutes off the bench . the knicks shot 53 percent from the field and 33 percent from three -point range . the knicks were led by carmelo anthony , who finished with 18 points ( 7 -16 fg , 2 -4 3pt , 2 -2 ft ) , one rebound , one assist and one block in 25 minutes . jose calderon added 10 points ( 4 -7 fg , 2 -3 3pt , 3 -3 ft ) , six assists , one rebound and one steal in 33 minutes . the knicks shot 53 percent from the field and 33 percent from three -point range . carmelo anthony was the high -point man for the knicks , with 18 points ( 7 -16 fg , 2 -4 3pt , 2 -2 ft ) , to go with one rebound , one assist , one steal and one block . jose calderon chipped in 10 points ( 4 -7 fg , 2 -3 3pt , 3 -3 ft ) , six assists , one rebound and one steal in 33 minutes . the knicks shot 53 percent from the field and 31 percent from three -point range . the knicks have now lost four of their last five games , as they continue to battle for the eighth seed in the eastern conference . they will look to get back on track as they play host to the bucks on sunday . the pacers have now lost four of their last six games , as they continue to battle for a playoff spot in the eastern conference . they will look to bounce back as they play host to the nets on sunday . proceedings of the 19th china national conference on computational linguistics, pages 881-891, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0