Market Comment Generation from Data with Noisy Alignments (2020)

Abstract: End-to-end models on data-to-text learn the mapping of data and text from the aligned pairs in the dataset. However, these alignments are not always obtained reliably, especially for the time-series data, for which real time comments are given to some situation and there might be a delay in the comment delivery time compared to the actual event time. To handle this issue of possible noisy alignments in the dataset, we propose a neural network model with multi-timestep data and a copy mechanism, which allows the models to learn the correspondences between data and text from the dataset with noisier alignments. We focus on generating market comments in Japanese that are delivered each time an event occurs in the market. The core idea of our approach is to utilize multi-timestep data, which is not only the latest market price data when the comment is delivered, but also the data obtained at several timesteps earlier. On top of this, we employ a copy mechanism that is suitable for referring to the content of data records in the market price data. We confirm the superiority of our proposal by two evaluation metrics and show the accuracy improvement of the sentence generation using the time series data by our proposed method.

## introduction in recent time, various industries such as finance, pharmaceuticals and telecommunications have increased opportunities to treat large-scale data. hence, there is an increasing demand to automatically generate a text from large and complex data. in recent studies, neural network-based models have achieved significant progress on the data-to-text which is a text generation task from input data @xcite @xcite . one important issue in constructing a dataset for data-to-text is to obtain the correct alignment between data and text. it is not very problematic when there is a clear correspondence between data and text, for example, when the text is manually provided by an annotator for each input, including e2e nlg challenge dataset @xcite . however, this is often not a trivial problem in the wild, in particular for the application of real-time text generation, such as sequential comment generation on sports games @xcite and stock markets @xcite , for which we can only obtain a loose alignment of data and text. this problem has been taken into account in a classical task @xcite , but has been overlooked in the recent neural-based models cited above. in @xcite , for example, they constructed a dataset for market comment generation in japanese from a chart of stock price trends and its market reports, wherein the alignments between data and texts sometimes deviate. figure 1 presents examples of data which are a five-minute chart of nikkei 225 (nikkei stock average) and comments describing the chart trends. all of the (i) to (iii) comments are about an event at "9:00 am, 29th of january". however, since there is often a delay in comment delivery time for an event (e.g., (ii) and (i ii)), if a different movement occurs during this time period, the expressions in comment may not exactly reflect the movement at the delivery time. the word "rebound" indicates a downward and then upward movement on three points (the closing prices of the last two days and the latest price). this is valid for the prices at (i) and (ii), but does not hold at (iii) because the latest price (17039.22 yen) is lower than the last closing price (17041.45 yen). in addition, the expression "gains 88 yen" is only valid at the opening time (i) and is not valid at (ii). to deal with these inconsistencies, the models have to be aware of these possible mismatch of data and text due to the delay, but a simple encoder-decoder-based delivery time price movement comment (bold text: movement expression) (i) 09:00 rebound gained 86 yen nikkei stock average starts with rebound, gains 86 yen. (ii) 09:05 rebound gained 22 yen nikkei stock average starts with rebound, gains 86 yen, a positive response in high price of crude oil. the stock price of toyota increased. (iii) 09:19 continuous fall dropped 2 yen tosho begins with rebound. a positive response in high price of crude oil, fanuc share prices dropped significantly. model of murakami et al. ( 2017 ), which does not tell the difference between the event and delivery times, may perform an undesirable generalization between data and text. in this study, we extend @xcite 's model with a new architecture to solve the problem due to the noisy alignments. the presented architecture is a multi-timestep architecture, which employs multiple input vectors to compensate the lack of information about the actual event time, treating it as a hidden variable and learning the correspondences from the ambiguous data. our model employs a copy mechanism to generate a price value in a text, in which an attention weight to the time can be regarded as an induced alignment between the text and actual event time. the experimental results reveal that our proposed model outperforms the existing model in terms of the correctness of market price movement expressions, in addition to the bleu scores. ## generating market comments in this section, we explain the model proposed by @xcite , which be used as a base model. they proposed a model for generating a market comment, which is a news headline about the movement of the nikkei 225, from the timeseries of the stock price of the nikkei 225. their model is based on the encoder-decoder @xcite . @xcite use both a long-term vector x long and a short-term vector x short . to capture the long-term price movement, they use a vector consisting of the closing prices of the preceding trading days represented as x @xmath0, 1 , long, 2 , . . . , long, ). similarly, to capture the short-term price movement, they use a vector consisting of the prices of previous timesteps on the five-minute chart, starting from the comment delivery time (e.g., 10:00 am), represented as x @xmath1, 1 , short, 2 , . . . , short, ). each of these vectors undergoes the following two preprocessing steps : ## related work the task of generating text describing input data, which is called data-to-text, has been worked on various domains, for instance weather forecasts @xcite , healthcare @xcite , and sports @xcite . traditionally, data-to-text is divided into two sub-problems @xcite : content selection, which is about "what to say", and surface realization, which is about "how to say". moreover, @xcite divides three modules, adding micro planning between the above sub-problems. in the early stage of this task, surface realization is often realized using templates @xcite or statistically learned models with hand-crafted features @xcite . in recent times, various industries such as finance, pharmaceuticals, and telecommunications have been increased providing opportunities to treat various types of large-scale data, so that they are interested in automatically learning a correspondence relationship from data to text and generating a description of this relationship. therefore, recent works have focused on generating text from data with neural networks, that can solve the above sub-tasks in one through. especially the models, which utilize an encoder-decoder model @xcite have proven to be useful @xcite . while text generation by neural network can describe the text fluently, they do not describe the exact entities or numbers. therefore, a copy mechanism @xcite , which provides a way to directly copy words from the input, has been utilized. by these neural networks, the works such as conditional language generation based on tables @xcite , short biographies generation from wikipedia tables @xcite @xcite and sports scoreboards @xcite @xcite are well performed. contrastingly, they can only generate the superficial contents that appear in their input table, and cannot generate contents that require arithmetic operations. however, @xcite and @xcite indicate that current neural models have difficulties in learning arithmetic operations such as addition and comparisons by neural program inductions. thus, there have been some methods to prepare the numerical values with arithmetic operations in advance. @xcite post-process the price by extending the copy mechanism and replacing numerical values with defined arithmetic operations after generation. @xcite utilizes information from pre-computed operations on raw data to consider incorporating the facts that can be inferred from the input data to guide the generation process. our model prepares numerical values with defined arithmetic operations as @xcite for copy and that copy target is guided by encoded input. ## conclusion in this paper, we have proposed an encoder-decoder model with multi-timestep data and a copy mechanism for generating the market comment from data with the noisy alignments. both bleu scores and our proposal evaluation showed the accuracy of sentence generation with time-series data has been improved by our proposed method, especially utilizing a copy mechanism.