IR & TM - NJUST @ CLS ci S umm 20 (2020)

Abstract: This paper mainly introduces our methods for Task 1A and Task 1B of CL-SciSumm 2020. Task 1A is to identify reference text in reference paper. Traditional machine learning models and MLP model are used. We evaluate the performances of these models and submit the final results from the optimal model. Compared with previous work, we optimize the ratio of positive to negative examples after data sampling. In order to construct features for classification, we calculate similarities between reference text and candidate sentences based on sentence vectors. Accordingly, nine similarities are used, of which eight are chosen from what we used in CL-SciSumm 2019 and a new sentence similarity based on fastText is added. Task 1B is to classify the facets of reference text. Unlike the methods used in CL-SciSumm 2019, we construct inputs of models based on word vectors and add deep learning models for classification this year.

## introduction the rapid growth of papers has provided scholars with various knowledge and methods, which can offer references for development or innovation of the research. but it makes difficult for researchers to get brief summaries quickly from such massive amount of papers @xcite . automatic summarization can solve this problem. researchers express their views on reference paper through citation text. so, citation text can be used to generate summary of paper @xcite . however, as a result of researchers' different views (citation), the quality of the summary is not guaranteed and the summary cannot fully restore the original  corresponding author. given: a topic consisting of a reference paper (rp) and citing papers (cps) that all contain citations to the rp. in each cp, the text spans (i.e., citances) have been identified that pertain to a particular citation to the rp. task 1a: for each citance, identify the spans of text (cited text spans) in the rp that most accurately reflect the citance. these are of the granularity of a sentence fragment, a full sentence, or several consecutive sentences (no more than 5). task 1b: for each cited text span, identify what facet of the paper it belongs to, from a predefined set of facets. task 2 (optional bonus task): finally, generate a structured summary of the rp from the cited text spans of the rp. the length of the summary should not exceed 250 words. in figure 1 , the blue text span in the citing paper shows the citation text, and the green text span in the reference paper shows the reference text which most accurately reflects the citance. our team has participated in the cl-scisumm competition in 2017 @xcite , 2018 @xcite and 2019 @xcite . for task 1a, a similarity-based negative sampling strategy is applied to construct the training set. nine similarity features and sentence vectors are used to represent citation text and candidate sentences. then we employ traditional machine learning methods and build mlp model to identify the reference text in reference papers. for task 1b, sentence vectors are generated based on word frequency and word vector. traditional machine learning models and deep learning models are built to identify the facets. as for task 2, cosine similarity is calculated between reference sentences and the original abstract based on their sentence vectors. then sentences are selected to construct summary according to their similarities, and length of the summary does not exceed 250 words. compared with previous work, we make changes in following steps. in task 1a, we optimize ratio of positive to negative examples after negative sampling. the structure and parameters of mlp model are adjusted to get better results. for task 1b, we first try to use word vector to construct inputs of models. and the result has been improved about 10% at accuracy score. ## methodology before introducing the methodology of each task, we define some concepts to avoid ambiguity in the following description. ## experiments and results analysis in this section, we report the results of different models in task 1a and task 1b. ## conclusion and future work in task 1a, training data and test data are constructed by negative sampling. and the ratio of positive to negative examples has been optimized. next, we use deep learning model (mlp) with the input of sentence vectors and traditional machine learning models based on nine similarity features to identify the reference text. the effect of mlp is proved to be better than that of traditional machine learning models. as for task 1b, we calculate different combinations of sentence vectors as input. traditional machine learning models and deep learning models have been evaluated on classifying the facets of reference text. in this process, the effect of using pre-training model (bert) to obtain word vector is worse than that of using fasttext to train word vector based on training set. and traditional machine models (lightgbm and xgboost) work better than deep learning models. generally, word vectors can reflect more semantic information compared to traditional machine learning features. we create a suitable number of training data by negative sampling in task 1a, so deep learning model (mlp) works better. while in task 1b, insufficient training data makes deep learning models inferior to traditional machine learning models. in future work, we can optimize training set through data augmentation technology and apply other deep learning models for task 1a. as for task 1b, its recognition result is affected by the imbalance of data. we will try to expand the training data for the facets with smaller data scale from other data sources, such as structured abstract.