Communicative-Function-Based Sentence Classification for Construction of an Academic Formulaic Expression Database (2021)

Abstract: Formulaic expressions (FEs), such as ‘in this paper, we propose’ are frequently used in scientific papers. FEs convey a communicative function (CF), i.e. ‘showing the aim of the paper’ in the above-mentioned example. Although CF-labelled FEs are helpful in assisting academic writing, the construction of FE databases requires manual labour for assigning CF labels. In this study, we considered a fully automated construction of a CF-labelled FE database using the top–down approach, in which the CF labels are first assigned to sentences, and then the FEs are extracted. For the CF-label assignment, we created a CF-labelled sentence dataset, on which we trained a SciBERT classifier. We show that the classifier and dataset can be used to construct FE databases of disciplines that are different from the training data. The accuracy of in-disciplinary classification was more than 80%, while cross-disciplinary classification also worked well. We also propose an FE extraction method, which was applied to the CF-labelled sentences. Finally, we constructed and published a new, large CF-labelled FE database. The evaluation of the final CF-labelled FE database showed that approximately 65% of the FEs are correct and useful, which is sufficiently high considering practical use.

## introduction formulaic expressions (fes), such as 'in this paper we propose', are a type of multi-word expressions and are repeatedly used in scientific papers. some fes convey a communicative function (cf) of a sentence, which represents intentions of authors. for example, 'in this paper, we propose' conveys the cf of 'showing the aim of the paper'. databases comprising cf-labelled fes are required from a pedagogical perspective @xcite , and a computer-based academic writing assistance system foot_0 that uses such cf-labelled fes has been proposed @xcite . several attempts have been made to extract fes from scientific corpora and categorise them based on cfs @xcite @xcite @xcite . a cflabelled fe database can be constructed using two main approaches: top-down and bottom-up approaches @xcite . by using the top-down approach, sentences are first assigned cf labels, and then fes are extracted, while in the case of the bottomup approach, fes are first extracted and then assigned cf labels. to date, both the approaches have been adopted because cf assignment is performed manually ( method for creating db db statistics approach cf fe discipl. #cfs #docs #fes simpson-vlach and ellis (2010) bottom-up manual corpus mixed 15 -200 morley (n. d.) -manual manual mixed 146 100 2, 000 mizumoto et al. (2017) top-down manual corpus specific 52 1,000 -lu et al. (2018) bottom-up manual corpus mixed 12 600 454 ours top-down automated sentence specific 32 61,728 86,931 for cf-based sentence classification, we created a dataset for supervised learning. the dataset consists of a small number of sentences that were assigned cf labels. we collected the sentences from scientific papers of multiple disciplines. by using this dataset, we fine-tuned scibert @xcite . additionally, because there are preferences for cf usage depending on disciplines and as the preparation and coverage of all cfs of every discipline are difficult, sentences to which any prepared cf label should not be assigned may appear in a corpus (no-cf sentences). these no-cf sentences will have a negative effect on the classification performance. based on the recent work on out-of-distribution detection in natural language processing @xcite , we used the maximum value of the softmax layer as the threshold to filter no-cf sentences in order to improve the final precision. the experimental results show that the maximum value of the softmax layer works well as the threshold to filter out undesirable sentences. we carefully considered multidisciplinary problems in the classification. although the development of a training dataset for every discipline in the world is obviously impossible, demonstrating a successful classification using a single disciplinary dataset is not sufficient for practical use. in this study, we determined whether a model trained on a corpus of one discipline can be applied to that of another discipline. moreover, the effects of a pre-training dataset were examined by comparing scibert and bert @xcite . the experimental results show that the classifiers performed fairly well in terms of both in-discipline and cross-discipline data, and the performance was only slightly affected when scientific papers were not used as pre-training data. for the fe-extraction process, one fe should be extracted from one sentence because cf labels are assigned to each sentence; this is termed as sentence-level approach (see section 2.2). therefore, we propose a sentence-level fe extraction method that is based on an existing method @xcite . the method consists of three steps: named and scientific entity removal, dependency-structure-based word removal, and word-association-measure-based word removal. finally, we created a new, large, multidisciplinary cf-labelled fe database and evaluated it by asking human evaluators whether each instance was assigned a correct cf label and whether an fe was useful for writing a paper. the results show that approximately 65% of the collected fes are appropriate. the contributions of our study are as follows: ## conclusion in this paper, we proposed the fully-automated construction of a cf-labelled fe database, by solving the problem of cf-based sentence classification. we carefully considered a practical case of creating a fe database of other disciplines. the experimental results showed that the proposed classifi-cation method and dataset can be utilised to construct fe databases for disciplines different from those that we used. we proposed the fe extraction method that utilised the named and scientific entity removal, dependency-structure-based word removal, and word-association-measure-based word removal. combining the proposed methods, we finally constructed the new cf-labelled fe database. the cf-labelled sentence database and the cflabelled fe database are available on our website 8 . we expect that the proposed database could be used by pedagogical practitioners and for computeraided academic-writing assistance such as sentence retrieval and automated proofreading. john malcolm swales. 1990. genre analysis: english in academic and research settings. cambridge university press. john malcolm swales. 2004. research genres: explorations and applications. cambridge university press. simone teufel. 1999. argumentative zoning: information extraction from scientific text. ph.d. thesis, university of edinburgh. simone teufel, advaith siddharthan, and colin batchelor. 2009. towards domain-independent argumentative zoning: evidence from chemistry and computational linguistics. in proceedings of the 2009 conference on empirical methods in natural language processing, pages 1493-1502, singapore. association for computational linguistics. benet vincent. 2013. investigating academic phraseology through combinations of very frequent words: a methodological exploration. journal of english for academic purposes, 12(1):44-56.