A Short Text Classification Algorithm Based on Semantic Extension

Chinese Journal of ElectronicsVolume 30, Issue 1 p. 153-159 Original Research PaperFree Access A Short Text Classification Algorithm Based on Semantic Extension Zhou Yajian, Corresponding Author Zhou Yajian yajian@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this authorDeng Dingpeng, Corresponding Author Deng Dingpeng dingpeng@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this authorChi Junhui, Corresponding Author Chi Junhui cjhwilliam714@edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this author Zhou Yajian, Corresponding Author Zhou Yajian yajian@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this authorDeng Dingpeng, Corresponding Author Deng Dingpeng dingpeng@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this authorChi Junhui, Corresponding Author Chi Junhui cjhwilliam714@edu.cn Beijing University of Posts and Telecommunications, Beijing, 100876 ChinaSearch for more papers by this author First published: 01 January 2021 https://doi.org/10.1049/cje.2020.11.014Citations: 1 This work is supported by the National Key Research and Development Project (No.2017YFB0802803). AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract A semantic-extension-based algorithm for short texts is proposed, by involving the Word2vec and the LDA model, to improve the performance of classification, which is frequently deteriorated by semantic dependencies and scarcity of features. For every keyword within a short text, weighted synonyms and related words can be generated by the Word2Vec and LDA model, respectively, and subsequently be inserted to extend the short text to a reasonable length. We not only have established a criterion by means of similarity estimation to determine whether a sentence should be extended, we designed a scheme to choose the number of extended words. The extended text will be classified. Experimental results show that, the classification performance of the proposed algorithm, in terms of the precision rate, is approximately 5% higher than that of the TF-IDF model and approximately 10% higher than that of the VSM method. I. Introduction Huge amounts of data emerges every second within social networks, e.g. Facebook, Weibo, Twitter, Instagram, etc., a large portion of which is reported to take the form of short texts and contain diverse user information, such as potential demand, interests, intentions and so forth[1]. Thus, how to extract useful knowledge has become a key issue in the field of natural language processing and machine learning, for the purpose of decision-making[2],topic tracking[3], public opinion detection[4], sentiment analysis[5] and personalized recommendation[6]. However, unlike ordinary texts, short texts are hard to be processed due to some particular features of their own, such as scarcity of features, complicated background noise, etc.[7]. As a result, satisfactory precision rate of classification can hardly be achieved. The Vector space model (VSM)[8], once the most popular method of text representation, treats each text as a random arrangement of a collection of terms, ignoring grammar and textual structure information, i.e., semantic information among words. Therefore, it is also called a bag-of-words. During the past several decades, VSM has played an important role in text classification for ordinary texts. But for short texts, VSM behaves no longer as effectively as they once did. Recently, two ideas have emerged to exploit the latent information contained within short texts. The first one tends to extend short texts based on the Search engine (SE)[9]. The SE can generate collections of related words and is capable of inducing high correlation between original features and extended ones. However, the idea of extension depends intimately on the search engine. As a result, the process of classification cannot be finished efficiently and timely. The second idea makes the choice to extend the features of short texts by loading large-scale external knowledge repositories, such as Wikipedia and CNKI, to enrich the semantic relationship among words. However, it is not a hard job to recognize that the effect of extension is determined by the quality of external knowledge bases. The contents that are absent in the bases will lose the opportunity of being extended and labeled samples of related fields are unavailable. What is worse, the method is computationally complex. On the basis of the aforementioned extensions, some scholars have attempted to use generative models to extract potential features from short texts, such as LSA (Latent semantic analysis), PLSA (Probabilistic latent semantic analysis) and LDA (Latent dirichlet allocation)[10-12]. Compared with previous methods, generation models are different in that they obtain the topic features of short texts by reasoning and fusing with the original features for training and classification. For example, Y. J. Hu et al.[13] utilize the LDA model to realize feature extension through the potential semantic association among words and documents; their results exhibit a certain degree of improvement compared to the traditional methods of applying LDA directly. Phan, Chen et al.[14] combine the two ideas mentioned above by using the LDA model to carry out implicit topic mining on Wikipedia to achieve short text classification. Although being capable of reducing the impact of scarcity of features on classification, the extension process proposed is time-consuming, and the classification effect is still dependent on the quality of the established corpus. Furthermore, such model construction methods operate at the "text" level[15], without taking into account the semantics of short texts. In 2013, Mikolov et al.[16] proposed the Word to Vector model (Word2vec) for calculating word vectors, which maps words to a low-dimensional space and generates corresponding word vectors by employing the semantic relationship among words and context. The more similar the words, the closer they are in the vector space. The application of Word2vec for natural language processing has been more and more popular in Chinese word segmentation, sentiment classification, and syntax dependence analysis, among others. The main contribution of this paper is that we propose a well-proven text extension mechanism, i.e., semantic extension, by efficiently merging the synonym calculation function of Word2vec and the related-word calculation function of the LDA model, which eventually leads to a novel short text classification method. The proposed method provides a solution to the urgent problem that original short texts are classified without any imperative attention on the scarcity of features, semantic dependencies of short texts. Experimental results offer a strong proof for the performance and the feasibility of our idea. The rest of the paper is organized as follows. In Section I, we explain the problem of short text classification and the purpose of this paper, followed by an overview of related works and a brief introduction to VSM, the Word2vec and LDA model in Section II. In Section III, the proposed short text classification algorithm is described in detail, whose performance is evaluated and analyzed by experiment in Section IV. Finally, we draw conclusions in Section V. II. Related Work 1. Vector space model and word2vec model The Vector space model (VSM)[17], proposed by Salton et al., is an algebraic model for text representation. It is widely used in information retrieval, information filtering, relevancy rankings and text classification, etc. The model transforms texts into vectors in order to degenerate texts analytics into conventional mathematical operations upon ordinary vectors, which is beneficial in that the complexity of the problem can be notably reduced. As for VSM, a document space is composed of a set of orthogonal term vectors. Here, we give some annotations of symbols: Vocabulary , where N denotes the total number of words; collections , where M represents the total number of collections. The vector representation of a document is W = w1,w2, …, wk, where wk indicates the weight of the word vk V in di and is usually estimated through the TF-IDF[18] weight evaluation function. The vector-space approach treats words in any sentence as independent features and ignores the relationship among words and the ordering of sentences. Moreover, the influence of position relationship on sentence semantics is neglected. Different from VSM, Word2vec is an open-source model, which is used to convert words into vector representations. The model has two main implementations: the Continuous bag-of-word model (CBOW model) and the Skip-gram model, both aiming to obtain high-quality word vectors and to simplify training steps to optimize the synthesis method, which apparently reduces the computational complexity. Either CBOW or Skip-gram model consists of the input layer, the hidden layer, and the output layer. The CBOW model utilizes the context wct of the word wt to predict the given word wt. However, the Skip-gram model predicts the context wct of a word provided that wt is known (see Fig.1). The Word2vec model can be employed for word extension representation. Furthermore, some research shows that the CBOW model is more suitable for small corpus, while Skip-gram performs better in large corpus[19]. Fig. 1Open in figure viewerPowerPoint CBOW and Skip-gram In this paper, we take the Skip-gram model as our choice to find synonyms and related words, followed by an insertion into the short texts. In addition, the advantage of the Word2vec is that words with similar meanings are also mathematically close in the vectors, thus the correlation among words can be maintained. 2. Latent Dirichlet allocation The LSA model does not solve the polysemy problem, while PLSA solves some problems existing based on LSA model. And the LDA model is improved on the basis of the PLSA model. As a result, the LDA model is selected in this paper to ensure better performance. In the LDA model, the process of word generation consists of two steps, as shown in Fig.2. The first step chooses a topic from the topic set, while the second one chooses a word according to the topic-word distribution under the selected topic. Some annotations of symbols are listed in Table 1. Fig. 2Open in figure viewerPowerPoint Diagram representation of LDA The probability of a topic, t, can be generateed as shown in Eq.(1). (1) Benefiting from the conjugate properties of the Dirichlet-polynomial distribution, we can eliminate the parameters θ and ϕ by integration. Then, the likelihood estimation of document m is shown in Eq.(2). (2) Because the documents are assumed to be independent of each other in the LDA model, the probability of generating the entire corpus is simply the multiplication of that generating each document, as shown in Eq.(3). (3) The two hyperparameters are generally set to , respectively. The number of parameters of the LDA model is only related to the number of topics and the number of words. Estimation of the parameters θ and ϕ requires calculation of the documents-topic probability distribution and the topic-word probability distribution successively. Indirect estimates for θ and ϕ can be obtained via Gibbs sampling. Table 1. Meaning of each symbol in the LDA model Symbol Quantity α Hyperparameter of θ β Hyperparameter of ϕ θ Collections-topic probability distribution ϕ Topic-words probability w Word N Number of words M Number of collections S Topic of word k Number of topic III. Short Text Classification Algorithm Based on Semantic Extension 1. Problem description As a representative traditional topic models, the LDA model has three levels, i.e., documents, topics, and words. It is likely that a single document may contain multiple topics, each of which is composed of words, using document-level co-occurrence information to reason the topic structure implicitly. When models are applied to long documents, words at the document level have more co-occurrence information and can be used to infer topic structures with higher quality. However, each short text contains limited number of words after preprocessing, e.g., deletion of stop words. When combined with LDA model, the available information, such as word frequency and word co-occurrence, is insufficient, leading to the problem of scarcity of features. As a result, for any inference algorithm, it is difficult to accurately estimate distribution parameters of topics, which may degrade the modeling quality. Therefore, we propose a classification algorithm for short texts based on semantic extension, i.e., extending the short text to a reasonably longer one, in order to improve the efficiency of classification. 2. Sentence extension based on Word2Vec A sentence is a basic type of linguistic unit, organized as a collection of words according to certain grammatical structures to express a specific meaning[20]. We introduce the concept of "local to global", in which the sentence similarity is based on the local similarity and the global similarity of sentences can be obtained via the local similarity. Thus, from another perspective, the calculation of sentence similarity should be attributed to the word similarity calculation. In this paper, we focus on the semantic structure of sentences in Chinese. Firstly, in the data pre-processing stage, we obtain the word collection via word segmentation and denoising. Secondly, the top N keywords with the largest TF-IDF (term frequency-inverse document frequency) weights can be singled out. Thirdly, the set of M synonym words is obtained by Word2vec for each word in the set. Finally, semantically extended words are added to the original text. To avoid adding noisy words due to an ill-founded corpus, we use a semantic similarity algorithm to calculate the text similarity value after semantic extension, which is compared with the original text. If the similarity value after extension is below the threshold value, it means that the semantic extension does not meet our requirement. The semantic similarity calculation method will be described in Section III.3. 3. Sentence similarity measure TF-IDF algorithm is usually considered to have three layers, and is commonly used for word frequency statistics. It also acts as a measure for keyword extraction, text summarization, text similarity, etc. It should be noted that TF-IDF is a computational method which does not take semantics into account. In this paper, we use this method to evaluate the similarity of sentences before and after text extension. In case the similarity value is above the threshold, semantic extension will be performed. Relevant parameters of TF-IDF include: 1) TF (Text frequency): The frequency with which a word appears in the document. 2) IDF (Inverse document frequency): The total number of documents divided by the number of documents in the corpus containing the word. 3) TF×IDF: It is proportional to the number of occurrences of a word in the document and inversely proportional to the number of occurrences of the word in the entire corpus. And the higher the importance of a word to a document, the larger the TF-IDF value will be. We use the TF-IDF algorithm to find the keywords of sentences. To avoid the influence of divergence in the length of sentences, a concept of relative word frequency is introduced, and the word frequency vector of two sentences can be calculated. Therefore, the word frequency vectors corresponding to all sentences are equal in length, and the elements in the same position correspond to the same word. Then, the cosine similarity of the two vectors are calculated, as shown in Eq.(4). A larger value means a higher similarity. (4) In this paper, the similarity is normalized to the interval [0, 1], so the cosine similarity is expressed, as shown in Eq.(5). (5) 4. Algorithm procedure In this paper, we trains the LDA model by the Sogou corpus, which will be introduced in detail later. The keywords of each class are obtained subsequently, and the corresponding class-topic dictionary is constructed. It is critical to select a well-edited set of large documents before training the LDA model. The algorithm framework is shown in Fig.3. Fig. 3Open in figure viewerPowerPoint Algorithm flowchart Based on the algorithm framework, the algorithm can be divided into five major modules. 1) Topic-dictionary training module: The Sogou corpus is used to train the word vectors based on the LDA model, which yields the topic and words probability distribution. 2) Word processing module: Word segmentation and denoising are carried out to generate word sets. 3) Semantic extension module: The algorithm mentioned in Section III.1 is used to extend the preprocessed keywords semantically. 4) Text similaity calculation module: In order to avoid the insertion of irrelevant noisy words into the extended text, it is necessary to calculate the similarity of the text before and after the extension. In this paper, the cosine similarity is used to calculate the similarity value. 5) Text classification module: The threshold of similarity value is set on the basis of experimental results, and it decides whether a short text should be extended or not. We also describe the algorithm in the form of pseudocode as illustrated by Algorithm 1, in order to make it more understandable. Algorithm 1 Short text classification Input: Short text to be classified trained classification model, corpus Output: Results of text classification Function Short_Text_Classification (Data): Initialization: Get a class-topic dictionary from the Sougou Corpus. While(true): get collection Data_A(Set A); break; for i=1 to DataA_N do: Find the words with the weight of the first N in set A if words in topic dictionaries: Extend the keyword use topic dictionaries. else: Extend the keyword use the Word2vec model end if end for while(true): Use the Eq.(8), get the similarity value k if : the extended text will be used for classification. else: the original text will be classified directly. return Results of text classification end function In Algorithm 1, we train the LDA model to generate six class-topic dictionaries, i.e., Education, Health, Military, Entertainment, Education and PE, denoted by C1, C2, C3, C4, C5, C6, respectively, as listed in Table 2. As for Sogou corpus, proportion of different classes is balanced, with approximately 10,000 texts for each category. IV. Experiments 1. Dataset We use the Sogou corpus, one of the most popular and largest corpora, as experimental data. Article titles are extracted as short texts, and the class to which each belongs are text labels. The dataset is presented in Table II. There are six classes, totally 63831 texts. And we split the dataset into training data and test data, and the ratio of training set and test set is 7:3. 2. Performance metrics In this paper, we employ the Softmax method as text classifiers, which are trained by the Sogou corpus. Classification performance will be measured by precision rate, Recall rate, and the F1 value. The performance metrics are defined as follows: 1) Precision rate The precision rate, Pr, is the ratio of the number of correctly classified samples in the classification results to the number of all classification samples. Table 2. Data collection Class Number of texts C1: Education 10773 C2: Health 10720 C3: Military 10600 C4: Entertaient 10500 C5: Finace 10438 C6: PE 10800 2) Recall rate Recall rate, denoted by Re, is the ratio of the number of correctly classified samples in the classification result to the actual number of texts in the classes. 3) F1 value The calculation formulas for the F1 value is as shown in Eq.(6). (6) 3. Parameter settings Number of keywords N: The outcome of a statistical analysis, with the purpose to investigate the length distribution of article titles within the Sogou corpus, is shown Fig.4, which demonstrates that short texts with 7-10 keywords account for at least 70% of the total number. Therefore, we set N to 7, which guarantees words with larger weights will be extended with priority. If , is the number of keywords of the text. Fig. 4Open in figure viewerPowerPoint The proportion distribution of number of keywords Number of semantic extension words M: The F1 values of the six classes with respect to different number of extension words are shown in Fig.5. From this figure, setting M to 4 seems to be an optimal decision. Choice of text similarity K: The K value measures the similarity of two short texts. The larger the K value, the more similar the two texts are. After semantic extension, the similarity value of the two texts may be reduced due to the possible involvement of some noisy words, which results in a lower precision rate of text classification. Therefore, it is a critical job to set the K value reasonably. Fig.6 illustrates the relationship between the K value and the classification performance. When the K value approaches 0.70, the best classification results can be achieved. So K is set to 0.70 in this paper. Fig. 5Open in figure viewerPowerPoint Effect of the M value on classification performance 4. Experimental result and analysis Classification performance of the proposed algorithm has been evaluated in terms of precision rate, recall rate and value of F1. The results are listed in Table 3. From Table 3, it can be found that the classification precision rate, recall rate and F1 value of the proposed method on the Softmax classifier are obviously improved compared with the TF-IDF and VSM model. Experimental results show that, in terms of precision rate, the proposed method is approximately 5% higher than that of the TF-IDF model and approximately 10% higher than that of the VSM method. Fig. 6Open in figure viewerPowerPoint Effect of the K value on classification performance We compare not only the performance of different algorithms, but also the performance improvement before and after words extension. Experimental results show that, in terms of precision rate, the proposed algorithm is approximately 12% higher than that without semantic extension. The results are listed in Table 4. Table 3. Recall, precision and F1 values of three models Class TF-IDF model Semantic extension model VSM model Pr(%) Re(%) F1(%) Pr(%) Re(%) F1(%) Pr(%) Re(%) F1(%) C1 81.12 87.32 82.53 89.25 83.49 86.27 75.31 74.13 74.72 C2 84.00 80.19 82.05 82.16 90.19 85.99 79.23 80.35 79.79 C3 75.31 82.77 78.86 79.36 88.38 83.63 73.90 78.56 76.16 C4 81.13 83.21 82.16 90.11 85.45 87.72 74.74 70.90 72.77 C5 80.10 83.15 81.60 85.21 90.21 87.64 78.45 77.00 77.72 C6 75.89 82.33 78.98 78.23 80.32 79.26 70.42 71.78 71.09 avg 79.59 83.16 81.03 84.05 86.34 85.09 75.34 75.45 75.38 Table 4. Recall, precision and F1 value of the extension model Class Not use extension Semantic extension model Pr(%) Re(%) F1(%) Pr(%) Re(%) F1(%) C1 73.24 76.00 74.59 89.25 83.49 86.27 C2 73.64 78.21 75.86 82.16 90.19 85.99 C3 68.38 71.35 69.83 79.36 88.38 83.63 C4 75.49 79.88 77.62 90.11 85.45 87.72 C5 68.40 74.87 71.49 85.21 90.21 87.64 C6 70.29 73.32 71.77 78.23 80.32 79.26 avg 71.57 75.60 73.53 84.05 86.34 85.09 V. Conclusions The semantic extension mechanism proposed in this paper focuses on the generation of appropriate synonyms and related words required to extend short texts. We can conclude that our attempt to realize extension by means of the Word2vec and the LDA model is successful. Meanwhile, it is feasible to determine whether a short text should be extended by the criterion and we designed a scheme to choose the number of extended words. The most important conclusion at which we arrive in this paper is that extended texts actually become more semantically distinguishable, which not only demonstrated by experimental results in terms of higher classification precision rate compared with the method without semantic extension, but also proved by an obvious improvement of classification efficiency compared with VSM and TF-IDF model. Biographies ZHOU Yajian obtained the Ph.D. degree in communications engineering from Xidian University at Xi'an in 2003. He received the M.S. degree in 1996 and B.S. degree in 1993, both from Beihang University. He is currently an associate professor in the School of Computer Science of Beijing University of Posts and Telecommunications (BUPT). His main research interests include mobile communications (e.g., LTE, D2D, and WLAN), security of wireless networks, security of databases, and cryptography theory and its application. He is now in charge of two state-level projects: one from the National Natural Science Foundation of China and another one from the National Science and Technology Major Project of China. He was once the project leaders of the National 863 High-Tech Research and Development Plan of China and a project from Beijing Municipal Natural Science Foundation. He has published approximately 50 technical papers and 6 books. DENG Dingpeng (corresponding author) received the B.S. degree in network engineering from Beijing Information Science and Technology University, China, in 2013, and the M.S. degree in big data from Beijing University of Posts and Telecommunications, China, in 2017. His current research interests include information processing, natural language processing, cloud computing, and network security. CHI Junhui received an M.S. degree in big data and intelligent information processing from Beijing University of Posts and Telecommunications, China in 2018. From 2018 to 2019, he was a teaching assistant of discrete mathematics. His research interests include natural language processing, statistical learning theory and brain-like computing. Mr. Chi is a student member of the China Computer Federation (CCF). References 1T. Naiem, W. Stephen, et al., "Big data: The next frontier for innovation in therapeutics and healthcare", Expert Rev Clin Pharmacol, Vol. 7, No.3, pp. 293– 298, 2015. 2M.J. Sousa, A.M. Pesqueira, et al., "Decision-making based on big data analytics for people management in healthcare organizations", Journal of Medical Systems, Vol. 43, No.9, pp. 1– 10, 2019. 3W.Tianshu, C.Shuyu, et al., "A feature optimized deep learning model for clinical data mining", Chinese Journal of Electronics, Vol. 29, No.3, pp. 476– 481, 2020. 4Norris R J and Mullinix K J, "Framing innocence: An experimental test of the effects of wrongful convictions on public opinion", Journal of Experimental Criminology, DOI:10.1007/s11292-019-09360-7, 2019. 5L.Bingyu, W.Cuirong, et al., "Microblog topic mining based on FR-DATM", Chinese Journal of Electronics, Vol. 27, No.2, pp. 334– 341, 2018. 6W. Zhou and W. Han, "Personalized recommendation via user preference matching", Information Processing & Management, Vol. 56, No.3, pp. 955– 968, 2019. 7H.B. Peng, H.Y. Shao, et al., "General improved SSD model for picking object recognition of multiple fruits in natural environment", Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering, Vol. 34, No.16, pp. 155– 162, 2018. 8K. Dorota and S. Vishvajit, "Improvement of manual assembly line based on Value stream mapping (VSM) and efectiveness coefficient", Quality Production Improvement-QPI, Vol. 1, No.1, pp. 537– 544, 2019. 9Z. Wang, "An improved method of short text feature extraction based on words cooccurrence", Proc. of the 2nd International Conference on Electric Technology and Civil Engineering, Wuhan, Hubei, China, pp. 18– 20, 2012. 10ST. Dumais, "Latent semantic analysis", Annual Review of Information Science and Technology, Vol. 38, No.1, pp. 188– 230, 2005. 11T. Hofmann, "Probabilistic latent semantic indexing", Proc. of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, CA, USA, pp. 50– 57, 1999. 12D.M. Blei, A.Y. Ng and M.I. Jordan, "Latent Dirichlet allocation", Machine Learning Research Archive, Vol. 3, No.6, pp. 993– 1022, 2003. 13Y.J. Hu, J.X. Jiang, et al., "A new method of keywords extraction for Chinese short-text classification", New Technology of Library and Information Service, Vol. 6, No.1, pp. 42– 48, 2013. 14M.G. Chen, X.M. Jin and D. Shen, "Short text classification improved by learning multigranularity topics", Proc. of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Spain, pp. 16– 22, 2011. 15W.Haitao, H.Jie, et al., "A short text classification method based on N-gram and CNN", Chinese Journal of Electronics, Vol. 29, No.2, pp. 248– 254, 2020. 16Q. Le and T. Mikolov, "Distributed representations of sentences and documents", Proc. of the 31st International Conference on Machine Learning, Beijing, China, pp. 1188– 1196, 2014. 17G. Salton, "A vector space model for auto-matic indexing", Communications of the ACM, Vol. 18, No.11, pp. 613– 620, 1975. 18M. Sahami and T.D. Heilman, "A Web - based kernel function for measuring the similarity of short text snippets", Proc. of the 15th Conference on World Wide Web, New York, USA, pp. 377– 386, 2006. 19T. Mikolov, T. Sutskever, K. Chen, et al., "Distributed representations of words and phrases and their compositionality", Proc. of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, pp. 3111– 3119, 2013. 20A. Omaki, "Grammatical constraints and reductionism in sentence processing", Linguistic Approaches to Bilingualism, Vol. 3, No.3, pp. 330– 334, 2013. Citing Literature Volume30, Issue1Special Issue: DATA-DRIVEN INDUSTRIAL INTELLIGENCEJanuary 2021Pages 153-159 FiguresReferencesRelatedInformation