Regularized Graph Convolutional Networks for Short Text Classification (2020)

Abstract: Short text classification is a fundamental problem in natural language processing, social network analysis, and e-commerce. The lack of structure in short text sequences limits the success of popular NLP methods based on deep learning. Simpler methods that rely on bag-of-words representations tend to perform on par with complex deep learning methods. To tackle the limitations of textual features in short text, we propose a Graph-regularized Graph Convolution Network (GR-GCN), which augments graph convolution networks by incorporating label dependencies in the output space. Our model achieves state-of-the-art results on both proprietary and external datasets, outperforming several baseline methods by up to 6% . Furthermore, we show that compared to baseline methods, GR-GCN is more robust to noise in textual features.

## introduction short-text classification is a common problem in information retrieval @xcite and has applications in several domains including e-commerce @xcite , social media @xcite , healthcare @xcite and cognitive-biometric recognition @xcite . in this paper, we develop a short text classification technique for solving two problems relevant to product search on e-commerce platform: 1) product query classification (pqc) -when the customer enters a free form query, it is important to understand their product type intent to recommend and advertise the relevant products. we classify customer search queries to one or more product types (e.g., shoe, televisions, skis), and 2) product title classification (ptc) -we classify billions of product titles to one or more product categories. this is important for sellers to place their items in the correct product category and retrieve it when a customer queries it. unlike traditional text classification, classifying short-texts poses additional challenges. first, short texts in e-commerce typically involve sentences with an average length of 3 (for queries) to 15 words (for product titles). second, unlike longer texts such as blogs or news articles, these customer queries or product titles lack "natural" language structure and are often plagued with spelling errors. for example, in pqc, queries like nike running, shoes size 9, nike shos (misspelling variants) all belong to the shoes category. in addition, queries contain non-target language text and non-language text (like model/part numbers), which introduces noise in the embedding. a similar challenge could also be faced in ptc problem, where products from the same class have high diversity in their title texts. for example, titles photofast microsd to ms pro duo cr-5300, kingston microsd card and 8gb card for blackberry storm 9530 all belong to the same genre of microsd card products and hence need to be listed under the same category. all these factors make it difficult to separate product-type classes by purely relying on text which is heterogeneous and contains noise. in this work, we propose to enhance the textual information by leveraging additional knowledge about relationships between input short-texts as well as among class labels. for pqc, we can derive similarities between input user queries from (anonymized) user logs, by looking at commonly purchased items in response to different queries. the intuition is that two queries that consistently lead to the purchase of a same set of items might have similar product-type intents. likewise, in ptc, we can estimate similarity between two input product titles from historical information such as co-views. similarly, in output space, relationships between product-type classes can be modeled using product-category taxonomies, which are typically hand-curated and readily available in e-commerce applications. such auxiliary information can be naturally represented in graphical form, where each node represents a short-text (input graph) or a class label (output graph), while an edge indicates magnitude of similarity between two nodes. we thus propose a graph-regularized graph convolution network (gr-gcn) approach, which augments the graph convolutional network @xcite to incorporate such graphical information in an end to end learning framework. the two key aspects of gr-gcn are: i) a gcn that leverages dependencies in the input space to learn more informative representations of nodes(input short texts), and ii) a graph-regularization (gr) term in the objective function that exploits label similarities to penalize contrasting predictions for similar class labels on each input sample, thereby restricting the solution space and making our approach more robust to noise in the data. we perform extensive experiments on one proprietary, and two public datasets and demonstrate the improvement in classification accuracy for gr-gcn upto 6% compared to text-based baselines. further, we add noise in the input data and show that the graph's presence makes our method more robust to noise as compared to baseline methods based on just textual features. ## proposed approach let @xmath0. here d is the dimension of embedding and l number of class labels. let g @xmath1, e i ) be the graph on input samples, with a corresponding adjacency matrix ## conclusion in this paper, we propose gr-gcn to classify short texts that capture dependency on two levels, i.e., within the text samples (input space graph) and amongst the output label (output space graph). we demonstrated its efficacy on two commercial e-commerce applications and its robustness to noise. we note that the proposed method can add value to other domains like medical science, where the input graph can capture drug similarity, and the output graph can capture the relationship among various types of illness; remote sensing, where the input graph can capture the distance, depth etc. between different ground points and output graph can capture similarity between similar labels such as pasture and vegetation as well as distinguish between entirely different labels such as river and residence.