Bootstrapping Named Entity Recognition in E -Commerce with Positive Unlabeled Learning (2020)

Abstract: In this work, we introduce a bootstrapped, iterative NER model that integrates a PU learning algorithm for recognizing named entities in a low-resource setting. Our approach combines dictionary-based labeling with syntactically-informed label expansion to efficiently enrich the seed dictionaries. Experimental results on a dataset of manually annotated e-commerce product descriptions demonstrate the effectiveness of the proposed framework.

## introduction the vast majority of existing named entity recognition (ner) methods focus on a small set of prominent entity types, such as persons, organizations, diseases, and genes, for which labeled datasets are readily available @xcite @xcite . there is a marked lack of studies in many other domains, such as e-commerce, and for novel entity types, e.g. products and components. the lack of annotated datasets in the ecommerce domain makes it hard to apply supervised ner methods. an alternative approach is to use dictionaries @xcite , but freely available knowledge resources, e.g. wikidata @xcite or yago @xcite , contain only very limited information about e-commerce entities. manually creating a dictionary of sufficient quality and coverage would be prohibitively expensive. this is amplified by the fact that in the e-commerce domain, entities are frequently ex-pressed as complex noun phrases instead of proper names. product and component category terms are often combined with brand names, model numbers, and attributes ("hard drive" → "ssd hard drive" → "wd blue 500 gb ssd hard drive"), which are almost impossible to enumerate exhaustively. in such a low-coverage setting, employing a simple dictionary-based approach would result in very low recall, and yield very noisy labels when used as a source of labels for a supervised machine learning algorithm. to address the drawbacks of dictionary-based labeling, @xcite propose a positive-unlabeled (pu) ner approach that labels positive instances using a seed dictionary, but makes no label assumptions for the remaining tokens @xcite . the authors validate their approach on the conll, muc and twitter datasets for standard entity types, but it is unclear how their approach transfers to the ecommerce domain and its entity types. contributions we adopt the pu algorithm of @xcite to the domain of consumer electronic product descriptions, and evaluate its effectiveness on four entity types: product, component, brand and attribute. our algorithm bootstraps ner with a seed dictionary, iteratively labels more data and expands the dictionary, while accounting for accumulated errors from model predictions. during labeling, we utilize dependency parsing to efficiently expand dictionary matches in text. our experiments on a novel dataset of product descriptions show that this labeling mechanism, combined with a pu learning strategy, consistently improves f1 scores over a standard bilstm classifier. iterative learning quickly expands the dictionary, and further improves model performance. the proposed approach exhibits much better recall than the baseline model, and generalizes better to unseen entities. algorithm 1: iterative bootstrapping ner input: dictionary d seed , corpus c, threshold k, max iterations i result: dictionary d + , classifier l d + ← d seed ; c dep ← dependency parse(c); i ← 0; while not converged(d + ) and @xmath0, d + ); c exp ← expand labels(c lab , c dep ); l ← train classif ier(c exp ); c pred ← predict(c exp , l); for e ← c pred do if e / ∈ d + and freq(e) > k then d + ← add entity(d + , e); end end i ← i + 1; end 2 ner with positive unlabeled learning in this section, we first describe the iterative bootstrapping process, followed by our approach to positive unlabeled learning for ner (pu-ner). ## experiments to evaluate our proposed model (pu), we compare it against two baselines: (1) dictionary-only labeling (dictionary), and (2) our model with standard cross-entropy loss instead of the pu learning risk (bilstm). the bilstm model is trained in a supervised fashion, treating all non-dictionary entries as negative tokens. the bilstm and pu models were implemented using allennlp @xcite . we use spacy foot_0 for preprocessing, dependency parsing, and dictionary-based entity labeling. we manually define seed dictionaries for product (6 entries), component (60 entries) and brand (13 entries). for attributes, we define a set of 8 regular expressions to pre-label the dataset. following previous works, we evaluate model performance using token-level f1 score. there are two options to estimate the value of the class prior π p . one approach is to treat π p as a hyperparameter which is fixed during training. another option is suggested in @xcite , who specify an initial value for π p to start bootstrapping, but recalculate π p after several train-relabel steps based on the predicted entity type distribution. in our work, we treated π p as a fixed hyperparameter with a value of π @xmath1.01. we run our bootstrapping approach for @xmath2, and report the f1 score of the best iteration. ## related work recent work in positive-unlabeled learning in the area of nlp includes deceptive review detection @xcite , keyphrase extraction @xcite and fact check-worthiness detection @xcite , see also @xcite for a survey. our approach extends the work of @xcite in a novel domain and for challenging entity types. in the area of ner for e-commerce, @xcite present an approach to extract product attributes and values from product listing titles. @xcite formulate missing attribute value extraction as a sequence tagging problem, and present a bilstm-crf model with attention. @xcite studies the problem of product name recognition, but uses a fully supervised approach. in contrast, our method is semi-supervised and uses only very few seed labels. ## conclusion in this work, we introduce a bootstrapped, iterative ner model that integrates a pu learning algorithm for recognizing named entities in a low-resource setting. our approach combines dictionary-based labeling with syntactically-informed label expansion to efficiently enrich the seed dictionaries. experimental results on a dataset of manually annotated e-commerce product descriptions demonstrate the effectiveness of the proposed framework.