BioCreative II Gene Mention Tagging System at IBM Watson

This paper describes our system developed for the BioCreati ve II gene mention tagging task. The goal of this task is to annotate mentions of genes or gene products in the g iven Medline sentences. Our focus was to experiment with a semi-supervised learning method, Alternating Structure Optimization (ASO) [1], by which we exploited a large amount ofunlabeled data in addition to the labeled training data provided by the orga nizer. The system is also equipped with automatic induction of high-order features, gene name lexicon lookup, classifier combination, and simple postprocessing. Our system appears to be competitive. All of our three official runs belong to the Quartile 1. 1 Gene mention tagging system Our gene mention tagging system was built on top of a named ent ity chunking system described in [1], which was used for annotating names of persons, organizations, an d so forth. This system casts the chunking task into that of sequential labeling, as is commonly done, by enc oding chunk information into token tags. It uses a regularized linear classifier with modified Huber loss and t he 2-norm regularization. That is, using the ‘oneversus-all’ scheme, we train binary classifiers, one for eac h token tag, usingn labeled data points {(xi, yi)} for i = 1, . , n by: ŵ = arg minw ∑ n i=1 L(wTxi, yi) + λ||w|| 2 . The regularization parameter λ is set to10. L is the loss function:L(p, y) = max(0, 1 − py) if py ≥ −1; and−4py otherwise. The optimization is done by stochastic gradient descent. Viterbi-style dynamic pro gramming is performed to find the token tag sequence with the largest confidence. Feature types are shown in Figur e 1. Using this framework, we experimented with additional resources and algorithms, which we describe bel ow. · words, parts-of-speech, character types, 4 characters at b eginning/ending in a 5-word window · words in a 3-syntactic chunk window. · labels assigned to two words on the left. · bi-grams of the current word and the left label. · labels assigned to previous occurrences of the current word . Figure 1:Feature types. 1.1 Exploiting unlabeled data through Alternating Structure Optimization (ASO) ASO is a multi-task learning algorithm that seeks to improve performance on individual tasks by simultaneously learning multiple tasks that are related to each other. The a pplication of ASO to semi-supervised learning involves automatic generation of thousands of prediction pro blems (called ‘auxiliary problems’) and their labeled data from unlabeled data, so that the multi-task learning al gorithm can be applied on the unlabeled data. To put this into perspective, ASO-based semi-supervised le arning can be viewed as learning new (and better) feature representation from unlabeled data. This i s done by learning auxiliary predictors that predict one part of the feature vectors from another part of the feature v ectors, which can be learned from unlabeled data. Under certain conditions, it can be shown that learning auxi liary predictors of this type can reveal the predictive structure (something useful for the target prediction prob lems) underlying the data. The final classifiers are trained with labeled data using the original features and th e new features learned from unlabeled data. Since modern classifiers based on empirical risk minimization are capable of ignoring irrelevant features to some degree, the risk of using unlabeled data this way is relative ly low, and its potential gain is large. [1] should be consulted for the details of ASO. Below, we only describe the sp cifics of our setting.