W142 Nucleic Acids Research, 2009, Vol. 37, Web Server issue return a list of records ranked by relevance. Based on two previous studies (9,10), we have implemented the MedlineRanker webserver, which allows a ﬂexible ranking in Medline for a topic of interest without expert knowl- edge. The user deﬁnes their topic of interest using their own set of abstracts, which can be just a few examples, and can run the analysis with default parameters. If the input contains closely related abstracts, the program returns relevant abstracts from the recent bibliography with high accuracy. The web interface also allows customization of other parameters and inputs, such as the reference set of abstracts, which is compared to the query. The use of the MedlineRanker webserver is free and requires no user registration. Our tool can process thousands of abstracts from the Medline database in few seconds, or millions in few minutes. MATERIALS AND METHODS Method and implementation The MedlineRanker method is derived from a supervised learning method which was tested on the subject of stem cells (9). Brieﬂy, noun usage is compared between a set of abstracts related to a topic of interest, called the training set, and the whole Medline or a subset, called the back- ground set. First, nouns are extracted from each English abstract, including the title, without counting multiple occurrences. The original supervised learning method was improved by using a linear naı¨ ve Bayesian classiﬁer which is applied by calculating noun weights with a refac- tored-for-speed dot product (10,11), which sums only the features that occur (12). We also use the split-Laplace smoothing scheme to counteract class skew (11) (see sup- porting information and http://mscanner.stanford.edu/ static/thesis.pdf for details). An abstract is scored by sum- ming the weights of each of its nouns, and P-values are deﬁned as the proportion of abstracts with a higher score within 10 000 recent abstracts. Scripts and web pages are programmed using HTML4, Perl 5.8.8 and R 2.8.1 (13). Extraction of nouns in English abstracts is performed using the TreeTagger program (Helmut Schmid, Institute for Natural Language Processing, University of Stuttgart) and stored in a local MySQL database (version 5.0.45) along with information from the Medline database (http://www.nlm.nih.gov/pubs/factsheets/medline.html). The source code is available from the authors upon request. Cross validations and manual evaluations Each MedlineRanker query result includes an estimation of its performance calculated using leave-one-out cross- validation. Abstracts are considered true positives if they relate to the topic of interest and negatives otherwise. For a given P-value cut-oﬀ, the web server produces a list of candidate abstracts, which are true positives if they really relate to the topic and false positives otherwise. We deﬁne the sensitivity of the tool as the number of true positives divided by the total number of positives, and the false positive rate as the number of false positives divided by the total number of negatives. A receiver operating characteristic (ROC) curve, which plots the sensitivity versus the false positive rate for several score cutoﬀs, is provided to facilitate comparisons with other classiﬁers. For comparisons, we have also used a 10-fold cross-validation procedure as previously described (10). First, the training and the background sets were divided into 10 equally sized parts. Then, nine parts from each set were used as input to MedlineRanker, and the remaining parts were used to calculate the accuracy. This was repeated 10 times to process all the abstracts and to cal- culate the mean accuracy. Manual evaluations were performed to count the number of true positives in a selection of the best abstracts ranked by MedlineRanker. In the ﬁrst benchmark the training set was composed of 12 291 abstracts annotated with the ‘Host–Pathogen Interactions’ MeSH term, the background set was the whole Medline, and the test set was composed of 20 052 abstracts annotated with the ‘Arabidopsis’ MeSH term (excluding abstracts from the training set). A second benchmark of MedlineRanker’s ability to retrieve abstracts related to dependent topics was performed using a training set related to the concept of phosphorylation-dependent molecular processes. A total of 136 abstracts were automatically selected for the train- ing set using a text mining facility (LAITOR, Barbosa- Silva, A. et al., in preparation). To be selected, abstracts had to show at least one sentence containing: two human protein names, a word related to a biological action in between (Bioactions AKS data source, http://schneider- www.embl.de/), and terms or synonyms indicative of phosphorylation-dependent processes (see supporting information). Then, following a leave-one-out procedure, MedlineRanker used all the following abstracts related to human PPI as background and test sets: 18 981 abstracts from the Human Protein Reference Database (HPRD) (14), 2549 abstracts from the Molecular INTeraction data- base (MINT) (15), and 3056 abstracts from the Database of Interacting Proteins (DIP) (16). For each benchmark, abstracts that scored among the top hundred were then subjected to manual evaluation by a team of researchers. Each abstract was classiﬁed as relevant or not by a majority out of three votes. RESULTS User inputs There are three diﬀerent sets of data that the user can provide to help them get the most relevant results from MedlineRanker: the training set, the background set and the test set. A user interested in ranked results related to a particular topic has to input some abstracts related to that topic as the training set. In the training set, an abstract is repre- sented by its PubMed identiﬁer (PMID). These identiﬁers can be easily retrieved from a PubMed search results page as explained in the webserver online documentation. Also, thanks to available Medline annotations the webser- ver can automatically construct the training set from a list of biomedical MeSH terms. Some example training