Positives Negatives Medline MScanner Ranker 24426 24689 47772 24777 24572 24609 24592 47746 47758 0.99977 0.9075 0.99795 0.8724 0.99748 0.9939 0.99601 0.9560 0.99421 0.9867 0.98812 0.9671 0.97041 0.9889 0.94179 0.9910 0.90237 0.9754 The MedlineRanker webserver provides a fast and ﬂex- ible way to rank the biomedical literature without expert knowledge. Querying is not limited by a complex query syntax, a controlled vocabulary, or any existing annota- tion of the literature. There is only one input required, a list of abstracts related to the topic of interest, to start the ranking of recent abstracts and the determination of discriminative words. This list of abstracts can be set directly by the user or automatically constructed using biomedical terms. Optionally, this list can be compared to a user-provided set of abstracts instead of the whole Medline database. This may produce a better ranking of closely related abstracts. Moreover, diﬀerent sets can be ranked, including the most recent months or years of Medline and also user-provided abstracts. The latter can for instance be used to focus on a given database or a given gene, by providing a list of abstracts related to that database or gene. Our tool can process tens of thou- sands of abstracts in a few seconds, and approximately one million per minute when ranking the most recent years of Medline. The MedlineRanker will produce more accurate results if the user provides a training set with enough abstracts to deﬁne the topic of interest. In our experience and as shown above, 100–1000 abstracts are appropriate for most of the topics, but providing more abstracts is likely to improve the method’s precision. Of course, the more homogeneous are the abstracts related to a topic, the better the ranking will be. One can get an idea of the predictive performance by observing the statistical output from the tool (Figure 1C). The MedlineRanker can be compared to two other Medline data mining tools that also use sets of Medline entries as input: PubFinder (17), which has not been updated for several years, and MScanner (10), which uses a diﬀerent method. The latter is diﬀerent in its way to select discriminative features: it uses mainly abstract annotations (MeSH terms and journal identiﬁers), whereas MedlineRanker uses only nouns extracted from abstract texts. As a result, MedlineRanker can be applied to all publications with an English abstract, including those with incomplete or missing annotations, while this is not possible with MScanner. This was illustrated with a benchmark ranking Arabidopsis-related abstracts according to host–pathogen interactions (Table 1 and sup- porting Table 1). Manual validations showed 99 true posi- tives within the best hundred abstracts, and only 17 were properly annotated. Very few (a total of 108) abstracts published for the plant model Arabidopsis thaliana received the tag ‘Host–pathogen Interactions’ in the MeSH Database. The results show that MedlineRanker can be also useful in the attribution of new MeSH terms. Comparing the speed of diﬀerent methods is compli- cated because there is often a trade-oﬀ between speed, capabilities and performance. MScanner, designed for maximum speed, sacriﬁces ﬂexibility by forcing all of Medline to be ranked. Ranking annotated abstracts from the whole Medline takes approximately one to three minutes using MScanner. MedlineRanker, designed for ﬂexibility, is not faster and processes approximately one million abstracts within a minute. Despite these Nucleic Acids Research, 2009, Vol. 37, Web Server issue W145 Table 2. ROC areas of various topics Topic Virus contamination 28 in Europe Microarray and protein 71 aggregation Radiology (10) 53 Text mining 312 Phosphorylation-dependent 136 processes Systems biology and pathway 407 Microarray and cancer 8327 AIDSBio (10) 4099 PG07 (10) 1611 MedlineRanker was compared to MScanner for various topics by the mean ROC area after 10-fold cross-validations (the two columns on the right). The same numbers of abstracts in the training set (positives) and in the background set (negatives) were used by both methods. diﬀerences, the two methods may be considered comple- mentary since both behave very well but diﬀerently for various topics (Table 2). Nevertheless, MedlineRanker seems to perform better when few abstracts are used to deﬁne the topic. The MedlineRanker webserver is more general than other comparable resources because it allows ranking user deﬁned sets of abstracts, and it also allows the user to deﬁne a particular set as the reference. For example, one can choose to rank only the abstracts associated to a given database which provides Medline references such as some PPI databases or other molecular databases. This was illustrated above with a benchmark ranking all refer- ences linked from three PPI databases according to a complex topic: phosphorylation-dependent molecular pro- cesses. Manual validation of the best 100 abstracts selected by MedlineRanker shows the relevance of our method which can lead to a positive predictive value of 0.90. Yet, the method used here to rank abstracts is not dedi- cated to detecting relationships between concepts. The pay-oﬀ is speed as it can retrieve many candidates in few seconds. Using, in a second step, co-occurrence analysis of diﬀerent concepts at the sentence level and semantic analysis, for which specialized tools are available, may help to focus on true positives in such complex situations. In conclusion, the MedlineRanker webserver provides a fast and ﬂexible tool to rank the biomedical literature without expert knowledge. It is not limited to any topic and can be useful for all scientists interested in ranking or retrieving relevant abstracts from the Medline database, including speciﬁc subsets like abstracts linked from partic- ular databases. SUPPLEMENTARY DATA Supplementary Data are available at NAR Online. ACKNOWLEDGEMENTS We thank Ellen Ramminger (MDC, Berlin), Ulrich Stelzl (Max–Planck Institute for Molecular Genetics, Berlin),