Question: Does Blumenstock play a very important role in the quality assessment of Wikipedia articles?
Evidence: Automatic quality evaluation of Wikipedia articles has been addressed in previous works with both unsupervised and supervised learning approaches. The common idea of most of the existing work is to identify a feature set, having as a starting point the Wikipedia project guidelines, to be exploited with the objective in mind to distinguish Featured Articles. In BIBREF21 , Stvilia et al. identify a relevant set of features, including lingual, structural, historical and reputational aspects of each article. They show the effectiveness of their metrics by applying both clustering and classification. As a result, more than 90% of FA are correctly identified.
Blumenstock BIBREF20 inspects the relevance of the word-count feature at each quality stage, showing that it can play a very important role in the quality assessment of Wikipedia articles. Only using this feature, the author achieves a F-measure of 0.902 in the task of classifying featured articles and 0.983 in the task of classifying non featured articles. The best results of the investigation are achieved with a classifier based on a neural network implemented with a multi-layer perceptron.
In BIBREF22 , the authors try to analyze the factors affecting the quality of Wikipedia articles, with respect to their quality class. The authors evaluate a set of 28 features, over a random sample of 500 Wikipedia articles, by weighing each metric in different stages using neural networks. Findings are that linguistic features weigh more in the lowest quality classes, and structural features, along with historical ones, become more important as the article quality improves. Their results indicate that the information quality is mainly affected by completeness, and to be “well-written" is a basic requirement in the initial stage. Instead, reputation of authors or editors is not so important in Wikipedia because of its horizontal structure. In BIBREF23 , the authors consider the quality of the data in the infoboxes of Wikipedia, finding a correlation between the quality of information in the infobox and the article itself.
In BIBREF7 , the authors deal with the problem of discriminating between two large classes, namely NeedWork, GoodEnough (including in GoodEnough both GA and FA), in order to identify which articles need further revisions for being featured. They also introduce new composite features, those that we have referred to as an “actionable model” in Section "Baseline: the actionable model" . They obtain good classification results, with a F-measure of 0.876 in their best configuration. They also try classification for all the seven quality classes, as done in this work, using a random forest classifier with 100 trees, with a reduced set of features. The poor results (an average F-measure of 0.425) highlights the hardness of this fine-grained classification. In this paper, we address this last task in a novel way, by introducing domain features, specially dealing with the medical domain. The results of the investigation are promising.
Recent studies specifically address the quality of medical information (in Wikipedia as well as in other resources): in BIBREF24 and BIBREF25 , the authors debate if Wikipedia is a reliable learning resource for medical students, evaluating articles on respiratory topics and cardiovascular diseases. The evaluation is carried out by exploiting DISCERN, a tool evaluating readability of articles. In BIBREF26 the authors provide novel solutions for measure the quality of medical information in Wikipedia, by adopting an unsupervised approach based on the Analytic Hierachy Process, a multi-criteria decision making technique BIBREF27 . The work in BIBREF28 aims to provide the web surfers a numerical indication of Quality of Medical Web Sites. In particular in BIBREF28 the author proposes an index to make IQ judgment of the content and of its reliability, to give the so called “surface markers” and “trust indicator”. A similar measurement is considered in BIBREF29 . where the authors present an empirical analysis that suggests the need to define genre-specific templates for quality evaluation and to develop models for an automatic genre-based classification of health information Web pages. In addition, the study shows that consumers may lack the motivation or literacy skills to evaluate the information quality of health Web pages. Clearly, this further highlights the cruciality to develop accessible automatic information quality evaluation tools and ontologies. Our work moves towards the goal, by specifically considering domain-relevant features and featuring an automatic classification task spanning over more than two classes.
Blumenstock [6] inspects the relevance of the word-count feature at each quality stage, showing that it can play a very important role in the quality assessment of Wikipedia articles.
Answer: Yes, it does.