Question: What kind of techniques do they use to evaluate the articles?
Evidence: As observed by a recent article of Nature News BIBREF0 , “Wikipedia is among the most frequently visited websites in the world and one of the most popular places to tap into the world's scientific and medical information". Despite the huge amount of consultations, open issues still threaten a fully confident fruition of the popular online open encyclopedia.
A first issue relates to the reliability of the information available: since Wikipedia can be edited by anyone, regardless of their level of expertise, this tends to erode the average reputation of the sources, and, consequently, the trustworthiness of the contents posted by those sources. In an attempt to fix this shortcoming, Wikipedia has recently enlisted the help of scientists to actively support the editing on Wikipedia BIBREF0 . Furthermore, lack of control may lead to the publication of fake Wikipedia pages, which distort the information by inserting, e.g., promotional articles and promotional external links. Fighting vandalism is one of the main goals of the Wikimedia Foundation, the nonprofit organization that supports Wikipedia: machine learning techniques have been considered to offer a service to “judge whether an edit was made in good faith or not" BIBREF1 . Nonetheless, in the past recent time, malicious organisations have acted disruptively with purposes of extortion - see, e.g., the recent news on the uncovering of a blackmail network of accounts, which threatened celebrities with the menace of inserting offending information on their Wikipedia pages.
Secondly, articles may suffer from readability issues: achieving a syntactical accuracy that helps the reader with a fluid reading experience is —quite obviously— a property which articles should fulfill. Traditionally, the literature has widely adopted well known criteria, as the “Flesch-Kincaid" measure" BIBREF2 , to automatically assess readability in textual documents. More recently, new techniques have been proposed too, for assessing the readability of natural languages (see, e.g., BIBREF3 for the Italian use case, BIBREF4 for the Swedish one, BIBREF5 for English).
In this paper, we face the quest for quality assessment of a Wikipedia article, in an automatic way that comprehends not only readability and reliability criteria, but also additional parameters testifying completeness of information and coherence with the content one expects from an article dealing with specific topics, plus sufficient insights for the reader to elaborate further on some argument. The notion of data quality we deal with in the paper is coherent with the one suggested by recent contributions (see, e.g., BIBREF6 ), which points out like the quality of Web information is strictly connected to the scope for which one needs such information.
Our intuition is that groups of articles related to a specific topic and falling within specific scopes are intrinsically different from other groups on different topics within different scopes. We approach the article evaluation through machine learning techniques. Such techniques are not new to be employed for automatic evaluation of articles quality. As an example, the work in BIBREF7 exploits classification techniques based on structural and linguistic features of an article. Here, we enrich that model with novel features that are domain-specific. As a running scenario, we focus on the Wikipedia medical portal. Indeed, facing the problems of information quality and ensuring high and correct levels of informativeness is even more demanding when health aspects are involved. Recent statistics report that Internet users are increasingly searching the Web for health information, by consulting search engines, social networks, and specialised health portals, like that of Wikipedia. As pointed out by the 2014 Eurobarometer survey on European citizens' digital health literacy, around six out of ten respondents have used the Internet to search for health-related information. This means that, although the trend in digital health literacy is growing, there is also a demand for a qualified source where people can ask and find medical information which, to an extent, can provide the same level of familiarity and guarantees as those given by a doctor or a health professional.
We anticipate here that leveraging new domain-specific features is in line with this demand of articles quality. Moreover, as the outcomes of our experiments show, they effectively improve the classification results in the hard task of multi-class assessment, especially for those classes that other automatic approaches worst classify. Remarkably, our proposal is general enough to be easily extended to other domains, in addition to the medical one.
Section "Dataset" first describes the structure of the articles present in the medical portal. Then, it gives details on the real data used in the experiments, which are indeed articles extracted from the medical portal and labeled according to the manual assessment by the Wikimedia project. Section "Baseline: the actionable model" briefly presents the actionable model in BIBREF7 : we adopt it as the baseline for our analysis. In Section "The medical domain model" , we present the domain-specific, medical model we newly adopt in this paper as an extension of the baseline. The extended model includes features specifically extracted from the medical domain. One novel feature is based on the article textual content. Section "Bio-medical entities" presents the process which its extraction relies on, with a non trivial analysis of natural language and domain knowledge. Section "Experiments and results" presents experiments and results, with a comparison of the baseline model with the new one. In Section "Related work" , we survey related work in the area and in Section "Conclusions" we conclude the paper.
We approach the article evaluation through machine learning techniques.
Answer: Machine learning techniques based on structural and linguistic features of an article.