# Mass Collaboration and Learning: Opportunities, Challenges, and Influential Factors
## 4 Analysis of Affecting Factors
### Quality of Knowledge Building and Learning
#### 4.5.2 Evaluating the Quality of Knowledge in Mass Collaboration  
There is a well-developed body of research on evaluation various aspects of data, information, and knowledge management in different contexts (e.g., management science, medicine, multimedia, information systems engineering) and different businesses. However, for mass collaboration in particular, we could not find any substantial study that provides a comprehensive list of recommended methods that can be broadly used for evaluating the quality of created knowledge. Moreover, no systematic study in this context seems to have been already carried out to address this important issue. Perhaps the exception is Wikipedia, a prominent example of mass collaboration, from which we can gain some insight that might be used for guiding other mass collaborative cases. Wikipedia is an open access, large scale, and continuously evolving collaborative knowledge workflow that harnesses the efforts of thousands of volunteer contributors. A number of research works appraised the quality of knowledge in Wikipedia from different points of view. For instance, [134] assesses Wikipedia content construction and qualification from the perspective of participatory journalism; [135] recommends a technology that called 'history flow' to visualize Wikipedia content appraisal by means of article version histories; and [136] appraises the quality of Wikipedia articles with lifecycle-based metrics (based on tracking the changes in the editing intensity throughout the entire existence of an article). In fact, there are at least three types of processes for quality assurance in Wikipedia, including:  
1. The process of evaluating the quality of an article by direct actions like, modifying, changing, or deleting the status;
2. The process of Wikipedia editor's performance evaluation and selection of quality assurance agents; and
3. The process of creating and maintaining the work coordination artifacts of Wikipedia. \begin{table}
\begin{tabular}{l l} \hline \hline \multicolumn{1}{c}{**Criteria**} & \multicolumn{1}{c}{**Consideration**} \\ \hline Authority & Is the author credible enough? Is he/she well-known? \\ \hline Accuracy & Is the knowledge free from errors, and can it be verified? \\ \hline Accessibility & Is the knowledge easily retrievable? \\ \hline Currency & Is the knowledge up to date? \\ \hline Coverage & Is the knowledge comprehensive and depth enough for respective audience? \\ \hline \multirow{2}{*}{Relevancy} & Is the knowledge respected to your need (your topic or answer of \\  & question)? \\ \hline Purpose & What is the knowledge served for (teaching, informing, selling, \\  & entertaining)? \\ \hline Objectivity/Point of view or bias & Are all perspectives presented in an unbiased manner and balanced \\  & viewpoint? Are opinions separated from facts? \\ \hline Soundness & Is the created knowledge reasonable for the intended application? \\ \hline Applicability and utility & Is the created knowledge suitable for the intended learner? \\ \hline Clarity and completeness & To what extent is the created knowledge clear and complete? \\ \hline Uncertainty and variability & To what extent is the created knowledge certain and variable? \\ \hline Safety & Are the privacy policies and data protection procedures presented? \\ \hline References & Are the qualifications of the owner, reference, or reviewer addressed? \\ \hline Policy & Which policy is the knowledge following up (advertising, political, etc.)? \\ \hline Technical criteria (e.g., links, \\  & navigation, proper operation) & Is the knowledge created in the structured way? \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Some classic criteria for evaluating created knowledge. Besides, to assess the quality of featured articles in Wikipedia, eight major criteria need to be considered--namely, (1) accuracy, (2) comprehensiveness, (3) stability, (4) well written, (5) uncontroversial, (6) compliance with standards of Wikipedia, (7) having appropriate style, and (8) having appropriate images.