# Mass Collaboration and Learning: Opportunities, Challenges, and Influential Factors
## 4 Analysis of Affecting Factors
### Quality of Knowledge Building and Learning
#### 4.5.2 Evaluating the Quality of Knowledge in Mass Collaboration
In addition to them, some popular methods (e.g., nominating featured article, peer review, feedback, and reputation mechanism) are also considered by Wikipedia community [137]. In order to compare these criteria and methods considered in Wikipedia with those proposed in collected papers, we summarized them in Table 14 along with brief explanations and respective references. From the collected papers it can be concluded that:  
* User feedback and expert evaluation were the most suggested methods for evaluation. \begin{table}
\begin{tabular}{l l l l} \hline  & **Suggested Methods** & **Explanations** & **Sources** \\ \hline - & Credit assignment & - & Using web of trust \\ - & Machine learning & - & Automatically propose to contributors for refining \\ - & User feedback & & their entries \\ - & Experts statements & & \\ \hline - & Initialization & - & Manually specify the correct topics from the title \\ - & Gaging user weights & - & Each user measures the quality of his or her feedback & [138] \\ - & User feedback & & \\ \hline  & & - & The type of contributor activity spent shaping an article for \\ - & Type of contributor activity & - & It shows the level of social presence (the degree to which \\ - & Number of anonymous contributors & contributors establish personal connections in a \\ - & Top contributor experience & & communication setting) & [139] \\  & & - & They often exert considerable influence over collaboration \\  & & & in online communities & \\ \hline - & User feedback & - & By comments and ratings \\ - & Ranking method & - & Measure both the value and transaction volume & [140] \\ - & Expert valuation & & of knowledge & \\ \hline - & CI model (lay citizens and & - & It relies on contributions, administration, reviewing, and \\  & traditional experts) & & consensus from both lay citizens and traditional experts & [124] \\ \hline - & Structure of groups & - & Systems should be designed in the way to foster \\ - & Structure of groups & - & conditions for producing collective intelligence & [141] \\ \hline - & Content facilitation & - & Direct intervention in the content to improve the quality \\ - & Process facilitation & - & Of contribution \\ - & Process facilitation & - & Procedural intervention by facilitator in structuring of \\  & & group processes & \\ \hline - & Argumentation & - & Submit arguments in support or oppose the topic \\ - & Consensus & - & Drive a community towards a consensus \\ - & Selection & - & All knowledge is ranked from best to worst based on \\ - & Expert evaluation & & the consensus \\ \hline - & Reputation & - & Editors who previously established records of \\ - & Peer review & & valuable contributions & [137] \\ - & User feedback & & \\ \hline - & Group observation & & [112] \\ \hline \end{tabular}
\end{table} 
Table 14: Suggested methods in collected papers for evaluating the quality of knowledge. * Top contributors (e.g., community managers, leaders) are also important, as they could bring their rich experiences to improve the process of evaluation. * Evaluation seems to be more effective when built on a combination of machine learning and human work. * When the results of evaluation are published for all learners, it could be helpful not only for error detection but also for error correction. * There is a pressing need for learners to be well trained for taking the advantages of knowledge evaluation. From our analysis, the evaluation of the quality of knowledge in mass collaboration contexts is still at its embryonic stage. Therefore, various issues still remain unaddressed, for example:  
* While endless knowledge is disseminated through the Internet and social media from different sources (while little is known about most of it) how can we really determine the quality of created and shared knowledge that might be readily altered, misrepresented, plagiarized, or built up anonymously under false claim? * How can communities gain a common sense for perceiving the expectations of evaluation? * How can we motivate learners to contribute to the evaluation process?