# Mass Collaboration and Learning: Opportunities, Challenges, and Influential Factors
## 4 Analysis of Affecting Factors
### Evaluating Learners' Performance
#### 4.4.2 Performance Evaluation Methods  
A number of methods for performance evaluation, and some concrete experiences have been reported which are appropriate for different situations and characteristics of an organization or community. Based on [103], there are three performance evaluation categories: (1) absolute standards (compare learners' performance to a standard, and the evaluation is independent of others); (2) relative standards (compare learners' performance against others); and (3) objective-based (learners are assessed on how well they fulfill a specific set of objectives). This taxonomy can be extended with three techniques borrowed from [98; 104], see Figure 6. In order to further clarify, some explanations for each method are presented in Table 11.  
Considering the collected papers, only 25 papers focused on methods of performance evaluation. The majority of these papers preferred to use absolute standards and added methods in their study. This category has the highest percentage of application (48%) and (16%) respectively among all three suggested categories. While objective methods have the lowest degree (4%) of application. The usage of relative standards methods was half of the absolute standards methods (32%). Among all proposed methods, ranking method was the most applied (24%). More details are illustrated in Figure 7. Some adopted methods and corresponding references are also addressed in Table 12. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline
**Categories and Methods** & **Descriptions** \\ \hline
**Absolute standards** & It compares learners’ performance to a standard, and the evaluation is independent of others. \\ \hline Check list methods & It provides a series of statements e.g., ’yes’ or ’no’ questions and their answers for matter to mark right answers. \\ \hline Critical incident methods & The focus of rather is on behaviors that make difference between performing a task in a noteworthy manner. \\ \hline Graphic Rating scale methods & Influential behaviors on performance are listed and learners are rated based on them. The rates are helpful for quantifying the behaviors. \\ \hline Essay methods & In order to improve learner’s performance, a narrative description is written by the rather about learner’s status. \\ \hline Behavioral Anchored Rating Scales (BARS) & It combines the benefits of narratives, critical incidents, and quantified ratings by anchoring a quantified scale with specific behavioral examples of good or poor performance. \\ \hline Forced choice methods & A learner is being evaluated and rated based on before written statements. One common method in this group involves positive and negative statements. \\ \hline
**Aided methods** & Outstanding, satisfactory and unsatisfactory are three established categories of worth for evaluation. \\ \hline Confidential report & A confidential report rates learner’s performance with respect to items such as, teamwork, attendance, reasoning and technical abilities, etc. \\ \hline Assessment centers & Methods such as social or informal events, tests, and exercises are used to evaluate learners’ performance for future responsibilities. \\ \hline
**Relative standards** & It compares learners’ performance against others \\ \hline Ranking methods & Ranks from the highest to the lowest are used for comparing learner with others. \\ \hline Paired comparison & Based on one trait or one- on one basis, learner is compared with others. \\ \hline
**Objectives** & Learmers are assessed on how well they fulfil a specific set of objectives \\ \hline Management by objectives (MBO) & Learmers are evaluated periodically based on defined objectives. \\ \hline
360\({}^{\circ}\) appraisal & Learmers are evaluated by people working around them through confidential, anonymous feedback from. \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Some partial descriptions of performance evaluation methods [98; 104; 105]. Figure 6: Methods for performance evaluation based on [105].