## 3 Grading Algorithm
### Grade Calculation  
The algorithm calculates a grade for each learner individually, assigning a partial score to each message posted by each learner in the course forum. The grade calculation considers both the number of messages posted by learners and their relevance (measured through direct quantifiable indicators). Learners who do not post any messages to the course forum are excluded from this analysis directly receiving a score of zero points. **(a)**: Quantitative dimension  
The quantitative dimension considers the total number of messages sent by each learner, regardless of the level of these messages (see Table 1). Due to the different durations and delivery modes in MOOCs (e.g., 4-7 weeks of duration, instructor-paced/synchronous vs. self-paced/asynchronous delivery mode [44]), the interval in which the messages posted in the forum are analyzed is considered of importance. Therefore, a new indicator is defined as part of the quantitative dimension: the _period_. The _period_ is the average time between messages posted by the same learner and can be computed for a certain time interval (e.g., the full duration of the MOOC, a calendar month, a week of the MOOC, etc.). This gives instructors the flexibility to calculate the grade for learners' contribution in the forum throughout the entire duration of the MOOC, or a sub-interval of this duration. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **ID** & **Level** \\ \hline Thread & Th. & 1 st \\ Response & Resp. & 2 nd \\ Comment & Com. & 3 rd \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Message types. * Relevance  
The relevance of each message is assessed individually, considering a series of direct quantifiable indicators. It is important to note that these indicators may change depending on the level of the message (see Table 1) since some indicators are not available for all levels. Table 2 presents the list of indicators used as well as the score received by each message according to its level (_'Thread'_, _'Response'_, and _'Comment'_). Each message is assessed in the range of 0 to 10 points. Four indicators are considered mandatory (length, votes, replies, and followers). The length of the message (number of words) is available at all levels and is the only mandatory quantifiable indicator in the case of _'Comments'_; the score for the length of each message is computed according to four ranges. The number of votes is available for _'Threads'_ and _'Responses'_ and is also computed according to four ranges. The number of followers is only available for _'Threads'_ and is computed according to two ranges. The upper and lower levels for each range in length, number of votes, number of replies, and the number of followers may take default values or be tuned by instructors. In addition, five indicators are optional (quotes, endorsed, pinned, views, and abusive) since they may or may not be present in the messages. The number of mentions (quotes) to other learners is available at all levels, although is computed differently depending on the level of the message. The endorsement by instructors is available for _'Threads'_ and _'Responses'_ and is also computed differently depending on the level of the message. Instructors can also pin a _'Thread'_ but not _'Responses'_ or _'Comments'_. _'Threads'_ can also be viewed by other participants, adding points with an upper limit. Finally, messages that are