# Probabilistic Models for Competence Assessment in Education
## 3 Results
### Probabilistic Models of Peer Assessment
#### 3.1.1 Personalized Automated Assessments
This way, we keep information about whether \(j\), for instance, underevaluates with respect to \(i\), or overevaluates, or shows any other possible pattern. Information about their evaluation dissimilarities is summarized in that probability distribution. The evaluation difference between two assessments performed by \(i\) and \(j\) is defined as  
\[\mathit{diff}(i,j)=z_{u}^{i}-z_{u}^{j}\]  
If \(\mathit{diff}(i,j)>0\), it follows from the above definition that agent \(i\) over rates \(u\) with respect to \(j\). The opposite occurs if \(\mathit{diff}(i,j)<0\). From the previous definitions, it is clear that a situation of complete agreement between two agents is represented by  
\[\mathbb{T}_{i,j}=\{0\mapsto 1\}\]  
In what follows, we will denote that situation as \(\mathbb{O}\). If the leader has no objects assessed in common with a student \(j\), a relation of indirect trust is computed. Since the model deals with probability distributions, it becomes necessary to define aggregation operators for probability distributions. These operators compute an indirect trust distribution from two trust distributions by combining differences in an additive way. The basic idea is that if the difference of opinions between \(k\) and \(j\) is \(x\), and the difference of opinions between \(k\) and the teacher is \(y\), then the difference of opinions between \(j\) and the teacher is \(x+y\). Translating the above to probabilities and assuming independence of opinions, a combined distance distribution operator \(\bigotimes\) can be defined as follows. Given trust distributions \(\mathbb{P}\) and \(\mathbb{Q}\) over the numeric interval \([-b,b]\) we define their combined distance distribution, noted \(\mathbb{R}=\mathbb{P}\otimes\mathbb{Q}\), as:  
\[ \begin{split}& r(X=x)=\\ &\begin{cases}\sum_{x_{1}+x_{2}=x}p(X=x_{1})*q(X=x_{2})&\text{ if }x\in(-b,b)\\ \sum_{x_{1}+x_{2}\leq-b}p(X=x_{1})*q(X=x_{2})&\text{ if }x\leq-b\\ \sum_{x_{1}+x_{2}\geq b}p(X=x_{1})*q(X=x_{2})&\text{ if }x\geq b \end{cases}\end{split}  \tag{3}\]  
This aggregation combines the distributions along a path between two peers. From that definition, it follows that \(\bigotimes\) is commutative, and its neutral element is the probability distribution representing the complete agreement between two peers (\(\mathbb{Q}\)). In case there are several possible paths from the teacher to a student \(j\), an aggregation operator \(\bigoplus\) is defined to combine the aggregations computed along the different paths:  
Given probability distributions \(\mathbb{P}\) and \(\mathbb{Q}\) over the numeric interval \([a,b]\), we define \(\mathbb{P}\oplus\mathbb{Q}\), as  
\[\mathbb{P}\oplus\mathbb{Q}=\arg\min_{\mathbb{T}\in\{\mathbb{P},\mathbb{Q}\}}( \text{EMD}(\mathbb{T},\mathbb{Q})) \tag{4}\]  
with EMD standing for Earth mover's distance (a.k.a Wasserstein distance). (Informally, if the distributions are interpreted as two different ways of piling up a certain amount of dirt over the region \(D\), the EMD is the minimum cost of turning one pile into the other, where the cost is assumed to be the amount of dirt moved times the distance by which it is moved [13]). This aggregation is optimistically performed; it assumes that the combined distance that is closer to \(\mathbb{Q}\) (i.e., the one that brings the student's and teacher's opinions closer) is the true one. This operator is both commutative and associative. Thus, the order in which distributions (i.e., paths) are combined is irrelevant. Incremental Updates  
Every time an object is assessed by a peer, PAAS launches an overall update of the trust values and hence of the student marks. This update can be simplified as follows:  
1. Initially, the default direct trust distribution \(T_{i,j}\) between any two peers \(i\) and \(j\) is the one describing ignorance (i.e., the flat equiprobable distribution \(\mathbb{F}\)). When \(j\) evaluates an object \(\alpha\) that was already assessed by \(i\), \(T_{i,j}\) is updated as follows:
2.