# Cognitively Driven Arabic Text Readability Assessment Using Eye-Tracking
## 5 Results and Discussion
### Readability Prediction at the Text Level  
In addition to extracted eye-tracking measures at the word level, we extracted several eye-tracking measures aggregated at the text level, as shown in Table 7.  
In addition, we took the common subjective readabilities from all the participants for all texts. Table 8 shows all extracted eye-tracking measures and common labels assigned by participants.  
Using the eye-tracking measures in Table 8 and taking their average as shown in Table 9, we tried to observe how collected readers' behaviors using their eye movements were correlated with their subjective readabilities and with the OSMAN scores assigned in Table 1.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Eye-Tracking Feature** & **Description** \\ \hline FIXATION\_COUNT (FC) & The average number of fixations in a text. \\ \hline FIXATION\_DURTATION\_MEAN (FDM) & The average duration of fixations (in milliseconds) a participant makes in a text. \\ \hline  & Average size (in degrees of visual angle) of all saccades in the trial group. “This is calculated by summing up the amplitude of all saccades in the trial group divided by the total number of saccades in the group” [78]. \\ \hline SACCADE\_COUNT (SC) & The average number of saccades in a text. \\ \hline RUN\_COUNT (RC) & The average count of how many times a participant makes runs of fixations in a text. \\ \hline \hline \end{tabular}
\end{table}
Table 7: Used eye-tracking features for each text.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Texts** & **FC** & **FDM** & **SAM** & **SC** & **RC** & **Participants’ Common Subjective Readability** \\ \hline Text 1 & 80.9 & 276.01 & 1.79 & 81.15 & 41.88 & E \\ Text 2 & 89.39 & 293.92 & 1.65 & 89.54 & 49.59 & M/E \\ Text 3 & 86.98 & 280.19 & 1.75 & 87.22 & 49.68 & E \\ Text 4 & 102.63 & 290.63 & 1.53 & 102.88 & 53.56 & M/D \\ Text 5 & 108.15 & 282.38 & 1.74 & 108.37 & 54.1 & M/D \\ Text 6 & 100.02 & 287.06 & 1.66 & 100.24 & 53.12 & M/D \\ \hline \hline \end{tabular}
\end{table}
Table 8: Texts’ aggregated eye-tracking measures and common subjective readabilities assigned by participants.  
As we can see from Table 9, the results validate our initial assumption about the readability level of the labeled texts based on the era and assigned OSMAN scores in Table 1 (thus, poems written in the Andalusian era are easier to read than poems written in the pre-Islamic era). There is a correlation between these scores, the texts' aggregated eye features, and the participants' feedback about these texts.  
It is clear that when comparing the two text categories, texts 1 to 3 had a lower fixation count, shorter fixation duration, larger amplitude of saccades (caused by the simplification of the texts [40]), lower saccade count, and lower run count, which indicated fewer regressions compared to texts 4 to 6. All these findings are well-matched with previous findings by [PERSON] [74], reported by [8], which conclude that readers' comprehension difficulties could be reflected by longer fixation duration, shorter saccades, and more revisits of parts in the text (regressions).  
Although in readability assessment research, text readability is done by having annotated data and then training an ML model to predict the readability of the document or sentence, in our approach, we did the opposite by starting with unlabeled texts, assessing the readability at the word level by collecting eye movement data and then labeling texts. Thus, we can say that texts 1 to 3 are easy to read, and texts 4 to 6 are medium near to difficult, or more generally more difficult than other texts.  
Because these texts are basically composed of the words that have been used at the word-level readability prediction and there was a correlation between the readability level of these texts (as measured by the OSMAN tool) and the eye-tracking measures obtained for those texts, we can infer that the generated readability scores at the word level are acceptable.