# Design and Applications of GLANCE: GLanceable Alarm Notification for a User Centered Experience
## 3 Results of the 1 st Macro-Phase: The GLANCE Project
### Third Iteration: Final Release of the Proof-of-Concept and Definition of a General Formal
#### 3.3.3 Evaluation and Learning
Only in one case was there a recorded distance of two levels down, but the associated observer's note indicated that it occurred when the user was distracted from the foreground task. Table 7 summarizes the results, differentiating between the two cases of \"level up\" (the display change grabbed more user attention than due) and \"level down\" (the display change grabbed less user attention than due). We notice that overall the display behaved as expected. It is worth nothing that, if we consider the total of inattention and divided attention display changes (90 changes), we detected unwanted interruptions only in 4% of cases. On the other hand, if we consider the total of focused attention and demand action display changes (105 changes), we did not get the desired interruptions in only 3.8% of cases. As to the rate of level down in the demand action notification level, it has to be noticed that more than 50% of cases occurred in the first appearance of a demand action display change (these users did promote the background task to foreground task but, without instructions, they forgot to click on the display as requested by the test). If we ignore the first demand action display change for all users and we limit the calculation to the remaining display changes, the rate drops from 12% to 6%. We also notice that, while in the administered test the requested action did not produce any particular benefit to the users, in real contexts it would be associated to some domain-specific meaningful effect that would push users to perform it. Actually, in the post-experiment debriefing, some users reported that they perceived the requested action as a \"chore\" and they would have liked, e.g., to access the source of the critical news by clicking. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Notification Level** & **Rate of Level Up** & **Rate of Level Down** \\ \hline Inattention & 13\% & 0\% \\ Divided attention & 8\% & 11\% \\ Focused attention & 4\% & 8\% \\ Demand action & 0\% & 12\% (6\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Rate of unexpected behavior. As to the users' _post-test_, the result of the questionnaires showed that the notifier did not affect at all the completion of the primary task (with a correctness rate of 100% on the questions about the proposed text). As to the secondary task, Table 8 summarizes the correctness rate (CR) of users' answers, rated according to four levels: _Excellent_ (CR > 90%), _good_ (70% < CR \(\leq\) 90%), _sufficient_ (50% < CR \(\leq\) 70%), and _poor_ (CR < 50%). Overall, the correctness rate of the secondary task was above 80% in 66% of the questions (1-4, 8-11), and around 60% in the remaining 33% (5-7, 12). It is worth noticing that user performances reflect somehow the results obtained by the observational study, which showed that some ambient notifications fell in the inattention zone. In particular:  
* There was no detection of statistically relevant differences based on age or computer experience;
* Notwithstanding that 8% of alerting notifications fell in the ambient zone, critical information was always correctly recalled;
* Most significant mistakes are related to the recall of some specific details of a low-to-medium severity notification (like, e.g., the name of a stock index in Question 5, the sender of an email message in Question 7, the background color of a panel in Question 12);
* Question 12 was asked just to check how many different information users were able to digest about a notification but remembering a notification color is not the objective of the system.