## 4 Results
### The Challenges of Including Users with Disabilities in User-Based Testing
\begin{table}
\begin{tabular}{p{42.7 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**ID** & **Challenge** & **Short Explanation** \\ \hline C2.2 & Presenting the instructions and tasks to the participants in a clear and accessible way & During testing, the researcher has to provide instructions and task presentations tailored to the specific abilities and limitations of the participants to ensure that the tasks are understandable and accessible to all participants. Participants find it difficult to visualize the problems as they need more real experiences (they cannot judge a game only by watching but must experience playing a game) [44]. Instructions being unclear and confusing [45] can lead to confusion and inaccurate feedback, impacting the validity of testing results. & While performing user testing, there is a risk of the tested environment malfunctions (freezing, failing to update, etc.) [26]. Any disturbance can affect the length of the testing process and, consequently, the well-being and experience of the participant. Poor accessibility of tools and instruments for measuring or data acquisition during testing procedures can affect the ability to fully capture the feedback of user testing participants with disabilities (e.g., visual disability [46]), resulting in potentially incorrect conclusions. Therefore, various measurement tools used during the testing procedures must also be tested for accessibility, primarily if these tools will be used by the users directly. & Products being tested that are not accessible and can be used only by users without disabilities (e.g., without visual disability [47]) lead to incomplete and inaccurate results. Therefore, the test results may not accurately reflect how the product or service will be used and accepted by the different end users. & Ensuring that the testing environment including all tools used for testing purposes is accessible for participants with different disabilities, the technology is integrated flawlessly into the testing process, preventing any additional barriers for participants with disabilities. For example, the lack of screen reader support on some devices (e.g., smart glasses), a rapidly draining battery used by the device, or dependency on an Internet connection can decrease the success of the testing process. [48] \\ \hline C2.6 & Acquiring unbiased feedback from participants & During the testing procedures, if others are present (e.g., the participant’s caregiver or the researcher) in the testing room, the participants often feel obliged to provide only positive feedback. Such impact can make participants’ opinions favorable to their caregivers or the researcher [36], leading to biased or incomplete feedback, as participants may be hesitant to provide negative feedback or may not express their true opinions or preferences. In a testing environment setting with observers present during the testing, users tend to increase their emotional control in disadvantageous conditions due to social desirability [49], conforming to the expectations of others rather than their accurate opinions or experiences. This impact can lead to erroneous testing results based on incomplete and dishonest feedback. & During the testing procedures, if others are present (e.g., the participant’s caregiver or the researcher) in the testing room, the participants often feel obliged to provide only positive feedback. Such impact can make participants’ opinions favorable to their caregivers or the researcher [36], leading to biased or incomplete feedback, as participants may be hesitant to provide negative feedback or may not express their true opinions or preferences. In a testing environment setting with observers present during the testing, users tend to increase their emotional control in disadvantageous conditions due to social desirability [49], conforming to the expectations of others rather than their accurate opinions or experiences. This impact can lead to erroneous testing results based on incomplete and dishonest feedback. & The behavior or feedback of children and users with disabilities who need caregivers (e.g., young adults with special communication needs [50] who cannot participate in testing alone) might be influenced by the presence of an accompanying person (e.g., parents, caregivers, and others). Sometimes, instead of the user with a disability, the caregiver gives input, leading to testing results that do not reflect the participant’s true experiences or feelings. & User-based testing in an online environment due to the pandemic or other reasons often results in unreliable findings [51]. Conducting remote user-based testing can bring new challenges, especially when performing research with participants with disabilities when too little support is provided for the users [52]. Participants’ technical barriers, such as bad internet connection, device limits, software compatibility issues, and other challenges, can make testing at home only sometimes feasible [53]. In addition to technical issues, there is an issue of environmental control (background noise or poor lighting affecting focus), which can lead to less engaged participants resulting in biased feedback. & Some user-based testing is connected to costs, logistics, scheduling, organizing space and testing equipment, time constraints, recruitment, and other travelling-connected inconveniences, which can discourage users’ participation. Therefore, it is necessary to encourage further and enable the conducting of on-site user testing, especially for users with disabilities that otherwise circumstances prevent them or make it harder to come to the laboratory setting. \\ \hline C2.9 & Organizing on-site testing for participants who have difficulties coming to the laboratory & Some user-based tests require strict laboratory conditions and rigorous protocols that cannot be performed remotely [33]. On-site user-based testing is connected to costs, logistics, scheduling, organizing space and testing equipment, time constraints, recruitment, and other travelling-connected inconveniences, which can discourage users’ participation. Therefore, it is necessary to encourage further and enable the conducting of on-site user testing, especially for users with disabilities that otherwise circumstances prevent them or make it harder to come to the laboratory setting. \\ \hline \hline \end{tabular}
\end{table}