## 2 Materials and Methods
### Assessment Method
#### 2.4.2 Accessibility Assessment  
Accessibility analysis is a process that allows for determining whether the interface of a given program or application is available to the widest possible audience, including users with various limitations.  
As part of the analysis, the Scratch, App Inventor, and StarLogo environments were automatically analyzed against compliance with the WCAG guidelines, using the WAVE tool [50]. This was possible as all of the visual programming environments were accessible through the web. The authors used WAVE as the tool for the automatic accessibility assessment, checking the compliance of the chosen visual programming environments with the newest WCAG guidelines. WAVE is a proven tool used by other authors for such types of analysis [43; 45]. WAVE produced a compliance report that indicated, e.g., errors and alerts related to particular guidelines. The environments were compared for the number of errors and their priority.  
The authors decided to check the WCAG compliance with and without the application code embedded (displayed) in the visual programming environment code editor. Testing the empty code editor allowed us to check the influence of displaying visual syntax symbols on the overall results obtained from the compliance checking tool. Moreover, each visual programming environment was tested with the code of three different applications embedded in the code editor area of the interface. Applications published by the community or the visual programming environment provider were selected, making the results more reproducible, as the visual code of applications can be easily found. Another goal was to select a balanced variety of applications to cover the wide spectrum of syntax offered by the visual programming environments, as it is not possible to test all the possible application codes that can be developed using the provided visual syntax.  
In the case of the App Inventor ([[http://appinventor.mit.edu/](http://appinventor.mit.edu/)]([http://appinventor.mit.edu/](http://appinventor.mit.edu/))) web-based visual environment, the following accessibility test cases were addressed:  
1. (AP) Visual environment without any application code embedded.
2. (AC1) Visual environment with the \"Where is my car?\" application code embedded.
3. (AC2) Visual environment with the \"CreativeProject\" application code embedded.
4. (AC3) Visual environment with the \"pongpub\" application code embedded.  
In the case of the Scratch ([[https://scratch.mit.edu/](https://scratch.mit.edu/)]([https://scratch.mit.edu/](https://scratch.mit.edu/))) web-based visual environment, the following accessibility test cases were addressed:  
1. (SP) Visual environment without any application code embedded.
2. (SC1) Visual environment with the \"Avocado Juggle v1.2\" application code embedded.
3. (SC2) Visual environment with the \"Maze Starter\" application code embedded.
4. (SC3) Visual environment with the \"Greeting Card\" application code embedded.  
In the case of the StarLogo ([[https://www.slnova.org](https://www.slnova.org)]([https://www.slnova.org](https://www.slnova.org))) web-based visual environment, the following accessibility test cases were addressed:  
1. (LP) Visual environment without any application code embedded.
2. (LC1) Visual environment with the \"Game starter project\" application code embedded.
3. (LC2) Visual environment with the \"First person action game\" application code embedded.
4. (LC3) Visual environment with the \"Biology Simulation\" application code embedded.  
The results obtained from the compliance check were discussed keeping in mind the following four WCAG areas: perceivable, operable, understandable, and robust. Moreover, the number of distinct problems categories and total number of detected problems in each test case were calculated.