title: A Multi-method Evaluation of Website Accessibility: Foregrounding User-centred Design, Data Mining and Using a Quantitative Metric
Creators: name: Alahmadi, Tahani J
Publisher: name: Griffith University
Year: 2019
Subjects: subject: Website
subject: Accessibility:
subject: User-centred Design
subject: UCD
subject: Data Mining

description: Accessibility evaluation is an important equity-oriented step in assessing the effectiveness and usefulness of online learning materials for users with disabilities. The popular uptake of blended and online learning warrants an assessment of the accessibility of university web-based systems. Previous studies indicated that such systems have become gradually inaccessible as the complexity of their content increases, yet higher education institutions continue to pay minimal attention to this issue. Administrators of university websites and companies that develop learning management systems usually focus on conforming to specific accessibility standards, such as the Web Content Accessibility Guidelines (WCAG) 2.1. The problem is that the systems of these institutions extensively feature media and document files, thereby necessitating more than mere adherence to common principles for users with disabilities to efficiently interact with the systems. The development of an efficient accessibility evaluation method for dynamic web-based systems poses a significant challenge to researchers. In this regard, the main shortcomings that were addressed in this thesis are as follows: (1) The creation of accessible and usable questionnaire statements to be answered by disabled groups. (2) The limitations of accessibility evaluations carried out through automated and human-directed approaches. (3) The assignment of fixed weights to accessibility problems without consideration for the prioritisation of each problem encountered by disabled populations. (4) The lack of a technique for validating the results of human and automated assessments. (5) The occurrence of false negative and false positive accessibility problems during evaluations performed by automated tools. (6) The insufficient quality of descriptive texts or captions for non-textual elements. (7) The absence of accessibility values for uploaded files, such as Word, PowerPoint, PDF, image and video files, in a quantitative metric. (8) The shortage of initiatives in highlighting potential influencing factors for accessibility evaluation by users. (9) The failure of current standards to categorise accessibility issues on the basis of disability characteristics and system features. The outcomes of this research are expected to influence the types of information incorporated into evaluated systems and the development plans formulated for such platforms. They should also drive universities to institute adequate mechanisms that increase accessibility awareness among developers and educators. All the data collected through the multi-evaluation method put forward in this work can serve as reference in the design of a quantitative metric intended to identify accessibility values for users with visual and hearing disabilities. The ultimate goal of the proposed approach is to enable the construction of fully accessible systems that simultaneously facilitate the creation of accessible content by end-users, such as educators. To the above-mentioned ends, this study involved measuring and analysing the accessibility of university web-based systems used by visually and hearing-impaired groups. The measurement and analysis, which covered the accessibility of image, video and document files, were aimed at casting light on the influence of university web-based systems on content accessibility for vision-, hearing- and vision–hearing-impaired individuals. They were also meant to elucidate how developers, educators and universities implement accessibility principles with respect to the usability of system webpages and uploaded files. A model for accessibility evaluation was developed in accordance with a multi-method approach, the characteristics of disabled groups and the features of university systems. The model considers subjective and objective measurements. The subjective evaluation conforms to user-centred design (UCD) theory and embodies a design informed by usability and accessibility statements derived from state-of-the-art questionnaires and standards. The specific foundational materials for this evaluation are WCAG 2.0, the Software Usability Measurement Inventory, the IBM Usability Evaluation, Section 508 and PDF and MS Office standards. The objective evaluation entails the use of automated tools, human judgement and a quantitative metric (A3). Mining techniques (NaiveBayes, BayesNet, JRip and J48) were likewise carried out in developing the evaluation. The core end-products of this thesis are strategies for ranking and prioritising accessibility problems, a UCD theory-grounded estimation approach to the weighing of each accessibility problem by system users with visual and hearing impairments (subjective evaluation), a list of accessibility problems for each checkpoint for webpages as well as image, video, Word, PowerPoint and PDF files and a list of accessible and inaccessible files. Additional insights were drawn from the comparison of the outputs of the automated and human evaluations and the comparison of the number of accessible and inaccessible instances identified via data mining techniques as validation for the results of the human evaluations and vice versa. In the end, accessibility values for visually and hearing-impaired groups were determined by integrating all the evaluation findings as regards webpages, images, videos and Word, PowerPoint and PDF files and then by computing the developed Accessibility Metric of Web-based system including uploaded files (AMWSUF) (objective evaluation). The results showed that the overall accessibility values of university web-based systems were low. The accessibility value for the visually impaired users was 0.88, indicating that 88% of the examined university webpages (including uploaded files) were inaccessible for this population. The accessibility value for the hearing-impaired users was 0.42, denoting that 42% of the webpages (including uploaded files) were inaccessible for this group. The group with visual disabilities faced web access challenges in interacting with such critical systems, while the hearing-impaired group was confronted with difficulties relating to videos and audio files, which are popular content replete with usability problems.