# Automated Adaptations for Improving the Accessibility of Public E-Services Based on Annotations

J. Eduardo Pérez, Sandra M. Espín-Tello, Aritz Sala, Myriam Arrue

► To cite this version:

J. Eduardo Pérez, Sandra M. Espín-Tello, Aritz Sala, Myriam Arrue. Automated Adaptations for Improving the Accessibility of Public E-Services Based on Annotations. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.373-382, 10.1007/978-3-030-85610-6_22. hal-04215494

HAL Id: hal-04215494

https://inria.hal.science/hal-04215494v1

Submitted on 22 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Automated adaptations for improving the
accessibility of public e-services based on
annotations

Aritz Sala, Myriam Arrue, J. Eduardo Pérez, and Sandra M. Espín-Tello

EGOKITUZ, University of the Basque Country (UPV/EHU)

asala020ikasle.ehu.eus, {myriam.arrue, juaneduardo.perez, sandramartina.espin}@ehu.eus

**Abstract.** The COVID-19 pandemic has changed the way we interact with public administration, making indispensable the use of online public services (e-services) to perform required administration procedures. All citizens should be able to interact with public e-services in an autonomous, satisfactory and easy way. However, these services are not always accessible to people with disabilities. In this paper, we present a descriptive model for e-services adaptation based on ontologies, which contains the e-service annotations, user features and techniques required for adapting e-services to the needs of people with disabilities. Based on this model, an automated adaptation system was developed and adapted versions of three different Spanish public e-services aiming at users with low vision were obtained as a proof of concept.

**Keywords:** e-Government · public e-services · annotation ontology · automated adaptation · low vision.

# 1 Introduction

Last year has been an unusual one due to the COVID-19 pandemic. The impossibility of performing the usual face-to-face administrative tasks has forced many people to change the way they interact with public administration. The "E-Government Survey 2020" from the United Nations [22] identified significant uptakes in digital services in different geographic regions, countries and cities as well as increased e-participation in relation to the previous surveys.

Public e-services are crucial tools for citizens to guarantee their e-participation. They are used for different tasks, such as making an appointment with the public administration, searching for employment and social care, etc. with online-forms as the predominant style of interaction [14]. However, some users may experience difficulties when using these services if they are not accessible[5].

The Web Accessibility Initiative (WAI) has made efforts for ensuring accessibility of websites. The Web Content Accessibility Guidelines (WCAG) are the most well-known and applied for accessibility evaluation. Current version (v2.1) [10], structures the guidelines into four principles: perceivable, operable, understandable, and robust; and includes success criteria for checking the fulfilment
of each principle. These guidelines are considered of mandatory conformance for public e-services in many countries and are included in their legislation [4]. Despite this, many people with disabilities experience difficulties when interacting with web interfaces [9]. These difficulties aggravate when referring to public e-services as many of them require the fulfilment of forms that contain complex elements to interact with (e.g., calendars, controls, captcha, etc.) and may present accessibility barriers of more severity than static web pages. Several studies about the experience of accessibility barriers by people with disabilities in governmental websites can be found in the literature [5][21].

The objective of this paper is to present an automated adaptation system for improving the accessibility of e-services as well as to structure the process for the fulfilment of online-forms so more logical and uniform interaction is provided to users. For this, we define a descriptive model through an ontology which contains the required features for e-services annotations as well as the user model and the adaptation techniques to apply. This automated adaptation system was applied to three different Spanish public e-services and adapted interfaces aimed at people with low vision were obtained as a proof of concept.

## 2 Related Work

Ontologies are used as a modeling tool that allows performing designs, establishing relationships, and formulating axioms to infer and deduce information within a specific process in a more personalized and flexible way [7]. Several studies have shown that ontology models are effective in the generation of adaptive web user interfaces at runtime. For example, Catwalk framework [12] that researched how web user interfaces could be dynamically selected, generated, and adapted according to user-aspects such as their profile, the specific task, or goal to be achieved or the location and time characteristics. Unlike our approach, where the adaptation is performed remotely to the e-service, the Catwalk framework was developed to be deployed in the web server, so the web interfaces presented to the user are dynamically adapted as they are requested.

Some of the developed projects using ontologies are aimed to improve web accessibility for people with disabilities, and specifically, for those with visual impairments, such as the one carried out by Bukhari and Kim [3]. They proposed an ontology called “Ontology based on the extraction of information for people with visual disabilities” (HOIEV), composed of 3 layers: interaction with the remote user, extraction of information from the core, and the programming interface (API). This is a comprehensive ontology for obtaining information about users interaction. However, it can not be applied for automated adaptation of e-services at run-time. Another example is the method developed by Rosales et al. [18], that proposed a prototype of a semantic platform to enhance web accessibility. This prototype was developed for a specific adaptation as it provides mechanisms for interacting with the interfaces through voice commands. The adaptation system presented in this paper aims at providing a comprehensive adaptation techniques pool for selecting and applying accordingly to different
<figure><img src="image_3.png" /><figcaption>Fig. 1: Diagram of the proposed ontology. Continuous lines correspond to 'has' relation type, while dashed lines correspond to 'is a' relation type.</figcaption></figure>

groups of users. The Egoki system [6] deployes a user ontology model to auto-
matically generate user-tailored interfaces in ubiquitous environments. However,
the web interfaces were created from models. This approach is not applicable for
adapting existing e-services interfaces at run-time.

There are some works aiming at adapting web interfaces at run-time for better fit to the needs of users such as the transcoding systems presented in [1, 8]. However, these systems are oriented to specific user needs, based on annotations of single web pages and do not provide a descriptive model applicable to the process of public e-service systems. Another research work worth mentioning is the one shown in [23]. They present a comprehensive web interface adaptation system. However, the adaptation process is individually performed for each web page so it is not applicable for adapting the process of public e-services.

3 Basis Of Automated Adaptation

To get an automated adaptation system that improves web accessibility of pub-
lic e-services, we have defined a descriptive model to annotate relationships be-
tween the e-service process and components, along with the user features and the
adaptation techniques. For this, we have implemented a knowledge base using
the Web Ontology Language (OWL) [13] and the ontology editor tool Protégé<fnref n="1" />.
Figure 1 shows the class diagram defined in the ontology.

**User Model.** The user model included in our ontology defines the interaction features to consider from each user. Thus, appropriate techniques can be selected to generate the adapted e-service. In particular, three general user groups are included (users with cognitive impairment, physical impairment, and sensory impairment) along with their corresponding features (e.g., colour blindness). This user model can be updated, adding new user groups and features.

<footnote n="1">¹ http://protege.stanford.edu</footnote>
In addition, users' preferences and the assistive technology used are also considered in the model, so more specific adaptation techniques may be applied [17], for example, adapting navigation for users of trackballs, adapting components for screen reader users, etc.

**Annotation Model Of Public E-services.** In most cases, accessing public e-services requires the fulfilment of online-forms, which usually include several steps. Therefore, it may be described as a transactional process [15].

Nowadays, public e-services are not uniform and present different fulfilment processes even if their target task is similar. However, it is possible to identify common steps within e-services, as well as similar data to be introduced by users. These common features among public e-services led us to define an annotation model that can be applied to annotate the process in general, the concrete steps of the process, the data required from users, etc. The objective of these annotations is to obtain valuable information in order to automatically apply the selected adaptation techniques depending on user features/preferences in order to obtain more accessible interfaces and enhance the structure of the process by relocating components when necessary.

Figure 1 shows the classes defined in the ontology and the relationships between them. The main classes for the annotation model are the following: eService, Page, Step, and HTMLComponent. One eService may have several Page instances. One Page may have several HTMLComponent instances. The Step class is related to eService, so it is possible to annotate the steps in the process of the eService. In addition, the Step class is also related to HTMLComponent, so it is possible to annotate the information to be required in each step. These relationships make it possible to split one e-service process into several steps assigning the necessary HTML components to each one and automatically creating the necessary interfaces and presenting them in the most adequate order.

There are subclasses for HTMLComponent: InteractionComponent and DescriptionComponent. The former are those HTML components requiring direct data introduction from the user (DataEntry), selecting data from a set of options (DataSelection) and submitting data to the server (DataSubmission). The latter are those HTML components presenting Textual, Auditory or Visual information. This model allows providing alternatives for components, so the most appropriate one is presented to the user. The Form subclass has been also defined as a compound component.

In addition, several properties were added for HTMLComponent class: dimming, stretch, hide and remove. 'Dimming', 'hide' and 'stretch' can be used to hide part of the content of the e-service, such as leaving only part of the instructions to make it simpler and smaller. Finally, 'remove' property is used to remove those elements that can bother the user. For instance, a flashing element is tagged as remove = 'flashing' so it would be removed when the user has photosensitivity. The 'priority' property has been defined for DescriptionComponent class and it can be used to mark the elements as being necessary, reordering them, or selecting the most appropriate for the user. The 'optional' property has
been defined for InteractionComponent in order to identify those fields that are not mandatory to fulfil in the process.

**Adaption Techniques.** The knowledge base models the adaption techniques and decides which ones are applicable, but it does not implement them. This approach enables the addition of new adaptation techniques, thus enabling system development to become a continuous process. In the current study, we have defined three groups of adaptation techniques: service adaptations, step adaptations and interaction component adaptations. The definition of these groups is based on a modification of the original classification of Knutov et al. [11]: navigation, content and presentation. In our perspective the navigation adaptations are applied to all the e-service as a whole (e.g., providing orientation components such as breadcrumb of steps, progress information, completed and to-do steps, etc.), content adaptations are mainly applied to each step of the service (e.g., only showing the required fields to fulfil by the users in each step, showing links instead of radio buttons for data selection components, etc.) and presentation adaptations are applied to all (service, steps and components) (e.g., using high contrast colors in forms, visually grouping the mandatory fields, over-sizing interaction components, etc.).

<table><caption>Table 1: Features of e-service selected.</caption><thead><tr><th>Features</th><th>DNI</th><th>SPE</th><th>SGS</th></tr></thead><tbody><tr><td>Number of pages</td><td>7</td><td>4</td><td>5</td></tr><tr><td>DataEntry components</td><td>10</td><td>6</td><td>7</td></tr><tr><td>DataSelection components</td><td>-</td><td>2</td><td>9</td></tr><tr><td>DataSubmission components</td><td>10</td><td>4</td><td>5</td></tr><tr><td>Mandatory Int. Comp.</td><td>9</td><td>6</td><td>6</td></tr></tbody></table>

# 4 Automated Adaptation System

The architecture of the adaptation system developed is shown in Figure 2. The Knowledge Base and the e-services annotations are located in a remote machine. The Adaptation engine is implemented as a Web Service whereas a Chrome extension is installed locally on the user computer.

This browser add-on is in charge of starting and completing the adaptation process performed by the system. It gets an e-service to adapt (steps 1 and 2 in Figure 2) and sends the source code with the user's credentials to the remote Web Service (step 3 in Figure 2). Then, it presents the adapted version of the e-service returned by the Web Service (step 6 in Figure 2). The Adaptation engine implements two main functionalities. The first is to query the Knowledge Base about adaptations corresponding to the current user and the e-service (step 4 in Figure 2).
Figure 2). The second is to perform adaptations resolved by the Knowledge Base according to the annotations of the e-service (step 5 in Figure 2) and return the adapted e-service to the browser add-on (step 6 in Figure 2). The e-services are adapted using JSON, JQuery, JavaScript and CSS technologies.

**Proof of concept.** Three different e-services from the Spanish public administration were selected to carry out the proof of concept of the adaptation system: from the Ministry of Home Affairs (DNI<fnref n="2" />), from the Public Service of State Employment (SPE<fnref n="3" />), and from the Social Security (SGS<fnref n="4" />). These e-services had already been tested in an experiment involving people with low vision [19, 20], identifying several accessibility barriers. The three e-services offer the same functionality (“make an appointment”) but with different features regarding online-forms (e.g., number of pages, data to be inserted, interaction components, etc.). Table 1 shows detailed information about the features of each e-service: number of pages, number of each type of fields implemented in the forms and number of mandatory fields (those dataEntry and DataSelection components users have to interact with).

<figure><img src="image_4.png" /><figcaption>Fig. 2: Proposed architecture for the adaptation system.</figcaption></figure>

On the other hand, we identified the following similar steps in the form fulfilment process: providing personal information (UserId), selecting the service (ServSel), selecting the location (LocSel), selecting the date and time (DateSel), answering security questions (SecQue). All these steps can be found in the e-services studied, except LocSel in SPE. However, in some cases, the information related to each step is required in different pages across the e-service. For

<footnote n="2">² http://www.interior.gob.es/</footnote>

<footnote n="3">³ https://www.sepe.es/</footnote>

<footnote n="4">⁴ http://www.seg-social.es/wps/portal/wss/internet/Inicio</footnote>
example, regarding the UserId step, the zip code and the identification (ID) card number is required in the first page of SPE service whereas the name and surname of the user are requested on the second page.

With the objective of providing an automatically adapted version to people with low vision, the three e-services have been annotated using the proposed ontology in the knowledge base. The annotation process was in this way: firstly, a reference to the e-service (for example, DNIeService), is stored as an eService class. Then, the web pages in the e-service are stored as Page classes and linked with the eService with a property assertion (e.g. 'DNIeService hasPage Page1'). The identified steps are also stored as Step classes and linked with the eService class (e.g. 'DNIeService hasStep UserId'). After that, HTML components have to be stored which are doubled linked with the Page and Step classes (e.g. 'Page1 hasHtmlComponent Name' and 'UserId hasHtmlComponent Name'). Each HTML component is identified as an interaction or description component and its type is stored accordingly linked with isA subclass relation ('Name isA InteractionComponent' and 'Name isA DataEntry'). The form is a particular HTML component consisted of the HTML components within (e.g., 'Form1 hasHtmlComponent Name'). Some properties are assigned to the components (e.g. 'aternum hasProperty optional', 'Advert hasProperty distractor'). Finally, the HTML components are accordingly identified with their name, id or uri. The annotation process is quite tedious as it is necessary to analyse the e-service to define the number of steps and decide the interaction components it is composed of. In addition, some constraints have to be considered as sometimes the existence of one component may depend on previous user input or the order in which steps are presented to users may be limited by the service program logic which can not be modified.

<figure><img src="image_5.png" /><figcaption>Fig. 3: Examples of original (a) and adapted interfaces (b, c and d) for SPE.</figcaption></figure>

The adaptation techniques applied in this proof of concept are oriented to
improve the navigation experience of people with low vision. The selection of
techniques to apply was based on a bibliographic revision:

1. Providing uniform style for e-services [16] [15]

2. Structuring the process in steps [16]

3. Ordering steps and grouping components in steps [16]

4. Identifying steps and overall progress of the process [16][14, 15]
5. Changing the presentation to high contrast colours [2]

6. Resizing components [2]

7. Replacing the selection components (DataSelection) [19, 20]

The rules for the adaptations were specified based on the ontology classes and stored in the knowledge base. For example, adaptation 7 was defined as:

`'HtmlComponent(?el), isInteractionComponent(?el), isDataSelection(?el), '`

`'HtmlComponent((?el) hasOption (?op), ⇒ isApplicable(convertToLink, ?el, ?op) '`

Figure 3(a) shows the original first web page of the SPE e-service. The fields to fill in are: ZIP code, ID card number, selection of the required service (radiobutton with 8 options) and CAPTCHA (with alternative audio and help link). All fields are mandatory and there is one button to send the input data. Based on the descriptive model proposed, these fields are annotated as InteractionComponents: ZIP code, ID card number and CAPTCHA as DataEntry, selection of the service as DataSelection with 8 Option elements and the button as DataSubmission. These fields correspond to different step in the process (UserID, ServSel, DateSel, SecQue) so we annotated the steps of the e-service and each field with the corresponding step: ZIP code and ID card number fields as UserID, CAPTCHA as SecQue and the radiobutton as ServSel. The steps order defined in the adaptation techniques is [SecQue, UserID, ServSel, DateSel]. The components contained in the 4 original web pages of this e-service are relocated according to this order into 8 adapted interfaces. The user input in each field is stored between interfaces and sent to the e-service when it corresponds depending on the program logic. The first three interfaces are shown in Figure 3 (b, c and d, they correspond to SecQue, UserID, ServSel respectively). The step in the process is identified showing a title at the top of the interfaces and the progress in the process of fulfilling the form is presented at the bottom of the interfaces. The same process was done to DNI and SGS e-services resulting on 10 adapted interfaces for the 7 web pages and 11 adapted interfaces for the 5 web pages respectively.

# 5 Conclusions

The developed automated adaptation system based on annotations may improve the experience of users with low vision when fulfilling forms in the public e-services, as the adaptation techniques applied aim to improve their accessibility. Although for the proof of concept this system was aimed at people with low vision and was only tested in three public e-services, in the short-term, we will also apply it for annotating and adapting other public e-services in order to test if it is useful for a broad range of e-services. Moreover, we plan to add more adaptation techniques applicable to users with diverse kinds of disabilities.

**Acknowledgments** This research was supported by the Basque Government, Department of Education, Universities and Research under grant IT980-16 and by the University of the Basque Country under grant US20/19. The authors
thank all the individuals who participated in the study, as well as to the Begiris
and Begisare associations of people with visual impairments for helping with the
recruitment of the participants.

References

[1] Akpınar, E., Yeşilada, Y.: Old habits die hard!: eye tracking based experiential transcoding: a study with mobile users. In: Proceedings of the 12th International Web for All Conference. ACM (may 2015). https://doi.org/10.1145/2745555.2746646

[2] Allan, J., Kirkpatrick, A., Henry, S.L.: Accessibility requirements for people with low vision (Mar 2016), https://www.w3.org/TR/low-vision-needs/

[3] Bukhari, A.C., Kim, Y.G.: Ontology-assisted automatic precise information extractor for visually impaired inhabitants. Artificial Intelligence Review 38(1), 9–24 (may 2011). https://doi.org/10.1007/s10462-011-9238-6

[4] European Commission: The adoption of a directive on the accessibility of the sector bodies' websites and mobile apps (Oct 2016), https://ec.europa.eu/digital-single-market/en/news/adoption-directive-accessibility-sector-bodies-websites-and-mobile-apps

[5] Gambino, O., Pirrone, R., Giorgio, F.D.: Accessibility of the italian institutional web pages: a survey on the compliance of the italian public administration web pages to the stanca act and its 22 technical requirements for web accessibility. Univers. Access Inf. Soc. 15(2), 305–312 (2016). https://doi.org/10.1007/s10209-014-0381-0

[6] Gamecho, B., Minon, R., Aizpurua, A., Cearreta, I., Arrue, M., Garay-Vitoria, N., Abascal, J.: Automatic generation of tailored accessible user interfaces for ubiquitous services. IEEE Transactions on Human-Machine Systems 45(5), 612–623 (oct 2015). https://doi.org/10.1109/THMS.2014.2384452

[7] Gruber, T.R.: A translation approach to portable ontology specifications. Knowledge Acquisition 5(2), 199–220 (jun 1993). https://doi.org/10.1006/knac.1993.1008

[8] Harper, S., Bechhofer, S.: SADIe: structural semantics for accessibility and device independence. ACM Transactions on Computer-Human Interaction 14(2), 10 (aug 2007). https://doi.org/10.1145/1275511.1275516

[9] Jaeger, P.T.: Assessing section 508 compliance on federal e-government web sites: A multi-method, user-centered evaluation of accessibility for persons with disabilities. Government Information Quarterly 23(2), 169–190 (jan 2006). https://doi.org/10.1016/j.giq.2006.03.002

[10] Kirkpatrick, A., Connor, J.O., Campbell, A., Cooper, M.: Web content accessibility guidelines (wcag) 2.1 (Jun 2018), https://www.w3.org/TR/WCAG21/

[11] Knutov, E., Bra, P.D., Pechenizkiy, M.: AH 12 years later: a comprehensive survey of adaptive hypermedia methods and techniques. New Review of Hypermedia and Multimedia 15(1), 5–38 (apr 2009). https://doi.org/10.1080/13614560902801608
[12] Lohmann, S., Kaltz, J.W., Ziegler, J.: Model-driven dynamic generation of context-adaptive web user interfaces. In: Models in Software Engineering, pp. 116–125. Springer Berlin Heidelberg (2006). https://doi.org/10.1007/978-3-540-69489-2_15

[13] McGuinness, D.L., van Harmelen, F.: Owl web ontology language (Feb 2004), https://www.w3.org/TR/owl-features/

[14] Money, A.G., Lines, L., Fernando, S., Elliman, A.D.: e-government online forms: design guidelines for older adults in europe. Universal Access in the Information Society 10(1), 1–16 (may 2010). https://doi.org/10.1007/s10209-010-0191-y

[15] Moreno, L., Martínez, P., Muguerza, J., Abascal, J.: Support resource based on standards for accessible e-government transactional services. Computer Standards & Interfaces 58, 146–157 (may 2018). https://doi.org/10.1016/j.csi.2018.01.003

[16] Norwegian Ministry of Trade and Industry: Elmer: User interface guidelines for governmental forms on the internet (Aug 2011), https://brukskvalitet.brreg.no/elmer21/elmer_2_1_english.pdf

[17] Pérez, J.E., Arrue, M., Valencia, X., Abascal, J.: Longitudinal study of two virtual cursors for people with motor impairments: A performance and satisfaction analysis on web navigation. IEEE Access 8, 110381–110396 (2020). https://doi.org/10.1109/ACCESS.2020.3001766

[18] Rosales-Huamaní, J., Castillo-Sequera, J., Puente-Mansilla, F., Boza-Quispe, G.: A prototype of a semantic platform with a speech recognition system for visual impaired people. Journal of Intelligent Learning Systems and Applications 07(04), 87–92 (2015). https://doi.org/10.4236/jilsa.2015.74008

[19] Sala, A., Arrue, M., Pérez, J.E., Espín-Tello, S.M.: Measuring complexity of e-government services for people with low vision. In: Proceedings of the 17th International Web for All Conference. ACM (apr 2020). https://doi.org/10.1145/3371300.3383350

[20] Sala, A., Arrue, M., Pérez, J.E., Espín-Tello, S.M.: Towards less complex e-government services for people with low vision. ACM SIGACCESS Accessibility and Computing 1(128), 1–10 (dec 2020). https://doi.org/10.1145/3441497.3441499

[21] Shi, Y.: The accessibility of chinese local government web sites: An exploratory study. Gov. Inf. Q. 24(2), 377–403 (2007). https://doi.org/10.1016/j.giq.2006.05.004

[22] United Nations, Department of Economic and Social Affairs: Un e-government survey 2020, digital government in the decade of action for sustainable development. Tech. rep., UN (2020), https://publicadministration.un.org/egovkb/Portals/egovkb/Documents/un/2020-Survey/2020UNE-GovernmentSurvey(FullReport).pdf

[23] Valencia, X., Pérez, J.E., Arrue, M., Abascal, J., Duarte, C., Moreno, L.: Adapting the web for people with upper body motor impairments using touch screen tablets. Interacting with Computers 29(6), 794–812 (aug 2017). https://doi.org/10.1093/iwc/iwx013