<figure><img src="image_1.png" /></figure>

HAL
open science

Model-Based Accessible User Interface Generation in
Ubiquitous Environments

Raúl Miñón, Julio Abascal, Amaia Aizpurua, Idoia Cearreta, Borja Gamecho, Nestor Garay

► To cite this version:

Raúl Miñón, Julio Abascal, Amaia Aizpurua, Idoia Cearreta, Borja Gamecho, et al. Model-Based Accessible User Interface Generation in Ubiquitous Environments. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.572-575, 10.1007/978-3-642-23768-3_85. hal-01597001

HAL Id: hal-01597001

https://inria.hal.science/hal-01597001

Submitted on 28 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Model-Based Accessible User Interface Generation in Ubiquitous Environments

Raúl Miñón, Julio Abascal, Amaia Aizpurua, Idoia Cearreta, Borja Gamecho, and
Nestor Garay

Informatika Fakultatea. University of the Basque Country
Manuel Lardizabal 1, 20018 Donostia, Spain
{raul.minon, julio.abascal, amaia.aizpurua, idoia.cearreta, borja gamecho,
nestor.garay}@ehu.es

**Abstract.** This paper presents a system that automatically generates accessible interfaces tailored to the users' capabilities and needs in order to provide them with access to ubiquitous computing environments. The aim is to ensure that people with disabilities are able to use ubiquitous services provided by intelligent machines, such as ATMs and vending machines. The tailored interfaces are generated from a formal description specified by a User Interface Description Language, and based on user and context models represented by ontologies.

**Keywords:** Adapted user interfaces, accessibility, ubiquitous computing.

## 1 Introduction

People with physical, sensory or cognitive disabilities are frequently excluded from digital services since they cannot use interactive systems that have been designed ignoring human diversity. Due to the advances in mobile technology, ubiquitous services have flourished in diverse contexts. In a typical ubiquitous scenario a user equipped with a mobile device enters a ubiquitous environment, where a software layer -called middleware- enables access to the ubiquitous system, and the local services are offered to her or him. If the user selects one of the available services, a user interface is downloaded to the user device. In this way, the user can access the service through the user interface using her or his own handheld device.

In order to improve the autonomy and inclusion of people with disabilities, “intelligent” machines (such as information kiosks, ATMs, vending machines, home appliances, etc.) must offer ubiquitous services in an accessible way. This will enable users to conveniently access these services through their mobile device, which is adapted to their needs and preferences. Nevertheless, service providers usually offer a unique user interface for all users, which usually does not cater to every user’s needs. Therefore, a user may experience accessibility problems when trying to access such services.
## 1.1 Related Work

Among the abundant works on adapted user interaction, only a few research works are devoted to adapting the content, presentation or the navigation scheme of the user interface to users with special needs. Nevertheless, some interesting works can be found in the literature [1, 2].

In the context of adaptive systems, the adaptations are generally applied to a previously created user interface. However, in the case of adaptive systems for ubiquitous environments, the adaptation process is not based on an existing user interface but on an abstract description of the functionality of the service [3]. This entails new challenges, such as determining the structure and organization of the elements that will comprise the final user interface. In addition, the accessibility of the interface can be considered before creating it. This approach avoids the need to find and eliminate the accessibility barriers that may exist in a previously designed user interface.

These two research lines (adaptive accessible user interfaces for people with special needs and automatic generation of user interfaces for ubiquitous environments) were combined in the Inredis<fnref n="1" /> project. This project was intended to provide users with disabilities with access to ubiquitous services through their own mobile devices in an accessible manner. One of the outcomes of the project was the software component called *Interface Generator* (IG). This module was responsible for creating a final user interface adapted to each user’s needs, enabling the use of the service without accessibility barriers.

This paper describes the approach that was followed in the project for automatically generating accessible user interfaces. The rest of the paper is organized as follows: in section two, the architecture of the Interface Generator is presented; the features of the user interfaces are explained in section three; finally, some conclusions are summarized in section four.

## 2 Architecture of the Interface Generator

The IG component follows a modular architecture, which is sufficiently flexible to accommodate new modules in the future. Each module of the IG performs a specific function, as explained below:

The **Orchestrator** module is in charge of organizing all the processes in the system. It plays a key role in the independence of the modules and it also enables the integration of new modules. The **Resource manager** is responsible for analyzing the resources offered by the provider of each service, and selecting the resource types (text, images, etc.) that are most appropriate for the user and her or his device. The **Constructor** creates a user interface, without any adaptations, from a User Interface Description Language. The **Selector** chooses the required adaptations by considering the users' capabilities, in order to provide them with an accessible user interface. Two

<footnote n="1">¹ http://www.inredis.es/Default.aspx</footnote>
examples of adaptations are “enlarging text size” and “changing the contrast”. The
adaptations are implemented with EXtensible Stylesheet Language<fnref n="2" /> (XSL)
transformations and Cascading Style Sheets<fnref n="3" /> (CSS) code fragments. The **Adapter**
applies the previously selected adaptations into the interface created by the
Constructor. In this way, an accessible interface that is also adapted to the users’
needs is obtained. The **Data Injector** first checks whether a user interface already
exists for the user, her or his device and the service. If so, the interface is downloaded
to the user device instead of repeating the whole generation process.

In addition to the aforementioned modules, the IG makes use of a *Knowledge Base* (KB) which is composed of different ontologies for each of the required models: user, access device and context. The ontologies are queried by the IG during the interface generation process in order to retrieve information from the models.

Apart from these ontological models, which are focused on the user side, the IG employs user interface abstract models [4]. These models enable the automatic generation of interfaces regardless of the underlying technology (e.g. system platform or markup language). Starting from an abstract description of the functions provided by the service, the system is able to generate a functional user interface that allows users to access ubiquitous services and devices. In the particular case of Inredis, the User Interface Markup Language<fnref n="4" /> (UIML) was chosen because it was the language that best suited the project requirements.

## 3 User Interfaces

The final user interfaces automatically generated by the IG are focused on certain user stereotypes. The system generates different interface alternatives for the same service depending on the person who is interacting. In order to demonstrate the viability of the system, three different web-based user interfaces were selected to be generated: a text-only interface, an iconic interface and a mixed interface that incorporates both textual and iconic elements.

The text-only user interface is divided into two main parts. In the first part the interaction commands are placed, whereas in the second part each command is described to facilitate the use and understanding of the service functions. This interface is aimed at blind people using screen readers, as well as users with mobile devices that do not support advanced resources.

All the elements in the iconic interface are represented as icons. The interaction
elements for each function of the service are grouped into sections to allow users to
better understand the interface. This version would be valid for people with cognitive
disabilities, prelingual hearing-impairments, and illiteracy.

The mixed interface would be useful for users with mild sensory restrictions; for
example, elderly people or users with little experience in using computing technology.

<footnote n="2">² http://www.w3.org/TR/xsl/</footnote>

<footnote n="3">³ http://www.w3.org/Style/CSS/</footnote>

<footnote n="4">⁴ http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=uiml</footnote>
The aim is to create a redundant interface that contains simple text to label the interaction elements, accompanied by icons that emphasize the meaning of those tags. For this reason, every element in the interface has both textual and iconic resources to be grouped together.

All the generated user interfaces comply with XHTML Basic 1.1⁵ syntax. To this end, the transformation and adaptation rules were implemented with XHTML syntax requirements. In addition, appropriate subsets of the Web Content Accessibility Guidelines<fnref n="6" /> (WCAG) were considered in the generation rules, according to the type of interface and the target stereotype.

# 4 Conclusions

In this paper we have described the Interface Generator software component, which was one of the outcomes of the Inredis project. The prototypes developed demonstrate that the approach followed works well for the automated generation of adapted accessible user interfaces for different types of users in the context of ubiquitous environments. Although the interfaces generated are fully functional, further user testing is required in order to verify the validity and appropriateness of the interfaces for each type of user, as well as to check the accessibility and usability of the generated interfaces for each target user.

**Acknowledgments.** This research work has been partly funded by the Department of Education, Universities and Research of the Basque Government. In addition, A. Aizpurua, B. Gamecho and R. Miñón enjoy PhD scholarships from the Research Staff Training Programme of the Department of Education, Universities and Research of the Basque Government. We thank the INREDIS project, which has been the foundation for this work.

# References

1. Fink, J., Kobsa, A., Nill, A.: Adaptable and Adaptive Information Provision for All Users, Including Disabled and Elderly People. New Review of Hypermedia and Multimedia, 4(1), pp. 163--188 (1998)

2. Gajos, K.Z., Weld, D.S., Wobbrock, J.O.: Automatically generating personalized user interfaces with Supple. J. Artificial Intelligence, 174(12-13), pp. 910--950 (2010)

3. Hervás, R., Bravo, J.: Towards the ubiquitous visualization: Adaptive user-interfaces based on the Semantic Web. Interacting with Computers, 23(1), pp. 40--56 (2011)

4. Guerrero-Garcia, J., Gonzalez-Calleros, J.M., Vanderdonckt, J., Munoz-Arteaga, J.: A theoretical survey of user interface description languages: Preliminary results. In: The Latin American Web Congress (LA-WEB '09), pp. 36--43. IEEE Computer Society, Washington, DC, USA, (2009)

<footnote n="5">⁵ http://www.w3.org/TR/xhtml-basic/</footnote>

<footnote n="6">⁶ http://www.w3.org/TR/WCAG20/</footnote>