# Methodology for the Development of Accessible User Interfaces Based on Meta-Model Transformations: The Case of Blind Users

Lamia Zouhaier, Yousra Bendaly Hlaoui, Leila Ayed

► To cite this version:

Lamia Zouhaier, Yousra Bendaly Hlaoui, Leila Ayed. Methodology for the Development of Accessible User Interfaces Based on Meta-Model Transformations: The Case of Blind Users. 16th IFIP International Conference on Computer Information Systems and Industrial Management (CISIM), Jun 2017, Bialystok, Poland. pp.73-84, 10.1007/978-3-319-59105-6_7. hal-01656208

HAL Id: hal-01656208

https://inria.hal.science/hal-01656208

Submitted on 5 Dec 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Methodology for the Development of Accessible
User Interfaces Based on Meta-model
Transformations: the Case of Blind Users

Lamia Zouhaier¹, Yousra Bendaly Hlaoui², and Leila Jemni Ben Ayed³

Laboratory LaTICE Higher School of Sciences and Technologies of Tunis University
of Tunis, Tunisia

**Abstract.** We propose a generic approach for the adaptation of user interfaces to the accessibility context. This approach is based on the Model-Driven Engineering MDE. Thus, we adapt any kind of UI to the accessibility context based on a generic adaptation process. We are based on meta-model transformations parametrized with the context of accessibility. The proposed approach generates modality-based UI adapted to each disability. We have developed an accessibility context related to the type of disability, the used platform, the user identity and his/her situation, the surrounded environment and modelled into ontology. The metamodel transformations were based on mapping rules implemented by the Kermeta language.

# 1 Introduction

The intelligent devices have enhanced the life of the humanity targeting all categories of users. These platforms vary considerably in their properties as the screen size, the resolution and the interaction devices [13]. That is why adapting a user interface (UI) for various user devices is still one of the most interesting topics. In addition, adapting an UI to the user's disability remains the most interesting problem of the accessibility of interactive systems. Users with disabilities usually experience barriers when they interact with UIs which are not adequately adapted to their needs [12, 24].

Unfortunately, no effective solution has been proposed considering the context of accessibility related to people with disabilities. This context is defined as a combination of the disability profile of the user, the physical environment of the used computing platform and the user mobility situation. Therefore, we have developed an accessibility context related to the user disability profile, the used platform, the user identity, his/her mobility situation and his/her surrounded environment [22].

In this paper, we present a generic approach for the adaptation of any type of UI to the user context of accessibility [24]. This solution has to be generic in order to allow the re-usability and to consider all kind of disability. Furthermore, instead of building an adaptation process for each given UI model, we develop
a unique adaptation process getting as input the UI meta-model. To ensure this
genericity, we are based on the Model Driven Engineering(MDE) [15].

Therefore and based on the MDE principals, our approach is defined of a
series of meta-model transformations between different UI models. Thus, we pro-
pose a PIM(Platform Independent Model) to PIM transformation representing
the abstract adaptation process which adapts any kind of UI model to the dis-
ability profile of current user. This transformation provides as output an adapted
UI model to the user disability profile. To ensure the genericity of this process,
we define all categories of user disability profiles with an ontology. As ontolo-
gies [2] [23] are used to specify and manipulate any kind of knowledge [22]. Hence,
the proposed transformation are defined by a set of mapping rules between non
adapted UI metamodel and adapted UI metamodel using the user accessibility
profile ontology as a parameter. That is why, we have considered these mapping
rules as adaptation rules.

Once the adapted UI is provided, we perform a PIM to PSM(Platform Spe-
cific Model) transformation. This transformation consists of a specific adaptation
process of the user disability profile adapted UI model to provide an adapted UI
model to the execution platform. This output is not only adapted to the given
user disability profile but also is adapted to the interaction modalities of the
used execution platform [25]. The interaction modalities (vocal, graphic, mul-
timodal(vocal and graphic), tactile, gestural) depend on the kind of the user
disability. This dependence is specified in our approach by a relationship be-
tween the user disability profile ontology and the technical context ontology.
This relationship allows in the PIM to PSM transformation the right selection
of the convenient and the effective mode of the interaction between the user with
disabilities and their execution platform. In the PIM to PSM transformation, we
have respected the accessibility norms such as the norm WCAG2.0 [7, 10] and
the norm ISO 9241-171 [9] using their directives in our mapping rules [26].

The remainder of the paper is organized as follows. In section 2, we provide
an overview of some related works dealing with accessibility of UI. The third
section is a description of the proposed approach. Section four is dedicated to
focus on the adaptation process and the specific transformation. The fifth section
is an illustration of the proposed approach and sixth one represents an evaluation
of this approach. Finally, in the conclusion section, we present some concluding
remarks and future work.

## 2 Related works

Several MDE approaches [3, 4, 17, 18, 20] have been proposed in the adaptation of the UIs. They have considered, in their adaptation process, only the user preferences and the platform technical context without considering the user accessibility context.

Some of UI context adaptation approaches [1, 6] have considered the acces-
sibility problem in their adaptation process. However, they have only addressed
a limited number of disabilities.
SUPPLE [6] have proposed an automatic personalization of UIs suitable for people with motor and vision impairments. They have adapted the UIs to person's abilities, devices, preferences, and tasks.

EGOKI [1] is an automatic generator of accessible UIs allowing people with disabilities to access supportive ubiquitous services. EGOKI follows a model-based approach and selects suitable interaction resources and modalities depending on users? capabilities. The major limitation of these automated approaches is that they have applied the personalisation of UIs on specific prototypes. However, in our approach we adapt any kind of UI to any kind of user disability considering the execution platform modalities.

## 3 The Adaptation Approach of UIs to the Accessibility Context

We propose a *Generic Model-Driven Adaptation Approach based on the Context of Accessibility*. Hence, we adapt any kind of UI to the developed context of accessibility based on a series of parameterized meta-model transformations. The approach focuses on the use of *modality-transformation* as a key solution to suit the capacities of users with special needs. In fact, the employability of the multi-modality facilitates, firstly, the interaction of the disabled user with the UI and secondly with the platform capacities considering the accessibility context. Furthermore, we have fixed adaptation strategies for each type of disability in order to develop the target meta-models. In this paper, we will be limited only to the case of *blindness disability* which will be presented afterwards.

The proposed approach includes four principal phases:

**Abstract modelling of the non adapted-UI using a Reverse Engineering technique:**

In this phase, we abstract the executed non-adapted UI from the implementation and the modality details by providing an abstract UI non adapted model. This model (PIM-model) represents different abstract and structural elements composing the UI and their relationships.

This phase is based on two steps:

- **Step 1** is the extraction of the different elements and their relationships from the executable non adapted UI to build a *UI-tree* using YED tool [21]. The *UI-tree* is a graph of the structured *UI-elements* which facilitates the development of the *non adapted PIM-model*.

- **Step 2** is the development of the abstract UI model from the *UI-tree*. It transforms the *UI-tree* to an *UML class diagram model*. Each class of this model represents a *tree-node* which is an *UI-element*. The associations of the class diagram model describe the relationships existing between UI-elements represented as edges in the *UI-tree*. The resulted model is conformed to the non adapted UI meta-model.
This metamodel has to describe any type of UIs. It includes the information
about the employed modalities (input, output) defined at the high level of ab-
straction.

**Development of an ontology describing the context of accessibility:**
We have developed an ontology representing knowledge about all accessibility concepts such as the user disability profile, the user situations, the user profile, the technical context and the environment context. This ontology will serves as input for the adaptation process and the specific transformation of the approach. The ontology instances represent the captured real values of the current accessibility context.

**Adaptation process of the non adapted-UI according to an instance of the accessibility context ontology:**

It represents the kernel of the approach. It ensures the adaptation of any type
of UIs to any accessibility context. The adaptation process transforms the *Non
Adapted UI Meta-model* to an *UI-Disability-meta-model* using a given acces-
sibility context ontology instance. It provides an adapted UI model. It is a
parametrized transformation.

**Specific transformation of the *adapted UI-PIM* to a specific platform of execution:**

In this phase, we transforms the *adapted UI-PIM* provided from the adaptation
process to a *PSM model* (*Modality-PSM*) depending on the modalities of the
execution platforms.

This phase includes two steps:

- **Step 1** is an exogenous transformation which represents the *PIM to PSM* transformation depending on the modality

- **Step 2** is an endogenous transformation which is the *PSM to PSM* transformation depending on the target platform web, desktop or mobile.

Table 1 recapitulates different intervening source and target meta-models
during the *PIM to PIM* and *PIM to PSM* transformations represented in figure
1. In this figure, we have only focused on the *PSM to PSM* transformation for
the web platform.

In this paper, we have considered only sensory impairments. We have been
limited to blindness, low vision and deafness impairments. For each category,
we have defined the corresponding target metamodel namely, *BlindnessAdapt-
edPIM*, *LowVisionAdaptedPIM*, *DeafnessAdaptedPIM* and *HardDeafnessAdapt-
edPIM* (Cf. table1)

## 4 The metamodel transformation

The meta-model transformation is based on a set of rules according to the set
of the given disability values. These rules are divided into two types:
<table><caption>Table 1: Source and target Meta-models intervening during the PIM2PIM and PIM2PSM transformation</caption><thead><tr><th>Transformation Type</th><th>Source Metamodel</th><th>Target Metamodel</th></tr></thead><tbody><tr><td rowspan="4">Transformation PIM2PIM</td><td rowspan="4">Non-AdaptedPIM-Metamodel</td><td>BlindnessAdaptedMetamodel</td></tr><tr><td>DeafnessAdaptedMetamodel</td></tr><tr><td>LowVisionAdaptedMetamodel</td></tr><tr><td>HardDeafnessAdaptedMetamodel</td></tr><tr><td rowspan="4">Transformation PIM2PSM</td><td>BlindnessAdaptedMetamodel</td><td>VocalPSM-Metamodel</td></tr><tr><td>DeafnessAdaptedMetamodel</td><td>GraphicalPSM-Metamodel</td></tr><tr><td>LowVisionAdaptedMetamodel</td><td rowspan="2">MultimodalPSM-Metamodel</td></tr><tr><td>HardDeafnessAdaptedMetamodel</td></tr></tbody></table>

- Mapping rules: are used to match an element from the source meta-model to its correspondent of the target meta-model.

- Adaptation rules: are used to structure the adapted UI model. In fact, during the adaptation process, these rules provide the UI framework of the correspondent element provided by the mapping rule. The UI framework is defined accordingly to the user disability context.

The specification of transformation rules is defined as follow:

For each Element *E* of Source Metamodel, match an element *E'* from Target Metamodel based on an instance of the ontology representing a given accessibility context *Instance(OO)*.

To introduce the syntax of mapping rules, we defined the adaptation function A which is described as follows:

$$A: MM_{non\_adapted} \times OO \rightarrow MM_{adapted} \\ (M_{UI_{non\_adapted}}, I_{OO}) \mapsto M^{UI_{adapted}}$$

*MM<sub>non Adapted</sub>* The input Meta-model

*OO*: The Accessibility Ontology Model

*MM<sub>adapted</sub>*: The Target Meta-model

*M<sub>UI<sub>non Adapted</sub></sub>*: the model of the non adapted UI conformed to the source meta-model

*I<sub>OO</sub>*: the instance from the ontology of a given accessibility context.

*M<sup>UI<sub>adapted</sub></sup>*:the adapted model of the UI conformed to the target meta-model

*MM<sub>adapted</sub>*
<figure><img src="image_2.png" /><figcaption>Fig. 1: Diagram of activity of the whole approach process from the adaptation process to the specific transformation cibling the web platform</figcaption></figure>

## 4.1 Modality Transformation

This step focuses on adapting the input (I) and output (O) modalities of the UI in an appropriate manner of the user disability. In the case of a blind user, we have defined two principal rules $R_{B1}$ and $R_{B2}$ which are defined as follows:

$$R_{B1}: A(InputModality, Blindness) = VocalModality$$

$$R_{B2}: A(OutputModality, Blindness) = VocalModality$$

In fact, whatever the input or the output modality, for a blindness, the modality is vocal.

## 4.2 Transformation PIM2PIM: Case of Blindness disability

In this transformation, we use three strategies:

- **Strategy 1:** Since the problem of blind users is related to graphical elements of the UI and in order to offer perceptible elements for the her/him, we will
adopt the vocal mode to represent the different type of elements of the
adapted UI. Hence, each graphical element will be transformed to a vocal
one. This will be done through mapping rules.

- **Strategy 2:** The use of vocal modality into a UI causes a disruption of the user's concentration and his attention. This problem occurs when the user is faced to fill out a form. For this purpose, we will minimize the number of input elements (*InputInteractor*)per grouping unit (*GroupingUnit*). Therefore, we propose to put only one *InputInteractor* in each grouping unit of type *GroupingUnit*. Each presentation unit (*PresentationUnit*) should contain only one *GroupingUnit*. This gives result to new generated UIs, each one contains a single *InputInteractor* element. This strategy will be based on *adaptation rules*.

- **Strategy 3:** Based on the result of the strategy 2, we obtain several *PresentationUNITs* of the adapted UI that are logically linked. This requires a strategy to ensure the dialogue between these different *PresentationUNITs*. For this reason, we will add some additional elements for navigation *NavigationInteractor* named *next* and *previous* allowing navigation between the different *PresentationUNITs*.

Using these strategies, we have obtained the Blindness-PIM-Metamodel for blind-
ness disability represented in Fig.2.

5 Case Study: Blind user

In this case study, we focus only on the adaptation process of an UI of an hotel
reservation application for a blind user using his laptop. Figure 3a depicts the UI
hotel reservation and its corresponding PIM model represented by the Ecore [16].
To start, we determined the type of the UI (desktop), its domain (tourism) and
the input and output interaction modalities (graphical). As shown in Fig.3a,
the PIM model of the UI contains only one *PresentationUnit* that includes two
main *GroupingUnits Formulaire* and *Date and validation*. The distribution of
the elements within each *GroupingUnit* is as follows:

1. The element *GroupingUnit Formulaire* contains three input elements of type *InputInteractor* and three associated elements of type *TextOutput*: called *Name*, *Telephone*, *Email* and a *GroupingUnit* called *Region and hotel* including a selection element *SelectionInput* called *Hotel*, an output element *TextOutput* called *Select Hotel*, a selection element *SelectionInput region* and an output element *TextOutput* called *Select region*.

2. The element *GroupingUnit* named *Date and validation* contains three selection elements *SelectionInput* called as follows: *Day of Arrival*, *Month of Arrival* and *Year of Arrival* to introduce the date of arrival. Each one is associated to an output element *TextOutput*. It includes, also, three others *SelectionInput Day of departure*, *Month of Departure*, *Year of Departure* associated to their correspondent *TextOutput*. It contains also, two activator elements *ActivatorInteractor*: *Reserve* and *Cancel*.
<figure><img src="image_3.png" /><figcaption>Fig. 2: Blindness-PIM-Metamodel specific to blindness disability</figcaption></figure>

Following the adapted Blindness PIM Metamodel and based on the transformation rules, the obtained result is the generation of adapted UIs represented into Fig.3b, Fig.3c and Fig.3d. The result is generated by Kermeta language [14].
<figure><img src="image_4.png" /><figcaption>Fig. 3: The adaptation of a partial from the Reservation Hotel UI (a) based on the metamodel transformation for blind users for generating several Presentation UNIT (b)UI-1, (c) UI-2 , (d) UI-3, and others.</figcaption></figure>

6 Evaluation, Results and Interpretation

To asses the results produced by our methodology, we have conducted an em-
pirical study to improve the satisfaction of users with disabilities towards the
proposed strategies and the interaction modalities. Users must express their real
feeling and their comfort when interacting with adapted UIs. For that end, we
have based on the interface UI-1 (see Fig.3a). As apparatus, we have used a PC
model ASUSN56 with a screen size 15,4" set to 1920 by 1080 pixels resolution,
with a 64bit color palette. The apparatus is equipped with a loud speaker that
will be used for voice output and a microphone for voice input. To evaluate the
usability of the approach, 42 participants are selected carefully to achieve the
desired results where 12 participants are blind, 14 are visually impaired and 16
are deaf. The picked subjects have different skills (low, medium, high) on using
computing platforms.

Two instruments were the basis of this empirical study:

- *System Usability Scale (SUS)*: we have based on the standardized SUS questionnaire [11, 19] to be filled by subjects. SUS is a reliable low cost usability scale used for global assessment of system's usability aiming to determine the level of user's satisfaction of the given system. It consists of 10 items [19] measuring two factors: the *usability* and the *learnability* [11]. The usability is described through 8 items which are Q1, Q2, Q3, Q5, Q6, Q7, Q8 and Q9. While the *learnability* is defined by the 2 items Q4 and Q10. Participants are asked to provide a 5-point scales numbered from 1 anchored with "Strongly Disagree" to 5 anchored with "Strongly Agree".

- *Test Level Satisfaction:* This metric measures the relative satisfaction and preference among the *adapted UI-1* and the adapted interaction modalities that match better each disability. Each participant is asked to evaluate its satisfaction by providing a 7-point scales numbered from 1 anchored with "Very Bad" to 7 anchored with "Excellent".
To estimate the reliability of the SUS questionnaire, we have used the coefficient *Cronbach's Alpha* [5] where the threshold of acceptability is 0.7. The results recapitulated in table 2 show that the *SUS score* = 73.92 which represents a good SUS score result. On the other hand, the value of the Alpha α = 0.839 > 0.8 which represents good internal consistency among the items. Besides, considering the two factors usability and learnability, it represents a reasonable and acceptable reability in term of usability factor (α = 0.781) however, it is considered a questionable scale for learnability factor (α = 0.592).

Figure 4 represents the result of test level satisfaction performed by all participant considering their experience with computing platforms.

These results indicate that all users find that it is usable to use the system but it is difficult to learn rapidly how to interact with the UIs using the different interaction modalities. It should be noted that this problem is remarkable for blind users using vocal modality whereas those who are deaf and visually impaired encounter fewer difficulties using the graphic modality. The rate of satisfaction is high when users are experienced with the use of computing platform specifically deaf and visually impaired users.

<table><caption>Table 2: Reliability Statistics</caption><thead><tr><th colspan="2">N of items Alpha's Cronbach SUS score Std.dev</th></tr></thead><tbody><tr><td>Usability</td><td>8 0.781 64.78 7.79</td></tr><tr><td>Learnability</td><td>2 0.592 9.14 4.14</td></tr><tr><td>Total</td><td>10 0.839 73.92</td></tr></tbody></table>

# 7 Conclusion

In this paper, we have presented an MDE approach for the generation of adaptable UIs considering the context of accessibility. The approach is based on meta-model transformations. Thus, we have defined a set of transformation rules (mapping and adaptation rules) for the different transformations. One case study has been described showing how this approach can support the adaptation process in the case of blind users. An empirical study has been conducted to evaluate the usability of the approach and the efficiency of the different developed meta-model transformations and their associated transformation rules. As future work, we would like to develop a graphical tool using the Eclipse Graphical Modeling Framework (GMF) [8] in order to easily draw UI models conforming to the proposed meta-models. We aim to make an automatic process of the whole approach.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Test Level Satisfaction</figcaption></figure>

## References

1. Abascal, J., Aizpurua, A., Cearreta, I., Gamecho, B., Garay, N. and Min, R.: *A modular approach to user interface adaptation for people with disabilities in ubiquitous environments*, In Internal Technical Report No. EHU-KAT-IK-01-11(2011).

2. Benaboud, R., Sahnoun, Z. and Maamri, R.:(2010) *User's preferences and experiences based web services discovery using ontologies*,in RCIS, , pp. 121-126 (2010).

3. Brossard, A., Abed, M. and Kolski, C.: *Taking context into account in conceptual models using a Model Driven Engineering approach*, Information and Software Technology, vol 53 (12), pp. 1349-1369(2011).

4. Bouchelliga, W., Mahfoudi, A., Benammar, L., Rebai, S., Abed, M.: *An MDE approach for user interface adaptation to the context of use*. In Proceedings of the 3rd international conference on Human-centred software engineering. Springer, 62-78 (2011).

5. Carricano, M., Poujol, F.: *Analyse de données avec SPSS*, Pearson Education France (2008).

6. Gajos, Z., Weld, D.S., Wobbrock, J.O.: *Automatically generating personalized user interfaces with Supple*, Artificial Intelligence, 174:910-950 (2010).

7. Henry, S.H.: Web Content Accessibility Guidelines (WCAG) Overview, http://www.w3.org/WAI (2012).

8. IBM, Eclipse Graphical Modeling Framework (GMF), http://www.eclipse.org/modeling/gmf.

9. ISO.: *ISO 9241-171, Ergonomics of human-system interaction, Part 171: Guidance on software accessibility* (2008).

10. ISO.: ISO/IEC DIS 40500 Information technology, W3C Web Content Accessibility Guidelines (WCAG) 2.0 (TC/SC: JTC 1)(2012).
11. Lewis, J.R. and Sauro, J.: *The Factor Structure of the System Usability Scale*, In Human Centered Design, Vol 5619, Lecture Notes in Computer Science, Springer, pp 94-103 (2009).

12. Minon, R., Paterno, F., Arrue, M., Abascal, J.: *Integrating adaptation rules for people with special needs in model-based UI development process*, Journal of Universal Access in the Information Society, Springer Verlag (2015).

13. Mitrovic, N., Bobed, C., Mena, E.: *A Review of User Interface Description Languages for Mobile Applications*, In the Tenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies UBICOMM (2016).

14. Muller, P.A., Fleurey, F., and Jezequel, J.M.: *Weaving executability into object oriented meta-languages*. Model Driven Engineering Languages and Systems, pp. 264-278 (2005).

15. OMG, *Model Driven Architecture (MDA)*, document number ormsc/2001-07-01 (2001).

16. OMG, *EcoreTools* - *Graphical Modeling for Ecore*, http://www.eclipse.org/ecoretools/

17. Oliveira, K., Bacha, F., Mnasser, H. and Abed, M.: Transportation ontology definition and application for the content personalization of user interfaces, Journal Expert System with Applications, vol 40, pp. 3145-3159 (2013).

18. Paterno, F., Santoro, C. and Spano, L. D.: "Model-based Design of Multi-Device Interactive Applications based on Web Services", Human-Computer Interaction, INTERACT, LNCS Volume 5726, pp 892-905 (2009).

19. Sauro, J.: *Measuring usability with the System Usability Scale (SUS)*, https://measuringu.com/sus/ (2011).

20. Thevenin, D.: *Adaptation en interaction homme-machine: Le cas de la plasticité*, Phd Thesis, Universit Joseph Fourier, Grenoble, France (2001).

21. yEd Graph Editor.: https://www.yworks.com/products/yed (2016).

22. Zouhaier, L., Hlaoui Bendaly, Y., Jemni Ben Ayed, L.: *Automatic Generation of UIs for Disabled Users Using Context-Aware Techniques and Reasoning*, In Proceedings of the 5th the International Conference on Knowledge Engineering and Ontology Development KEOD (2013).

23. Zouhaier, L., Hlaoui Bendaly, Y., Jemni Ben Ayed, L.: *A Global Approach of Accessible Context-sensitive User Interface*, Journal of Data Processing (2013).

24. Zouhaier, L., Hlaoui Bendaly, Y., Jemni Ben Ayed, L.: *A Model Driven Approach for Improving the Generation of Accessible User Interfaces*, ICSOFT-PT (2015).

25. Zouhaier, L., Hlaoui Bendaly, Y., Jemni Ben Ayed, L.: *Generating Accessible Multi-modal User Interfaces Using MDA-Based Adaptation Approach*, COMPSAC (2014).

26. Zouhaier, L., Hlaoui Bendaly, Y., Jemni Ben Ayed, L.: *A MDA-based Approach for Enabling Accessibility Adaptation of User Interface for Disabled People*, In Proceedings of the 16th International Conference on Enterprise Information Systems, ICEIS, pages 120-127 (2014).