# Usability Testing Practice at MIMOS Usability Lab

Norfarhana Abdollah, Ashok Sivaji, Masitah Ghazali

► To cite this version:

Norfarhana Abdollah, Ashok Sivaji, Masitah Ghazali. Usability Testing Practice at MIMOS Usability Lab. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.628-629, 10.1007/978-3-319-22723-8_78 . hal-01610796

HAL Id: hal-01610796

https://inria.hal.science/hal-01610796

Submitted on 5 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Usability Testing Practice at MIMOS Usability Lab

Norfarhana Abdollah¹, Ashok Sivaji², and Masitah Ghazali³,

Usability/User Experience Lab

MIMOS Berhad

Technology Park Malaysia

57000 Kuala Lumpur, Malaysia

`{'norfarhana.abdollah, 'ashok.sivaji, 'masitah.ghazali}@mimos.my`

³Faculty of Computing

Universiti Teknologi Malaysia

Skudai 81310 Malaysia

masitah@utm.my

**Abstract.** This paper presents one of our practices in conducting usability testing. Accredited with ISO/IEC 17025:2005 software testing laboratory, we consider ISO usability sub-characteristics as the metrics for the usability evaluation.

**Keywords.** Usability Evaluation, User Experience Testing, Software Testing

## 1 Introduction

Practically, there are many different ways to apply by user experience practitioners to evaluate product's usability; depending on the type of the product; be it a website, a system, a standalone program, hardware devices and many more. We have our own approach to elicit usability of certain product, which depends on several conditions such as number of evaluators/resources available, project schedule, cost, product cycle stage, etc. By referring to [1] which defines usability as a subset of quality in use model consisting of effectiveness, efficiency and satisfaction for consistency with its established meaning, we present one of our usability testing methods we used which incorporates the combination of these three important metrics.

## 2 Usability Testing Practice

### 2.1 Setting up task to measure effectiveness, efficiency, satisfaction

In order to evaluate each usability metrics in a more practical manner, setting up a task requires certain technique [2]. For example, to evaluate effectiveness of a product usage, the task will be setup as of how successful the user completed the task, how often the user produces errors and how easy the user can recover. Meanwhile, to eval-
uate efficiency, the task will be setup with enough repetitions of typical tasks to create realistic work rhythm, or by observing the users at their daily work to look for situations that interrupt or slow them down. For satisfaction, interview or survey will normally be part of the evaluation, or by performing a comparative preference testing.

## 2.2 Giving score

Effectiveness and efficiency are measured by the successful completion of criteria breakdown from a scenario or task. For a task that matches totally to the set criteria, the moderator will mark the score to ‘Yes’. A success mark is given the full credit of 100%. Criteria that does not match at all will be given a ‘No’ mark, with credit of 0%. ‘No’ is normally given for unsuccessful task criteria which may include events such as user giving up, user requires a lot of assistance from moderator, user incompletes the task etc. While partial credit is given based on moderator’s discretion, for e.g. moderator decides that the mistake should be given at least 50% (partial) rather than 0% (no) mark. Measures of satisfaction are taken using post questionnaires with users. The questions will appear each time the user completed or abandons the pre-setup task.

## 2.3 Calculating individual metrics and usability score

Following are the calculations for each metric score:

* Effectiveness (%) = (yes + (partial x 0.5)) / total x 100%

* Efficiency (%) = (yes + (partial x 0.5)) / total x 100%

* Satisfaction (%) = answer point / total point x 100%

The way we calculate the overall usability score is as follows:

* Usability (%) = (effectiveness % + efficiency % + satisfaction %) / 3

The total usability score is the sum of the three metrics scores divided by three, i.e. the average.

# 3 Conclusion

We have applied the method discussed in this paper for many different case studies and tested usability for many different types of products. Comfortable with the method, we have developed and configure our own software tool called Mi-UXLab to evaluate usability which incorporates the discussed method.

## References

1. Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models, Ref. No.: MS ISO/IEC 25010 (2011)

2. Quesenbery, Whitney. "Balancing the 5Es of Usability." *Cutter IT Journal* 17.2 (2004): 4-11.