# Using Virtual Reality to Enable Individuals with Severe Visual Disabilities to Read Books

Kurtis Weir, Fernando Loizides, Vinita Nahar, Amar Aggoun

► To cite this version:

Kurtis Weir, Fernando Loizides, Vinita Nahar, Amar Aggoun. Using Virtual Reality to Enable Individuals with Severe Visual Disabilities to Read Books. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.680-684, 10.1007/978-3-030-29390-1_62. hal-02878636

HAL Id: hal-02878636

https://inria.hal.science/hal-02878636

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Using Virtual Reality to Enable Individuals with Severe Visual Disabilities to Read Books

Kurtis Weir<fnref n="1" />[0000-0002-7094-4600], Fernando Loizides<fnref n="2" />[0000-0003-0531-6760], Vinita Nahar<fnref n="1" />[0000-0002-0519-5260], and Amar Aggoun<fnref n="1" />[0000-0002-0823-6363]

**Abstract.** In this work, we present a bespoke assistive tool for people with severe visual disabilities. We are able to download text from books and present these books to our users in a virtual reality environment. This gives them specific capabilities to manipulate the text and factors such as brightness, size and contrast, in order for them to gain a comfortable reading experience.

**Keywords:** Reading · Virtual Reality · Visual Disabilities

## 1 Introduction

The rise of Virtual Reality (VR) and Augmented Reality (AR) technologies has bred a new competing market that is focused primarily on entertainment. This technology advances rapidly with newer head mounted displays (HMD), featuring higher resolutions, improved ergonomics, and better portability. We are concerned however, with a lack of attention to the needs of disabled users, and specifically those with severe visual impairments. Exploration between current available VR app marketplaces between different VR devices show that some early attempts into digital eBooks already exist [4], yet they do not focus on reasonable accessibility features that would make said applications usable for disabled persons. In previous research we investigated the potential for specialised VR equipment for visually impaired users and conducted 24 case studies that identified VR potential in improving their visual acuity (under review). The user group tested were what we would consider legally blind, requiring daily assistance and having little to no reading ability without severe aids, many whom were surprised at their increased acuity during testing over their current aids. During that study, we also elicited requirements from the participants on assisted living capabilities they wish to utilise VR/AR technology for. One dominant need was to improve (or in some cases restore) the ability to read. Currently, such software does not exist, and book readers that do exist are extremely limited in their accessibility. The aim of this paper is to present the development of specialised software that can accommodate the reading needs of severely visually

<footnote n="1">¹ University of Wolverhampton, UK</footnote>

<footnote>{K.Weir,Vinita.Nahar,A.Aggoun}@wlv.ac.uk</footnote>

<footnote n="2">² National Software Academy, Department of Computer Science and Informatics, Cardiff University, UK</footnote>

<footnote>loizidesf@cardiff.ac.uk</footnote>
impaired users and explain its features. The presented work will feed towards
a full series of integrated disability software aimed to promote disability focus
within VR systems.

## 2 Related Work

To highlight the potential and need for e-reading software on VR technology, we give a background foundation of similar electronic reading research and past findings. Low-vision aids are currently necessary for reading accessibility [5]. When looking at the history of electronic approaches to reading, extensive research on the comparative reading ability between physical paper and computer screen recreations already exist, with many authors mimicking results that computer screen reading has been slower between studies [2, 3, 6, 9]. Causes for these decreases in reading time via computer screens were credited to technical limitations such as display qualities, and even psychological aspects such as meta-cognitive regulation [1]. Although these findings surround typical computer screen readings (i.e. LCD monitors read at a distance), they serve to both demonstrate limitations with current technology as well as question the feasibility of newer alternatives to traditional reading. A recent study [7] looked at performing similar comparisons between both VR and AR reading speeds against digital screen reading. This research utilised the Oculus Rift CV1 (https://www.oculus.com/rift/ - Accessed April 2019) for VR, the Microsoft HoloLens (https://www.microsoft.com/en-CY/hololens - Accessed April 2019) for AR, and a LCD monitor to present a series of questions that required participants' responses via multiple choices. Results found that responses within both VR and AR devices were 10% slower than computer display, similar to results shown between computer screen and physical paper read speeds. Although results from this study suggest that time to perform tasks was diminished by 10% within VR and AR environments, it is important to note that many other factors could have attributed to this, such as unfamiliarity with new digital environments and interfaces, as well as device variables, such as distance to eyes, field of view, or backgrounds used. These studies discussed thus far have looked at standard reading ability between participants with normal vision, but what of those with severe visual disabilities, where the drawbacks of electronic reading are more than negated by the technology allowing for increased visual acuity over natural vision. Investigations into modern VR and AR systems have explored different visual enhancement techniques, such as the successful manipulation of magnification and contrast via ForeSee [10]. Specialist aid tools do exist that focus on providing reading capability, such as the OrCam device [8] which uses OCR (Optical Character Recognition) to read text via a camera mounted on one's glasses. This is limited by the need for voice feedback and lack of variable manipulation such as brightness and magnification. Finally, the human element of reading is also missing, both in the sense of a user being independently able to 'read' but also to make sense of OCR that may fail.
## 3 Artefact Description

This project utilises a Oculus Rift CV1 as the chosen VR HMD, and Unity (https://unity.com/ - Accessed April 2019) as the primary development platform for its strong VR documentation and compatibility, alongside the C# programming language. The goal was allowing the application to read standard UTF-8 formats and translate them onto a virtual reading panel for users to make their own reading adjustments. This common format allows greater compatibility with most text and books. For our testing we utilised Project Gutenberg (http://www.gutenberg.org/), a free to use eBook website offering over 58,000 royalty free books. Books were downloaded directly from their service and inserted into our application to be automatically translated. Once text is translated into our application, it is displayed in front of the user via a floating panel in a darkened environment. Graphical distractions are kept to a minimum to avoid accessibility issues, so only essentials are displayed to the user. Utilising the Oculus' touch controllers to simulate virtual hands, the user may freely grab a panel with lines of text (Figure 1) by squeezing down the grip button to pick up and re-position the text, and releasing this grip freezes the new position in place. By default the first 5 lines of text from the chosen book are displayed to the user, which is adjustable. The user manipulates environmental variables via voice control, allowing them to change how the text and parts of the virtual world are displayed.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Left: A user grabbing the book with their right hand. Right: Example of the book and font being enlarged, along with colour changes</figcaption></figure>

Currently voice commands allow the user to manipulate these settings: Change the numbers of lines displayed (“more lines”), flip to next page (“next page”), change the font size (“bigger font”), change the size of the book display (“larger book”), change the font type (“font *x*”)<fnref n="3" />, change the lighting/light sources (“brighter light”, “darker light”, “light *x*”), and multiple colour combinations.

<footnote n="3">³ *x* represents number or colour (e.g. Background Black, Font 2)</footnote>
The panel or book itself is split between 3 different planes that can change colour, being the background of the book itself ("Panel x"), the border edges around the book ("Border x"), and the colour of the text itself ("Text x"). Additionally, the background virtual space can change colour from its default black if a voice command is used. The user may also move lights around the environment by utilising their virtual hands, as well as using voice commands to manipulate or enable/disable lighting variables, allowing them to use light sources from specific angles as virtual torches (Figure 2), or simulate directional light sources for greater clarity depending on their conditions.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Left: The torch tool being shined onto the text for clarity. Right: An example of further background colour combinations</figcaption></figure>

This fully interactive functional prototype application serves as the foundation for developing a reader for severely visually disabled individuals. We aim to use the interactive elements of the technology as a stepping stone to also allow for further text interaction, such as subtitles in videos as augmented reality text reading through real time OCR. We now aim to test the application with patients suffering with macular degeneration, and also other similar limiting conditions to develop the interface to be as adaptive as possible.

# 4 Acknowledgements

This research is funded by Beacon Centre for the Blind (Registered Charity No. 216092) and the University of Wolverhampton.

# References

1. Rakefet Ackerman and Morris Goldsmith. Metacognitive regulation of text learning: On screen versus on paper. *J. of Exp. Psych.: Applied*, 17(1):18, 2011.

2. Andrew Dillon. Reading from paper versus screens: A critical review of the empirical literature. *Ergonomics*, 35(10):1297-1326, 1992.

3. John D Gould and Nancy Grischkowsky. Doing the same work with hard copy and with cathode-ray tube(crt) computer terminals. *Hum. Fac.*, 26(3):323-337, 1984.
4. Jim F Hahn. Virtual reality learning environments: Development of multi-user reference support experiences. *Inf. & Learning Science*, 119(11):652–661, 2018.

5. Keziah Latham. Benefits of low vision aids to reading accessibility. *Vision research*, 153:47–52, 2018.

6. Anne Mangen, Bente R Walgermo, and Kolbjørn Brønnick. Reading linear texts on paper versus computer screen: Effects on reading comprehension. *International journal of educational research*, 58:61–68, 2013.

7. Pei-Luen Patrick Rau, Jian Zheng, Zhi Guo, and Jiaqi Li. Speed reading on virtual reality and augmented reality. *Computers & Education*, 125:240–245, 2018.

8. Michael Waisbourd, Osama Ahmed, Linda Siam, Marlene R Moster, Lisa A Hark, and L Jay Katz. The impact of a novel artificial vision device (orcam) on the quality of life of patients with end-stage glaucoma. *Investigative Ophthalmology & Visual Science*, 56(7):519–519, 2015.

9. RT Wilkinson and Helen M Robinshaw. Proof-reading: Vdu and paper text compared for speed, accuracy and fatigue. *BIT*, 6(2):125–133, 1987.

10. Yuhang Zhao, Sarit Szpiro, and Shiri Azenkot. Foresee: A customizable head-mounted vision enhancement system for people with low vision. In *Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility*, pages 239–249. ACM, 2015.