# Braillestick: A Game Control Proposal for Blind Users
Based on the Braille Typewriter

Kayo C. Santana, Abel R. Galvão, Gabriel S. Azevedo, Victor T. Sarinho,
Claudia P. Pereira

► To cite this version:

Kayo C. Santana, Abel R. Galvão, Gabriel S. Azevedo, Victor T. Sarinho, Claudia P. Pereira. Braillestick: A Game Control Proposal for Blind Users Based on the Braille Typewriter. 19th International Conference on Entertainment Computing (ICEC), Nov 2020, Xi'an, China. pp.142-147, 10.1007/978-3-030-65736-9_13 . hal-03686024

HAL Id: hal-03686024

https://inria.hal.science/hal-03686024

Submitted on 2 Jun 2022

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Braillestick: A Game Control Proposal for Blind Users Based on the Braille Typewriter*

Kayo C. Santana<fnref n="1" />[0000-0003-2374-796X], Abel R. Galvão<fnref n="2" />[0000-0002-6303-9362], Gabriel S. Azevedo<fnref n="2" />[0000-0003-1596-2141], Victor T. Sarinho<fnref n="1" />[0000-0002-5653-8390], and Claudia P. Pereira<fnref n="1" />[0000-0002-2552-2089]

**Abstract.** Nowadays, modern technology is increasingly present in society. However, for the visually impaired people, this technological advance is not so inclusive, especially in the learning processes. This article discusses the development of Braillestick, a joystick similar to the braille machine but with a low cost and greater portability for different platforms. This joystick aims to be a pathway for the development and use of serious games for audiences with visual problems.

**Keywords:** Game Joystick · Braille Device · Assistive Technology.

## 1 Introduction

By the growth of technology in the 21st century, the use of the internet in routine activities has become practically indispensable [1]. As a result, common activities like scheduling an event on the smartphone, until something more complex like developing natural language bots, has become simpler and possible for people in general. However, when considering people with visual impairments, although some research has focused on the development of learning environments for them [2–4], they do not have in most cases a viable access to current Information Technology (IT) solutions.

Regarding the development of learning environments for visually impaired, the use of digital games can help in amused ways, with different applications in education, culture, training and practicing [4–6]. However, due to the constant IT evolution, the teaching and learning process through games needs to be constantly updated to find accessible ways for all audiences. As a result, it is necessary to develop alternative and inclusive technologies for people with visual impairments that do not deprive the game experience of the final user.

<footnote n="1">¹ Computer Science Post-Graduate Program - State University of Feira de Santana.
Feira de Santana, Bahia, Brazil</footnote>

<footnote n="2">² Department of Exact Sciences - State University of Feira de Santana. Feira de
Santana, Bahia, Brazil
{kayosantana94, abel.ramalho18, gabrielsilvadeazevedo}@gmail.com,
{vsarinho, and claudiap}@uefs.br</footnote>

<footnote>* Supported by organization CAPES.</footnote>
Looking for an accessible and portable alternative for people with visual impairments to play serious games related to the Braille writing system, this paper presents the Braillestick proposal. It is an adapted game joystick that works like a Braille Typewriter (BT) and allows people to learn and practice the Braille writing while playing.

## 2 Related Work

The Braille System is a valuable tool for people with congenital or acquired blindness, as long as it allows them to read and write. Braille was created by Loius Braille, in 1825, and today is still the most precious instrument of writing and reading for visually impaired people, allowing direct contact with words [7, 8].

Over the years, some researches have been responsible for spreading the Braille writing process across different devices, such as smartphones, computers and tablets. As an example, Hamzah and Fadzil [9] proposed the Voice4Blind application, which allows users to type using a keyboard adapted for Braille when connected to mobile devices. Per Lee and others [10] and Yang and others [11] proposals, devices that work like a glove were developed, allowing the writing of characters by the movement of the hands, where position and movement of the fingers are similar to the form used on the BT. Similarly, Chakraborty and others [12] proposed the FLight, a system of reading and writing that uses a low-cost wearable glove device that reads Braille printed texts and a special ruler containing blocks for easy writing.

To help visually impaired children, Gadiraju [13] proposed the BrailleBlocks learning system. It consists of a set of wooden blocks and pins representing a Braille cell that can be used by children, together with an interface computer with games for their parents. Google also launched a Braille keyboard for Talk-Back [14], a screen reader application for blind Android users. This proposal seeks to work similarly to the standard systems of this format, with 6 buttons distributed on the sides of the Android device [14].

Similar to those proposals, Braillestick offers an input mechanism that can be used in different applications, and also as a game control for serious games related to the Braille writing system. Unlike such proposals, Braillestick is a hardware solution that works like the BT, using its written mode to convert the Braille writing system to an equivalent in the latin alphabet.

## 3 Methodology

The Design Thinking methodology was applied in the development of the game device. It was chosen because of its potential to not only look into the product's development, in this case the Braillestick, but also in the target public needs and feedback.

The Design Thinking can be carried out according to 5 main steps [15]: Empathize, Define, Ideate, Prototype and Test. In Empathize, the team
needs to recognize the target audience of the product in order to create a meaningful innovation for them. After that, in the **Define** step, users' problems must be thoroughly reviewed to be overcome, leading to different ideas that will be explored in the **Ideate** step. Finally, the **Prototype** step is reserved to the design of artifacts that will help to get closer to the final solution, being redesigned according to users' experiences and opinions observed in the **Test**.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Braillestick design process methodology. Adapted from [15].</figcaption></figure>

Fig. 1 illustrates the Design Thinking steps applied in the Braillestick's development process. As a result, it is possible to visualize at the Ideate step two modeling proposals for the Braillestick prototype (Fig.2). The suggested finger-ing position can also be seen in Fig. 2 by the codes (LT, LI, LM, LR, RT, RI, RM, RR, and RP), where the first letter represents the Left or Right hand, and the second the finger name (Thumb, Index, Middle, Ring, and Pinky).

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Braillestick modeling proposals.</figcaption></figure>

These two proposals offer two main differences for users. While the prototype idea on the left side is a proposal to use holding with the hands (similar to a smartphone-joystick), which allows its use in different places, the second one on
the right side is a proposal that needs to be over a surface, being more similar
to the BT.

4 Obtained Results

For the development of an initial prototype, the Arduino UNO R3 was chosen, which is an open source hardware platform that is ideal for the creation of devices that allow interaction with the environment using buttons and sensors as input [16]. Arduino has a bootloader implemented on the board and an interface with the computer that uses the Processing language, based on the C/C++ language [17]. Among the attributes of the platform, it is possible to highlight the ease of use and the cost-benefit, since it is affordable.

For the input of Braillestick, the BT was initially observed. It has 6 buttons responsible for writing the characters in Braille. Each pressed button creates a relief in a different place in the Braille cell, representing different characters according to the pressed buttons simultaneously.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Braillestick’s use scheme.</figcaption></figure>

Fig. 3 shows how the Braillestick works. At 1st moment, the user presses simultaneously buttons from 1 to 6, according to the Braille character he wants to write. After that, at the 2nd moment, the Arduino checks the states of the buttons and sends to a device (e.g. computer), through serial port (USB), a code to identify which buttons were pressed. Finally, at the 3rd moment, the device receives that code and processes it through a driver, developed in Python programming language, for reading the Arduino output and to match the received code with the corresponding letter of the Latin alphabet. For each obtained letter, the **Braillearning** [18] software illustrates the displayed text and the respective braille representation. Moreover, in an entertainment perspective, the **The Braille Typist** game shows the obtained braille training and learning by matching letters according the game logic.

At this point, Braillestick's prototype (Fig. 3) is already working, according
tests performed with the development team in the Braillearning software, that
investigates not only the process of writing but also the shape and dimensions
of the case, providing an ideal finger position.
Until now, the tests were developed by using Braillestick as a input device in Braillearning Software, this way, we were able to test if all possible combination was working fine, according the Braille Alphabet. Besides that, it was also tested Graphic accentuation and other symbols used to perform numbers and capital letters.

# 5 Conclusions and Future Work

Braillestick's development and research refers not only to the production of a game joystick but also to the development of a low-cost assistive technology to help the social and digital inclusion of visually impaired people.

While a Braille Typewriter machine costs around $800.00, Braillestick's production costs were around $60.00 (Arduino, 3D print, and other components), representing a price reduction of 75%. Besides that, Braillestick is also lightweight and easy to transport, attending 3 of the main problems related by users about the Braille Typewriter.

Until then Braillestick's case and its logical part are already developed and working fine. The next steps of the research target the test by visually impaired people, with its use on the Braillearning learning object that is already developed and is working, and with the blind game The Braille Typist, that is currently under development.

As future works, we will also explore the input and precision rates of visually impaired users in typing games using Braillestick and other devices (regular joystick and keyboard, both mapped to work similar as a braille typewriter) to see the benefits of the proposed console when compared with other devices. Furthermore, the development of new games is also expected to disseminate the joystick use and even the Braille learning and application.

# References

1. Hoffman, D.L., Novak, T.P., Venkatesh, A.: Has the internet become indispensable? Commun. ACM 47(7), 37–42 (Jul 2004). https://doi.org/10.1145/1005817.1005818

2. Bhowmick, A., Hazarika, S.M.: An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends. Journal on Multimodal User Interfaces 11(2), 149–172 (2017)

3. Ferreira, F., Cavaco, S.: Mathematics for all: A game-based learning environment for visually impaired students. In: 2014 IEEE Frontiers in Education Conference (FIE) Proceedings. pp. 1–8 (2014)

4. Jaramillo-Alcázar, A., Luján-Mora, S.: Mobile serious games: An accessibility assessment for people with visual impairments. In: Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM 2017, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3144826.3145416

5. Leporini, B., Hersh, M.: Games for the rehabilitation of disabled people. In: Proceedings of the 4th Workshop on ICTs for Improving Patients Rehabilitation Research Techniques. p. 109–112. REHAB ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/3051488.3051496
6. Santana, K., Pereira, C.P., Fernandes, A., Santos, A.J.O.S., Macedo, R.: Blinds, basic education: jogo digital inclusivo para auxiliar o processo de ensino-aprendizagem das pessoas com deficiência visual. In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE). vol. 28, p. 877 (2017). https://doi.org/10.5753/cbie.sbie.2017.877

7. Machado, E.V.: A importância do (re) conhecimento do sistema braille para a humanização das políticas públicas de inclusão. International Studies on Law and Education 9, 49–54 (2011)

8. Roth, G.A., Fee, E.: The invention of braille. American Journal of Public Health 101(3), 454–454 (2011). https://doi.org/10.2105/AJPH.2010.200865, pMID: 21307377

9. Hamzah, R., Fadzil, M.I.M.: Voice4blind: The talking braille keyboard to assist the visual impaired users in text messaging. In: 2016 4th International Conference on User Science and Engineering (i-USEr). pp. 265–270 (2016). https://doi.org/10.1109/IUSER.2016.7857972

10. Lee, S., Hong, S.H., Jeon, J.W.: Designing a universal keyboard using chording gloves. In: Proceedings of the 2003 Conference on Universal Usability. p. 142–147. CUU ’03, Association for Computing Machinery, New York, NY, USA (2002). https://doi.org/10.1145/957205.957230, https://doi.org/10.1145/957205.957230

11. Yang, T.J., Chen, W.A., Chu, Y.L., You, Z.X., Chou, C.H.: Tactile braille learning system to assist visual impaired users to learn taiwanese braille. In: SIGGRAPH Asia 2017 Posters. SA ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3145690.3145710, https://doi.org/10.1145/3145690.3145710

12. Chakraborty, T., Khan, T.A., Al Islam, A.B.M.A.: Flight: A low-cost reading and writing system for economically less-privileged visually-impaired people exploiting ink-based braille system. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. p. 531–540. CHI ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025646, https://doi.org/10.1145/3025453.3025646

13. Gadiraju, V.: Brailleblocks: Braille toys for cross-ability collaboration. In: The 21st International ACM SIGACCESS Conference on Computers and Accessibility. p. 688–690. ASSETS ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3308561.3356104, https://doi.org/10.1145/3308561.3356104

14. Silva, C.C.: Google lança teclado braile gratuito integrado ao android (2020), https://www.tecmundo.com.br/software/151961.google-lanca-teclado-braile-gratuito-integrado-android.htm

15. Institute of Design at Stanford: An introduction to design thinking: Process guide (2010)

16. Souza, A.R.d., Paixão, A.C., Uzêda, D.D., Dias, M.A., Duarte, S., Amorim, H.S.d.: A placa arduino: uma opção de baixo custo para experiências de física assistidas pelo pc. Revista Brasileira de Ensino de Física 33(1), 01-05 (2011). https://doi.org/10.1590/S1806-11172011000100026

17. Arduino: Arduino reference (2020), https://www.Arduino.cc/reference/en?from=Reference.Extended

18. Santana, K., Pereira, C.P., de Santana, B.S.: Braillearning: Software para simular a máquina de escrever em braille. In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE). vol. 30, p. 1101 (2019). https://doi.org/10.5753/cbie.sbie.2019.1101