九州地区

教育システム情報学会 2019年度学生研究発表会

The Prototypes of Braille to Speech Application
as a Self-Learning Support Media for Visually Impaired Person*1

Jevri Tri ARDIANSAH*1, Hisaru TANAKA*1, Yasuhisa OKAZAKI*1
Faculty of Science and Engineering, Department of Intelligent Information Systems, Saga University
Email: 19704007@edu.cc.saga-u.ac.jp

Abstract：Visually Impaired Person (VIP) is a person who has a reduction of eyesight. VIP uses Braille as a written
communication tool. This application is developed to let VIP learn Braille by themselves so that they could masterthe beginner braille faster. By Prototyping method, two mains mechanism has been developed, they are the brailletranslation mechanism and the scanning mechanism. The translation mechanism could translate braille to words
from the taken braille picture. Since taking the braille picture manually is impossible for VIP, the braille scanningmechanism is developed. By scanning and letting the square pattern in frame the camera, the prototype will take thepicture automatically which the braille is located inside that square pattern.
Key words：Visually Impaired Person (VIP), Braille, Mobile App, OpenCV, Speech

1.

Introduction

help VIP in discovering the meaning of Braille written in
the public facilities by phone with embedded camera.
Based on the problem and the potential solution, asmartphone application to help VIP in learning Braille isneeded to develop. The main idea is the system will
convert the Braille to speech to help learners inunderstanding and practicing Braille reading by
themselves instead of the help from instructor just by
capture to the Braille written on their book by their own
smart phone.

According to World Health Organization (WHO),around 1.3 billion people around the world are VisualImpaired Person (VIP) (1). In general, visual impairment
is defined as the reduction of the eyesight that cannot becorrected by standard glasses or contact lens and it
reduces a person’s ability to perform some daily tasks (2).
Braille is tactile format of written communication forpeople with low vision and blindness worldwide, since
its inception by Louis Braille in 1829 (3). Braille was asystem of embossed or raised signs which used 6 dots torepresent different characters, which were arranged in 3
rows and 2 columns (4) as shown in Figure 1. Uniquecombinations of raised dots would represent different
characters (5).

2.

System Overview

To help VIP in learning braille by themselves, theapplication will translate the scanned braille picture tospeech sound produced by the application. By thatspeech, it checks the content of the braille so that the VIP
will be able to know whether they are correct or wrong inlearning braille. The mechanism of the application isshown as the Figure 2.

Figure 1. The braille written on paper

Blind Braille readers were better than sighted controls
at tactile recognition of dot patterns (with similar dot
diameter and spacing to Braille). It was thought that theBraille-reading experience of the blind subjects
accounted for these findings. Performance of thelate-blind subjects was somewhat worse than theearly-blind subjects but substantially better than that of
the sighted subjects (6).
There are two types of VIP: by born (study braille atschool) and by adult (study braille at private teaching andseminars) (7). VIP learn Braille by practice and assisted
by instructor whether to know they are correct or not inreading (8) because the shape memorizing (1st step inlearning braille) is the most difficult step (7). These pastyears, VIP are using smartphone as well like normal
people by the help of screen reader application which
describe what are in the screen so that VIP could
navigate the smartphone well. The survey says that thepercentage of VIP who are using smartphone in Japan in2007 is 78 % and in Malang city Indonesia is 60% in2017 (8). The smartphone-based for supporting VIP wasalso developed (9) in Japan. That system was designed toFigure 2. The Application Overviews

Based on Figure 2, there are three main steps:
1. Input
The input of the application is a braille picture.
Because VIP are not able to take the braille picture
manually, this application captures the brailleautomatically by detecting the square mark on the book.
The input of the application is based on the beginnerbraille book for VIP by adult in Japan. The main materialof the beginner braille is the shape memorizing which is
the most difficult step (7) and takes a long time to learn as
shown in Figure 3.

Figure 3. The shape memorizing braille236

九州地区

教育システム情報学会 2019年度学生研究発表会

In the implementation, the book will be made which theconcept is based on the beginner braille book (10) withsome custom designs, like the rectangle mark for having
communication with the application.
2. Process
In processing step, application translates the input
braille picture into corresponded characters. Thismechanism uses OpenCV (Open Source Computer
Vision Library).
3. Output
The output of the application is a speech. The speech
will be produced in the upcoming developing by using
Android to Speech Engine or Google Speech based on
the character translation in the previous step.
The developing process uses Android Studio with
prototyping method since the project's requirement are
not known. By prototyping method, two mains
mechanism has been developed, they are the brailletranslation mechanism and the scanning mechanism.
The braille translation mechanism prototype has beendeveloped and it translates braille to corresponded
characters from the taken braille picture. The translation
prototype is shown as Figure 4.

frame by using OpenCV. If there is no rectangle in theframe, the application will skip that frame and processwith the next frame until the rectangle is detected andthen put the cross sign.
Rectangle detection

Figure 5. The rectangle scanning mechanism prototype

3.

Conclusion

There are two prototypes developed in this research,
the braille translation prototype and the rectangle
scanning prototype. The next developing is combining
those two prototypes, adding speech engine and creating
custom beginner braille book. By those all mechanisms,
the application is expected to support VIP in learning
beginner braille by themselves.

References
WHO: “Visual Impairment and Blindness”, World HealthOrganization (2018).
(2) Foong, Oi Mean, Suziah Sulaiman, Kiing Kyu Ling: “Text
Signage Recognition in Android Mobile Device”, Journal ofComputer Science 9 (12) 1793-1802 (2013).
(3) Shreekanth, T. and Udayashankara, V.: “A Review on Software
Algorithms for Optical Recognition of Embossed Braille
Characters”, International Journal of Computer Applications,81(3) (2013).
(4) Dubus, J.P., Benjelloun, M., Devlaminck, V., Wauquier, F. andAltmayer, P.: “Image Processing Techniques to Perform an
Autonomous System to Translate Relief Braille into Black-ink,Called: Lectobraille”, Proceedings of the Annual International
Conference of the IEEE (pp. 1584-1585) IEEE (1988).
(5) Ritchings, R.T., Antonacopoulos, A. and Drakopoulos, D.:“Analysis of Scan Braille Documents”, In Document Analysis
Systems (Vol. 14, p. 413) World Scientific (1995).
(6) Grant, A.C., Thiagarajah, M.C. and Sathian, K.: “Tactile
Perception in Blind Braille Readers: A Psychophysical Study ofAcuity and Hyperacuity Using Gratings and Dot Patterns”,
Attention, Perception, & Psychophysics, 62(2), pp.301-312(2000).
(7) Ohta M., Oda, T., Kita, S., et al.: "Research and Development ofBraille E-learning Program for The Visually Impaired and Its
Learning Effect", Bulletin of Kobe Tokiwa University, Vol. 9，
pp.35-42 (2016).
(8) Cahya, R. A. D., Handayani, A. N., & Wibawa, A. P.: “MobileBraille Touch Application for Visually Impaired People using
Double Diamond Approach”, MATEC Web of Conferences (Vol.
197, p. 15007) (2018).
(9) Zhang, S. and Yoshino, K.: “A Braille Recognition System by The
Mobile Phone with Embedded Camera”, Innovative Computing,Information and Control ICICIC'07, SecondInternational
Conference on (pp. 223-223) IEEE (2007).
(10) Antonacopoulos, A. and Bridson, D.: “A Robust Braille
Recognition System”, International Workshop on Document
Analysis Systems (pp. 533-545) Springer, Berlin, Heidelberg(2004).
(11) Al-Saleh, A., El-Zaart, A. and Al Salman, A.: “Dot Detection ofOptical Braille Images for Braille Cells Recognition”,
International Conference on Computers for Handicapped Persons(pp. 821-826) Springer, Berlin, Heidelberg (2008).
(1)

Figure 4. The braille translation mechanism prototype

The translation mechanism has many steps to translate
the taken picture to corresponded characters. The firststep is detecting the template of braille which is the blackrectangle. It has fixed 5 rows and 25 columns formapping 8 braille characters (10). The app crops thepicture by the rectangle and processes it to the next
image processing steps.
The image processing steps use the OpenCV. Thesteps are gray scaling image, erosion, dilation, Gaussian(to remove noises), and for the last is applying
thresholding (to get the black and white result) (11). The
black and white image is used to check the presence ofdots by using Find Contours algorithm from OpenCV. By
the fixed cell mapping of the template, the presence ofdots in each braille can be detected and converted toarray data. The array then translated to corresponded
character based on the braille translation (10).
Since taking the braille picture manually isimpossible for VIP, the prototype of braille scanningmechanism is developed as well. By scanning and letting
the square pattern in frame the camera, the prototype willtake the picture automatically which the braille is locatedinside that square pattern. As we can see in Figure 5, theprototype detects the presence of a rectangle by showing
cross sign in the rectangle. This prototype works withframe. The app checks the presence of the rectangle in a237