# Inclusive Production of Tactile Graphics

Jens Bornschein, Denise Prescher, Gerhard Weber

► To cite this version:

Jens Bornschein, Denise Prescher, Gerhard Weber. Inclusive Production of Tactile Graphics. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.80-88, 10.1007/978-3-319-22701-6_7. hal-01599649

HAL Id: hal-01599649

https://inria.hal.science/hal-01599649

Submitted on 2 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Inclusive Production of Tactile Graphics

Jens Bornschein, Denise Prescher, and Gerhard Weber

Technische Universität Dresden, Institut für Angewandte Informatik,
Nöthnitzer Straße 46, 01187 Dresden, Germany
{jens.bornschein, denise.prescher, gerhard.weber}@tu-dresden.de

**Abstract.** In this article a collaborative workstation for creating audio-tactile graphics is presented. The system is based on a common open source office suite and supports a transcriber for tactile graphics with several tools. In addition the system allows a blind reviewer to get involved at every stage of the creation process. This is achieved through a refreshable two-dimensional tactile display. The blind participant can independently manipulate graphical objects and make annotations in parallel. As a result, a tandem team of a sighted graphic creator and a blind partner may create a tactile graphic with better quality.

**Keywords:** Tactile graphics, blind users, collaboration, pin-matrix device.

## 1 Introduction

While the visual world gets more connected and data gets more complex, visual presentation is an effective means to convey information. Problems arise if participants cannot access the presented information, in particular blind people. Non-visual access to graphics is often provided by a verbal description, which is not always a satisfying solution. To create an alternative textual description can be challenging for some graphics, for example maps, complex diagrams or images containing some kind of graphical notation. In addition, verbal descriptions can be incomplete or can contain wrong interpretations. This can change the meaning of an image or can lead to misunderstanding its intention. For this reason, a combination of a description and a transcribed tactile representation should be provided for important graphical data as it improves comprehension by blind people [1]. For some photographs or art it is hard to create even a suitable tactile counterpart, access to them remains to be limited.

Creating tactile graphics is also not an easy task, especially if the creator has only little experience. In a survey in German speaking countries, we found that only 5% of the 78 blind and visually impaired participants generally prefer image descriptions over tactile graphics [2]. Conversely, most blind people wish to get more tactile graphics as they not only allow for an independent exploration, but also for a better spatial comprehension of the content. However, a basic requirement for an effective reading is a sufficient quality of tactile graphics. In most institutions it is common to check some of their produced materials by visually impaired colleagues to ensure a good quality [2]. In the conventional production process it is necessary to produce a digitally mastered tactile graphic for every change again to show it to a blind reader.
In contrast, we use a refreshable tactile pin-matrix device to include a blind user in early stages of the production process within a collaborative workstation.

## 2 Tactile Graphics

Tactile graphics allow readers to feel presented structures, graphic elements or Braille text, through their tactile perception. Several different production methods exist. They all differentiate in their ability of presenting tactile structures. Differing features are, for example, possible resolution, the ability of allowing different raise levels, stability, speed and costs for reproducibility. While microcapsule paper and matrix embossers can produce tactile graphics from digital data without big additional effort, reliefs and other production methods need some manual post-processing or long production times. In this paper we focus on digital production methods, such as normal printing and embossing. Tactile embossers allow resolutions form 10 dpi for Braille embossers up to 20 dpi with different height levels for matrix embossers. Microcapsule paper is able to present detailed tactile structures printed with a normal printer on a special paper that rises on dark areas while heated under a special lamp.

Instead of conventional static production media refreshable two-dimensional tactile displays can be used to interactively present tactile graphical and textual information. For instance, the BrailleDis 7200 [3] allows to show content on a 120 x 60 pins large display area as binary tactile image with a resolution of about 10 dpi. Because of its zooming and panning possibilities, theoretically, each type of graphic can be shown on the pin device. However, there are some limitations. For instance, it is hard to handle graphics that use colors or different height levels. Very fine-grained structures, rounded objects and non-orthogonal lines can also lead to problems in their recognizability on such low-resolution devices. Reducing the complexity is a main requirement for tactile graphics; therefore these kinds of structures should be avoided in any case.

### 2.1 Tools for Preparing Tactile Graphics

The creation of a tactile graphic is not the challenging part of a transcription process. Almost every source image can be directly transformed into a grayscale or binary image which can then be produced as tactile version, for example, on microcapsule paper or with a matrix-embosser. The main problem is the readability and recognition of objects. Structures and shapes often get lost due to the low resolution. Therefore, a transcriber needs to prepare the image with respect to simplification and readability.

Several drawing tools exist to support transcribers. They contain functions for filling objects with different tactile patterns or allow redrawing image parts in a way that they fit into the embossing matrix of the related embosser. Some of them also apply graphic filters for grayscale or binary transformation as well as for finding and highlighting the outlines of objects (e.g. PictureBraille<fnref n="1" />). Creating Braille labels is another

<footnote n="1">¹ http://www.pentronics.com.au/index_files/PictureBraille.htm</footnote>
task for transcribers. The TGA system [4], for example, supports the translation of
text in images by semi automatic OCR-mechanisms. However, it seems that mostly
standard office or image processing software is used [2].

In contrast, suppliers for tactile graphic production hardware often deliver their
own production software. For example, the audio-tactile graphic exploration system
IVEO<fnref n="2" /> comes with a touch-sensitive image frame and its own tactile graphic produc-
tion software for creating scalable vector graphic (SVG) images with the feature to
add a two-level description to graphic elements. Technically, SVG is utilized for an-
notating a title and description of tactile objects.

## 2.2 Tactile Graphics by Blind People

Drawing graphics without vision is difficult. Blind pupils learn to create drawings by different techniques. Most of them are based on handicraft techniques or combine tactile primitives to more complex structures [2]. Creating freehand drawings can be done by scratching tactile perceptible structures into special foils or wood plates. These are all analog methods and, therefore, hard to reproduce. The digital creation of a drawing by a blind user is more complex and with limited tactile feedback. Often Braille characters form drawings or command languages define graphical structures. All analog techniques have the disadvantage that an error correction is almost impossible for the creator. For the digital methods this is also very exhausting because it needs a lot of mental imagination.

## 2.3 Quality of Tactile Graphics

The dissemination of tactile graphics among blind people lacks behind their im-
portance. Due to lack of training many blind users have difficulties in recognizing
complex tactile graphics [5]. Reasons may be, for instance, the overloading of a
graphic with too much information, the reduced distinctiveness among different ob-
jects as well as the limited preservation of orientation [2]. To minimize such problems
some kind of quality management for tactile graphics is important.

There are several guidelines (see [6]), which should improve the quality of tactile
materials. However, such guidelines are often complex and general, which compli-
cates their practical application - especially for non-expert sighted persons who com-
monly produce tactile graphics [2]. We propose the inclusion of blind users into the
creation process. Therefore, we have developed a collaborative workstation as de-
scribed in the following section.

The main purpose of our system is to allow a simplified inclusion of blind users in
the production process as well as to support inexperienced sighted users. Besides, the
system can also be used for sketching difficult concepts with and for a blind user, for
instance in learning or working scenarios. First user observations in a collaborative
graphic production scenario have shown that it is not effective to let the blind user
follow the creation of a graphic from the beginning. It is more effective to include the

<footnote n="2">² http://www.viewplus.com/products/software/hands-on-learning/</footnote>
blind user after a first draft of the graphic is produced. The blind reviewer can than
check the graphic and make some annotations and changes if necessary. The sighted
user can also ask for specific problems which can lead to further discussions and im-
provements of the resulting tactile graphic.

3 Collaborative Workstation for Tactile Graphics

Compared to cooperation, where each individual works separately on his own part,
collaboration requires direct interaction and discussions among the participants [7].
For an efficient collaboration between a sighted and a blind user, the visual and the
non-visual workspace have to remain consistent and have to be synchronized in a
multimodal sense. The Tangram collaborative workstation not only has to allow for
the creation and modification of tactile and visualized graphics by both users. It also
has to support the communication between the partners by focusing on objects of
interest jointly. Beyond basic graphics operations, the workstation also has to include
functions for annotation and quality management. In the following, the general archi-
tecture of the Tangram workstation as well as the mechanisms for supporting the col-
laborative production of tactile graphics by sighted and blind users are presented.

**3.1 Architecture of the Tangram Workstation**

The Tangram workstation consists of two parallel I/O workspaces connected to one computer (see Fig. 1). The sighted user uses mouse and standard keyboard as input and a monitor screen for the visual output. The blind user on the other hand uses the dynamic pin device BrailleDis 7200 for tactile output and its 28 buttons as well as its touch-sensitive display area as input device for keying in commands, Braille text and gestural input [3]. In addition, auditory feedback is given, consisting of sound and text-to-speech output. With the exception of non-visual drawing, all basic tasks for transcription can be mapped to interactive operations at which the two partners do not hinder each other while working on the same document independently and in parallel.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Schematic representation of the Tangram workstation</figcaption></figure>
As basic drawing system the open source office suites OpenOffice<fnref n="3" /> and LibreOffice<fnref n="4" /> can be used. These programs are free to use, even in a commercial context, and produce vector graphics which can be exported to several different document formats. Multiple tools have been added to implement the working process for creating audio-tactile graphics.

For the blind user a special screen reader has been implemented that allows access to the accessible interface of the office suite as well as to the document object structure [8]. This enables the blind user to manipulate the graphical and textual document elements. The presentation of the graphical content is realized by rendering a tactile screenshot of the working area window on the pin device. Thereto, a configurable filter is applied to the screenshot, converting lighter points to lowered pins and darker points to raised ones. The user can freely explore the tactile screenshot by zooming and panning operations. Additional information is added to the screenshot image, such as a dotted frame resembling the size and position of the sheet to draw on within the working area. The blind user can retrieve further information on document objects by pointing at them while pressing a button.

The blind users' input and output system is realized with our open-source framework BrailleIO [9]. This framework allows for a fast and easy implementation of a basic windowing system for two-dimensional tactile pin devices. Furthermore, the framework also allows the Tangram workstation software to be applied to other pin device types besides the BrailleDis 7200.

## 3.2 Supporting the Sighted User

Especially for sighted producers with little experiences, the provision of a specialized toolbar by the graphic editing software itself can reduce some of the deficits when designing tactile graphics on a visual screen. This can include the specification of predefined sets of filling patterns, line styles or elementary graphical primitives, such as arrows or tactile symbols. Moreover, an easier access to important properties of graphical objects, such as position, size and textual information (title and description), can also be helpful for enhancing the productivity of a sighted user. The implemented toolbar supports the following tasks (compare Fig. 2):

<figure><img src="image_3.png" /><figcaption>Fig. 2. Toolbar for supporting the sighted user in producing tactile graphics</figcaption></figure>

<footnote n="3">³ www.openoffice.org</footnote>

<footnote n="4">⁴ www.libreoffice.org</footnote>
- position properties: edit x and y position of the selected object.

- size properties: specify width and height of the selected object.

- keep ratio toggle: enable and disable preservation ratio of width and height.

- unit selection: select measuring unit for presentation of position and size.

- line styles: select a tactile perceivable line style (solid, dotted and dashed) for the selected object.

- pattern styles: select a texture for filling the selected graphical object. We have tested the suitability and discriminability of the provided patterns for utilization with different tactile graphics technologies (microcapsule paper, matrix embossed printing and pin-matrix device) in an unpublished study with 12 blind users.

- pattern-border spacing: add or remove white space between the filling pattern and the border of the selected object to increase distinctiveness and outline perception.

- title & description dialog: open or hide a docking window for annotating title and description of graphical objects.

- open Tangram workstation: start the Tangram workstation for the blind reviewer. Besides, document properties are adapted, such as standard colors and font style.

- open check tool: open a dialog for reviewing tactile graphics based on our authoring guidelines.

## 3.3 Supporting the Blind User

The blind user manipulates graphic objects independently on the pin-matrix device by a rotating menu with five different editing modes. Input is realized by the buttons of the right cursor key pad of BrailleDis 7200 (see Fig. 3). If an element of the graphic is selected, the user can start its editing by pressing the center key. Initially, the move mode is activated. By pressing one of the four direction buttons the element is moved into the chosen direction. The current editing mode can be changed by pressing the center key iteratively. In this way an element can be moved, scaled, rotated, and its texture and line style can be changed. The assignment of functions to the cursor key pad buttons is dependent on the current mode (see Fig. 3). Therefore, the user doesn't have to remember many button combinations. Furthermore, one hand may remain on the cursor key pad while the other hand can monitor the changes on the element in real time. Bi-manual operation has been found to be effective for most interaction tasks [10]. One drawback is the reduced efficiency due to the modal menu. Nevertheless, an informal discussion with four blind users showed that this is acceptable if there are not too many modes.

Besides the editing of the appearance of graphic objects, the blind user also can annotate the tactile graphic, for instance, by adding a title and description for each element. Thereto, he can open a dialog in the detail region of the pin-matrix device, where reading and editing of Braille content is performed. Either a text field for the title or for the description is shown. The user can switch between these two by using the up or down button of the right cursor key pad. For writing, the Braille input keys on the BrailleDis are used. Finally, the data have to be saved by calling the save menu, which is realized by pushing the center button of the right cursor key pad. Using the left or right button the user can choose between the following menu options:
save, not save or cancel. All options are presented in Braille, where the current selected option is underlined and also is spoken out.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Usage of the cursor keys within the five editing modes in the non-visual user interface</figcaption></figure>

## 3.4 Collaboration

The blind user can edit the graphic elements independently from the sighted user, as mentioned above. Therefore, it is necessary to have two separate foci: the standard GUI focus, which will be called mouse focus in the following, and the pin device focus which refers to the DOM objects of the drawing document. The latter will be called Braille focus. To enable an effective collaboration, the two users have to be aware of these focus concepts.

Allowing the blind user to follow his sighted partner's activities, the selected GUI element can be highlighted on the pin device, and a follow mouse focus mode can be activated. In this mode, the element which is currently selected by the sighted user, is not only spoken by audio output, but also is signaled by a blinking pin frame and is shown in the center of the pin device. If the sighted user selects another element, the tactile output is refreshed automatically. In this way the blind user can easily track the mouse focus. If he wants to edit the element which has the mouse focus, he can set the Braille focus to that element and quit the follow mouse focus mode with a single button command. By this means, the sighted is able to show the blind user an element and ask him to do some improvements, for example.

On the other side, the blind user can also highlight the Braille focus for his sighted partner on the GUI to indicate the element he is currently editing. Therefore, a blinking overlay window is shown for some seconds above the corresponding element.

# 4 Conclusion and Outlook

A collaborative workstation for producing tactile graphics was implemented. This allows blind users to actively take part in the transcription process for tactile graphics from early stages. As they are the target user group of the materials created, this should increase the quality of the resulting images. First prototyping tests with four blind users showed that the editing functionalities on the pin device were rated as
useful and intuitive. Nevertheless, a high learning curve was mentioned. The focus concepts were rated as understandable, but the effective usage in collaboration with a sighted partner needs a lot of training, practice and coordination.

As the next step a real scenario user study with pairs of blind and sighted subjects reworking a previously transcribed tactile graphic will be conducted. Thereby, we will be able to assess usability and accessibility of the Tangram workstation's approach to collaboration. It already became clear that blind reviewers would like to do some changes in existing tactile graphics. In our studies, we want to find out if materials produced by professionals will be improved by a blind user, too.

**Acknowledgements.** We thank our blind participants for the rich discussion about the concepts of our tactile user interface. The Tangram project is sponsored by the Federal Ministry of Labour and Social Affairs (BMAS) under the grant number R/FO125423.

## References

1. Edman, P.K.: Tactile Graphics. AFB Press, New York (1992)

2. Prescher, D., Bornschein, J., Weber, G.: Production of Accessible Tactile Graphics. In: Miesenberger, K., Fels, D., Archambault, D., Penáz, P., Zagler, W. (eds.): Computers Helping People with Special Needs, ICCHP 2014, Part II, LNCS 8548, pp. 26-33. Springer International Publishing (2014)

3. Prescher, D.: Redesigning Input Controls of a Touch-Sensitive Pin-Matrix Device. In: Zeng, L., Weber, G. (eds.): Proceedings of the International Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets (2014)

4. Jayant, C., Renzelmann, M., Wen, D., Krisnandi, S., Ladner, R., Comden, D.: Automated tactile graphics translation: in the field. In: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, pp. 75-82. ACM (2007)

5. Bentzen, B.L.: Tangible graphic displays in the education of the blind persons. In: Schiff, W., Foulke, E. (eds.): Tactual perception: a sourcebook, pp. 387-404. Cambridge University Press (1982)

6. Braille Authority of North America and Canadian Braille Authority: Guidelines and Standards for Tactile Graphics 2010, http://www.brailleauthority.org/tg/web-manual/index.html

7. Kozar, O.: Towards better group work: Seeing the difference between cooperation and collaboration. English Teaching Forum, vol. 48, no. 2, pp. 16-23 (2010)

8. Bornschein, J., Prescher, D.: Collaborative Tactile Graphic Workstation for Touch-Sensitive Pin-Matrix Devices. In: Zeng, L., Weber, G. (eds.): Proceedings of the International Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets (2014)

9. Bornschein, J.: BrailleIO - a Tactile Display Abstraction Framework. In: Zeng, L., Weber, G. (eds.): Proceedings of the International Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets (2014)

10. Morash, V. S., Pensky, A. E. C., Tseng, S. T., Miele, J. A.: Effects of using multiple hands and fingers on haptic performance in individuals who are blind. Perception, vol. 43, no. 6, pp. 569-588 (2014)