# Dial-based Game Interface with Multi-modal Feedback

Wanjoo Park, Laehyun Kim, Hyunchul Cho, Sehyung Park

► To cite this version:

Wanjoo Park, Laehyun Kim, Hyunchul Cho, Sehyung Park. Dial-based Game Interface with Multi-modal Feedback. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.389-396, 10.1007/978-3-642-15399-0_42. hal-01055620

HAL Id: hal-01055620

https://inria.hal.science/hal-01055620

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Dial-based Game Interface with Multi-modal Feedback

Wanjoo Park, Laehyun Kim, *Member*, IEEE, Hyunchul Cho and Sehyung Park,

Intelligence & Interaction Research Center,
Korea Institute of Science and Technology, Seoul, Korea
{ wanjoo, laehyunk, hccho, sehyung}@kist.re.kr

**Abstract.** This paper introduces a dial-based haptic interface for a brickout game. Conventionally brickout games are played through a mouse or a keyboard. However, these input devices cannot provide sufficient intuitive interface to move the game paddle or provide multi-modal feedback for the user. We developed a haptic game device that gives the user haptic feedback during the game as well as visual and sound feedback. The user can move the position of the paddle by spinning the dial knob and feels various multi-modal effects according to the game context. Basic haptic effects include detent, vibration, friction and barrier. We can generate any combinations of these effects by adjusting the amount, frequency, and direction of torque along the rotational path. The result of a user-study showed that the proposed haptic dial interface made a simple brickout game more fun and more interesting. Additionally, the users were able to focus on the game more easily than when they played using a mouse.

**Keywords:** haptic interface, brickout game, dial knob, tactile feedback.

## 1 Introduction

Recently haptic interfaces have been developed that allow users to interact with digital information via the sense of touch. Haptic senses include force/tactile feedback, pressure, vibration, heat, and even pain. Haptic technology has various applications such as medical simulations [1], rehabilitation [2], mobile phones, automobiles, and games. Especially, the success of the Wii [3] and the haptic phone [4] in providing tactile feedback has led to the integration of a haptic interface into many game devices. Most game devices such as Wii, PSP, and Nintendo DS give simple vibration effects to the user. PC-based video games are played by a mouse, a keyboard, or a joystick without tactile feedback.

Over the past several years, there have been a number of studies of haptic feedback controllers. Karon E. MacLean et al. introduced a variety of haptic devices and design parameters [5-7]. They considered characteristics of the touch sense and designed various tactile signals on their experimental devices. Several studies have been conducted of the 1 DOF (Degree of Freedom) dial knob. Scott S. Snibbe et al. suggested several haptic techniques for manipulating digital media based on intuitive physical metaphors [8]. Mircea Badescu et al. developed a single force feedback knob to imitate the senses of various conventional control knobs in motor vehicles [9].
Laehyun Kim et al. used a haptic dial system for multimodal prototyping in the early phase of product development [10]. They applied the haptic technology to prototype the dial module of a washing machine.

There are several works to apply the haptic interface to games. Yuichiro Sekiguchi applied the haptic effect to a game, proposing a device that gives a user the illusion of a virtual object inside the device when shaking it using accelerometers and actuators [11]. Jukka Linjama et al. developed a bouncing ball game that uses accelerometer sensors and tactile feedback actuators [12]. Wanjoo Park et al. proposed a brickout game using a haptic dial interface and some haptic effects for the game [13].

## 2 Brickout Game and Hardware Configuration

The main objective of the brickout game is to remove all the bricks from the wall by hitting them with the ball (the right side of Fig. 1). In our brickout game, conventional bricks are replaced by banana bricks, a game ball is replaced by a monkey, and a paddle is replaced by people holding up boards. In addition, a new item is added, a cloth wrapper.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Block diagram of the dial-based haptic system (**left upper side**), Modules of motor, control unit PCB and touch screen (**left down side**), Implementation of a brickout game (**right side**)</figcaption></figure>

When the ball hits a cloth wrapper, the haptic item falls down. If the user catches the item by moving the paddle, a predefined tactile effect is felt that lasts a few seconds. The game's visual information helps the user understand the current game context.

We will explain in detail the hardware configuration of the haptic dial knob. A block diagram of the system is shown in the left upper side of Fig. 1. The main
processor is an AVR ATMEGA-128 processor and the DAC (Digital to Analog
Convertor) is a DAC0800 we obtained from National Semiconductor. A DC motor is
used to generate various haptic patterns. We use an RE25 with a gear box of 5:1 ratio
obtained from Maxon motor. The motor works at 24V/0.6A and 28.8mNm torque.
The Dial knob is installed on the motor gear box, allowing the user to rotate the knob
and to feel various haptic effects which are programmed along the angular position.
The Encoder to measure the angular position is an MR 128 with 1000 pulse per turn
from Maxon Motor. The encoder counter is an LS7166 manufactured by LSI
Computer Systems. It has a 24-bits quadrature counter and a DC set to 25MHz count
frequency. A LCD Module (an EZ-TFT350T manufactured by Alls Technology) is
used to display visual information. It displays 3.5 inch QVGA 320 x 240 pixels with
24 bits true color.

When the user rotates the dial, the encoder generates the pulses. Then the encoder
counter IC counts the pulses and the AVR processor reads the encoder value. The
processor calculates the angle and drives the motor to generate haptic dial effects
described in section 3. The user feels the programmed haptic effects according to the
game context.

3 Haptic dial effects

In this section, we describe how to implement various haptic effects for a brickout
game. Basic haptic dial effects include detent, vibration, friction, and barrier.

**3.1 Detent effect**

The detent effect simulates small notches or a rugged surface along the angular
position using sine functions. We call it the angle-based effect since the motor's
torque function is implemented based on the dial's rotation angle. Eq. (1) shows the
detent effect which is defined by a sine function.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Detent haptic effect (**left side**), these are paddle images for the different friction types **right side**: Normal condition (**right upper side**), high friction (**right middle side**), low friction (**right down side**).</figcaption></figure>
$$f_d(\theta) = A_{\theta} \sin(b \cdot \theta) \qquad (1)$$

where, $A_{\theta}$ is the amplitude, $b$ is the number of notches per turn, and $\theta$ is the rotation angle of the dial knob.

The torque profile for the detent effect is shown in the left side of Fig. 2. When a game player gets a detent haptic item, a gravel image appears under the paddle and an audio clip of walking on a gravel road is heard. The user senses the rugged surface through the multimodal stimulation of visual, aural, and haptic feedback. This effect makes control more difficult but makes the game more interesting and allows the player to focus on the game with a realistic physical feeling.

## 3.2 Vibration effect

The vibration effects generate vibrotactile feedback as a time-based effect. This is represented by Eq. (2). We designed paddle vibrations and item vibrations. The paddle vibration has a small amplitude ($A_v$) and high frequency ($s$). When the paddle hits the game ball, the user feels the impact between the paddle and the ball according to the paddle's vibration. During this effect, the dial knob shakes weakly and rapidly and an audio clip of bouncing strings is played. This gives the user the sense that the ball is actually bouncing physically.

On the other hand, the item vibration is implemented by a large amplitude and low frequency. This effect generates a larger torque and longer period than the paddle vibration effect. When the user takes the cloth wrapper containing the item vibration effect, he/she feels several big vibrations lasting several seconds. As a result, paddle control becomes more difficult. For audio feedback, an audio clip of a warning siren adds tension for the gamer.

$$f_v(t) = A_v \sin(2\pi \cdot s \cdot t) \qquad (2)$$

where, $A_v$ is the amplitude constant, $s$ is the frequency scaling constant, and $t$ is time

## 3.3 Friction effect

The friction effects generate resistant torque opposite to the direction of movement as a movement-based effect. This is implemented based on the friction con model [14]. When the dial knob is rotated slowly, the rotational friction is low. On the other hand, when the user spins the dial fast, he/she feels more strong friction. There are 3 different types of friction mode. The first is the friction mode for normal conditions. In this mode, the friction level $L_f$ in Eq. (3) is small and the player feels relatively weak friction force. The paddle is represented by five walking people holding up a board. For the audio effect, an audio clip of many noisy people is used. The right side images of Fig. 3 show the paddle images for the different friction types. The second friction mode is the high friction effect. The paddle's image is
transferred to only one person for the visual effect. An audio clip of the sound of a single person walking is used for the audio effect. Rotational friction force of dial knob is very high due to high $L_f$ in Eq. (3). Accordingly, the player feels strong friction force as he/she moves the paddle. The third friction mode is a low friction effect. The paddle is transferred to a vehicle image and an audio clip of moving vehicle is used. In addition, the rotation friction of dial knob is near zero because of $L_f$ is near zero in Eq. (3). Then the player can control the paddle's movement very easily and quickly.

$$ 
\begin{array}{l}
\begin{aligned}
P_{\text{curr\_f}}(n) &= P_{\text{prev}}(n-1) + (P_{\text{now}}(n) - P_{\text{pre}}(n-1)) \cdot S_f \\
P_{\text{diff}}(n) &= (P_{\text{now}}(n) - P_{\text{curr\_f}}(n)) \cdot L_f \\
P_{\text{prev}}(n) &= P_{\text{curr\_f}}(n) \\
\text{if} \quad P_{\text{diff}}(n) > T_{f\_\text{max}} \quad \text{then} \quad T_f(n) = T_{f\_\text{max}} \\
\text{else if} \quad P_{\text{diff}}(n) < T_{f\_\text{min}} \quad \text{then} \quad T_f(n) = T_{f\_\text{min}} \\
\text{else} \quad T_f(n) = P_{\text{diff}}(n)
\end{aligned}
\end{array}
\qquad (3) $$

where $P_{\text{curr\_f}}$ is the current position, $P_{\text{now}}$ is the angular position of the dial knob, $P_{\text{pre}}$ is the previous position, $S_f$ is the scaling factor, $P_{\text{diff}}$ is the difference of position, $L_f$ is the friction level, and $T_f$ is the friction torque.

## 3.4 Barrier effect

The barrier effects simulate the hard stop as location-based effects because the barrier effect restricts the movement of the paddle in a certain location. Eq. (4) is a second order equation to calculate the barrier torque function. There are 2 types of barriers. In the normal condition, the user feels the barriers around the left and right wall. The barrier effect for the normal condition is calculated by $b=180°$, $a=1$ and $w=6$. When $b=180°$ it means that barriers are located on -180° and 180°. Variables of $a$ and $w$ in Eq. (4) are used to determine an edge function. With this, the player can feel a paddle cling to the wall like iron clings to magnet on near edge zones. If the player rotates the dial knob over the b point, the barrier effect blocks this rotation. The item barrier effect restricts the movement of a paddle inside a certain range for a certain time. In this case, b in Eq. (4) is set to 90, half of the normal condition. This effect is the most challenging item. If the game ball falls into the limited zone, the user cannot move the paddle inside that zone.

$$ 
\begin{aligned}
f_b(\theta) &= \frac{-(\theta+b-w)^2}{w^2} + a, \quad (\theta \le -b+2w) \\
&= 0, \quad (-b+2w < \theta < b-2w) \\
&= \frac{(\theta-b+w)^2}{w^2} - a, \quad (b-2w \le \theta)
\end{aligned}
\qquad (4) $$

where $\theta$ is the rotation angle of the dial knob, $b$ is the start point of barrier, $a$ is the amplitude of edge function and $w$ is the range of edge function.
## 5 User-study

We performed a user study to compare the dial-based haptic interface with a mouse interface when playing the brickout game and to evaluate the interface's usability by using a heuristic evaluation method.

At the beginning, 130 words related to game evaluation were collected and then representative words were chosen by 4 experts. The similarities between the words were analyzed by cluster analysis. The cluster analysis is a statistic method of a common technique for statistical data analysis used in many fields. It is the assignment of a set of observations into subsets-clusters so that observations in the same cluster are similar in some sense. Finally 4 representative words were selected such as interesting, immersed, easy to control and fatigued (Table 1).

A usability survey was conducted with 35 undergraduate students regarding the 4 classified items. The survey processed in 2 steps. First, a volunteer was asked to play the brickout game with a mouse for 5 minutes, and to fill out the questionnaire. Additionally, the volunteer was asked to play the game with the dial-based haptic device for another 5 minutes and to answer the other questionnaire. The questions on the questionnaire were 5-point Likert scales for 4 adjectives: 'Interesting', 'Immersed', 'Easy to control', and 'Fatigued' (Table 1) and the volunteers gave the 4 adjectives scores from 1 (low) to 5 (high). A Likert scale is a psychometric scale commonly used in questionnaires, and is the most widely used scale in survey research. When responding to a Likert questionnaire item, respondents specify their level of agreement to a statement.

<table><caption>**Table 1. Result of cluster analysis**</caption><thead><tr><th>Interesting</th><th>Immersed</th><th>Easy to control</th><th>Fatigued</th></tr></thead><tbody><tr><td>fascinating</td><td>concentrated</td><td>easy</td><td>tired</td></tr><tr><td>absorbing</td><td>focused</td><td>simple</td><td>exhausted</td></tr><tr><td>attractive</td><td>centralized</td><td>freedom</td><td>weary</td></tr><tr><td>exciting</td><td>absorbed</td><td>efficiency of use</td><td>strenuous</td></tr><tr><td>fun</td><td>passionate</td><td>flexibility of use</td><td>arduous</td></tr><tr><td>pleasant</td><td>engrossed</td><td>clarity of goals</td><td>stressor</td></tr><tr><td>interesting</td><td>engaged</td><td>easy to use</td><td>fatigued</td></tr><tr><td>curiosity</td><td></td><td>intuitive</td><td>obstacles</td></tr><tr><td>wonder</td><td></td><td>easy to control</td><td>tension</td></tr><tr><td>fooling</td><td></td><td>casual</td><td></td></tr></tbody></table>

<table><caption>**Table 2. Result of T-test**</caption><thead><tr><th></th><th>F</th><th>p-value</th><th>T</th></tr></thead><tbody><tr><td>Interesting</td><td>8.9711</td><td>0.0038*</td><td>-9.0440</td></tr><tr><td>Immersed</td><td>4.3130</td><td>0.0416*</td><td>-5.8874</td></tr><tr><td>Easy to control</td><td>0.2710</td><td>0.6044</td><td>5.0556</td></tr><tr><td>Fatigued</td><td>0.5828</td><td>0.4478</td><td>-4.6697</td></tr></tbody></table>

* P < 0.05
The questionnaire was analyzed by *t*-test and numerical statement. A *t*-test is any statistical hypothesis test in which the test statistic follows a Student's *t* distribution if the null hypothesis is true. It is most commonly applied when the test statistic would follow a normal distribution if the value of a scaling term in the test statistic were known. When the scaling term is unknown and is replaced by an estimate based on the data, the test statistic follows a Student's *t* distribution.

<figure><img src="image_4.png" /><figcaption>Fig. 3 Mean value of questionnaire.</figcaption></figure>

In Fig. 3, each bar indicates the average value of each item. The results show that the users felt more interest and more immersed in the game using the haptic interface than the mouse. However, the users could control the game paddle more easily with the mouse and felt somewhat more fatigued when playing with the haptic dial interface than with the mouse interface. According to a *t*-test (Table 2), though, it was concluded that the haptics dial control was more interesting and players felt more immersed in the game than when using the mouse control (P<0.05). It is less meaningful that the mouse interface is easier to control and less fatiguing because of the high p-value. Therefore, it is concluded that the game using the dial-based haptic interface is more interesting and immersed than with mouse interface. It is recommended that a dial knob be designed with an ergomechanic grip that is easier to control and less fatiguing.

# 5 Conclusion

This paper describes a dial-based haptic interface and various haptic effects for a brickout game. Conventional game devices such as Wii and PSP provide simple effects like vibration. However, the haptic effects we proposed give the user more active and various physical experiences. We designed several different haptic effects which have different amplitude, direction, and frequency of torque along the angular position or movement. These include detent, vibration, friction, barrier, slow motion, and combinations of these effects. The barrier effect can block the movement of the dial at specific points and the detent effect makes the user feel a rugged surface while moving the game paddle. These haptic effects, which can be used as penalties or rewards, can
make the game more fun. In addition, the dial knob gives a very intuitive interface to
change the position of the paddle in the brickout game compared with conventional
input devices such as a keyboard, button, or mouse. We conducted a user-study to
compare a conventional mouse interface with our newly developed dial-based haptic
interface. According to the results of the survey, the haptic interface made the game
more fun and helped the game player focus on the game.

In the future we will work on improving the haptic interface to reduce fatigue and facilitate greater control by designing an ergomechanic grip.

References

1. M. Arbabtafti, et al., "Haptic and Visual Rendering of Virtual Bone Surgery: A Physically Realistic Voxel-Based Approach", *IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2008)*, pp. 30--35 (2008)

2. Atif Alamri, et al., "Haptic Virtual Rehabilitation Exercises for Poststroke Diagnosis", *IEEE Transactions On Instrumentation and Measurement, Vol. 57, No. 9*, pp.1876--1884

3. Nintendo Wii
http://www.nintendo.com/wii

4. Behold Mobile Phone by Samsung Electronics
http://www.samsungmobileusa.com/Behold.aspx

5. Karon E. MacLean, "Designing with Haptic Feedback" *Symposium on Haptic Feedback in the Proc. of IEEE Robotics and Automation (ICRA 2000)*, pp.22--28 (2000)

6. Vincent Hayward and Karon E. Maclean, "Do It Yourself Haptics: Part 1", *IEEE Robotics & Automation Magazine December 2007*, pp. 88--104 (2007)

7. Karon E. MacLean, "Foundations of Transparency in Tactile Information Design", *IEEE Transaction on Haptics*, Vol. 1, No. 2, pp. 84--95 (2008)

8. Scott S. Snibbe, et al., "Haptic Techniques for Media Control", *Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST 2001)*, pp.199-208 (2001)

9. Mircea Badescu, Charles Wampler and Constantinos Mavroidis, "Rotary Haptic Knob for Vehicular Instrument Controls", *Haptics'02, Haptic Interfaces For Virtual Envir & Teleoperator Systs*, pp. 342--343 (2002)

10. Laehyun Kim, Manchul Han, Sang Kyun Shin, Se Hyung Park, "A Haptic Dial System for Multimodal Prototyping", *18th International Conference on Artificial Reality and Telexistence (ICAT 2008)* (2008)

11. Yuichiro Sekiguchi, Koichi Hirota and Michitaka Hirose, "The Design and Implementation of Ubiquitous Haptic Device", *Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint*, pp. 527--528 (2005)

12. Jukka Linjama and Topi Kaaresoja, "Novel Minimalist Haptic Gesture Interaction for Mobile Devices", *NordiCHI 2004 ACM*, pp. 457--458, 2004

13. Wanjoo Park, Laehyun Kim, Hyunchul Cho and Sehung Park, "Design of Haptic Interface for Brickout Game", *IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2009)*, pp. 64--68 (2009)

14. N. Melder and W. S. Harwin, "Extending the Friction Cone Algorithm for Arbitrary Polygon Based Haptic Objects", *Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS'04)*, pp. 234--241 (2004)