Temporal and amplitude aspects in sonically simulating walking over bumps, holes and flat surfaces

Abstract: This paper describes an experiment whose goal is to assess the role of temporal andamplitude variations in sonically simulating the act of walking over a bump or a hole. In particular, ithas been investigated whether the timing between heel and toe and the timing between footsteps, aswell as variations in the amplitude of heel and toe aﬀect the perception of walking on unﬂat surfaces.Forty ﬁve subjects participated to three between-subjects experiments where they were asked tointeract with a desktop system simulating bumps, holes and ﬂat surfaces by means of auditory cues.Results show that it is possible to simulate a bump or a hole by only using temporal information in theauditory modality. Furthermore results show that the proposed amplitude variations are not suﬃcientto provide the information concerning uneven surfaces.Keywords: Footsteps, Bumps, Holes, AmplitudePACS number: 43.66. Jh, 43.66. Lj, 43.66. MK [doi:10.1250/ast.33. 291] 1. INTRODUCTION Footstep sounds represent important elements in multi-media productions such as movies and videogames. Chionet al. write of footstep sounds as being rich in what herefers to as materializing sound indices—those featuresthat can lend concreteness and materiality to what is on-screen, or contrarily, make it seem abstracted and unreal[1]. In the auditory perception and sound design andsynthesis community, sonic properties of footsteps havebeen extensively investigated. From the perceptual point ofview, previous research has shown that it is possible torecognize the gender of a human walker only by listeningto recorded footsteps [2]. Moreover, footstep sounds ofa person walking on a wooden ﬂoor provide informationabout the gender, age, size, and emotional intention of theperson, and hardness and material of both shoes and ﬂoor[3]. Other studies demonstrated the possibility of recog-nizing simulated surfaces subjects are walking upon [4], aswell as emotional intentions of the walker [5,6]. To our knowledge, all previous research on walkingsounds from a simulation point of view has focused on theact of walking on ﬂat surfaces [7–11]. On the other hand, the human-computer interactioncommunity has explored the possibility of simulatingbumps and holes, by implementing a so-called pseudo-haptic simulation [12]. The main idea of such research wasto investigate whether it is possible to simulate a bump or aholeonascreenbyonlyusingvisualfeedback.Thisillusionwas achieved by creating a visual interface where thecontrol-display ratio between the motion of the mouse andthe cursor is not linear. In particular, when simulating abump, the cursor on the screen was decelerating untilreaching the top of the bump and then accelerating, whilewhen simulating a hole the cursor ﬁrst accelerated and thendecelerated. Experiments showed that subjects could suc-cessfully recognize a hole or a bump with this system [12]. Such research has recently been extended in [13],where the authors investigated whether it is possible tosimulate the illusion of walking on a hole or a bump onlyby using visual feedback. Three parameters were consid-ered in such simulation: orientation, velocity and height,and their combination. The experiments were run bothactively, having users wear an head mounted display, aswell as passively, having users interact with a desktopsimulation. Results showed that such visualization tech-niques successfully simulate bumps and holes located inthe ground.It has also been shown that it is possible to simulate abump or a hole only by using auditory cues [14]. In this