Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USA

Workshop on Amplification andAugmentation of Human Perception
Albrecht Schmidt,
Stefan Schneegass
University of Stuttgart
Stuttgart, Germany<first.last>@vis.uni-stuttgart.deJun Rekimoto
University of Tokyo
Tokyo, Japanrekimoto@acm.org

Kai Kunze
Keio University
Yokohama, Tokyo, JapanJST Presto
kai.kunze@gmail.com

Woontack Woo
KAIST
Daejeon, Korea
wwoo@kaist.ac.kr

Abstract
Technical capabilities increase steadily while human evolution is slow. Technical sensors improve while human senses
remain largely the same. Many tasks are becoming mucheasier when our perception is augmented with additionalinformation. In this workshop, we want to bring researchers
together that are working on the augmentation and amplification of human perception. For many augmented reality (AR) use cases the fundamental design concept is toimprove human perceptual abilities. This area, becoming
more and more central to human-computer-, human-data,and human-environment interaction is so far not well researched and we lack conceptual foundations, theoreticalmodels, and best practice examples. This workshop will bea forum to jointly catalog the state of the art and to discuss
future directions for amplifying and augmenting human perception in a human centered way.

Author Keywords
Figure 1: Examples of amplifying human perception by showing
multiple views (left) [2], actuated navigation (center) and artificial
vision (right).
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s).
CHI 2017 Extended Abstracts, May 06–11, 2017, Denver, CO, USA.
ACM 978-1-4503-4656-6/17/05.
http://dx.doi.org/10.1145/3027063.3027088

Amplifying Perception; Augmenting Perception; AugmentedHuman; Augmented Reality Interaction.

ACM Classification Keywords
H.5.m [Information interfaces and presentation (e.g., HCI)]:
Miscellaneous

668

Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USA

Introduction
Amplification of human perception, physiology, and cognition aims at using technical systems to enhance existing human abilities or to create new capabilities. Technicalsensors, such as cameras and microphones, are in temporal and spatial resolution superior to the human sensesof vision and hearing. Artificial intelligence is outperforming humans in games such as chess and go. These facts
are often used to paint a dark vision of the future in whichmachines take over the world. In this workshop, we want
to bring people together who look at opportunities of using
technologies to amplify and augment human perception tokeep up with technical advances. In this the interaction between humans and technology and their interplay are thecore scientific challenges. The CHI community has demonstrated the feasibility of amplifying human perception byextending the human vision (e.g., sensory substitution [7]or change the perspective [2]) and by adding novel sensors and learn them as new senses (e.g., color vision [1]and spidersense [4]). Amplification can be extended beyond perception to cognition and physiology. Adding additional control to muscle movement (e.g., in the Hand [6, 3]
or while walking [5]) allows to create new motor abilities.
This growing research area poses many fundamental questions in human-technology collaboration and sets toughtechnical, ethical, and societal challenges.

Amplifying Perception: Fundamental Challenges
In order to create systems that amplify human capabilities
we face fundamental challenges in different areas.
Data transfer from Sensor to Human. When adding
new sensors and when acquiring new information throughtechnical systems the big questions is: how to provide thisinformation to the user without overloading. Given a microphone that extends the hearing into higher ultrasonic fre-quencies the question is how to present this to the user in apleasant way without reducing the ability to hear the original
frequencies and without information overload?
Mixing Conventional Senses with new Sensor Information. With more information available we have to findways to combine our naturally perceived stimuli with new
information from technical sensors. Here a central questionis, how this can be done in a way that humans can easily
learn mappings and acquire information subconsciously.
Intuitive Control for Amplified and Augmented Senses.
Once we have created an amplified sense it becomes achallenge to intuitively control them. We can easily control
our natural visual perception, for example, by head and eyemovement. If we add further camera perspectives and if weextent the visual spectrum we have to find mechanisms tocontrol these additional features implicitly.
Understanding the Implications for Human Self-Awareness. If we chance the input channel in human perception we can expect that this changes how we perceive ourselves. It is a central question how altering the way we perceive our everyday world will impact us and how we candesign for this to be a positive change. This includes ethicalquestions in many different areas.
Many augmented reality systems are a first step towards
amplifying and augmenting human perception. These fundamental challenges need to be addressed in these ARSystems already, but become much harder as the amountof information available increases and augmentation becomes ubiquitous. These fundamental challenges are becoming more important across many areas in human computer interaction.

669

Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USAKey Questions for the Workshop
Given these developments and the new challenges it is surprising that there is so far little practical, theoretical, andconceptual work beyond examples and cases studies. AsAR is becoming ubiquitous and as we move more and more
towards amplifying our sense we still have a very limited understanding of how to create a seamless and intuitive user
experience for interacting with such systems. At the workshop will discuss the following questions:• Understanding what is out there. In a first step, weaim at gaining an overview of the types of existingsystems and forms of interactivity that have beencreated over the last years and to discuss systems
currently under development. Our approach to this isto analyses and discuss a broad set of case studies
participants are willing to share at the workshop.

central open issues and how to further study and design user experiences for augmenting and amplifying
human perception.

Topics of Interest
We focus on different topics within the context of this workshop. These topics range from application scenarios, to implementations of novel interaction concepts or novel sensing and actuating means. We also highly welcome reflections and discussions. In particular, we focus on but are notlimited to:• Amplifying Human Senses• Creating Novel Senses• Actuating Humans through EMS• Understanding the Impact on Users. To make such
systems usable and useful it is important to understand what kinds of experiences are created. How
additional information impacts perception, and how
it effects the experienced cognitive load. The impact
may be intended by the system designer or unintentionally happening.

• Brain-Computer Interfaces and Eye-wear Computers
for Implicit Input• Charting the Design Space. This design spaceshould describe all known factors that impact theuser experience of amplification of human perception. Such a design space could become a conceptual foundation and a tool for reflecting on the designof such systems.

• Alternative or Novel Feedback Techniques• Research Agenda. Based on insights from the current systems and experiments as well as factors identified in the design space we would create a research
agenda. In particular, it is assessed which are the• Ethical Implications of Amplified Senses• Super Human Sport• Augmented Reality for increasing Human Perception• Creating Seamless Interactions with Novel Senses• Interacting with Augmented Reality• Models, Theories, and Concepts of Digitally Augmented Human Perception

670

Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USA

Date

Action

1st December
9th December

Release of the Webpage
Release of the Call for Papers & selection of program committee
Early Submission Deadline
Early Notification of Acceptance
Submission Deadline
Notification of Acceptance

15th December
21st December
20th January
10th February

Table 1: Important workshop dates.

Workshop Structure
Dates
The most important dates are summarized in Table 1. Priorto the workshop, the call for participation will be distributed
through various channels such as mailing lists, social media, and personal contacts. We will also distribute advertisement material to different research institutions and conferences. Concurrently, we will recruit a set of experts inthe field serving as program committee. Approximately 20submissions will be accepted based on the quality and novelty of the submitted position paper. We will particularly
allow participants to submit positions paper in 2016 to leverage 2016 budget. We will shortly after the early submission
deadline accept strong submissions.
Prior to the Workshop
The following steps are planned prior to the workshop.
Participants. We will recruit participants through related
mailing lists as well as through personal contacts with experts working in the field of amplifying human perception.
We strive for a workshop size of 20 participants.
Submissions. Workshop participants are requested tohand in a position paper of up to 4 pages in the ACM SIGCHI

Extended Abstract format. The position paper could contribute the authors opinion on the workshop theme but also
provide specific ideas for a prototype allowing the amplification of humans perception. The submission of the position
paper will be handled through a conference management
system.
Webpage & Social Media. We will create a webpage(hosted on our institutional servers) and social media accounts (e.g., facebook group) to foster the communication
between participants before and after the workshop. The
webpage will contain general information on the workshop
such as the important dates but also provide hints to literature on the topic of amplifying human perception. In particular, we will focus on literature published outside the CHI
community (e.g., background on human senses in psychology). This will support the participant in gaining a foundation of the background of the overall workshop theme.
Detailed Plan of the Workshop
After a brief opening and introductory round, the workshop
will be start with a keynote speaker to motivate the overall
topic of the workshop. Afterwards, the submitted position
papers will be presented in brief presentation slots of 5 minutes with additional time for questions. Next, we will conduct group work activities to generate concepts of amplified
human senses. This will be done in small groups of 3-4 participants. Material to visualize the ideas will be provided(e.g., cardboard, clay, mock-ups of wearable devices, etc.).
The ideas generated in the workshop will be presented afterwards. Finally, we will wrap-up the workshop and discuss
future research directions. Furthermore, we will highlightthe possibility of submitting to the planned special issue on
amplifying human perception and encourage the participants to work together on the ideas and concepts createdduring the workshop.
671

Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USA

Time

Activity

9:00-9:10
9:10-10:00
10:00-12:30

Opening & Introduction
Keynote on Augmented Perception
Presentation of the Workshop Submissions(incl. coffee break)
Lunch
Generating concepts of novel and amplified human senses in groups (incl. coffeebreak)Presentation of the generated concepts
Wrap-up and discussion on future research
directions

12:30-14:00
14:00-17:00

17:00-18:00
18:00-18:30

Table 2: Detailed plan of the workshop.

After the Workshop
We plan to create a special issue on the workshop theme
to provide participants a platform to publish their work. We
will encourage the participants of the workshop to realize
the ideas and concepts generated during the workshop andsubmit them to the special issue.

Organizers
Albrecht Schmidt is a professor of human-computer interaction at the University of Stuttgart. His primary researchinterest is at the crossroads of human-computer interaction
and human perception. Albrecht received his PhD in computer science from Lancaster University in the UK.
Stefan Schneegass is a research associate within theHuman-Computer Interaction group at University of Stuttgart.
His current research interest centers on ubiquitous computing and human-computer interaction (HCI) particularlywearable computing.

Kai Kunze works as an associate project professor at KeioMedia Design, Keio University. Beforehand, he held an assistant professorship at Osaka Prefecture University. His
major research contributions are in pervasive computing, especially in sensing, physical and cognitive activity
recognition. Recently, he focuses on tracking and improving
knowledge acquisition activities. He is also a JST Presto
grant recipient working on Eyewear Computing.
Jun Rekimoto is a professor at the University of Tokyo anddirector of the Interaction Lab, Sony Computer ScienceLaboratories. He was appointed to the SIGCHI Academyin 2007. Rekimoto‘s research interests include humancomputer interaction, computer augmented environments
and computer augmented human (human-computer integration).
Woontack Woo is a Professor in the Graduate School ofCulture Technology (GSCT) at Korea Advanced Instituteof Science and Technology (KAIST), Daejeon, Korea. The
main thrust of his research has been implementing ubiquitous virtual reality in smart space, which includes Contextaware Augmented Reality, 3D Vision, HCI, and Culture
Technology.

Conclusion
Amplifying the perception of humans with technology hasthe potential to highly impact our everyday life. One of thecore challenges will be to identify ways to seamlessly interact with these technology so that the novel sensors havethe potential to become senses. In this workshop, we explore the foundations of how technology can in the future beused for amplifying and augmenting humans and in particular the perception.

672

Workshop

CHI 2017, May 6–11, 2017, Denver, CO, USA

Acknowledgments
This work was partly conductedwithin the Amplify project which received funding from the European
Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 683008) and was supported by JST PRESTO.

References[1] Neil Harbisson. 2012. I listen to color. TEDXTalk,
http:// www.ted.com/ talks/ neil_harbisson_i_listen_to_color(2012).
[2] Shunichi Kasahara, Mitsuhito Ando, Kiyoshi Suganuma, and Jun Rekimoto. 2016. Parallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing. In Proceedings of the2016 CHI Conference on Human Factors in Computing
Systems (CHI ’16). ACM, New York, NY, USA, 1561–1572. DOI:http://dx.doi.org/10.1145/2858036.2858495[3] Pedro Lopes, Patrik Jonell, and Patrick Baudisch.
2015. Affordance++: Allowing Objects to Communicate Dynamic Use. In Proceedings of the 33rd Annual
ACM Conference on Human Factors in Computing
Systems (CHI ’15). ACM, New York, NY, USA, 2515–2524. DOI:http://dx.doi.org/10.1145/2702123.2702128[4] Victor Mateevitsi, Brad Haggadone, Jason Leigh, Brian

Kunzer, and Robert V. Kenyon. 2013. Sensing theEnvironment Through SpiderSense. In Proceedings ofthe 4th Augmented Human International Conference(AH ’13). ACM, New York, NY, USA, 51–57. DOI:
http://dx.doi.org/10.1145/2459236.2459246[5] Max Pfeiffer, Tim Dünte, Stefan Schneegass, Florian Alt, and Michael Rohs. 2015. Cruise Control forPedestrians: Controlling Walking Direction Using Electrical Muscle Stimulation. In Proceedings of the 33rd
Annual ACM Conference on Human Factors in Computing Systems (CHI ’15). ACM, New York, NY, USA,
2505–2514. DOI:http://dx.doi.org/10.1145/2702123.
2702190[6] Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011.
PossessedHand: Techniques for Controlling HumanHands Using Electrical Muscles Stimuli. In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (CHI ’11). ACM, New York, NY,USA, 543–552. DOI:http://dx.doi.org/10.1145/1978942.
1979018[7] Jamie Ward and Peter Meijer. 2010. Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition 19,1 (2010), 492 – 500. DOI:http://dx.doi.org/10.1016/j.
concog.2009.10.006

673