「様式３」課題番号 P-9

Multimedia Interaction Interfaces
in Collaborative E-learning Environments[1] Organization
Leader: Noriyuki Matsuda (Faculty of Systems
Engineering, Wakayama University, Japan)
Representatives at RIE: Kamen Kanev & Hiroshi
Inokawa (Research Institute of Electronics,Shizuoka University)
Participants:
Tsukasa Hirashima (Graduate School ofEngineering, Hiroshima University, Japan)
Julita Vassileva (University of Saskatchewan,Canada)Bill Kapralos (Faculty of Business andInformation Technology, University of Ontario
Institute of Technology, Canada)
Karen Collins (Canadian Centre of Arts andTechnology at the University of Waterloo,Canada)
Michio Yoneya (National Institute of Advanced
Industrial Science and Technology, Tohoku
Center, Sendai, Japan)[2] Progress of the Research
The objective of this cooperative research projectis to explore possibilities for combining expertisefrom different areas and fostering the design anddevelopment of innovative interfaces andenvironments for e-learning based on naturalinteraction paradigms that is adjustable to theneeds of specific user groups.
First, sound interface design for smart tablecomputer interaction was addressed in (1) whereusers position themselves around a horizontal
computer screen and a combined audio/visionbased interface is employed for sound source anduser tracking in real-time. Second, the SONIDO
sonification tool for network traffic and security
that is capable of generating distinguishable
sounds based on traffic patterns, and has thepotential to be of use in teaching and learning ofintrusion detection was presented. In (2)SONIDO’s architecture and report on the current
state of usability studies was described. Third, in(3) an ontology-based annotations for test
interpretation and scoring was considered. Imagebased and CLUSPI enhanced annotation andscoring approaches were proposed andpossibilities for their integration were investigated.
During the project period the following researchmeetings were held:z August 27-28, 2009- Prof. Kanev from RIE
visited Wakayama University, and conductedjoint research collaborative meetings with Prof.
Matsuda, Prof. Taki, and other faculty andstudents.
z December 11, 2009- Prof. Matsuda, Prof.
Collins, Prof. Kapralos, Prof. Kanev, and Prof.
Inokawa had a collaborative research meetingat the Research Institute of Electronics.
z February 3, 2009- Prof. Matsuda fromWakayama University visited RIE and had acollaborative research meeting with Prof.
Kanev and Prof. Inokawa.
Project members were involved in organizing andpresenting intermediate research results stemmingfrom the project activities at the 12th International
Conference on Humans and Computers as follows:z Prof. Kanev worked as a Program Committee
Chair
z Prof. Matsuda made a presentation on
ontology-based
annotationsfor
testinterpretation and scoring (3)z Prof. Kapralos made presentations on soundinterface design for smart table interactions(1) and on using sonification to enhanceteaching and learning of network intrusion
detection (2)z Prof. Collins made a presentation onarchetypal sound effect icons for improved
multimedia accessibility (4)[3] Results(3.1) Research results
The project thus far highlights the need for further
research into new interfaces for collaborative
learning. We speculate that new modes ofinteracting that technologies, such as a multi-touch
table afford require new understandings of how tobest incorporate sound and visuals to enhancelearning. At this preliminary stage, we arebuilding prototypes to explore these newinterfaces, and plan to complete the building andtesting of these prototypes in the coming years.
In regards to sound interface design for smart tablecomputer interaction (1), we proposed the setup ofa table-top touchscreen computer and described asimple amplitude panning method to convey
sound amongst four loudspeakers. Althoughsimple, the method is independent of user position

and doesn’t require extensive computational
resources. Future work will explore further, more
complex sound output/ interaction techniques andnew methods for testing sound perception by users,
combining elements from a broad range of
disciplines. In examining the specific problems,
this work will lead to a significant broadening of
our understanding of sound interaction design, ofmulti-user interaction, of how users perceive
sound on horizontal surface devices, of how to
implement sound for games on multi-touchcomputers, and about methodologies we can use to
test user preferences of sound in computer
applications. Since sonic interaction design is avery new and emerging field, this research will
make a significant contribution towards methods
and practice. The prototypes will be built andtested and results published in the coming years.
In regards to sonification in teaching and learning(2) we described an analysis of educational gains
on sonification, a number of challenges withrespect to non-speech sound applications forteaching and learning about Network Intrusion
Detection Systems (NIDS), and guidelines for
future work on NIDS-based sonification ineducational settings. We also presented theSONIDO system, a sonification tool for network
traffic that is capable of generating distinguishable
sounds based on traffic patterns, and has thepotential to be of use in teaching and learning ofintrusion detection. The SONIDO system iscurrently going through two phases of evaluation.
The first phase of testing takes place during
production and consists of usability studies tobring the system to acceptable levels ofpracticality. This phase of evaluation is being
conducted with instructors of security-related
courses at the undergraduate and graduate level of
our institution. These tests will ensure that the tool
is usable and that it accurately reflects network
activity with sounds.
In regards to ontology-based annotations for test
interpretation and scoring (3) we have proposed an
innovative ontology-based approach which
supports examiners and ensures properinterpretation of student answers. For better
coverage of the variety of examination problems
and question types our ontology still needs
development and extension with additionalannotations. An image-based and CLUSPI-based
annotation and scoring have been discussed andpossibilities for their integration in a unifiedsystem have been considered. The system
discussed in this work is under continuing
development; its functionality is being extended

and further experiments and evaluation of its
effectiveness are still to be conducted.
(3.2) Future work
We are planning to add more members to ourproject, to enhance our research, and to reportobtained results at international conferences on aregular basis. Higher level results will be
submitted to renown international journals forconsideration and publication. New team members
will include (amongst others) Dr. Michael Jenkin(Computer Science and Engineering, YorkUniversity, Toronto, Canada), and Dr. Andrew
Hogue (Game Development Program, UOIT),
both of which have extensive computer visionbased experience.
We have discussed arranging for exchange
students between Japan and Canada to train thesestudents in cooperative research projects and incommunication skills in addition to the specificsof the technology, methods, and practice.
We are currently developing a more complete
research plan based on our preliminary findings,
and will be submitting further grant applications inCanada to facilitate a larger collaborative project.
Potential grant applications include theInternational Strategic Opportunities Program
offered by the Ontario Ministry of Research andInnovation. This grant specifically emphasizes
international collaboration and is thus particularly
relevant to our established collaboration.
We are submitting a joint paper to the ACM
FuturePlay 2010 conference, held in Vancouver,
British Columbia, Canada May 6-7 2010.
[4] Publications(1) Karen Collins, Bill Kapralos, Kamen Kanev:"Sound Interface Design for Smart TableComputerInteraction,"InternationalConference on Humans and Computers(HC'2009), 2009.
(2) Miguel Vargas Martin, Bill Kapralos, Miguel
A. Garcia-Ruiz, Mark Green, Adrienne
Brown: "SONIDO: Using Sonification toEnhance Teaching and Learning of Network
Intrusion Detection," International Conferenceon Humans and Computers (HC'2009), 2009.
(3) Noriyuki Matsuda, Kamen Kanev, Tsukasa
Hirashima, Hirokazu Taki: "Ontology-based
Annotations for Test Interpretation andScoring," International Conference onHumans and Computers (HC'2009), 2009.
(4) Karen Collins, Peter Taillon: "Archetypal
Sound Effect Icons for Improved Multimedia
Accessibility," International Conference onHumans and Computers (HC'2009), 2009.

Traveling report
Name:
Affiliation:
Period of time:
Destination:
Purpose:Name of receiver:Kamen Kanev
Research Institute of Electronics, Shizuoka University, Japan
August 27, 2009 – August 28, 2009
Wakayama University, JapanTo carry out joint research and project organization meetings and to schedule future
research activities
Prof. Noriyuki Matsuda

Name:
Affiliation:
Period of time:
Destination:
Purpose:Name of receiver:

Karen Collins
Canadian Centre of Arts and Technology at the University of Waterloo, Canada
December 5, 2009 – December 13, 2009
Shizuoka University, Japan(Please see the description in the bottom of the page)Prof. Kamen Kanev

Name:
Affiliation:
Period of time:
Destination:
Purpose:Name of receiver:Bill Kapralos
Faculty of Business and Information Technology, University of Ontario Institute ofTechnology, Canada
December 5, 2009 – December 13, 2009
Shizuoka University, Japan(Please see the description in the bottom of the page)Prof. Kamen Kanev

Name:
Affiliation:
Period of time:
Destination:
Purpose:Name of receiver:

Noriyuki Matsuda
Faculty of Systems Engineering, Wakayama University, Japan
February 2, 2010 – February 3, 2010
Shizuoka University, JapanTo discuss the research and project organization and to schedule future publications
Prof. Kamen Kanev

The purpose of the visit of Prof. Collins and Prof. Kapralos was two-fold:i) To attend the Humans and Computers 2009 conference and present three papers including a paper
outlining the research being conducted by Karen Collins, Bill Kapralos (Canada) and collaborators at
Shizuoka University (led by Kamen Kanev).
ii) To further build upon and expand the research collaborations between the University of Waterloo andthe University of Ontario Institute of Technology (UOIT) which are both in Ontario, Canada and theResearch Institute of Electronics, Shizuoka University, Japan.
A portion of their time was dedicated to meeting the project leader Prof. Matsuda from Wakayama
University and the representatives at RIE Prof. Kanev and Prof. Inokawa, and discussing collaboration
details, including the future direction of the current collaborative project that investigates audio-visualinteraction techniques for surface computers, and grant opportunities. Both Karen Collins and Bill
Kapralos also had the opportunity to meet many researchers from Shizuoka University as well as
researchers from other universities that attended the conference.
They discussed the potential of a two-way visiting opportunity between themselves (the researchers) andstudents at Shizuoka University, Waterloo, and UOIT. Finally, in preparation for a future extended visit,
they also had the opportunity to learn about the University itself (resources, etc.) as well as life in
Hamamatsu. The goals of their visit were not only met but exceeded. Their collaboration with Shizuoka
University continues and they are currently preparing a submission to the ACM FuturePlay conference andexploring grant collaborative opportunities.