Malcolm Doering(updated April 7, 2021)

Dept. of Social Informatics, Graduate School of Informatics, Kyoto University
Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, JAPAN
Email: doermalc@gmail.com
Cell: +81 80-1395-0265
Education
PhD Engineering (Robotics), 2019
Osaka University, Osaka, Japan• Dissertation: Robot Data-Driven Imitation Learning of Human Social Behavior with Time-PersistentInteraction Context• Advisor: Dr. Hiroshi Ishiguro
MS Computer Science, 2015
Michigan State University, East Lansing MI, USA• Thesis: Verb Semantics as Denoting Change of State in the Physical World• Advisor: Dr. Joyce Y. Chai
BS Computer Science with Honors, 2013
Michigan State University, East Lansing MI, USA
BA Linguistics and Japanese Language with Honors, 2013
Michigan State University, East Lansing MI, USA
Intensive Japanese Language Course, 2011-2012
Japan Center for Michigan Universities, Hikone, Japan
Research Experience
Specially-appointed Assistant Professor, April 2021-present
Specially-appointed Researcher, April 2020-March 2021
Technical Assistant, April 2019-March 2020
Human-Robot Interaction (Kanda) Lab., Kyoto University, Kyoto, Japan• Research focus: Imitation-learning for human-robot interaction, manly focusing on developing
methods for integrating database knowledge into neural conversational systems.
• Collaborated with / supervised visiting student (Amal Nanavati) in one-to-many imitation learning incamera shop domain – including interaction design, data-collection, neural architecture design, andevaluation.
• Supervising a PhD student on their research project.
Cooperative Researcher 2020-present
Interaction Science Laboratories, Advanced Telecommunications Research Institute International (ATR),Kyoto, Japan
Cooperative Researcher 2019-2020
Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research InstituteInternational (ATR), Kyoto, Japan
Research Intern, 2015-2019

1Hiroshi Ishiguro Laboratories, Advanced Telecommunications Research Institute International (ATR),
Kyoto, Japan• PhD dissertation research focus: Imitation-learning for human-robot conversational systems with
time-persistent interaction context. Developed neural-network-based and rule-based conversation
systems and evaluated them in off-line evaluations and user studies. Novel contributions include
integration of time-persistent interaction context to these systems, which enables more appropriate
responses to ambiguous human actions and personalization of behavior to individual users. The work
was published in the ACM THRI and IEEE THMS journals. It was supported in part by the JSTERATO Ishiguro Symbiotic Human-Robot Interaction Project, Grant Number JPMJER1401 and inpart by the Japanese Government MEXT Scholarship.
Graduate Research Assistant, 2013-2015
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• MS thesis research focus: Exploring verb semantics denoting change of state in the kitchen domain.
Crowdsourcing was used to collect perceived changes of state denoted by verbs in the kitchen domain
for the purpose of training a robot to ground language to perception. The roles of linguistic and visualfeatures in predicting change of state were explored in both machine learning models and humans.
This work lead to a publication in ACL 2016.
•Other research: Collaborated in referring expression generation (REG) for human-robot interaction.
Contributed in implementing a client-server system that enabled an REG model to interact withworkers in the crowd so it could be trained via reinforcement learning. Implemented the trained REG
model on a Nao robot, integrated eye-gaze feedback, and aided in conducting a user study. The work
was published in AAAI 2014 and HRI 2015.

Undergraduate Research Assistant, Evolutionary Robotics, 2010-2013
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Designed and implemented evolutionary algorithms for simulated swarms of e-puck robots to evolve
synchronization behaviors for error detection in distributed robot networks.
• Designed and implemented a distributed evolutionary algorithm in an interactive robot art exhibit forevolving interesting responses to human touch and sound. Wrote embedded software to integrate a
RaspberryPi with 5 peripheral Atmel microcontrollers, pressure/temperature sensor, touch sensor, andethernet chip.
Undergraduate Research Assistant, Natural Language Processing, 2009, 2011
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Assisted in annotating the thematic roles of constituents in thousands English sentences for researchon semantic role-labeling of implicit arguments. The resulting work won Best Long Paper (Gerberand Chai. “Beyond NomBank: A Study of Implicit Arguments for Nominal Predicates.” ACL 2010,
acceptance rate 25%).
• Applied insights from Centering Theory of discourse coherence to increase the accuracy of adecision-tree based implicit-argument identification and classification system.
Undergraduate Research Assistant, Sociolinguistics, 2008-2009
Michigan State University Department of Linguistics and Languages, East Lansing, USA• Wrote regular expression queries to find instances of pre-auxiliary adverbs in the American National
Corpus for the purpose of comparing adverb use in American and British English. Our poster won theLinguistics Undergraduate Research Award.
Journal Publications
M Doering, D Brščić, T Kanda. Data-driven Imitation Learning for a Shopkeeper Robot with Periodically
Changing Product Information, ACM Transactions on Human-Robot Interaction (THRI), (in press).

2

M Doering, T Kanda, H Ishiguro. Neural-network-based memory for a social robot: Learning a memory
model of human behavior from data, ACM Transactions on Human-Robot Interaction (THRI) 8 (4), 24,December, 2019.
M Doering, P Liu, DF Glas, T Kanda, D Kulić, H Ishiguro. Curiosity Did Not Kill the Robot: A
Curiosity-based Learning System for a Shopkeeper Robot, ACM Transactions on Human-RobotInteraction (THRI) 8 (3), 15, July, 2019.
M Doering, DF Glas, H Ishiguro. Modeling Interaction Structure for Robot Imitation Learning of Human
Social Behavior. IEEE Transactions on Human-Machine Systems (THMS) 49 (3), 219-231, February,2019.
Refereed Conference Publications
A Nanavati, M Doering, D Brščić, T Kanda. Autonomously Learning One-To-Many Social Interaction
Logic from Human-Human Interaction Data. 15th ACM/IEEE International Conference on Human-RobotInteraction (HRI 2020), March, 2020.
DF Glas, M Doering, P Liu, T Kanda, H Ishiguro. Robot's Delight: A Lyrical Exposition on Learning byImitation from Human-human Interaction, 12th ACM/IEEE Conference on Human-Robot Interaction(HRI 2017), March, 2017. (Video submission) (24% acceptance rate) (Best Video Award)
Q Gao, M Doering, S Yang, J Chai. Physical causality of action verbs in grounded language
understanding. 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016),
August, 2016. (25% acceptance rate)
R Fang, M Doering, JY Chai. Embodied collaborative referring expression generation in situated humanrobot interaction. 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2015),
March, 2015. (25% acceptance rate)
R Fang, M Doering, JY Chai. Collaborative models for referring expression generation in situateddialogue. Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI 2014), July, 2014. (28%acceptance rate)
Workshop Papers and PostersP Liu, M Doering, DF Glas, T Kanda, D Kulić, H Ishiguro. Personalizing crowdsourced human-robot
interaction through curiosity-driven learning. 14th ACM/IEEE International Conference on Human-Robot
Interaction (HRI 2019), March, 2019.
M Doering, DF Glas, H Ishiguro. Modeling Interaction Structure for Robot Imitation Learning of Human
Social Behavior. 1st International Symposium on Systems Intelligence. Osaka, Japan. January, 2018.
A Evrard, M Doering, K Fraser, S Giuffre. I Probably Would Put Probably Second: Pre-Auxiliary
Adverbs in American English. Undergraduate Research & Arts Forum. East Lansing, USA. 2009.
(Linguistics Undergraduate Research Award)
Media Appearances“The Rise of Social Robots.” TechKnow. Al Jazeera English. December 24, 2017. Television.
Awards and Honors•

MEXT Scholarship for full funding of Graduate Studies in Japan, 2015-2018

3••
•
•
•
•
•

"Botsker" award for Best Picture, 8th annual Robot Film Festival, July 2018. Robot's Delight - A
Lyrical Exposition on Learning by Imitation from Human-Human Interaction
http://robotfilmfestival.com/rff2018-films
Best Video Award, ACM/IEEE International Conference on Human-Robot Interaction, 2017
Urban Science Sigma Award for best overall capstone project, Michigan State University, 2013
Bridging Scholarship for Study Abroad in Japan, US-Japan Bridging Foundation, 2011
Marion L. & Harriette I. Bandeen Endowed Scholarship, Michigan State University, 2010
Linguistics Undergraduate Research Award, Michigan State University, 2009
Alumni Distinguished Scholars Semi-finalist Commended Scholarship, Michigan StateUniversity, 2007

Teaching and Mentoring Experience
Graduate Teaching Assistant, Senior Design Course (CSE 498), 2013-2015
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Supervised semester-long group projects (12 per semester) via weekly status meetings, offering
guidance, and providing feedback on project proposals and other documents.
Graduate Teaching Assistant, Discrete Structures in Computer Science (CSE 460), 2015
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Answered questions during help room hours and graded homework, tests, and quizzes.
Mentor, NSF Research Experience for Teachers Summer Program, Summer 2014
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Mentored Mrs. Brandy Butcher in research on verb categorization and child language acquisition.
Insights from her research were used to design an experiment to explore how humans naturally teachnovel actions in noisy conditions.
Students Supervised
Yongqiang Jiang (PhD student, 2020-present)
Amal Nanavati (Fulbright Fellow research student, 2018-2019)
Other Relevant Experience
Senior Design Course Project (CSE 498), 2013
Michigan State University, Dept. of Computer Science and Engineering, East Lansing, USA• Developed a sign-language learning application for the Microsoft Surface tablet in collaboration with
two other students and a corporate client (TechSmith, East Lansing, USA).
• Exhibited and presented the finished project to judges on ‘Design Day’ and received Urban ScienceSigma Award for best project.
Professional ServiceReviewer• International Conference on Human-Agent Interaction (HAI)• ACM/IEEE International Conference on Human-Robot Interaction (HRI)• IEEE Transactions on Cognitive and Developmental Systems (TCDS)• ACM Transactions on Human-Robot Interaction (THRI)• International Journal of Social Robotics• IEEE Transactions on Cybernetics
Student Volunteer Chair, 201914th ACM/IEEE International Conference on Human-Robot Interaction,4•

Helped to recruit and organize the student volunteers to assist with conference registration,
telepresence-robot operation, room-setup, etc.

Technical Skills•••••Programming languages: Python, R, Java, C#, C++, C, Matlab
Experiencing in programming: Deep learning with TensorFlow and Theano, (Atmel) microcontrollers
and peripheral devices, RaspberryPi, Android and Windows RT mobile applications, YouTube API via
RESTful, IBM Watson Natural Language Understanding API, Google Speech API, ASL Mobile Eye
eye-tracking system, Vizard game engine, Webots robot simulator
Video editing with Adobe Premiere Pro and Blender
Japanese language: Intermediate
Subject areas: Human-robot interaction, natural language processing, machine learning (clustering,classification, dimensionality reduction), evolutionary computation, child language acquisition,
linguistics, verb semantics, neural networks (DNN, RNN, GRU, LSTM, Memory Networks, etc.),
deep learning, utterance and topic clustering, interaction modeling, biologically-inspired computation,
experiment design, user studies

5