2020年度日本認知科学会第37回大会

OS10-1

Theory of Cooperation:
Exploring Human and Machine Capabilities
Naoto Iwahashi

Hiroyuki Okada

Kotaro Funakoshi

Okayama Prefectural Univ

Tamagawa Univ

Tokyo TechAbstract

2

Cooperation is one of the most fundamental humancharacteristics, and several interdisciplinary studies
have been conducted to understand this aspect. However, despite the numerous studies conducted on thesubject, a comprehensive theory regarding cooperation remains elusive. In this study, the theoretical andpractical aspects of human–human, human–machine,and machine–machine cooperation are explored. Cooperation is theoretically analyzed from behavioral,mathematical, and cognitive viewpoints. The theoretical principles were applied to certain unsolved problems. Furthermore, it is demonstrated that such cooperative machines can be used to investigate the humancapabilities of cooperation. The developed versatileinteractive software RoCoCo, which can be used asa research tool, is presented. Finally, the possibilities and prospects of this research framework, whichis termed the theory of cooperation, are discussed.

We analyze cooperation from behavioral, mathematical, and cognitive viewpoints to explore the principles
that establish cooperation.

Keywords: cooperation, theory, practice, human,machine, RoCoCo

1

Introduction

Cooperation is a fundamental and crucial human characteristic, regarding which numerous interdisciplinary
studies (including philosophy, science, and technology) have been conducted. Despite the recent rapid
research progress, a comprehensive theory regarding
cooperation has not been established yet. In thisstudy, we introduce a framework, termed as the theoryof cooperation, which we proposed to explore human–
human, human–machine, and machine–machine cooperation, both via theoretical research and developmentof practical machines.
The theory of cooperation aims to answer the following questions:• What is cooperation?

2.1

Principles of cooperation

Behavioral

From a behavioral viewpoint, cooperation generallycomprises the following three interdependent and hierarchical behavioral functions:
Action This function enables each agent to act according to the given dynamics and constraints.
Interaction This function enables agents to mutually predict each other s behaviors and act undera given role assignment.
Role coordination This function enables agents tomutually find and select their own role assignments online.
Among these three functions, role coordination is particularly important and has been investigated in several studies. However, this function is not yet completely understood with regard to both cognitive andmathematical aspects.

2.2

Mathematical

We propose hierarchical equilibrium dynamics (HED)
as a mathematical formulation of general cooperation.
It is based on control, game theory, and physics, andcharacterized by the structure of dynamics and equilibrium of coupled dynamics.
The structure comprises the following three-layered
dynamics according to the behavioral hierarchies(Fig. 1):
Action and task dynamics Modeling of the actions and tasks executed by the agents.
Interaction equilibrium dynamics Modelingof the interactions among agents, which aredescribed via the occurrence of an equilibrium inthe coupling of dynamical systems.

• How does cooperation work?
• What is human cooperation ability?
• How do we apply cooperation to artificial intelligence systems?

Role equilibrium dynamics Modeling of role coordination, which controls the action equilibriumdynamics.

1007

2020年度日本認知科学会第37回大会

OS10-1Figure 2: Belief configuration of an agent for cooperation

Figure 1: Structure of dynamics that establish cooperation
The dynamics of the role coordination for two agents,
f role , and the dynamics of the action of the ith agent,
fiact , can be expressed as follows:
at+1 , bt+1=

f role (at , bt , ct )yi,t+1 , xi,t+1=

fiact (yi,t , xi,t , ui,t , at , zt )The task dynamics, f task , can be expressed as follows:
zt+1 , vt+1 = f task (zt , vt , y0,t , y1,t )For optimization, the objective for the control of theith agent is expressed as follows:
Jioverall=J role (A, B, C) + Jiact (Yi , Xi , Ui ) + J task (Z)where J role , Jiact , and J task denote the objectives forthe control of the dynamics f role , fiact , and f task , respectively.
HED specializes or generalizes a wide range of formulas for synchronization, such as non-cooperative
game [1], differential game [2], decentralized control[3], pulse-coupled biological oscillators [4], chaotic synchronization [5], coupling of recurrent neural networks[6, 7], coupling of statistical dynamical models [8], andrecursive reasoning [9].
Previous studies on cooperative dynamics have addressed the action and interaction functions but not
the role coordination function. In particular, studies on the interaction function focused on convergenceto an equilibrium point. However, owing to the existence of multiple or infinite equilibrium points intypical problems, emphasis should be placed on theselection of an appropriate equilibrium point among
the available points; this selection is defined as rolecoordination.

2.3

Cognitive

Similar to the equilibrium in game theory that is basedon mutual belief, the equilibrium in HED is also basedon mutual belief. Moreover, because the equilibrium
in HED is hierarchical, a hierarchical structure of mutual beliefs is required. To meet this requirement, wepropose a mutual belief hierarchy with three layers,
each of which is represented by the corresponding dynamics as follows:
Belief The dynamics used by an agent to make decisions.
Mutual belief The dynamics of the coupling between the belief of an agent and those of others;
not mutual beliefs themselves.
Meta mutual belief The dynamics of the coupling
between the mutual beliefs of an agent and thoseof others; not meta mutual beliefs themselves.
The belief configuration of an agent with this belief
hierarchy system is depicted in Fig. 2.
The proposed hierarchical mutual belief system differs from the traditional mutual belief system [10] in
two ways. First, the coordinating agents’ different
mutual beliefs are emphasized. As our mutual beliefs
never become same, the most important problem to besolved should be the achievement of cooperation evenunder different mutual beliefs. However, traditional
theories of joint activities do not solve this problem.
The cognition of an agent should allows the agent tocoordinate her/his mutual belief with the mutual beliefs of others. Meta mutual belief is set at the top of
the hierarchy of the proposed system and provides theimportant ability to coordinate roles. Second, mental
self-recursive reflection is represented by the equilibrium of dynamics. The traditional mutual belief system is represented by the metaphor of self-recursivereflection by mirrors. However, it seems impossible toconstruct the mechanism of self-recursive reflection in
the brain from the viewpoints of information and control theories, and physics. Therefore, we use the concept of equilibrium between dynamics instead, which
seems feasible in the brain.

3Machine study

We applied the above-mentioned principles to thefollowing unsolved problems: autonomous driving[11], human–robot physical interaction, multi-modallanguage acquisition [12, 13], generation [14], andunderstanding[15], and robot-directed utterance detection [16]. From these applications, the validity ofthe principles was confirmed.

1008

2020年度日本認知科学会第37回大会

Humans-r machine
Cooperative machine

Cooperation
feasible
infeasible
feasible

OS10-1

References

Control
infeasible
feasiblefeasible[1] J. Nash, “Non-cooperative games,” Annals ofmathematics, pp. 286–295, 1951.
[2] D. Wishart, “Differential games. a mathematical theory with applications to warfare and pursuit, control and optimization,” Physics Bulletin,
1966.

Table 1: Comparison of the characteristics of threedifferent experimental frameworks in which subjects
interact with humans, s-r (stimulus-responsive) machines, and cooperative machines, respectively.

4[3] N. Sandell, et al., “Survey of decentralized control methods for large scale systems,” AutomaticControl, 1978.

Human study with cooperative machines[4] R. E. Mirollo and S. H. Strogatz, “Synchronization of pulse-coupled biological oscillators,”
SIAM Journal on Applied Mathematics, 1990.

The above-mentioned cooperative machines can beused to investigate the human capabilities of cooperation. In conventional experiments, human subjects interact with human or stimulus-responsive (sr) machines. However, there are some limitations toachieving coordinated operation and reproducible conditional control of the experiments (Table 1). By using
cooperative machines, we can reduce such limitations,and explore human capabilities more accurately anddeeply.

5[5] L. M. Pecora and T. L. Carroll, “Synchronizationin chaotic systems,” Physical review letters, 1990.
[6] T. Ikegami and M. Taiji, “Imitation and cooperation in coupled dynamical recognizers,” in European Conference on Artificial Life. Springer,
1999.
[7] J. Tani, et al., “Self-organization of distributedlyrepresented multiple behavior schemata in a mirror system: reviews of robot experiments using
rnnpb,” Neural Networks, 2004.
[8] T. Sasaki, N. Iwahashi, and K. Funakoshi, et al.,“Learning and generation of collaborative actions
by mdl coupled hmms,” National Convention ofIPSJ, 2018.

Research tool

We developed the interactive software, RoCoCo, as a
research tool for exploring human and machine capabilities. RoCoCo has the following features:• Real time interactive[10] H. H. Clark, Using language. Cambridge university press, 1996.

• Versatile and expandable[11] N. Iwahashi, “Equilibrium selective role coordination for autonomous driving,” in Int. Conf.
Awareness Science and Technology, 2019.

• Multi-purpose (human and machine studies)• Flexible configuration (number of agents, humanmachine, human-human, and machine-machine)• Flexible task setting (autonomous driving andphysical interaction)• Various quantitative evaluations
The RoCoCo is now expanding in its versatality andflexibility.

6[9] Y. Wen, et al., “Probabilistic recursive reasoningfor multi-agent reinforcement learning,” in Proc.
ICLR, 2019.

Conclusions

On the basis of the fundamental analysis of cooperation, cognitive and mathematical principles were proposed. The validity of the principles was verified byapplying them to certain unsolved tasks. The potential of cooperative machines for exploring the humancapabilities of cooperation was discussed. A humanstudy using the interactive software RoCoCo is underway.

[12] ——, “Language acquisition by robots - towards
new paradigm of language processing -,” Journalof JSAI, 2003.
[13] N. Iwahashi, et al., “Robots that learn to communicate,” in AAAI Fall Symposium: Dialog withRobots, 2010.
[14] S. Nakamura, N. Iwahashi, and T. Nagai,“Mutully-adaptive generation of utterances basedon estimation of belief shared by human androbots in real world,” Journal of Japan Society for Fuzzy Theory and Intelligent Informatics,vol. 21, no. 5, pp. 663–682, 2009.
[15] K. Sugiura and N. Iwahashi, et al., “Object manipulation dialogue by estimating utterance understanding probability in a robot language acquisition framework,” Journal of the Robotic Society of Japan, vol. 28, no. 8, pp. 978–988, 2010.
[16] X. Zuo, N. Iwahashi, and K. Funakoshi, et al.,“Detecting robot-directed speech by situated understanding in physical interaction,” Informationand Media Technologies, 2010.

1009