National Institute for Basic Biology

Neurobiology

LABORATORY OF NEUROPHYSIOLOGY

Associate Professor

WATANABE, Eiji

Specially Appointed Assistant Professor:
KOBAYASHI, Taisuke
JSPS Postdoctoral Fellow: NISHIUMI, Nozomi
Visiting Scientist:
KOMATSU, HidehikoTechnical Assistant:
WATABE, Mihoko
TSURUMARU, Mayumi
SUGINAGA, Tomomi

We believe that the best way to understand mechanisms underlying a certain system is to reconstruct the system as it exists. Byutilizing computer technologies, we are accordingly trying tobuild the systems based on psychophysical and ecological viewpoints in particular to understand systems of animal perception
and behavior. This type of methodology based on computerswill pave the way for biology in the future.

ing predator-prey interactions in aquatic environments. To
understand the elements of motion that cause such selective
predation behavior, we used a virtual plankton system where
we analyzed the predation behavior in response to computergenerated plankton. As a result, we confirmed that medaka
exhibited predation behavior against several characteristic
movements of the virtual plankton, particularly against aswimming pattern that could be characterized as a pink noise
motion (Matsunaga & Watanabe, 2012). As a side-story,
we analyzed the swimming behaviors of adult water flea
Daphnia magna, and found apparent sexual differences:laterally biased diffusion of males in contrast to the nondirectional diffusion of females (Toyota et al., 2022).
Many fish species are known to live in groups, and visualcues have been shown to play a crucial role in the formationof shoals. By utilizing biological motion stimuli, which inthis case was the depiction of a moving creature by using justa few isolated points, we examined whether physical motion
information is involved in the induction of shoaling behavior.
We consequently found that the presentation of virtual biological motion can clearly induce shoaling behavior, andhave shown what aspects of this motion are critical in theinduction of shoaling behavior. Motion and behavioral characteristics can be valuable in recognizing animal species,
sex, and group members. Studies using biological motion
stimuli will further enhance our understanding of how nonhuman animals extract and process information which is vitalfor their survival (Nakayasu & Watanabe, 2014).
Additionally, we have developed a novel method for
behavior analysis using 3D computer graphics (Nakayasuet al., 2017). The fine control of various features of living
fish has been difficult to achieve in studies of their behavior.
However, computer graphics allow us to systematicallymanipulate morphological and motion cues. Therefore, wehave constructed 3D computer graphic animations based on
tracking coordinate data and photo data obtained from actual
medaka (Figure 1). These virtual 3D models will allow us
to represent medaka more faithfully as well as undertake amore detailed analysis of the properties of the visual stimuli
that are critical for the induction of various behaviors. This
experimental system was applied to studies on dynamic
seasonal changes in color perception in medaka (Shimmuraet al., 2017), and on underwater imaging technology (Abe etal., 2019 and Yamamoto et al., 2019).

I. Psychophysical study of medaka fish

One of our major subjects is the psychophysical and computational study of medaka (Oryzias latipes, Matsunaga& Watanabe, 2010). We have made progress in studiesof prey-predator interaction using both these organisms
and zooplankton. Visual motion cues are one of the mostimportant factors for eliciting animal behaviors, includ-44

Figure 1. Virtual medaka fish.

Ⅱ. Psychophysical study of human & AI vision

Another of our major subjects is the psychophysical andtheoretical studies of the visual illusions experienced byhuman and artificial intelligence (AI). One recent focus of
this debate is the flash-lag effect, in which a moving object
is perceived to head towards a flashed object when bothobjects are aligned in an actual physical space. We developed
a simple conceptual model explaining the flash-lag effect(Watanabe et al., 2010).
In recent years, we have made more developed novel visualillusions, such as the shelf-shadow illusion (3rd Award of theIllusion Contest), the Monstre Benham illusion (2ndAward), the Morning Monster Illusion (10th Award), and theGenkikan Illusion (Figure 2).

static images where humans were unable to perceive any
illusory motion. Additionally, illusion-like designs withrepeating patterns were generated using areas where anomalous vectors were detected, and psychophysical experiments
were conducted, in which illusory motion perception in thegenerated designs was detected (Figure 3). The observed
inaccuracy of the networks will provide useful information
for further understanding information processing associatedwith human vision (Kobayashi et al., 2022)

Figure 3. A visual illusions extracted by AIⅢ. Ecological study of tactics in predators andprey.

Figure 2. Okazaki Genkikan.

Watanabe (NIBB) and Sinapayen (Sony CSL) got 2nd placeat the 13th Visual Illusion Contest, for the Okazaki Genkikan
Illusion. It is one of the Poggendorff illusions. (http://www.
psy.ritsumei.ac.jp/~akitaoka/sakkon/sakkon2021.html). It
was found in Okazaki Genkikan Facility.
In 2018, we successfully generated deep neural networks(DNNs) that represent the perceived rotational motion
for illusion images that were not physically moving, yetsimilar to what we experience in human visual perception.
(Watanabe et al., 2018). These DNN computer models will
help to facilitate our future work on perception science. This
experimental system was applied to studies on evolutionary
illusion generator in collaborating with Dr. Lana Sinapayen(SONY CSL) (Evolutionary Generation of Visual Motion
Illusions, https://arxiv.org/abs/2112.13243, Sinapayen
and Watanabe, 2021). Currently, a variety of research isbeing conducted using these AI systems (https://arxiv.org/abs/2106.12254).
In 2022, we examined the properties of the network using a
set of 1500 images, including ordinary static images of paintings and photographs and images of various types of motion
illusions (Visual Illusions Dataset, https://doi.org/10.6084/
m9.figshare.9878663). Results showed that the networks
clearly classified a group of illusory images and others andreproduced illusory motions against various types of illusions
similar to human perception. Notably, the networks occasionally detected anomalous motion vectors, even in ordinally

We are interested in behavioral interactions between predators and prey concerning the ecological aspect. In the processof the co-evolution, predators and prey have developed
various tactics to overcome each other. To elucidate thesophistication of these tactics, we have examined the mechanisms and efficacies of predatory and antipredator behaviorsof several animals such as fish, dragonflies, bats and birds.
In addition, we have developed experimental system forthose studies, which present interactive virtual predators toprey animals based on animation (Figure 4) or robotics. This
experimental system consists of three modules as a sensing, acommand generating and a virtual animal module. According
to the spatial scale, the system employs appropriate technologies in each module. For example, in a small aquarium
scale, we use high speed cameras as a sensing module andvirtual reality animation as a virtual animal module. On theother hand, in a large outdoor scale, we use real-time GPS
as a sensing module and unmanned aerial vehicle as a virtual
animal module. Therefore this system are able to cover wider
range of animals and expected to strongly support the studyof animal interactions.

Figure 4. A computer animation of the virtual predatory fish.

45

National Institute for Basic Biology

Neurobiology

Publication List:〔Original paper〕

T., Kitaoka, A., Kosaka, M., Tanaka, K., and Watanabe,• Kobayashi,
E. (2022). Motion illusion-like patterns extracted from photo and art
images using predictive deep neural networks. Sci. Rep. 12, 3893. DOI:
10.1038/s41598-022-07438-3

46