東北大学 電気通信研究所

研究室外部評価資料(2013 年度-2018 年度)

Activity Report of Research Laboratory
for External Review
April 2013 - March 2019(FY.2013-2018)

Research Institute of Electrical Communication
Tohoku University

高次視覚情報システム研究室
Visual Cognition and Systems

April 2013 - March 2019 (FY. 2013-2018)
A. 研究室名 / Research Laboratory
高次視覚情報システム研究室
Visual Cognition and Systems

B. 構成員 / Faculty and Research Staff (as of May 1, 2019)
Professor
Associate Professor Kuriki, Ichiro (Dr. Eng.)
Associate Professor Tseng, Chia H, Ichiro (Dr. Eng.)
産学官連携研究員：1 名※ 欄を適宜追加削除等調整して下さい．期間内に異動等があった場合には，在籍期間を記載して下さい．

教授 / Professor
氏名
塩入 諭Satoshi Shioiri
Name
分野名
高次視覚情報システム研究室
Research Field Visual Cognition and Systems
准教授 / Associate Professor
氏名
栗木 一郎
Ichiro Kuriki (April 2015 - March 2017)
Name
分野名
知覚脳機能研究分野
Cognitive Brain Functions
Research Field
准教授 / Associate Professor
氏名曽 加蕙Chai-huei, Tseng (September 2016 - present)
Name
分野名
注意・学習研究分野
Attention and Learning Systems
Research Field
助教 / Assistant Professor
氏名
羽鳥 康裕
Yasuhiro Hatori (May 2018 - present)
Name
分野名
高次視覚情報システム研究室
Visual Cognition and Systems
Research Field
他 / Others
博士研究員/Postdoctoral fellows
産学官連携研究員: 1 名 (April 2010 - March 2013)
産学官連携研究員: 1 名 (May 2011 - March 2014)
産学官連携研究員: 1 名 (April 2015 - August 2015)
産学官連携研究員: 1 名 (April 2014 - March 2016)
学振特別研究員: 1 名 (April 2015 - January 2018)

C. 研究目的 / Research Purpose
本研究室では、脳機能について特に視覚系の働きの研究から探求し、それに基づく人間工学、画像工学などへの応用的展開を目的としている。人間の視覚特性を知るための心理物理学的実験を中心に脳機能測定やコンピュータビジョン的アプローチを利用して、視覚による空間知覚、立体認識、注意による選択機構のモデルの構築、視触覚統合機構に関する研究をしている。
Human brain is one of the most adaptable systems in the world. Understanding the brain functions
is one of the most important issues for evaluating and designing things around us to improve thequality of life. We investigate the brain through visual functions to apply the knowledge to humanengineering and image engineering.

D. 主な研究テーマ / Research Topics
１1. 初期視覚の基礎研究と応用
2. 注意の神経機構の研究
3. 能動的視覚の研究
4. 無意識的空間表象獲得の研究
5. 色カテゴリーの文化差・個人差
1. Investigation of early vision and its application.
2. Neuro-mechanisms of attention
3. Study of active aspect of visual system
4. Implicit learning of visual representations
5. Cultural and individual difference of color categories

２E. 学術論文等の編数 / The Number of Research Papers(1) 査読付学術論文
Refereed journal papers(2) 原著論文と同等に扱う
査読付国際会議発表論文
Full papers in refereed conference
proceedings equivalent to journal papers(3) 査読付国際会議
Papers in refereed conference proceedings(4) 査読なし国際会議・シンポジウム等
Papers in conference proceedings(5) 総説・解説
Review articles(6) 査読付国内会議
Refereed proceedings in domestic conferences(7) 査読なし国内研究会・講演会
Proceedings in domestic conferences(8) 著書Books(9) 特許Patents(10) 招待講演
Invited Talks

３2013

2014

2015

2016

2017

2018

Total

841886650

100000181425149239300000220000000000000025611101448000022400000002894121449

F. 特筆すべき研究成果 / Significant Research Achievements (FY.2013-2018)
See Ref. 1. “#” mark indicates research carried out at a former organization.
2013-2018 年度の研究成果（論文・特許など）のうち，前半（2013-2015 年度）と後半（2016-2018 年度）それぞれで代表的な数件（2-3 件程度ずつ）について，参考資料を引用して，その特徴と学術的意義などを簡単に紹介する．英文のみ，もしくは和文と英文で記載．
要約は 300 字程度．論文誌の要約/Abstract のコピー可．学術面での国際的インパクトならびに社会的影響を 100 字程度で記載．
必ずしも当該期間内に発表・出版したものに限るのではなく，例えば過去に発表したものでもこの期間内に成果が得られたり，評価されるようになったりしたものも含むものとする．
インパクトファクターや被引用件数など，できる限り第三者が定量的に評価できる指標を用いてアピールすること．それらの指標にはそぐわない場合には，その事情とそれに変わる適当な評価指標・尺度を示すこと．

[2013-2015]
1. Eye-head coordination for visual cognitive processing
Y Fang, R Nakashima, K Matsumiya, I Kuriki, S Shioiri
PloS one 10 (3), e0121035, 2013, [Scopus on 2019 July 19, Citation 22, FWCI 2.0]
Summary:
We investigated coordinated movements between the eyes and head (“eye-head coordination”) in relation to vision
for action. Several studies have measured eye and head movements during a single gaze shift, focusing on themechanisms of motor control during eye-head coordination. However, in everyday life, gaze shifts occur
sequentially and are accompanied by movements of the head and body. Under such conditions, visual cognitive
processing influences eye movements and might also influence eye-head coordination because sequential gaze shifts
include cycles of visual processing (fixation) and data acquisition (gaze shifts). In the present study, we examined
how the eyes and head move in coordination during visual search in a large visual field. Subjects moved their eyes,
head, and body without restriction inside a 360° visual display system. We found patterns of eye-head coordinationthat differed from those observed in single gaze-shift studies. First, we frequently observed multiple saccades during
one continuous head movement, and the contribution of head movement to gaze shifts increased as the number ofsaccades increased. This relationship between head movements and sequential gaze shifts suggests eye-headcoordination over several saccade-fixation sequences; this could be related to cognitive processing because saccadefixation cycles are the result of visual cognitive processing. Second, distribution biases of eye position during gaze
fixation was highly correlated with head orientation. The distribution peak of eye position was biased in the same
direction as head orientation. This influence of head orientation suggests that eye-head coordination is involved ingaze fixation, when the visual system processes retinal information. This further supports the role of eye-headcoordination in visual cognitive processing.

International impact on both academic and social aspects：
This work showed new patterns of eye-head coordination for continuous visual search with head and bodymovements. Since the experimental condition is much closer to everyday life than previously studies of eye-headcoordination with a single gaze shift, the findings of the study have great impact for understanding human visualfunctions in natural environments. The article has been cited 22 times and the number is increasing during the period
between 2015 and 2018.

[2016-2018]
1. Visual attention spreads broadly but selects information locally, S Shioiri, H Honjyo, Y Kashiwase, K Matsumiya,
I. Kuriki, Scientific reports 6, 35513[Scopus Citation 6; FWCI 0.44]
４Summary:
Visual attention spreads over a range around the focus as the spotlight metaphor describes. Spatial spread ofattentional enhancement and local selection/inhibition are crucial factors determining the profile of the spatialattention. Enhancement and ignorance/suppression are opposite effects of attention, and appeared to be mutuallyexclusive. Yet, no unified view of the factors has been provided despite their necessity for understanding thefunctions of spatial attention. This report provides electroencephalographic and behavioral evidence for theattentional spread at an early stage and selection/inhibition at a later stage of visual processing. Steady state visualevoked potential showed broad spatial tuning whereas the P3 component of the event related potential showed local
selection or inhibition of the adjacent areas. Based on these results, we propose a two-stage model of spatial attention
with broad spread at an early stage and local selection at a later stage.
.
International impact on both academic and social aspects：
This work showed that there are two different aspects of spatial attention. These two can be attributed to the different
stages of visual processes, such as one at an occipital area and one at a parietal area. The results shown in the article
are clear pieces of evidence of multiple processes of spatial attention, and this would shine a light on the concept ofmultiple stages of attentional control. The number of citations of the related articles (Shioiri et al, Vision Research,2002 (CI 25), 2000 (CI 25); Matsubara et al, Optical Review 2007 (CI 6); Shioiri et al, Journal of Vision 2010 (CI15), 2015 (CI 0); Harasawa & Shioiri, Brain and Cognition, 2011 (CI 6); Kashiwase et al, Journal of Cognitive
Neuroscience, 2012 (CI 22); Kashiwase et al, PLoS One , 2013 (CI 4)), is 103 in total while the citation of the article
is 6 yet.

2. Spatial representations of the viewer’s surroundings, S Shioiri, M Kobayashi, K Matsumiya, I Kuriki
Scientific reports 8 (1), 7171, 2018[Scopus Citation 3; FWCI 1.03]
Summary:
Spatial representation surrounding a viewer including outside the visual field is crucial for moving around the threedimensional world. To obtain such spatial representations, we predict that there is a learning process that integrates
visual inputs from different viewpoints covering all the 360° visual angles. We report here the learning effect of thespatial layouts on six displays arranged to surround the viewer, showing shortening of visual search time onsurrounding layouts that are repeatedly used (contextual cueing effect). The learning effect is found even in the time
to reach the display with the target as well as the time to reach the target within the target display, which indicatesthat there is an implicit learning effect on spatial configurations of stimulus elements across displays. Since,
furthermore, the learning effect is found between layouts and the target presented on displays located even 120°apart, this effect should be based on the representation that covers visual information far outside the visual field.
International impact on both academic and social aspects：
This study showed that visual system learn implicitly spatial information in the surroundings through repeatedobservation and the information can be used to identify the location of task related object. This indicates that
representations of the surrounding space are used to control attention which controls motor commands for action inthe space. Visual spatial attention is not only the process for current visual input, but it is also the process for
information outside the visual field. These findings have strong impact on the field of investigating and modeling５human visual attention.

3. Yang J., Kanazawa S., Yamaguchi M.K., and Kuriki I. (2016) Cortical response to categorical color perception ininfants investigated by near-infrared spectroscopy. Proceedings of The National Academy of Sciences of U.S.A.
113(9), 2370-2375. doi: 10.1073/pnas.1512044113
Summary:
Perceptual color space is continuous; however, we tend to divide it into only a small number of categories. It is
unclear whether categorical color perception is obtained solely through the development of the visual system orwhether it is affected by language acquisition. To address this issue, we recruited prelinguistic infants (5- to 7-moolds) to measure changes in brain activity in relation to categorical color differences by using near-infrared
spectroscopy (NIRS). We presented two sets of geometric figures to infants: One set altered in color between green
and blue, and the other set altered between two different shades of green. We found a significant increase inhemodynamic responses during the between-category alternations, but not during the within-category alternations.
These differences in hemodynamic response based on categorical relationship were observed only in the bilateral
occipitotemporal regions, and not in the occipital region. We confirmed that categorical color differences yield
behavioral differences in infants. We also observed comparable hemodynamic responses to categorical color
differences in adults. The present study provided the first evidence, to our knowledge, that colors of different
categories are represented differently in the visual cortex of prelinguistic infants, which implies that color categories
may develop independently before language acquisition.
International impact on both academic and social aspects:
This study first showed the presence of neural activity, corresponding to categorial color perception in prelingual
infants, and that the foci of activity seem to be irrelevant from language processing areas. Presence of such
categorical responses in the infants of 5-7 month years old implies that the development of color categorization isoccurring independent of language acquisition. This is a strong counter evidence for the famous psychologicaltheory called Sapir-Warf hypothesis, in which language is considered to define sensations. This study was reported
in periodically published magazines for general readers (La Repubblica, Il Venerdi, Italy, and Scientific American
Mind, U.S.A). This study, together with the following article on Japanese color lexicon, led to the publication of areview article, and 13 invited talks (including two keynote lectures) at international conferences.

4. Kuriki I., Lange R., Muto Y., Fukuda K., Tokunaga R, Lindsey D.T., Brown A.M., Uchikawa K., and Shioiri S.
(2017) The modern Japanese color lexicon. Journal of Vision, 17(3), 1, 1-18.
Summary:
Despite numerous prior studies, important questions about the Japanese color lexicon persist, particularly about thenumber of Japanese basic color terms and their deployment across color space. Here, 57 native Japanese speakers
provided monolexemic terms for 320 chromatic and 10 achromatic Munsell color samples. Through k-means cluster
analysis we revealed 16 statistically distinct Japanese chromatic categories. These included eight chromatic basic
color terms plus eight additional terms. Of these additional terms, mizu was used by 98% of informants, and emerged
as a strong candidate for a 12th Japanese basic color term. Japanese and American English color-naming systems
were broadly similar, except for color categories in one language (mizu, kon, teal, lavender, magenta, lime) that had６no equivalent in the other. Our analysis revealed two statistically distinct Japanese motifs (or color-naming systems),
which differed mainly in the extension of mizu across our color palette. Comparison of the present data with anearlier study by Uchikawa & Boynton (1987) suggests that some changes in the Japanese color lexicon haveoccurred over the last 30 years.
International impact on both academic and social aspects:
The separation of light blue category from blue category is already found in other languages (e.g., Russian, Italian,and Spanish), however, it is quite rare to observe the separation of a basic color category in several decades. Also,by the use of k-means clustering analysis, the separation of category became free from linguistic point of arguments,
and it also made it possible to compare categories beyond the language barrier. The story about the separation ofbasic color term (blue -> blue & light blue) drew public attention. Several newspapers, including a local paper anda paper for primary school students, picked up this result in their article. It led to 13 invited talks including twokeynotes at international conferences, together with the infant study (#3), and it also led to be invited to a TV
program to give comments on color names in Japanese.

5. Tsui, A. S. M., Ma, Y. K., Ho, A., Chow, H. M., & Tseng, C. H. (2016). Bimodal emotion congruency is criticalto preverbal infants’ abstract rule learning. Developmental science, 19(3), 382-393.
Tseng, C. H., Chow, H. M., Ma, Y. K., & Ding, J. (2018). Preverbal infants utilize cross-modal semantic congruency
in artificial grammar acquisition. Scientific reports, 8(1), 12707.
Summary:
These two reports investigate how visual-audio integration modulates pre-verbal infants learning of abstract rules.
The 2016 study demonstrated that 8-to10-month old infants used emotional congruency between face and voice asa pre-cursor to subsequent learning. The 2018 study examines the generality of visual-audio integration limitations
on infant learning. The results show that infants are also sensitive to visual-audio object file registration (e.g. humanvs machine) and sensory correspondence (e.g. visual motion vs audio slide), implying a more general gating
mechanism in early learning during infancy.
International impact on both academic and social aspects：
The series of infant learning research offered a new research method for sensory integration studies to thedevelopmental science community which enables new research topics to be studied. The most common methods ininfant multi-sensory studies are (1) uni-modal vs bi-modal comparison, and (2) synchronous vs asynchronouscomparison. These methods are useful but limited to probe the properties of low-level inputs (i.e. spatial-temporalcharacteristics of bimodal pairs). The new simultaneous bimodal presentation method here enables exploration ofhigh-level effects (e.g. semantically related bimodal pairs) during developmental maturity. They provide insights toinfant rule-extraction learning abilities before linguistic acquisition.

７

G. 特筆すべき活動 / Significant Activities (FY.2013-2018)
See Ref. 2-9.

“#” mark indicates research carried out at a former organization.

研究室外部評価参考資料の 2 以降を参照しながら，2013-2018 年度のなどの活動の中から特筆すべきものを取り出し，前半（20132015 年度）と後半（2016-2018 年度）に分けて簡単に紹介する．英文のみ，もしくは和文と英文で記載．

1. International Collaboration
Our activities of international collaboration are listed below.
1) We contributed to organize 8 international meeting between 2013 and 2015, and we organized 10 international
meetings between 2016 and 2018.
2) We published 8 journal papers from international collaboration between 2016 and 2018.
3) We had 2 international collaboration projects via RIEC Cooperative Research Project between 2013 and 2015,and 3 projects between 2016 and 2018. In 2018, we started international cross-institutional collaboration (S type)with National Taiwan University.
4) The number of total international collaboration projects between 2013 and 2015 is 6 and that between 2016 and2018 is 6.

2. Leading the field of vision science in Japan
Our activities as a lab leading the field of vision science in Japan are listed below.
1) The number of RIEC Cooperative Research Project between 2013 and 2015 is 5 and that between 2016 and 2018is 10. Contributors to the projects were from all over Japan and from foreign countries.
2) We held monthly internet meeting for young vision science researchers in Japan (internet vision meeting or IVM)
since 2010. Currently it is attended by visual scientists from 10 universities. This is a continuation of a similar
meeting we organized between 1998 and 2008, using a satellite communication system for remote education (spacecollaboration system or SCS), which was terminated in 2008.
3) Our group senior members served in leadership roles of in academic societies as a president, a secretary, a meetingorganizer and so on.
4) Our group member served in government committee to establish industrial display standards and in non-profit
organization (NHK) as research adviser.

3. Outreach activities
Our outreach activities are listed below.
1) We reached out to the general public to promote vision sciences by offering lectures and interactive activities.
2) We also offered seminars on vision-related topics to industrial partners and medical association.
3) We visited middle and high schools to provide lectures and hand-on workshops (3 times in 2017, 3 times in 2018,
2 times in 2016, 2014, and 2013).
4) We hosted hundreds of community visitors every year at the Annual Open Day.
5) Our research members received TV interviews/commentaries and contributed to newspaper column as experts.

4. Research grants
1) JSPS Kakanehi Projects
During the period of 2013-3015, we achieved 4 Kakenhi projects as principal investigators and 2 projects as co８

principal investigator. The total amount of grant was 66,930k JPY. During the period of 2016-2018, we achieved 6Kakenhi projects as principal investigators and the total amount of grant during this period was 31,350k JPY.
2) Other research grants
During the period of 2013-2015, we received 3 research programs in Japan (58,080k JPY), including a JST CREST
project as a Major co-investigator. During the period of 2016-2018, we received a support from MEXT as a principal
investigator (30,000k JPY/year).

5. Education
Our activities of education are listed below.
1) The number of doctoral degree students supervised between 2013-2015 is 1 and that between 2016 and 2018 is1.
2) The number of master degree students supervised between 2013-2015 is 5 and that between 2016 and 2018 is 12.
3) The number of bachelor degree students supervised between 2013-2015 is 6 and that between 2016 and 2018 is9.
4) The number of lectures we had between 2013-2015 is 8 and that between 2016 and 2018 is 5.

９