+RESEARCH ARTICLE

Inter-Hemispheric Desynchronization of the Human MT+
during Visually Induced Motion Sickness
Jungo Miyazaki1, Hiroki Yamamoto2*, Yoshikatsu Ichimura1, Hiroyuki
Yamashiro3, Tomokazu Murase4, Tetsuya Yamamoto5, Masahiro Umeda4, Toshihiro
Higuchi6
1. Corporate R&D, Canon Inc., Tokyo, Japan
2. Graduate School of Human and Environmental Studies, Kyoto University,
Kyoto, Japan
3. Department of Medical Engineering, Aino University, Osaka, Japan
4. Department of Medical Informatics, Meiji University of Integrative Medicine,
Kyoto, Japan
5. Graduate School of Engineering, Kyoto University, Kyoto, Japan
6. Department of Neurosurgery, Meiji University of Integrative Medicine, Kyoto,
Japan*Correspondence to: Dr. Hiroki Yamamoto
Graduate School of Human and Environmental Studies, Kyoto University
YoshidaNihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan
E-mail: yamamoto@cv.jinkan.kyoto-u.ac.jp
Tel: +81-75-753-2978; Fax: +81-75-753-6574

Short Title: Impact of VIMS on the human MT+
Keywords: human visual cortex, optokinetic nystagmus (OKN), vection, functional
magnetic resonance imaging (fMRI), visual motion,

ABSTRACTVisually induced motion sickness (VIMS) is triggered in susceptible individuals bystationary viewing of moving visual scenes. VIMS is often preceded by an illusion ofself-motion (vection) and/or by inappropriate optokinetic nystagmus (OKN), responses
associated with increased activity in the human motion-sensitive middle temporal area(MT+). Neuroimaging studies have reported predominant right hemispheric activationin MT+ during both vection and OKN, suggesting that VIMS may result fromdesynchronization of activity between left and right MT+ cortices. However, thispossibility has not been directly tested. To this end, we presented VIMS-free and
VIMS-inducing movies in that order while measuring the temporal correlations between
corresponding left and right visual cortices (including MT+) using functional magnetic
resonance imaging (fMRI). The inter-hemispheric correlation was reduced significantly
during the viewing of the VIMS-inducing movie compared to the control VIMS-freemovie in the MT+ of subjects reporting VIMS, but not in insusceptible subjects. In
contrast, there were no significant inter-hemispheric differences within VIMS-free or
VIMS-inducing movie exposure for visual area V1, V2, V3, V3A, or V7. Our findings
provide the first evidence for an association between asynchronous bilateral MT+
activation and VIMS. Desynchronization of left and right MT+ regions may reflect
hemispheric asymmetry in the activities of functional networks involved in eye
movement control, vection perception, and/or postural control.

  

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  

INTRODUCTION
Motion sickness (MS) is a physiological response to certain types of movement, as
experienced during sea, air, or space travel. About a third of the population issusceptible to MS (Griffin 1990), although susceptibility varies with gender and age(Golding 2006). The symptoms and signs of MS are classified broadly into the three
types: (1) oculomotor disturbances (eyestrain, blurred vision, headache), (2) nausea(stomach awareness, increased salivation, cold sweating, pale skin) and (3)disorientation (dizziness, drowsiness, fatigue) (Shupak & Gordon 2006; Kennedy et al.
2010). These negative effects can last for hours or longer (Kennedy et al. 1993a).
The vestibular system is presumed to play a central role in MS generation sincebilateral labyrinthine-defective individuals are immune (Kennedy et al. 1968; Cheung etal. 1991). Several hypotheses have been proposed to explain the physiological basis of
MS (see Flanagan et al. 2004 for a review), including the sensory conflict hypothesis(Reason & Brand 1975; Reason 1978; Oman 1990), the eye movement hypothesis(Ebenholtz 1992; Ebenholtz et al. 1994), the subjective vertical conflict theory (Bos &Bles 1998), and the postural instability hypothesis (Riccio & Stoffiregen 1991), all ofwhich include the vestibular system as a key component. However, none of these
theories is supported by conclusive evidence.
In addition to actual movement, stationary viewing of a moving visual scene canprovoke MS in susceptible observers. This visually induced motion sickness (VIMS)
has been reported in a variety of virtual environments (Ellis 1991). Virtual reality has
important applications in testing, research, and skill development, so future research on
the physiological causes of VIMS could allow for the development of useful virtual
applications with reduced risk of VIMS. To address this issue, however, we first need tounderstand the visual processes that encode the motion signals inducing VIMS. The
present study was conducted to examine the possible association of VIMS with specificpatterns of activity in visual cortices as revealed by functional magnetic resonanceimaging (fMRI).
The putative visual cues for VIMS are explicitly mentioned in two hypotheses for MS,
the sensory conflict hypothesis (Reason & Brand 1975; Reason 1978; Oman 1990) andthe eye movement hypothesis (Ebenholtz 1992; Ebenholtz et al. 1994). The sensory
conflict hypothesis postulates that VIMS results from a conflict or mismatch between
motion signals generated by different sensory systems and/or between actual motion
signals and expected motion signals stored in sensory memory. In support of thissensory conflict hypothesis, VIMS is often preceded by a compelling illusion of
apparent self-motion called vection (Hettinger et al. 1990), which is generally producedby large-field visual motion. Vection may occur when the motion cues provided byvisual input and the stationary cues provided by vestibular and/or proprioceptive inputs
conflict. On the other hand, the eye movement hypothesis (Ebenholtz 1992; Ebenholtz
et al. 1994) postulates that the stretching and/or traction of extraocular muscles induced
1  

42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  

by optokinetic nystagmus (OKN), reflexive eye movements produced by moving visual
scenes, produce afferent signals that ultimately stimulate the brainstem vomiting centervia the vestibular nuclei.
The two cues for VIMS, vection and OKN, are both associated with changing activity
within the human motion-sensitive middle temporal area (MT+). During vection, earlyvisual areas and parieto-insular vestibular cortex were deactivated (Brandt et al. 1998;
Kleinschmidt et al. 2002), whereas MT+ was either broadly activated (Brandt et al.
1998) or exhibited transient activation associated with shifts in perception(Kleinschmidt et al. 2002). Similarly, OKN was associated with activity in the MT+ as
well as V1, frontal eye field, supplementary eye field, ventrolateral premotor cortex,
posterior parietal cortex, and cerebellum (Dieterich et al. 1998, 2003; Konen et al. 2006).
Notably, hemispheric laterality during both vection and OKN was reported in MT+.
Kovacs et al. (2008) reported that the right MT+ was more active during vection thanduring true object motion while Dieterich et al. (1998) reported prominent righthemispheric activation in the MT+ during OKN.
The close link between these VIMS precursors and right MT+ predominance suggeststhat divergent activity between left and right MT+ may trigger VIMS. Previous imaging
studies could not address this issue because the stimulation periods employed were too
short to induce VIMS. Recently, Napadow et al. (2012) measured fMRI activity as
subjects viewed translating large stripes for a time sufficient to induce VIMS and foundthat the subjective feeling of nausea was associated with activation of a broad cortical
network including insular and anterior cingulate cortices, but this study did not examine
hemispheric laterality.
We hypothesize that VIMS is associated with asymmetric activity in the left and right
MT+. This hypothesis was tested by presenting a motion picture acquired by a rapidly
moving camera to induce VIMS while measuring the temporal correlation of brain
activity in corresponding bilateral cortical regions by fMRI. The present study focusedon brain activity directly reflecting visual processing of incoming sensory signals. We
isolated this activity component from lower frequency components that reflect
functional connectivity between hemispheres by high-pass filtering the fMRI signal (>0.1 Hz). We predicted that the inter-hemispheric correlation of MT+ activity evoked by
the movie would be reduced selectively when subjects experienced VIMS.
METHODS
Subjects
Fourteen healthy adults (12 men, 2 women; mean age 32.9 years, range 25–48 years)
with normal or corrected-to-normal vision, including four of the authors, participated inthis study. Subjects had no history of psychiatric or neurological disease, and all
provided written informed consent to participate. The study protocol was approved by
the Human Studies Committee of the Graduate School of Human and Environmental
2  

83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  

Studies at Kyoto University and the Department of Neurosurgery at Meiji University ofOriental Medicine.
As described in the Results section, subjects were divided into 2 groups based on
their self-reported response to the visual stimuli: a VIMS group (n=8, all men including
two authors; age: 32.3 ± 7.2, mean ± SD) who reported motion sickness while viewing a
VIMS-inducing movie, and a healthy group (n=6, two women and four men including
two authors; age: 33.3 ± 7.2, mean ± SD) who did not report motion sickness. In
addition to group division based on self-reports, a median split criterion, which divided
subjects based on the median of the Simulator Sickness Questionnaire (SSQ) score(described below), was employed to separate subjects into groups more objectively.
Visual stimuli
Two motion stimuli were used. The first was a 6-min video clip consisting of three
2-min scenes separated by a 1-s fade-out and fade-in. The scenes were shot with amoving video camera mounted on a hand cart using a special jig consisting of a camera
mount and a stepping motor. During image acquisition, the hand cart meanderedthrough a room as the video camera was rotated 60º/s around the axis by the stepping
motor, resulting in optical flow consisting of both a translational component due tomovement of the hand cart and a rotational component due to the roll of the video
camera (Fig. 1A and Online Resource 1). The stimulated area (screen size) subtended
50º by 36º in visual angle. We call this video clip a global motion stimulus since it
contained global rotational and translational motion signals. The second motion
stimulus, which we call a local motion stimulus, was edited from the global motion
stimulus. It was the same size as the first stimulus (50º by 36º) but consisted of 64
panels (in 8 rows and 8 columns), each an identical reduced version of the originalvideo clip (Fig. 1B and Online Resource 2). This movie editing technique was used toequate low-level visual characteristics across stimuli and reduce vection induced by theoriginal global motion stimulus (Wall & Smith 2008), thereby reducing the incidence of
VIMS because vection may be a necessary precondition (Hettinger et al. 1990). Thus,
the local motion stimulus acted as a control.
The stimuli were projected onto a screen mounted to the top of the subject's headusing a digital light processing projector (LVP-HC3800; Mitsubishi, Japan) with aspatial resolution of 1080 × 720 pixels and refresh rate of 60 Hz. Subjects were supine
and viewed the stimuli through a planar total reflection mirror at a distance of around 25cm from the eyes.
-------Figure 1 about here-----Experimental procedure
For each fMRI session, a within-subjects design was used with two stimulus
conditions (6 min each) and three intervening resting periods (5 min each). During each
resting period, subjects maintained fixation on the center of a gray backgroundprojected from the digital projector. After the first resting period, the local motion
3  124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  

stimulus (Fig.1B) was presented, followed in sequence by the second resting period,
then the global motion stimulus (Fig.1A), and finally the third resting period. The orderof stimulus presentation was not counterbalanced because a preliminary observationsuggested that VIMS-susceptible subjects could still be experiencing symptoms during
the local motion condition if preceded by the global motion condition.
VIMS severity was assessed before and after each stimulus and resting period (6times in total) using the Simulator Sickness Questionnaire (SSQ) (Kennedy et al.
1993b). The SSQ consists of 16 items rating the severity of specific symptoms (e.g., eyestrain, headache, etc.) on a 4-point scale (none, slight, moderate, and severe). Before
and after movie presentations and resting periods, the questionnaire was displayed on
the screen and the subjects responded using a 4-button response pad (HH-1x4L; Current
Design Inc., Philadelphia, PA). All stimulus presentations were controlled and subject
responses were recorded by Presentation software (Neurobehavioral Systems Inc.,
Albany, CA) on a personal computer (ThinkPad T500; Lenovo, Morrisville, NC).
Brain image acquisition
Functional MR images were acquired on a standard clinical 3-T MRI scanner (Trio
TIM; Siemens, Germany) using T2Ü-weighted gradient-echo echo planer sequence(repetition time [TR] = 2000 ms; echo time [TE] = 30 ms; flip angle [FA] = 90°; voxel
size [VS] = 3 × 3 × 4 mm; field of view [FOV] = 192 × 192 mm; 37 slices; axial
orientation parallel to the AC-PC plane; interleaved acquisition) with a 20-channel
phased-array head coil. The functional volumes were continuously scanned from 10 s
before the beginning of the first resting condition until the end of the third resting
condition. The total duration of a functional run was about 35 min. To facilitate
anatomical registration, structural images were acquired using a 3D
magnetization-prepared rapid gradient-echo T1-weighted sequence with following
parameters: TR = 1800 ms; TE = 3.03 ms; inversion time [TI] = 650 ms; FA = 9°; VS =0.8 × 0.8 × 0.8 mm; FOV = 205 × 205 mm; 37 slices; axial orientation parallel to the
AC-PC plane; interleaved acquisition. All functional volumes were anatomicallyregistered to a standard structural data set recorded in a separate session, which was also
used to reconstruct individual cortical surfaces.
Data analysis
We performed region of interest (ROI) analysis on visual areas V1, V2, V3, V3
accessory (V3A), MT+, and V7. All fMRI data were analyzed with AFNI software(Cox 1996) and in-house software (Yamamoto et al. 2008, 2012). Figure 2
schematically illustrates the procedure for ROI analysis.
ROI definition. After reconstructing each individual’s cortical surface from thestructural brain volume acquired beforehand, the locations of the retinotopic visual areas
were identified individually for each subject by fMRI using multiple retinotopic
mapping procedures. The procedures comprised two steps. In the first step, areas V1,
4  165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
187  
188  
189  
190  
191  
192  
193  
194  
195  
196  
197  
198  
199  
200  
201  
202  
203  
204  

V2, V3, V3A, and V7 were localized using a standard phase-encoding method (Engel etal. 1994; Sereno et al. 1995; De Yoe et al. 1996) with relatively small and thin stimuli(rotating 24° wedge and 2° expanding ring). At this step, area MT+ was only tentatively
defined since the MT+ boundaries are somewhat ambiguous owing to the large neuronal
receptive fields. Thus, in the second step, we confirmed that the MT+ region included
sub-regions MT and MST (medial superior temporal area) as reported in previous
studies (e.g. Huk et al. 2002; Smith et al. 2012). This was confirmed using a newphase-encoding stimulus devoid of central visual fields (T. Yamamoto et al. 2009) in
conjunction with a standard motion localizer and an ipsilateral motion localizer (Huk et
al. 2002). Further details are described in the Online Resource 3 and elsewhere (H.
Yamamoto et al. 2008, 2012).
-------Figure 2 about here-----ROI analysis. All fMRI data were first subjected to slice-timing correction and headmotion correction. The ROI analysis was then conducted in the following steps: (1)sampling fMRI time series data during each 6-min stimulus condition (180 fMRI
volumes per condition), (2) high-pass filtering with a 0.1-Hz cut-off, (3) conversion topercent signal change, (4) removal of outlier voxels showing > 10% signal change, (5)calculation of the average time series for each ROI from non-outlier voxels within the
ROI, (6) computation of the inter-hemispheric correlation coefficient between thetime-series signals of corresponding left and right ROIs using Pearson’s correlation, and(7) Fisher z transformation of the inter-hemispheric correlation to improve normality forstatistical analyses. Our use of the relatively high 0.1-Hz cut-off (instead of the morecommon 0.01-Hz cut-off) was motivated by two factors. Firstly, we were interested insensory-driven neural activity rather than spontaneous activity. Secondly, prior works
have shown that spontaneous activity and fMRI noise have 1/f-like power spectra, withmost of the power concentrated below 0.1 Hz (Nir et al. 2008; Bulmore et al. 2001). We
confirmed through additional analyses that essentially similar results were obtained with
cut-off frequencies lower than 0.1 Hz, including 0.01 Hz (Online Resource 4).
Statistical Analyses
The R package (R Development Core Team 2013) was used for all analyses. A linear
mixed-effects model analysis was performed using the function lmer() from the lme4
package in R (Bates, Maechler, Bolker, & Wolter 2013) to evaluate the association
between VIMS (as determined by subjective reports and SSQ scores) and theinter-hemispheric synchronization of cortical activity measured by fMRI. For the SSQ
total score and three SSQ subscores (nausea, oculomotor, and disorientation) measured
after local and global motion stimulus conditions, linear mixed-effects model analyses
were conducted with a within-subjects fixed effect CONDITION (local vs. global
5  205  
206  
207  
208  
209  
210  
211  
212  
213  
214  
215  
216  
217  
218  
219  
220  
221  
222  

223  

motion stimulus condition) and a between-subjects fixed effect GROUP (VIMS groupvs. the non-responsive or healthy group). The model also included by-subject random
adjustment for the intercept, allowing us to generalize the conclusions from the sample
to the population since we can treat subjects as randomly drawn from a population of all
potential subjects. To assess inter-hemispheric correlations during both local and global
motion conditions, linear mixed-effects model analyses were conducted separately foreach visual area ROI with the fixed effects CONDITION and GROUP and random
intercepts of subjects.
The significance of the CONDITION × GROUP interaction was assessed by alikelihood-ratio test that compares the models with and without the CONDITION ×
GROUP interaction term. If there was a significant CONDITION × GROUP interaction,
p values are reported for only the interaction because the significance of the main effectis uninterpretable (Zar 1999). In these cases, the simple effect of CONDITION isreported instead as yielded by the function testInteractions() from the phia package in R
(Rosario-Martinez, Fox, & R Core Team 2013). If there was no significant interaction,
the main effect of CONDITION was evaluated by a likelihood-ratio test comparing
models with and without the CONDITION term. For each main and simple effect, a
pseudo R2 was calculated in order to quantify the effect size according to (1)⎧ 2⎫R 2 = 1 − exp⎨− (Lm − L0 )⎬
⎩ n
⎭

(1)

224  
225  
226  

where 𝑛 is the number of observations, and 𝐿! and 𝐿! are the log-likelihood of themodels with and without the target variable, respectively.
For testing the relationship between the inter-hemispheric correlation and SSQ score,

227  
228  

Kendall rank correlation coefficient (τ) was calculated and the statistical significancewas evaluated by the function Kendall() from the Kendall package in R.

229  
230  
231  
232  
233  
234  
235  
236  
237  
238  

The significance level for all statistical tests was 0.05. The reported p values for theinteraction, simple effect, and main effect were Bonferroni-corrected to account for
multiple comparisons across SSQ scores, visual area ROIs, and groups.
RESULTS
Visually induced motion sickness
Eight of the 14 subjects (all men) stated that they felt motion sick while viewing the
global motion video (Fig. 1A, the global motion condition) but not while viewing the
local motion video (Fig. 1B, the local motion condition). The remaining 6 subjects
6  239  
240  
241  
242  
243  
244  
245  
246  
247  
248  
249  
250  
251  
252  
253  
254  
255  
256  
257  
258  
259  
260  
261  
262  
263  
264  
265  
266  
267  
268  
269  
270  
271  
272  
273  
274  
275  
276  
277  
278  
279  

reported no VIMS under either condition. The 8 subjects reporting VIMS wereclassified as the VIMS group and the other 6 subjects as the healthy group. The
relationships between individual subjective reports and SSQ scores are shown in Figure
3. The rank order of the individual subject total SSQ scores was consistent with thegroup classification, and the two groups were clearly separated by a cut-off total score
around 20, which corresponds to the SSQ score empirically proposed for VIMS
diagnosis (Kennedy et al. 1993b).
-------Figure 3 about here-----Figure 4A compares the total SSQ scores between the VIMS and healthy group at the
6 SSQ test points during the experimental session. In the VIMS group, the mean SSQ
total score was markedly higher after the global motion condition compared to the local
motion condition, while the mean total SSQ score of the healthy group remained nearbaseline throughout the experimental session. Similar results were obtained for the SSQ
subscores nausea, oculomotor, and disorientation (Figure 4B). These results wereanalyzed by a repeated-measures linear mixed-effects model including the fixed effects
CONDITION (local vs. global motion condition) and GROUP (VIMS vs. healthygroup), and the random effect of subject (Table 1). For the SSQ total score, there was asignificant CONDITION × GROUP interaction (χ2(1) = 8.45, p = 0.015, Bonferroni
corrected). Post hoc tests for the simple effect of CONDITION showed a significant
increase in SSQ total score between local and global motion conditions only for the
VIMS group (VIMS: χ2(1) = 28.71, p < 0.001; healthy: χ2(1) = 0.03, p > 1, Bonferroni
corrected). Similar results were obtained for the SSQ subscores (oculomotor:
CONDITION × GROUP interaction: χ2(1) = 7.87, p = 0.02; the simple effect of
CONDITION: VIMS: χ2 (1) = 29.91, p < 0.001; healthy: χ2(1) = 0.19, p > 1;
disorientation: CONDITION × GROUP interaction: χ2(1) = 10.46, p = 0.004; the simple
effect of CONDITION: VIMS: χ2(1) = 29.20, p < 0.001; healthy: χ2 (1) = 0, p > 1,
Bonferroni corrected), except nausea (CONDITION × GROUP interaction: χ2(1) = 4.91,
p = 0.107; the main effect of CONDITION: χ2(1) = 5.53, p = 0.075, Bonferroni
corrected).
-------Figure 4 about here------------Table 1 about here-----Inter-hemispheric MT+ desynchronization during visually induced motion
sickness
To test our hypothesis that left and right MT+ activities desynchronize during VIMS,
we compared the inter-hemispheric correlation of fMRI signals from MT+ and othervisual areas (V1, V2, V3, V3A, and V7) between the local and the global motion
7  280  
281  
282  
283  
284  
285  
286  
287  
288  
289  
290  
291  
292  
293  
294  
295  
296  
297  
298  
299  
300  
301  
302  
303  

conditions (Figure 5). For all 8 VIMS subjects, the inter-hemispheric temporalcorrelation of MT+ signals decreased from the local to the global motion condition (Fig.
5A) as predicted by the hypothesis, whereas the change in inter-hemispheric correlation
between conditions showed no consistent pattern for the 6 healthy subjects (Fig. 5D). A
decrease in the inter-hemispheric correlation between the local and global motion
condition was also observed in V1 for all VIMS subjects except one (Fig. 5B), but again
there was no consistent change among the healthy subjects (Fig. 5E).

304  
305  
306  
307  

For MT+, there was a significant CONDITION × GROUP interaction (χ2 (1) = 11.66, p= 0.004, Bonferroni corrected). Post hoc tests for the simple effect of CONDITION
showed that the inter-hemispheric correlation of MT+ activity decreased significantly inthe VIMS group during the global motion condition compared to the local motion

308  
309  

condition (χ2(1) = 28.71, p < 0.001, Bonferroni corrected), whereas there was no
significant change in inter-hemispheric correlation of MT+ activity for the healthy

310  
311  
312  
313  

group between conditions (χ2(1) = 3.47, p = 0.751, Bonferroni corrected). For V1, theinteraction effect was not significant (χ2(1) = 4.92, p = 0.159, Bonferroni corrected) andthe VIMS-specific decrease in inter-hemispheric correlation was non-significant (χ2(1)= 6.59, p = 0.062, Bonferroni corrected). For the other visual areas V2, V3, V3A, and

314  
315  
316  

V7, there was no significant interaction between CONDITION and GROUP (V2: χ2(1)= 1.74, p > 1; V3: χ2(1) = 2.78, p = 0.573; V3A: χ2(1) = 0.45, p > 1; V7: χ2(1) = 5.04, p= 0.149, Bonferroni corrected), and no significant change in inter-hemispheric-------Figure 5 about here-----The mean inter-hemispheric correlation coefficient for MT+ was significantly lower inthe VIMS group while viewing the global motion video compared to the local motion
video (Figure 6A). In contrast, the mean inter-hemispheric correlation coefficient did
not differ between conditions for the healthy group. The mean inter-hemisphericcorrelation coefficient for V1 was also lower in the VIMS group while viewing the
global motion video compared to the local motion video, but the difference did notreach statistical significance (Fig. 6B).
-------Figure 6 about here-----As in the analysis of SSQ scores (Table 1), a linear mixed-effects model including the
fixed effects CONDITION (local vs. global motion condition) and GROUP (VIMS vs.
healthy group), and the random effect of subject was used to compare inter-hemisphericactivity correlations for corresponding ROIs between conditions and groups (Table 2).

8  317  
318  
319  
320  
321  
322  
323  
324  
325  
326  
327  
328  
329  

correlation from the local to global motion condition (V2: χ2(1) = 5.19, p = 0.137; V3:
χ2(1) = 1.86, p > 1; V3A: χ2(1) = 0.05, p > 1; V7: χ2(1) = 0.02, p > 1, Bonferroni
corrected). The above results were replicated using the median split criterion, in which
subjects were divided into seven healthy group members and seven VIMS groupmembers based on the median SSQ total score (see Online Resource 5). In sum, theinter-hemispheric correlation of left and right MT+ activities decreased during VIMS,
whereas other visual areas did not show such inter-hemispheric asymmetry in activity
during VIMS.
-------Table 2 about here------

330  
331  

from the degree of inter-hemispheric MT+ activity desynchronization. To this end, we
determined the change in inter-hemispheric correlation and the change in SSQ total

332  
333  

score from local to global motion conditions (global - local), and then computed theKendall rank correlation between the two changes for each visual area. The rank

334  
335  
336  
337  

correlation showed a non-significant negative trend for MT+ and V1 (MT+: τ = -0.54, p= 0.057; V1: τ = -0.54, p = 0.057, Bonferroni-corrected), indicating that the greater thedecrease in inter-hemispheric correlation from the local to global motion condition(greater desynchronization), the higher the SSQ total score (Figure 7). For other areas,

338  
339  
340  
341  
342  
343  
344  
345  
346  
347  
348  
349  
350  
351  
352  
353  
354  

the rank correlation was not significant (V2: τ = -0.33, p = 0.656; V3: τ = -0.50, p =0.106; V3A: τ = -0.18, p > 1; V7: τ = -0.45, p = 0.188, Bonferroni-corrected).

Correlation analysis
Can we predict the degree of motion sickness from the inter-hemispheric correlation?
To answer this, we examined whether the severity of motion sickness can be predicted-------Figure 7 about here------DISCUSSION
We hypothesized that VIMS would differentially affect the neural responses in left
and right MT+, and tested this by acquiring fMRI signal time series from multiple
visual cortical regions, including MT+, while subjects viewed VIMS-free and
VIMS-inducing videos. The inter-hemispheric temporal correlation of MT+ activity
decreased significantly during VIMS. A decrease was also observed for V1 but theeffect was not as robust as in MT+, and there were no effects of VIMS oninter-hemispheric correlations in the other extrastriate areas examined, V2, V3, V3A,
and V7. Moreover, correlation analysis suggested the possibility of predicting the
subjective severity of motion sickness from the inter-hemispheric correlation of MT+.
9  355  
356  
357  
358  
359  
360  
361  
362  
363  
364  
365  
366  
367  
368  
369  
370  
371  
372  
373  
374  
375  
376  
377  
378  
379  
380  
381  
382  
383  
384  
385  
386  
387  
388  
389  
390  
391  
392  
393  
394  
395  

To our knowledge, this is the first report demonstrating an association between VIMS
and asymmetric activation of left and right MT+ regions, although the sample-size here
was relatively small.
Compared to model-based approaches for fMRI analysis, time series correlation
analysis of fMRI data offers an unbiased method for revealing unexpected changes incortical activity associated with various neurological conditions. In contrast to our
results, Napadow et al. (2012) identified a VIMS-related network broadly distributed
over interoceptive, limbic, and somatosensory regions, but found no association
between VIMS and visual cortex. Presumably, the effects of VIMS on the temporalcharacteristics of activity in visual cortices were more complex than assumed in thatstudy. Alternatively, inter-hemispheric correlation analysis can detect such
unpredictable effects since the analysis makes no assumptions about the temporalcharacteristics of the BOLD signal. Indeed, Hasson et al. (2009) found reduced
inter-subject correlations among multiple fMRI time series in autistic subjects comparedto healthy controls while watching the same popular movie, underscoring the power ofinter-hemispheric and inter-subject correlation analyses to detect unpredictable changes
in brain activity as may occur in neurological diseases.
All subjects in the VIMS-resistant healthy group exhibited high inter-hemisphericcorrelations in visual cortex during both the local and global motion stimuli. Similar
results were also obtained for the VIMS group in most visual cortical regions, except forMT+ during VIMS induced by global motion. High inter-hemispheric correlation is
expected from the retinotopic organization of the visual areas and the stimulus
configurations. The local motion stimulus would elicit similar responses in left and rightvisual areas because the right and left halves of the stimulus were comprised of amosaic of identical frames depicting the same local motion. Similarly, although the twosides of the global motion stimulus were not identical, they had similar local and global
motion components, so again it is not surprising that most bilateral visual areas showed
relatively high activity correlations.
Why then did MT+ not maintain high temporal correlation between hemispheres
during VIMS? It seems unlikely that the effect can be explained by side effects of
VIMS such as fatigue, reduced arousal, or disrupted attention, effects that would alter
activity in many visual areas. Moreover, desynchronization was observed selectively in
MT+, a region responsible for encoding visual motion, which is the very cause of VIMS.
It is therefore more plausible that inter-hemispheric desynchronization of MT+ activity
reflects hemispheric asymmetry within functional networks related to the incidence of
VIMS.
The relevant networks and processes are a matter of conjecture. One possibility is the
network related to optokinetic nystagmus (OKN), a principal component of the eye
movement hypothesis of MS (Ebenholtz 1992; Ebenholtz et al. 1994). As laterality of
MT+ activity with right dominance was reported during OKN (Dieterich et al. 1998),
right MT+ activity may reflect or cause aberrant behavior of the eye movement control
10  396  
397  
398  
399  
400  
401  
402  
403  
404  
405  
406  
407  
408  
409  
410  
411  
412  
413  
414  
415  
416  
417  
418  
419  
420  
421  
422  
423  
424  
425  
426  
427  
428  
429  
430  
431  
432  
433  
434  
435  

system during the global motion stimulus, leading to unusual stretching and/or tractionof extraocular muscles in both eyes. Consequently, the afferent signals generated byextraocular muscles may stimulate the area postrema and vagus nerve, resulting in thesymptoms of VIMS.
The second possibility is a network related to vection activated by sensory conflict(Reason & Brand 1975, Reason 1978; Oman 1990). The right MT+ was found to be
more active during vection than during real object motion (Kovacs et al. 2008),
suggesting that MT+ activation is involved in networks associated with the generationof vection. Although the precise mechanism remains unclear, the right dominance of
MT+ suggests distinct roles for right and left MT+ in networks integrating sensory
information to detect mismatches.
The third possibility is that desynchronization of MT+ activity may be caused byinteractions within sensory integration networks. There are at least two possible
interactions. One is with the anterior insular cortex, which is thought to play animportant role in monitoring activity in the sensory cortex that could lead to
disturbances of homeostasis (Sterzer & Kleinschmidt 2010). In the course of VIMS,
fMRI activity in right fronto-insular cortex (including the anterior insula) was correlatedwith the severity of nausea (Napadow et al. 2012), and the white matter tract connecting
the right insular cortex and right MT+ was associated with VIMS susceptibility(Napadow et al. 2013). Another possible interaction is with the parieto-insularvestibular cortex. According to sensory conflict theory (Reason & Brand 1975; Reason
1978; Oman 1990), motion sickness can arise from a conflict or mismatch between
motion signals generated by different sensory systems, including visual-vestibular
conflict, suggesting some form of interaction between the visual and vestibular cortices.
Brandt et al. (1998) proposed a reciprocal inhibitory interaction between these systems
based on observed reciprocal activation-deactivation patterns. We speculate that theright and left MT+ regions are differentially modulated by these visual-vestibularinteractions or are uniquely involved in these networks, leading to inter-hemisphericdesynchronization.

Acknowledgement
We thank Dr. Yoshimichi Ejima and Shigeko Takahashi for their helpful comments.
We also thank anonymous reviewers for their constructive comments and suggestions.
This study was supported by a Grant-in-Aid for Scientific Research on Innovative Areas“Shitsukan” (23135517, 25135720) from the Ministry of Education, Culture, Sports,
Science and Technology of Japan and Grants-in-Aid for Scientific Research (22530793)
from the Japan Society for the Promotion of Science (JSPS) to H. Yamamoto. The final
publication is available at Springer via http://dx.doi.org/10.1007/s00221-015-4312-y

11  REFERENCES
Bos JE, Bles W (1998) Modelling motion sickness and subjective vertical mismatch detailed forvertical motions. Brain Res Bull. 47(5), 537-542.
Brandt T, Bartenstein P, Janek A, Dieterich M (1998) Reciprocal inhibitory visual-vestibularinteraction. Visual motion stimulation deactivates the parieto-insular vestibular cortex. Brain
121(Pt 9): 1749-1758.
Bullmore E, Long C, Suckling J, Fadili J, Calvert G, Zelaya F, Brammer M (2001) Colorednoise and computational inference in neurophysiological (fMRI) time series analysis:
resampling methods in time and wavelet domains. Human brain mapping 12(2) 61-78.
Cheung BSK, Howard IP, Money KE (1991) Visually-induced sickness in normal and bilaterallabyrinthine-defective subjects. Aviat Space Environ Med. 62: 527-531.
Cox RW (1996) AFNI: software for analysis and visualization of functional magnetic resonanceneuroimages. Comput Biomed Res. 29: 162-173.
De Yoe EA, Carman GJ, Bandettini P, Glickman S, Wieser J, Cox R, Miller D, Neitz J (1996)
Mapping striate and extrastriate visual areas in human cerebral cortex. Proc Nat Acad Sci.
(USA) 93: 2382-2386.
Dieterich M, Bucher SF, Seelos KC, Brandt T (1998). Horizontal or vertical optokinetic
stimulation activates visual motion-sensitive, ocular motor and vestibular cortex areas withright hemispheric dominance. An fMRI study. Brain 121(8), 1479-1495.
Dieterich M, Bense S, Stephan T, Yousry TA, Brandt T (2003). fMRI signal increases anddecreases in cortical areas during small-field optokinetic stimulation and central fixation.
Exp Brain Res. 148(1), 117-127.
Ebenholtz S (1992) Motion sickness and oculomotor systems in virtual environments.
Presence: Teleoperators and Virtual Environments. 1:3, 302-305.
Ebenholtz SM, Cohen MM, Linder BJ (1994) The possible role of nystagmus in motion
sickness: a hypothesis. Aviat Space Environ Med. 65(11), 1032-1035.
Ellis SR (1991) Nature and origins of virtual environments: A bibliographical essay. Comput
Syst Eng. 2(4), 321-347.
Engel SA, Rumelhart DE, Wandell BA, Lee AT, Glover GH, Chichilnisky EJ, Shadlen MN(1994) fMRI of human visual cortex. Nature 369: 525.
Flanagan MB, May JG, Dobie TG. (2003) The role of vection, eye movements and postural
instability in the etiology of motion sickness. J Vestib Res. 14(4), 335-346.
Golding JF (2006) Motion sickness susceptibility. Auton Neurosci. 129(1), 67-76.
Griffin MJ (1990) Handbook of human vibration. Academic press.
Hasson U, Avidan G, Gelbard H (2009) Shared and idiosyncratic cortical activation patterns in
autism revealed under continuous real-life viewing conditions. Autism Res. 2: 220-231.
Hettinger LJ, Berbaum KS, Kennedy RS, Dunlap WP, Nolan MD (1990) Vection and simulator
sickness. Mil Psychol. 2(3):171-81.
Huk AC, Dougherty RF, Heeger DJ (2002) Retinotopy and functional subdivision of humanareas MT and MST. J Neurosci, 22(16), 7195-7205.
Kennedy RS, Graybiel A, McDonough RC, Beekwith FD (1968) Symptomatology under storm
conditions in the North Atlantic in control subjects and in persons with bilateral labyrinthine
defects. Acta-Oto-Laryngologica 66: 533-540.
Kennedy RS, Fowlkes J, Lilienthal M (1993a) Postural and performance changes following
exposures to flight simulators. Aviat Space Environ Med. 64:10, 912 - 920.

  Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993b) Simulator sickness questionnaire:
an enhanced method for quantifying simulator sickness. Int J Aviat Psychol. 3(3): 203-220.
Kennedy RS, Drexler J, Kennedy RC (2010) Research in visually induced motion sickness.
Appl Ergon. 41(4): 494-503.
Konen CS, Kleiser R, Seitz RJ, Bremmer, F (2005) An fMRI study of optokinetic nystagmus
and smooth-pursuit eye movements in humans. Exp Brain Res. 165(2), 203-216.
Kovacs G, Raabe1 M, Greenlee MW (2008) Neural correlates of visually induced self-motion
illusion in depth. Cereb Cortex 18 (8): 1779-1787.
Napadow V, Li A, Loggia ML, Kim J, Schalock PC, Lerner E, Tran TN, Ring J, Rosen BR,
Kaptchuk TJ, Pfab F (2012) The brain circuitry underlying the temporal evolution of nausea
in humans. Cereb Cortex. 23(4): 806-813.
Napadow V, Sheehan J, Kim J, Dassatti A, Thurler AH, Surjanhata B, Vangel M, Makris N,
Schaechter JD, Kuo B (2013) Brain white matter microstructure is associated withsusceptibility to motion-induced nausea. Neurogastroenterol Motil. 25(5):448-e303
Nir Y, Mukamel R, Dinstein I, Privman E, Harel M, Fisch L, Malach R (2008) Interhemisphericcorrelations of slow spontaneous neuronal fluctuations revealed in human sensory cortex.
Nature neuroscience 11(9) 1100-1108.
Oman CM (1990) Motion sickness: A synthesis and evaluation of the sensory conflict theory.
Can J Physiol Pharmacol. 68(2), 294-303
Reason JT, Brand JJ (1975) Motion sickness. London Academic Press.
Reason JT (1978) Motion sickness adaptation: A neural mismatch model. Journal of the Royal
Society of Medicine, 71(11), 819-829.
Sereno MI, Dale AM, Reppas JB, Kwong KK, Belliveau JW, Brady TJ, Rosen BR, Tootell RB(1995) Borders of multiple visual areas in humans revealed by functional magnetic
resonance imaging. Science 268: 889-893.
Smith AT, Wall MB, Thilo KV (2012) Vestibular inputs to human motion-sensitive visual cortex.
Cereb Cortex 22(5), 1068-77.
Sterzer P, Kleinschmidt A (2010) Anterior insula activations in perceptual paradigms: often
observed but barely understood. Brain Struct Funct. 214(5-6): 611-22.
Shupak A, Gordon CR (2006). Motion sickness: advances in pathogenesis, prediction,
prevention, and treatment. Aviat Space Environmen Med. 77(12), 1213-1223.
Wall MB, Smith AT (2008) The representation of egomotion in the human brain. Curr Biol. 18:
191-194.
Yamamoto H, Ban H, Fukunaga M, Umeda M, Tanaka C, Ejima Y (2008) Large- andsmall-scale functional organization of visual field representation in the human visual cortex.
In: Visual Cortex: New Research, edited by Portocello TA, Velloti RB. New York: Nova
Science Publisher, 195-226.
Yamamoto H, Fukunaga M, Takahashi S, Mano H, Tanaka C, Umeda M, Ejima Y (2012)
Inconsistency and uncertainty of the human visual area loci following surface-based registration:
probability and entropy maps. Hum Brain Mapp. 33: 121-129.
Yamamoto T, Yamamoto H, Mano H, Umeda M, Tanaka C, Kawano K (2009) A new fMRI method
for subdividing the human middle temporal complex into retinotopic areas. Neuroscience
Research 65, S173.

  Table 1. Summary statistics for linear mixed-effects model analysis of SSQ scores
The results of the linear mixed-effects model analysis with the fixed effects CONDITION (local vs. global motion) and GROUP (VIMS vs. healthy) for Simulator Sickness Questionnaire (SSQ) scores. If there was a significant
interaction, a post hoc test was conducted for the simple effect of CONDITION. The effect size for the simple effect is a pseudo R2, indicating the goodness of fit of the model with the CONDITION term. If there was no significant
interaction, the main effect of CONDITION was examined. Both the interaction and the main effect p values are Bonferroni-corrected for multiple comparisons among the four SSQ scales (total and three subscores), and the post
hoc simple effect p values are corrected for multiple comparisons between the two subject groups (VIMS and healthy).

CONDITION × GROUP interaction
ParameterScale

coefficientestimate ± standard error

statistics
2
p value (χ)

Main effect of CONDITION

Simple effect of CONDITION
coefficientestimate ± standard error

statistics
p value (χ2)

Effect size
R2

VIMS

Healthy

VIMS

Healthy

VIMS

Healthy

46.75 ± 11.45

1.87 ± 1.92

< .001 (28.71)> 1 (0.03)

0.47

0.07

coefficientestimate ± standard error

statistics
p value (χ2)

Effect size
2
R

N/A

N/A

N/A

24.53 ± 9.77.075 (5.53)

0.17

Total score44.88 ± 13.33.015 (8.43)

Nausea40.15 ± 16.56.107 (4.91)

Oculomotor

24.95 ± 7.68.020 (7.87)

27.48 ± 6.28

2.53 ± 2.92

< .001 (29.91)> 1 (0.19)

0.46

0.06

N/A

N/A

N/A

Disorientation

62.64 ± 17.71.004 (10.46)

62.64 ± 14.87

0.00 ± 3.28

< .001 (29.20)> 1 (0)

0.53

0

N/A

N/A

N/A

N/A

N/A

N/A

SSQ scoreTable 2. Summary statistics for linear mixed-effects model analysis of regional inter-hemispheric time series correlations
The results of the linear mixed-effects model analysis with the fixed effects CONDITION (local vs. global motion) and GROUP (VIMS vs. healthy groups) for the inter-hemispheric correlation of visual cortical areas MT+, V1,
V2, V3, V3A, and V7. Since there was a significant interaction for MT+, a post hoc test was conducted for the simple effect of CONDITION. The effect size for the simple effect is a pseudo R2, indicating the goodness of fit of
the model with the CONDITION term. By contrast, there were no significant interactions for other visual areas. The main effect of CONDITION was thus examined. Both the interaction and the main effect p values are Bonferroni-corrected for multiple comparisons between the six pairs of visual areas, and the post hoc simple effect p values are corrected for multiple comparisons between subject groups.
Simple effect of CONDITION

CONDITION × GROUP interaction
Parameter

Inter-hemisphericcorrelation

Visual area

coefficientestimate ± standard error

statistics
2
p value (χ)

coefficientestimate ± standard error

statistics
p value (χ2)

Main effect of CONDITION
Effect size
R2

VIMS

Healthy

VIMS

Healthy

VIMS

Healthy−0.20 ± 0.02

0.10 ± 0.08

< .001 (28.71).751 (3.47)

0.77

0.11

coefficientestimate ± standard error

statistics
2
p value (χ)

Effect size
2
R

N/A

N/A

N/A

MT+

−0.30 ± 0.07

.004 (11.66)

V1−0.32 ± 0.13

.159 (4.92)

N/A

N/A

N/A

−0.22 ± 0.08

.062 (6.59)

0.21

V2

−0.16 ± 0.11

> 1 (1.74)

N/A

N/A

N/A

−0.15 ± 0.06

.137 (5.19)

0.17

V3

−0.21 ± 0.12.573 (2.78)

N/A

N/A

N/A

−0.09 ± 0.07

> 1 (1.86)

0.06

V3A

−0.08 ± 0.13

> 1 (0.45)

N/A

N/A

N/A

−0.01 ± 0.06

> 1 (0.05)< 0.01V7−0.23 ± 0.09

.149 (5.04)

N/A

N/A

N/A

0.01 ± 0.06

> 1 (0.02)< 0.01abGlobal motion condition Local motion condition
Figure 1. The visual stimuli. (a) A sample frame from the video clip used in the
global motion condition. A camera mounted on a cart with a swivel axis is subjected
to both translational and rotational motion. (b) A sample frame used in the local
motion condition.

fMRI time-series

Left vs. Right

V7

V7

V3A

V3A

MT+
MT+

V1

V1
V2
V3

V2

V3

Figure 2. A schematic of the analyses used to investigate inter-hemispheric temporal activity correlations in thehuman visual cortex. Left and right panels are posterior views of the left and right inflated cortical surfaces, respectively. The six visual areas investigated (V1, V2, V3, V3A, V7, and MT+) are colored (blue: left areas, red: rightareas). For each visual area, the functional magnetic resonance imaging time series was averaged across all
non-outlier voxels within the area and Pearson correlation coefficients calculated for corresponding time series(blue and red lines in the middle panel).

SSQ total score

150

100

50

0

1

2

3

4

5

6

7

8

9 10 11 12 13 14

Subject ID
Figure 3. Individual Simulator Sickness Questionnaire (SSQ)
total scores measured after the global motion condition. Graybars indicate the scores of the subjects who self-reported
motion sickness after the experimental session (visuallyinduced motion sickness [VIMS] group) and white bars indicate
scores of subjects who did not (healthy group).

a

b
80

VIMS
Healthy

60
40
20
0

0

500

1000
1500
Time (s)2000

Averaged SSQ subscore

Averaged SSQ total score

VIMS

80

nauseaoculomotor
disorientation

Healthy60

nauseaoculomotor
disorientation

40
20
0

0

500

1000
1500
Time (s)2000

Figure 4. SSQ scores and subscores over the time course of the experimental session. The scores wereaveraged among subjects in the VIMS group (solid lines) and in the healthy group (dotted lines). The left
gray bar indicates the presentation period of the local motion stimulus and the right bar the period of the
global motion stimulus (as denoted by the image icons above the bars). (a) SSQ total score. (b) SSQ
subscores for nausea, oculomotor, and disorientation.

a

b

c
V1

Behavior
150

0.5

1.0

0.1

0.2

d

e

1.0

2.0

SSQ total score

2.0

1.0

0.1

40

0.2
Local

Global

50

f

1.0

0.5

100

0

SSQ total score

Inter-hemisphericcorrelation (Z score)
Inter-hemisphericcorrelation (Z score)

Healthy group

VIMS group

MT+

30
20
10
0

Local

GlobalLocal

GlobalFigure 5. Comparison of individual inter-hemispheric correlations and SSQ scores between the local
and the global motion conditions for the VIMS group (a-c) and the healthy group (d-f). Time series arefrom MT+ (a, d) and V1 (b, e).

a

b
MT+

Inter-hemispheric Correlation(Z score)

1.0

V1

2.0
p < 0.001

0.5

0.5

0.0

0.0
Local Global
VIMS

Local Global
Healthy

Local Global
VIMS

Local Global
Healthy

Figure 6. Average inter-hemispheric correlations between the local and global motion conditions. (a)Inter-hemispheric correlations of MT+ during the local (open bars) and the global (gray bars) conditions for
the VIMS group (left) and healthy group (right). (b) Results for area V1. Error bars indicate mean ± standard
error of the mean (SEM).

a

b
MT+

Change in SSQ total score140

τ = −0.54
p = 0.057

120
100

100
80

60

60

40

40

20

20

0

0

-0.2

-0.1

0

0.1

0.2

0.3

τ = −0.54
p = 0.057

120

80

-20
-0.3

V1

140

0.4

Change in inter-hemispheric correlation(Z score)

-20
-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

Change in inter-hemispheric correlation(Z score)

Figure 7. The relationship between inter-hemispheric correlation and SSQ score. A scatter plot of thechange in inter-hemispheric correlation vs. change in SSQ total score for each subject between the local
and global motion conditions (global - local). Each point represents the data of one subject (n = 14). Kendall’s correlation coefficient (τ) and Bonferroni-corrected p value are shown. (a) for MT+ and (b) for V1.

Experimental Brain Research, Online Resource 3
Inter-Hemispheric Desynchronization of the Human MT+ during Visually Induced
Motion Sickness
Jungo Miyazaki1, Hiroki Yamamoto2*, Yoshikatsu Ichimura1, Hiroyuki Yamashiro2, Tomokazu
Murase3, Tetsuya Yamamoto5, Masahiro Umeda4, Toshihiro Higuchi
1
Frontier Research Center, Canon Inc., Tokyo, Japan
2
Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
Departments of 3Neurosurgery and 4Medical Informatics, Meiji University of Oriental
Medicine, Kyoto, Japan
5
Graduate School of Engineering, Kyoto University, Kyoto, Japan
E-mail: yamamoto@cv.jinkan.kyoto-u.ac.jp
Tel: +81-75-753-2978; Fax: +81-75-753-6574

Localization of visual areas based on retinotopy measurements
Visual areas, V1, V2, V3, V3A, and V7
After reconstructing each individual’s cortical surface, locations of the retinotopic
visual areas were identified by fMRI and standard retinotopic mapping procedures. The
surface regions delimiting areas V1, V2, V3, V3A, and V7 were determined byphase-encoded retinotopic mapping methods (Engel et al.1994; Sereno et al. 1995;
DeYoe et al. 1996), allowing visualization of the polar angle and eccentricity
components of the retinotopic map. The retinotopic map was constructed using aphase-encoding technique where the receptive field centers were temporally codedusing polar coordinates (Engel et al. 1994). The polar angle component of the map was
measured by performing fMRI while the subject viewed a small wedge-shaped
checkered pattern (24° center angle) that rotated counterclockwise around the fixation
point, making one rotation in 60 s. The eccentricity component was measured while thesubject viewed a thin checkered annulus (2° width) that expanded from the fovea to 16°
peripherally over 50 s and then disappeared for 10 s. Each stimulus underwent color(black/white, red/green, or blue/yellow) pattern reversal (1 Hz) and was presented in six
cycles, evoking a periodic response at a given point on the retinotopy map, where thecorresponding position in the visual field was encoded in the phase of the response. The
response phase for each of the polar angles or eccentricity components was computed
using Fourier analysis and mapped onto the cortical surface. More details of the surface
reconstruction, the retinotopy mapping, and the parcellation schema are described
elsewhere (H. Yamamoto et al. 2008, 2012).
Visual area MT+
It is often difficult to localize MT+ with retinotopy measurements alone since MT+    Figure S1 Visual stimuli used to localize MT and MST separately. In all stimuli, dots were 0.25° indiameter and maintained at a 15% areal density. (A) A fan-shaped phase-encoded stimulus rotatingin the far peripheral visual field. The fan shape (24° center angle) was defined by moving dots on a
background filled with stationary dots. Dots in the fan shape moved toward and away from fixation(8°/s), alternating direction once per second. The stimulus was rotated counterclockwise around a
fixation point at 12° every 2 s (for a full rotation every 60 s) and confined in an eccentricity of
15°–16°. (B) A motion localizer stimulus. The stimulus alternated in time between moving andstationary dot fields (16° eccentricity) every 16 s and 12 times in a run. Dots moved in a way similar
to those in A. (C) An ipsilateral localizer stimulus. The stimulus was positioned 10° away from afixation point in either the left or right visual field and alternated between moving and stationary dot
fields (15° diameter) every 16 s and 12 times in a run. Dots moved toward and away from the centerof the field, alternating direction once per second.

has large receptive fields (RFs). This difficulty was alleviated by our small and thinphase-encoding stimuli described above. Nevertheless, there remained ambiguities anduncertainties about the exact localization. To remove this uncertainty, MT+ wascarefully delineated in two steps. In the first step, MT+ was tentatively localized using
the phase-encoding method described above. We noted that MT+ is presumed to include
at least two sub-regions, MT and MST, the latter of which contains neurons with verylarge receptive fields that extend into the ipsilateral visual field (Huk et al., 2002). So, inthe second step, we confirmed MT+ localization by additional measurements thatallowed us to localize MT and MST separately using a new phase encoding stimulus (T.
Yamamoto et al. 2009) in conjunction with an ipsilateral motion localizer (Huk et al.
2002). This confirmation step was performed for most of the subjects (12 of 14). The
details of this method are described below.  
In the second step, in place of a conventional rotating wedge, we used a rotating
fan-shaped stimulus localized in the far peripheral visual field (Figure S1A). This new
stimulus is advantageous for mapping the locations of large RFs such as those ofipsilateral-responsive neurons (IRNs) in MST because it does not stimulate central
visual fields covered by the RFs of any IRNs, thereby evoking a periodic response from
IRNs. This is not possible with the standard wedge that always stimulates central visual  fields. The fan shape (24° center angle) was defined by moving dots on a backgroundfilled with stationary dots (Figure S1A). The dots moved toward and away from thefixation point at a speed of 8° per second and alternated direction once per second. All
dots maintained a 15% density in area. The stimulus was rotated counterclockwise
about the fixation point at 12° every 2 s (1 rotation per minute, 6 rotations in a run) andconfined in an eccentricity of 15°–16°.
To check the validity of the new method, we also employed two kinds of localizer
experiments, one to determine motion-sensitive regions generally and the other todetermine ipsilateral-responsive motion-sensitive regions. In these experiments, subjects
alternately viewed dynamic and stationary dot stimuli (an eccentricity of 16 visualdegrees in radius; Figure S1B). In the ipsilateral stimulation, as in Huk et al. (2002), a
peripheral dot patch was presented in either the left or right visual field (Figure S1C).
The 15° diameter field of dots was alternated between moving and stationary, while
subjects maintained fixation on a colored point 10° from the nearest edge of the dot
patch. The statistical significance and phase of fMRI activation were computed andmapped by the standard Fourier method described above.  
Figure S2 and S3 show the results of these experiments for MT+ localization fromtwo representative subjects. The results obtained with the conventional wedge stimulus
are shown in Figures S2A and S3A for angular maps and Figures S2B and S3B for
eccentricity maps. These maps reveal relatively clear retinotopic organization in theposterior ascending limb of the inferior temporal sulcus (ITS). It can be seen that theeccentricity maps show gradual changes from inferior to superior in this ITS region.
However, the angular maps show ambiguities, particularly in the anterior part. Area
MT+ was thus tentatively delineated using these maps, but with emphasis on
consistency in the eccentricity representation (solid white lines in Figures S2A and
S3A).  Figures S2C and S3C show the results obtained using our new phase-encoding
stimulus. The new stimulus revealed clear contralateral hemifield retinotopic maps
within the MT+ region. Notably, there are two mirror-symmetric angular maps in theposterior and the anterior regions, respectively, which share the upper vertical meridian(UVM) of the visual field as their border. The posterior and anterior regions arepresumed to correspond to areas MT and MST, respectively. This was confirmed by thetwo types of motion localizers. While the general motion localizer activated the tworegions together (Figures S2D and S3D), the ipsilateral localizer activated the anterior
region alone (Figures S2E and S3E), as predicted from previous studies on MT and
MST (e.g. Huk et al. 2002; Smith et al. 2012).

    Figure S2 Locations of retinotopic areas MT and MST on the occipito-temporal cortex of arepresentative subject’s (S1) hemisphere and their relations to retinotopic maps and loci activated bymotion localizers. Areas surrounded by colored borders on inflated cortical surfaces are MT (blue)
and MST (yellow). (A) Angular retinotopic maps acquired with a conventional rotating wedge
stimulus. The color overlay on the cortex indicates the preferred stimulus angle at each cortical point,in accordance with the color code to the right. The more saturated the color, the higher the statisticalsignificance of retinotopic activity, as shown in the rainbow-like color bar. (B) Eccentricity
retinotopic maps acquired with a conventional expanding ring stimulus. Data are shown in the sameformat as in A. (C) Angular retinotopic maps acquired with a far peripheral rotating fan-shaped
stimulus. (D) Motion-sensitive regions. The yellow region indicates fMRI activity evoked by amotion localizer. (E) Ipsilateral-responsive regions. The yellow region indicates fMRI activity
evoked by an ipsilateral motion localizer.

    Figure S3 Locations of retinotopic areas MT and MST of another representative subject (S2) andtheir relations to retinotopic maps and loci activated by motion localizers, shown in the same format
as in Figure S2.

References
De Yoe EA, Carman GJ, Bandettini P, Glickman S, Wieser J, Cox R, Miller D, Neitz J(1996) Mapping striate and extrastriate visual areas in human cerebral cortex. Proc
Nat Acad Sci. (USA) 93: 2382-2386.

  Engel SA, Rumelhart DE, Wandell BA, Lee AT, Glover GH, Chichilnisky EJ, Shadlen
MN (1994) fMRI of human visual cortex. Nature 369: 525.
Huk AC, Dougherty RF, Heeger DJ (2002) Retinotopy and functional subdivision ofhuman areas MT and MST. The Journal of Neuroscience, 22(16), 7195-7205.
Sereno MI, Dale AM, Reppas JB, Kwong KK, Belliveau JW, Brady TJ, Rosen BR,
Tootell RB (1995) Borders of multiple visual areas in humans revealed by functional
magnetic resonance imaging. Science 268: 889-893.
Smith AT, Wall MB, Thilo KV (2012) Vestibular inputs to human motion-sensitive
visual cortex. Cerebral Cortex (New York, N.Y. : 1991), 22(5), 1068-77.
Yamamoto H, Ban H, Fukunaga M, Umeda M, Tanaka C, Ejima Y (2008) Large- andsmall-scale functional organization of visual field representation in the human visualcortex. In: Visual Cortex: New Research, edited by Portocello TA, Velloti RB. New
York: Nova Science Publisher, 195-226.
Yamamoto H, Fukunaga M, Takahashi S, Mano H, Tanaka C, Umeda M, Ejima Y(2012) Inconsistency and uncertainty of the human visual area loci following
surface-based registration: probability and entropy maps. Hum Brain Mapp 33:
121-129.
Yamamoto T, Yamamoto H, Mano H, Umeda M, Tanaka C, Kawano K (2009) A newfMRI method for subdividing the human middle temporal complex into retinotopic
areas. Neuroscience Research, 65, S173.

  Experimental Brain Research, Online Resource 4Inter-Hemispheric Desynchronization of the Human MT+ during Visually Induced
Motion Sickness
Jungo Miyazaki1, Hiroki Yamamoto2*, Yoshikatsu Ichimura1, Hiroyuki Yamashiro2, Tomokazu
Murase3, Tetsuya Yamamoto5, Masahiro Umeda4, Toshihiro Higuchi
1
Frontier Research Center, Canon Inc., Tokyo, Japan
2
Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
Departments of 3Neurosurgery and 4Medical Informatics, Meiji University of Oriental
Medicine, Kyoto, Japan
5
Graduate School of Engineering, Kyoto University, Kyoto, Japan
E-mail: yamamoto@cv.jinkan.kyoto-u.ac.jp
Tel: +81-75-753-2978; Fax: +81-75-753-6574

To confirm whether the choice of cut-off frequency can affect our results, we
performed analysis on the relationship between the cut-off frequency and theinter-hemispheric correlation of MT+, as well as a linear mixed-effects model analysis
on the correlation data obtained for high-pass filtered fMRI signals with a 0.01-Hz
cut-off. Furthermore, we conducted a simulation of the dependency of fMRI
signal-to-noise ratio (SNR) on cut-off frequency.
Taken together, these analyses suggested that our findings were not meaningfully
affected by the choice of cut-off frequency, and also suggested the possibility that theuse of the 0.1-Hz cut-off would lead to comparable or even higher SNR in our
experimental conditions. The inter-hemispheric correlation again decreased for therange of cut-off frequencies between 0.01 Hz and 0.1 Hz, and the statistical analysis
using the typical 0.01-Hz cut-off again found a decrease in inter-hemispheric correlationof MT+ for only the VIMS group; however, the statistical significance did not remain
after Bonferroni correction for multiple comparisons. To understand why this was thecase, we performed a simulation that took account of the actual stimulus movie, the MTneurons’ receptive fields, and sources of noise. The simulation showed that it would bepossible to obtain an adequate, and potentially better, SNR with a 0.1-Hz cut-off. Moredetails of these analyses are presented below.
Analysis on the relationship between cut-off frequency and decrease ininter-hemispheric correlation of MT+
As shown in Fig.S4, the inter-hemispheric correlation of MT+ was found to be3

decreased for the lower cut-off frequencies than 0.1 Hz. The magnitude of the decrease
was 0.16 (mean) ± 0.09 (SD), 0.12 ± 0.08, and 0.20 ± 0.05 for 0.	
  01 Hz, 0.032 Hz, and
0.1 Hz, respectively.

	
  Figure S4. Decrease in inter-hemispheric correlation of MT+ for the VIMS group. The horizontal
axis indicates the cut-off frequency of high-pass filtering.

Linear mixed-effects model analysis on inter-hemispheric correlations when a0.01-Hz cut-off was used
The table S1 shows the results of statistical analysis before the Bonferroni correction
for the 6 visual areas. The results were essentially similar to those obtained using the
0.1-Hz cut-off, in that the most remarkable effect of VIMS was observed for MT+. A
significant CONDITION (global/local motion) × GROUP (VIMS/healthy) interaction
was found only for MT+. However, the interaction was not significant after Bonferroni
correction (p = 0.168). As evidenced by post hoc tests, the inter-hemispheric correlationof MT+ activity decreased significantly for only the VIMS group during the global
motion condition, and for the other visual areas, such a VIMS-specific decrease ininter-hemispheric correlation was not observed. 	
  4

Table S1 Summary statistics for linear mixed-effects model analysis on inter-hemispheric correlation
when a 0.01-Hz cut-off was used
CONDITION × GROUP
interaction
Visualarea

coefficientestimate ±standard
error

statistics
p value (χ 2)

Simple effect of CONDITION
coefficientestimate ± standard error
VIMS

statistics
p value (χ 2)

Healthy

VIMS

Main effect of CONDITION
Effect size
R2

coefficientestimate ±standard
error

statistics
p value (χ 2)

Effect
size
R2

0.13

Healthy

VIMS

Healthy0.48< 0.01

0.09 ± 0.04 .050 (3.85)

MT+0.17 ± 0.07 .028 (4.83)

0.17 ± 0.03 0.01 ± 0.07

.001 (11.99) > 1 (0.03)

V10.02 ± 0.11 .840 (0.04)

0.25 ± 0.07 0.23 ± 0.05

< .001 (13.26).008(8.33)

0.47

0.50

0.24 ± 0.05

V2

0.01 ± 0.09 .930 (< 0.01)

0.13 ± 0.06 0.12 ± 0.07

.069 (4.48).172(2.95)

0.21

0.17

0.12 ± 0.05 .015 (5.95)

0.19

V3

0.07 ± 0.06 .260 (1.27)

0.08 ± 0.03 0.02 ± 0.05

.049 (5.05)> 1 (0.18)

0.25

0.01

0.06 ± 0.03 .075 (3.17)

0.11

V3A

0.07 ± 0.08.425 (0.64)

0.02 ± 0.05 0.04 ± 0.07

> 1 (0.15).932(0.53)

0.01

0.03

0.01 ± 0.04 .859 (0.03)< 0.01V7

0.05 ± 0.08 .478 (0.50)

0.07 ± 0.06 0.02 ± 0.04

.315 (2.00)> 1 (0.07)

0.08

0.01

0.05 ± 0.04 .232 (1.43)

0.05< .001(13.02)

Simulation analysis of fMRI SNR
We estimated SNR as a function of cut-off frequencies by a simulation, in which thesignal component was synthesized as the fMRI responses of motion-sensitive neurons to
the video stimulus, using a motion-energy encoding model (Nishimoto et al. 2011),
whereas noise components were synthesized as cardiac-related noise and pink (1/f)
noise, with various proportions (see the Methods below for more details). The
simulation showed that, as expected, the SNR with 0.1-Hz cut-off was not so bad. The
SNR for a 0.1-Hz cut-off was approximately 80% on average as compared to that for
0.01-Hz cut-off (Figure S5). The simulation also showed that the higher the proportionof cardiac noise, the greater the SNR (Figure S6). This may be a reason for the
somewhat pronounced effect with the 0.1-Hz cut-off.

	
  Figure S5. Relative SNR as a function of the cut-off frequency as compared to that for 0.01-Hz
cut-off. The gray region shows the interquartile range (25–75th percentile, 56% and 135%,
respectively, at 0.1Hz). The dashed vertical lines indicate the cut-off of 0.01 and 0.1 Hz.
50.37

Figure S6. The relationship between the relative SNR for 0.1-Hz cut-off and the power of the cardiac
noise relative to overall noise power. The black thick line indicates median SNR of 200,000
simulations, and the gray region indicates the 5th to 95th percentile range. Spearman’s rankcorrelation coefficient (ρ) and P-value are shown.

Methods
The simulation consisted of signal generation and noise generation. Signals weregenerated using the video stimulus processed with motion energy filters (MEFs)(Nishimoto et al. 2011). The MEF model was a bank of 3D spatiotemporal Gaborwavelet filters, with 8 spatial frequencies (0.625, 1.25, 2.5, 5, 10, 20, 40, and 80
cycles/image), 6 temporal frequencies (0.25, 0.5, 1, 2, 4, and 8 Hz), and 8 motion
directions (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). For simplicity, quadraturepair filters (0° and 90° phase) were used. The movie was submitted to one of the MEFs
and the outputs of the quadrature pair were squared, summed, and log-transformed togenerate local motion energy signals. The motion energy signal was z-transformed,
clipped at 3 (i.e. 3 sigma), and convolved with hemodynamic impulse response function(HIRF) (Boynton et al. 1996), yielding local fMRI signal time-series. The fMRI signals
from all of the filters were averaged in the frequency domain to obtain a signal power
spectrum.
Secondly, noise composed of two components—pink noise (1/f noise) and cardiac
noise—were generated. The proportions of the power of these components wererandomly generated and scaled by their sum so that the sum was 1 (100 combinations
for each iteration). The pink noise component was randomly generated so that the power
spectral density was inversely proportional to the frequency of itself. The cardiac noise
component was generated by convolving HIRF to a cardiac pulse time-course that was
6randomly sampled for a continuous 360 s from those measured in two pilot experimentsby pulse oximeter, each of which lasted for 840 s. Finally, the noise power spectrum
was computed via the weighted average of the power spectrum of each component,
which was z-transformed beforehand. This procedure was repeated 2000 times, yielding
200,000 (100 × 2000) samples.
The SNR was computed as a ratio of the power of the signal to that of the noise,
where the powers for the signal and noise were the sum of the spectra above the cut-off
frequency. In Figure S5, the relative SNR was calculated as the ratio of the SNR foreach cut-off frequency to that for 0.01 Hz. In Figure S6, the relative SNR for 0.1-Hz
cut-off is shown as a function of the relative power of cardiac noise (the ratio of thepower of cardiac noise to that of overall noise). Such conversions into the relative SNR
allow us to cancel out the differences in absolute powers of the signal and noise,enabling the direct comparison of SNRs for various cut-off frequencies.
References
Aguirre GK, Detre JA, Zarahn E, Alsop DC (2002) Experimental design and therelative sensitivity of BOLD and perfusion fMRI. Neuroimage 15(3) 488-500.
Boynton GM, Engel SA, Glover GH, Heeger DJ. (1996) Linear systems analysis offunctional magnetic resonance imaging in human V1. J. Neurosci. 16(13) 4207-4221.
Bullmore E, Long C, Suckling J, Fadili J, Calvert G, Zelaya F, Brammer M (2001)
Colored noise and computational inference in neurophysiological (fMRI) time series
analysis: resampling methods in time and wavelet domains. Human brain mapping
12(2) 61-78.
Nir Y, Mukamel R, Dinstein I, Privman E, Harel M, Fisch L, Malach R (2008)
Interhemispheric correlations of slow spontaneous neuronal fluctuations revealed inhuman sensory cortex. Nature neuroscience 11(9) 1100-1108.
Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, Gallant JL (2011)
Reconstructing visual experiences from brain activity evoked by natural movies.
Current Biology 21(19) 1641-1646.

7

Experimental Brain Research, Online Resource 5Inter-Hemispheric Desynchronization of the Human MT+ during Visually Induced
Motion Sickness
Jungo Miyazaki1, Hiroki Yamamoto2*, Yoshikatsu Ichimura1, Hiroyuki Yamashiro2, Tomokazu
Murase3, Tetsuya Yamamoto5, Masahiro Umeda4, Toshihiro Higuchi
1
Frontier Research Center, Canon Inc., Tokyo, Japan
2
Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
Departments of 3Neurosurgery and 4Medical Informatics, Meiji University of Oriental
Medicine, Kyoto, Japan
5
Graduate School of Engineering, Kyoto University, Kyoto, Japan
E-mail: yamamoto@cv.jinkan.kyoto-u.ac.jp
Tel: +81-75-753-2978; Fax: +81-75-753-6574

Results using a median-split criterion to define subject group
For more objective group division, we also employed a median-split criterion todivide subjects into VIMS and healthy groups based on the median SSQ total score. As
two subjects (S7, S8 in Figure 3) had the same score, we considered the two kinds ofseparation and applied the mixed-effect model analysis for each case, as described
below. In these new analyses, 13 of 14 subjects were placed in the same group as
defined by self-report. Both analyses again showed desynchronization of MT+.
(1) In the case of changing S7 alone to the healthy group
For MT+ and V1, there was a significant CONDITION × GROUP interaction(MT+: χ2 (1) = 9.10, p = 0.015; V1: χ2 (1) = 10.59, p = 0.006; Bonferroni corrected).
Post hoc tests for the simple effect of CONDITION showed that theinter-hemispheric correlation of MT+ and V1 activity decreased significantly in the
VIMS group during the global motion condition compared to the local motion
condition (MT+: χ2(1) = 15.00, p = 0.001; V1: χ2(1) = 33.71, p < 0.001, Bonferroni
corrected), whereas there was no significant change in inter-hemispheric correlationof MT+ and V1 activities for the healthy group between conditions (MT+: χ2(1) =1.41, p > 1; V1: χ2(1) = 0.03, p > 1, Bonferroni corrected). For the other visual areas
V2, V3, V3A and V7, the interaction effect was not significant (V2: χ2(1) = 3.76, p =0.315; V3: χ2(1) = 3.71, p = 0.325; V3A: χ2(1) = 0.37, p > 1; V7: χ2(1) = 5.19, p =0.136, Bonferroni corrected) and the main effect of CONDITION was
non-significant (V2: χ2(1) = 5.19, p = 0.137; V3: χ2(1) = 1.86, p > 1; V3A: χ2(1) =0.05, p > 1; V7: χ2(1) = 0.02, p > 1, Bonferroni corrected). In sum, the results weresimilar to those reported in the manuscript except for V1.
(2) In the case of changing S8 alone to healthy group  For MT+, there was a significant CONDITION × GROUP interaction (χ2 (1) =9.15, p < 0.015, Bonferroni corrected). Post hoc tests for the simple effect of
CONDITION showed that the inter-hemispheric correlation of MT+ activity
decreased significantly in the VIMS group during the global motion condition
compared to the local motion condition (χ2(1) = 15.10, p = 0.001, Bonferroni
corrected), whereas there was no significant change in inter-hemispheric correlationof MT+ activity for the healthy group between conditions (χ2(1) = 1.43, p > 1,
Bonferroni corrected). For the other visual areas, the interaction effect was not
significant (V1: χ2(1) = 5.43, p = 0.119; V2: χ2(1) = 1.69, p > 1; V3: χ2(1) = 3.65, p= 0.337; V3A: χ2(1) = 0.38, p > 1; V7: χ2(1) = 4.36, p = 0.221, Bonferroni corrected)
and the main effect of CONDITION was non-significant (V1: χ2(1) = 6.59, p =0.062; V2: χ2(1) = 5.19, p = 0.137; V3: χ2(1) = 1.86, p > 1; V3A: χ2(1) = 0.05, p > 1;
V7: χ2(1) = 0.02, p > 1, Bonferroni corrected). In sum, the results were similar to
those reported in the manuscript.