  Multimodal imaging has transformed neuroscience research. While it presents
unprecedented opportunities, it also imposes serious challenges. Particularly,
it is difficult to combine the merits of interpretability attributed to a
simple association model and flexibility achieved by a highly adaptive
nonlinear model. In this article, we propose an orthogonal statistical
inferential framework, built upon the Neyman orthogonality and a form of
decomposition orthogonality, for multimodal data analysis. We target the
setting that naturally arises in almost all multimodal studies, where there is
a primary modality of interest, plus additional auxiliary modalities. We
successfully establish the root-$N$-consistency and asymptotic normality of the
estimated primary parameter, the semi-parametric estimation efficiency, and the
asymptotic honesty of the confidence interval of the predicted primary modality
effect. Our proposal enjoys, to a good extent, both model interpretability and
model flexibility. It is also considerably different from the existing
statistical methods for multimodal data integration, as well as the
orthogonality-based methods for high-dimensional inferences. We demonstrate the
efficacy of our method through both simulations and an application to a
multimodal neuroimaging study of Alzheimer's disease.
