Experiences with Matlab and VRML in Functional Neuroimaging Visualizations

IntroductionNeuroinformatics is the task of organizing, analyzing and presenting the knowledge of neuroscience. In the part of neuroin-formatics that is associated with functionalneuroimaging (“brain mapping”)visualization plays an important role. F unctionalneuroimaging utilizes brain scanners to examine the inner workings of the human brain, usually by mapping the relationshipbetween cognitive components and specialized brain regions. fMRI and PET brain scanners can produce sequential volu-metric scalar data. The functional neuroimaging study sometimes incorporate a complicated design making the “time axis”multi-dimensional. This data is seldom viewed directly, but instead it is put through rigorous processing, the result beingone or more scalar volumes. Center of mass points of “hot spot” areas are tabelized and included in scientiﬁc publication sas so-called “Talairach-coordinates” [1]. A database exists with these coordinates: The BrainMap database [2]. The typicalvisualization task is to display the scalar result volumes sometimes in comparison with other volumes and almost alwayswith some kind of reference, e.g., anatomical images or “Talairach” grid. Our experience is with VRML visualizations andsome minor experiences with Matlab for 3D visualization. We aim at a GUI Matlab program able to display several volumestogether and integrating BrainMap data, rendered both in Matlab and in VRML, with possible extended display on virtualreality devices. Both original data, program and “ﬁnished” visualizations should be distributable over the Internet.VRMLWe have used VRML for visualization of functional neuroimagingresults. Priorto Matlab 5.3 we used ourownprogram polyrforisosurfacegeneration. How-ever, a simple isosurface program could not sufﬁce for the in teractive visual-izations we would like. Small controllers can be built, but as VRML, e.g.,lacks keyboard interface complex interaction is limited, and CGI-script andjava/javascript did not provide a consistent interface. Our conclusion is thatVRML is not a good scientiﬁc visualization development environment, but itshould serve more as a kind of Postscript for 3D interactive visualizations. Vir-tual reality headset displays with VRML could be obtained, and the VRML-ﬁledid not have to be “extended” in any way for it to be displayed w ithin the vir-tual reality setup. We had access to WorldToolKit, but as the package did not fully integrate VRML and it is certainly notwidespread in the functional neuroimaging community we did not pursue it further.MatlabMatlab is widespread in the functional neuroimaging community partly due to the program SPM [3]. We have had someexperiences with building graphical user interfaces and simple 2D visualizations in connection with our lyngby toolbox [4]and found Matlab to be a quite convenient environment for this. As BrainMap data is quite heterogenous the visualizationenvironment calls for small-scale database-like capabilities, which has to some extent been honored with Matlab 5. Fromversion 5.3 Matlab includes some proper 3D visualization functions, e.g., isosurface generation, and data pipes easily toVRML: The ﬁgure shows a screenshot from a VRML-browser with a ﬁle constructed by a non-GUI Matlab program. Datafrom the BrainMap database is displayed together with some “context”, e.g., a part of the cortex. We have not tried tointegrate any virtual reality setup with matlab and have not examined if matlab is suited for realtime visualization. (Displayupdate in matlab is done when the mouse-button is released, contrary to typical VRML-browsers which continuously renderthe visualization)User experiencesOur user experience has been sparse: The visualizations have been appreciated for their visual appearance, but, though thetypical functionalneuroimagingresearcherwill be preparedto write small Matlab scripts, she is not likely to edit VRML ﬁles,e.g, to include the small library of VRML components we have made. A GUI interface is necessary (with what humorouslyhas been called “doctor’s buttons”) before it will become us eful for the typical functional neuroimaging researcher, and wewill pursue that further with a combination of Matlab and VRML.References1. Talairach, J., Tournoux, P., Co-planar Stereotaxic Atlas of the Human Brain, Thieme Medical Publisher Inc, 1988.2. Fox, P. T. , Lancaster, J. L. , Science, November 1994, 266(5187):994–996.3. Friston, K. J., et al., Human Brain Mapping, 1995, 2:189–210.4. Hansen, L. K. , et al., B. R. Rosen, R. J. Seitz, J. Volkmann,eds., NeuroImage, vol. 9, Academic Press, 1999, S241.