The evolving technology of computer auto-fabrication ("3-D printing") now makes it possible to produce physical models for complex biological molecules and assemblies. We report on an application that demonstrates the use of auto-fabricated tangible models and augmented reality for research and education in molecular biology, and for enhancing the scientific environment for collaboration and exploration. We have adapted an augmented reality system to allows virtual 3-D representations (generated by the Python Molecular Viewer) to be overlaid onto a tangible molecular model. Users can easily change the overlaid information, switching between different representations of the molecule, displays of molecular properties such as electrostatics, or dynamic information. The physical model provides a powerful, intuitive interface for manipulating the computer models, streamlining the interface between human intent, the physical model, and the computational activity.  With the prevalence of structural and genomic data, molecular biology has become a human-guided, computer-assisted endeavor. The computer assists the essential human function in two ways: in exploration of scientific data, searching for and testing scientific hypotheses; and in collaboration between two or more scientists, to share knowledge and expertise. As databases grow, as our structure and process models become more complex, and as software methods become more diverse, access and manipulation of digital information is increasingly a critical issue for research in molecular biology. Currently, exploratory research in structural molecular biology is dominated by 3-D representations via computer graphics. Collaboration, both remote and local, is aided by shared viewing of these interactive visual representations of molecular data. Yet, recent advances in the field of human-computer interfaces have not been applied to the technology used by molecular biologists --most work in biomolecular structure and genomics is performed in front of a workstation using a mouse and keyboard as input devices. The tactile and kinesthetic senses provide key perceptual cues to our ability to understand 3-D form and to perform physical manipulations, but are currently under-utilized in molecular biology. Early structure research relied heavily on physical models: Pauling used his newly-invented spacefilling models to predict the basic folding units of protein structures [1] and Watson and Crick used brass-wire molecular models to help them determine the structure of DNA [2], which reconciled decades of genetic data. These researchers "thought with their hands" to produce important scientific results. Current research in molecular biology now focuses on larger assemblies and more complex interactions, for which traditional atomic models are inadequate. Merging physical and virtual objects into an "augmented reality" (AR) environment [3] enables new modes of interaction through the manipulation of tangible models and the complex information they represent [4] . The evolving technology of computer auto-fabrication ("3D printing") now makes it possible to produce physical models for complex molecular assemblies. In this paper we report on an application that demonstrates the use of auto-fabricated tangible models and AR for research in molecular biology to enhance the scientific environment for collaboration and exploration. The physical models are integrated into an augmented reality environment to streamline the interface between human intent, the physical model, and the computational activity. We have developed an AR system that allows virtual 3-D representations generated by our Python Molecular Viewer (PMV) [5] to be overlaid on an auto-fabricated model of the molecule. The precise registration of the virtual objects with the real world is done using the ARToolKit library developed at the University of Washington [6] . While using the system, users can easily change the representation shown, and, for example, access information about molecular properties of the molecules. We will first describe how we create 3D tangible models of a molecular structure from a known atomic structure, then explain the integration of ARToolKit in our Python framework, and finally present some examples. We use PMV [5] both to create our virtual objects and to design our tangible models, simplifying the integration of the models with the virtual environment. PMV is a modular software framework for designing and specifying a wide range of molecular models, including molecular surfaces, extruded volumes, backbone ribbons, and atomic ball-and-stick representations. It allows the design of models at different levels of abstraction for different needs: using representations that focus on molecular shape when large systems and interactions are presented, and incorporating atomic details when needed to look at function at the atomic level. October 10-15,
In the field of molecular biology, Gillet et al. REF augmented physical molecular models with a virtual overlay: The user could change the representation of the digital layer or create new combinations of molecules.
Augmented Reality with Tangible Auto-Fabricated Models for Molecular Biology Applications
IEEE Visualization 2004
IEEE Visualization 2004
Computer Science