# UnityMol-Lite: Cross-platform Collaboration and Interactive Methods for Molecular Modeling in XR

Mariano Spivak, Bastien Vincke, Hubert Santuz, Jérôme Hénin, Antoine Taly, Marc Baaden, Nicolas Férey

► To cite this version:

Mariano Spivak, Bastien Vincke, Hubert Santuz, Jérôme Hénin, Antoine Taly, et al.. UnityMol-Lite: Cross-platform Collaboration and Interactive Methods for Molecular Modeling in XR. EuroXR 2024: Proceedings of the 21st EuroXR International Conference, Patrick Bourdot; Mónica González; Jerome Perret; Gabriel Zachmann; Arcadio Reyes-Lecuona, Nov 2024, Athens, Greece. hal-05163459

HAL Id: hal-05163459

https://hal.science/hal-05163459v1

Submitted on 16 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# UnityMol-Lite: Cross-platform Collaboration and Interactive Methods for Molecular Modeling in XR

Mariano Spivak¹, Bastien Vincke², Hubert Santuz³, Jérôme Henin³, Antoine Taly³, Marc Baaden³, and Nicolas Ferey¹

¹Laboratoire interdisciplinaire des sciences du numérique, Université Paris-Saclay, Paris, France

²Laboratoire SATIE, CNRS, Paris, France

³Laboratoire de Biochimie Théorique, CNRS, Paris, France

Corresponding author: mariano.spivak@universite-paris-saclay.fr

**Keywords:** Extended Reality, Tangible Interface, Molecular Dynamics

## Introduction

Molecular modeling consists of a battery of theoretical and computational methods to model the behavior of molecules. With the growth of computing power, molecular modeling has become a cornerstone of material, pharmaceutical and biomedical research. Among the computational methods, molecular dynamics simulations have shown tremendous predicting power and have become a perfect complement to experimental methods. Interactive Molecular Dynamics (IMD) is an advanced simulation technique that provides real-time interaction with the molecules, through picking and pulling of individual pieces (e.g. atoms, residues). However, this interface mode makes the technique slow and imprecise. As such, IMD has become a great tool for educational and dissemination purposes, but not for advanced research (Lanzerac, Férey et al., 2022).

On the other hand, the usage of extended reality (XR) technologies in molecular modeling have shown great promise, because it provides a framework for natural 3D-visualization and intuitive interaction. (Kuták et al., 2023) Examples of interactive visualization and IMD in XR include (Deeks et al., 2020) and (Lwindow et al., 2019). Finally, XR collaboration frameworks improve the exchange of ideas and discussion, so ever present in research.

In this work, we use state-of-the-art technologies to create a collaborative experience of an interactive molecular dynamics simulation (Figure D. 24). We propose an efficient interaction technique based on the concept of a tangible interface, which could greatly improve IMD capabilities. Finally, we present a cross-platform solution improving ease of use and dissemination.
UnityMol-Lite

UnityMol-Lite is an optimized version of the molecular visualization application UnityMol, (Laureanti et al., 2020) that was built to support new XR technologies and hardware. In particular, the application offers support for both PC-based and standalone HMDs, with a focus on cross-platform functionality, collaborative sessions and advanced interactive methods. In order to target standalone HMDs (e.g. Meta Quest 2 and 3) and their new technologies (video see through and hand tracking), a complete rework of the XR/interaction components (using OpenXR, XR Interaction Toolkit) was done, compared to the original UnityMol application. In contrast, many of the advanced visualization features of UnityMol, e.g. raytracing, were removed to support compatibility and performance.

<figure><img src="image_2.png" /><figcaption>Figure D. 24. Snapshot of an IMD experience on UnityMol-Lite (left). The user can interact with a virtual twin of a small molecule (on the front in black, red and white), defining a target conformation for the running simulation. Multiplayer session showing player avatar and tangible interface (right)</figcaption></figure>

In addition to the traditional banner user interface (UI), (Figure D. 25, center) we designed a new XR UI with accessibility at its core, providing an intuitive and immersive experience for users. This UI is integrated directly into the XR environment, by being anchored to the controller (Figure 2, left) or hand (Figure 2, right), allowing for natural interactions. It can be toggled and it ensures that frequent controls are always within reach, reducing the need for users to break their immersion and possible sight issues. This design allows users of all skill levels, including those new to XR, to engage with the interface in a more organic and user-friendly manner.

Collaborative sessions are supported through Photon Unity Networking. Avatars are used to display players' positions. The UI allows for alternate control of the scene elements.

The interactive molecular dynamics (IMD) method is achieved by the MDDriver library (Lanzerac, Laurent et al., 2022). Next section presents the complementary development done on the MDDriver library to support the new interface for IMD.
## New Interface for Interactive Molecular Dynamics

The novel interface method is inspired by the concept of the tangible interface, whereby a physical object is used to model and interact with its virtual twin. This is particularly useful in molecular modeling because many drug discovery studies involve a molecular system characterized by a small component (e.g. ligand) embedded in a large one (e.g. protein).

<figure><img src="image_3.png" /><figcaption>Figure D. 25. Composite of the user interfaces (UI) for UnityMol-Lite. A traditional banner UI, center, provides full functionality and information of the system. The new XR UI allows easy access to frequently used controls, and it is anchored to the controller (left) or the hands (right)</figcaption></figure>

Therefore, in addition to the traditional interface method of pushing and pulling the individual components, we define a virtual "tangible" interface of the small component in the scene, as a way to interact with the simulation.

Figure D. 24 (left) shows a snapshot of the IMD experience where the user manipulates the virtual interface, by changing the position, orientation and local conformation (e.g. angles, dihedrals) in real time. MDDriver compares the virtual interface coordinates with the simulation twin, and utilizes the collective variables library (COLVARS, Fiorin et al., 2013) to compute the forces required by the running simulation. Thus, the virtual interface is used as the target for the running molecular dynamics simulation, resulting in a smooth and simple experience. This combination of COLVARS and MDDriver can also be used to interact with larger components: by defining appropriate collective variables the user can impose large conformational changes to a protein, akin to those obtained using other biasing methodologies (e.g. Steered MD, Targeted MD).

Finally, this development sets the algorithmic foundations for the usage of real physical objects as tangible interfaces for interactive molecular dynamics. Next section discusses the current progress on supporting a tangible molecular interface.

## Support for Tangible Molecular Interface

Our group is currently developing a physical, modular and articulated interface to use in molecular modeling. (Vincke et al., 2020) The latest prototype features sensors to monitor adding, deleting and
connecting objects (atoms). In addition, the interface tracks rotation along connections, e.g. defining bond angles and dihedrals of the molecular model.

UnityMol-Lite supports these features, making the physical interface work as a molecular builder. In addition, the physical model can be used together with the interactive molecular dynamics method to define the target conformations. Figure D. 24 (right) shows a snapshot of the combined user experience with the physical interface and the AR mode (video see through). Support for tracking of absolute position and orientation will be revisited on the next prototype of the physical interface.

## Conclusion and Outlook

A novel interface method for interactive molecular dynamics (IMD) based on the tangible interface concept was developed. To support the use of a physical model in extended reality, we redesigned the molecular visualization application UnityMol to support standalone HMD and video see through. UnityMol-Lite is optimized for accessible android devices (e.g. Meta Quest 2 and 3) and cross-platform collaboration using Photon Unity Networking. Next, we plan to integrate our work with tutorial and demo examples from Molplay (Baaden 2024) and distribute the application through our public repository. In addition, we will continue adding support for the tangible interface model.

## Acknowledgements

This project has received funding from the Agence Nationale de la Recherche (ANR) under the grant ANR-21-CE45-0014.

## References

* Baaden, M. (2024). MolPlay: Democratizing Interactive Molecular Simulations and Analyses with a Portable, Turnkey Platform. The Journal of Physical Chemistry B. https://doi.org/10.1021/acs.jpcb.4c04712

* Deeks, H. M., Walters, R. K., Hare, S. R., O'Connor, M. B., Mulholland, A. J., & Glowacki, D. R. (2020). Interactive molecular dynamics in virtual reality for accurate flexible protein-ligand docking. Plos one, 15(3), e0228461. https://doi.org/10.1371/journal.pone.0228461

* Fiorin, G., Klein, M. L., & Hénin, J. (2013). Using collective variables to drive molecular dynamics simulations. Molecular Physics 111(22-23), 3345-3362. https://doi.org/10.1080/00268976.2013.813594

* Kuťák, D., Vázquez, P.-P., Isenberg, T., Krone, M., Baaden, M., Byška, J., Kozlíková, B. & Miao, H. (2023). State of the Art of Molecular Visualization in Immersive Virtual Environments. Computer Graphics Forum 42, e14738. https://doi.org/10.1111/cgf.14738

* Lanrezac, A., Férey, N. & Baaden, M. (2022). Wielding the power of interactive molecular simulations. WIREs Comput Mol Sci. 12, e1594. https://doi.org/10.1002/wcms.1594

* Lanrezac, A., Laurent, B., Santuz, H., Férey, N. & Baaden, M. (2022). Fast and Interactive Positioning of Proteins within Membranes. Algorithms 15, 415. https://doi.org/10.3390/a15110415

* Laureanti, J., Brandi, J., Offor, E., Engel, D., Rallo, R., Ginovska, B., ... & Baker, N. A. (2020). Visualizing biomolecular electrostatics in virtual reality with UnityMol-APBS. Protein Science, 29(1), 237-246. https://doi.org/10.1002/pro.3773

* L window, N., Baum, D., Leborgne, M. & Hege, H.-C. 2019. Interactive Visualization of RNA and DNA Structures. IEEE Transactions on Visualization and Computer Graphics 25(1), 967-976. https://doi.org/10.1109/TVCG.2018.2864507

* Vincke, B., Ghaoui, M. A., Férey, N. & Martinez, X. (2020). Physical, Modular and Articulated Interface for Interactive Molecular Manipulation, Sensors 20(18), 5415. https://doi.org/10.3390/s20185415