# Creating Collaborative and immersive Modular Audio Installations with WAM Jam Party

Michel Buffa, Marco Winckler, Quentin Escobar, Samuel Demont, Ayoub Hofr, Adam Mir-Sadjadi

## ► To cite this version:

Michel Buffa, Marco Winckler, Quentin Escobar, Samuel Demont, Ayoub Hofr, et al. Creating Collaborative and immersive Modular Audio Installations with WAM Jam Party. IS 2025 - IEEE International Symposium on the Internet of Sounds, Oct 2025, L'aquila, Italy. . [hal-05388549](https://hal HAL Id: hal-05388549)

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Creating Collaborative and immersive
Modular Audio Installations
with WAM Jam Party

Michel Buffa, Marco Winckler
Université Côte d'Azur,
I3S, CNRS, INRIA
Sophia-Antipolis, France
firstname.lastname@univ-cotedazur.fr

Quentin Escobar, Samuel Demont,
Ayoub Hofr, Adam Mir-Sadjadi
Université Côte d'Azur,
I3S, CNRS,
Sophia-Antipolis, France
firstname.lastname@etu-univ-cotedazur.fr

***Abstract***— WAM Jam Party is a collaborative real-time WebXR-based platform that allows users to create interactive 3D musical installations using WAM (Web Audio Modules) plugins. Building on previous work presented in the early phases of the project, this article presents substantial improvements focused on the affordance, usability, and expressive potential of the system, rather than its fundamental synchronization mechanisms, which remain unchanged and have been covered in previous publications. A 3D GUI editor has been introduced, enabling users to design and add interactive 3D graphical interfaces to existing 2D WAM plugins without the need to access or modify their source code. These 3D graphical interfaces offer custom representations—such as a synthesizer visualized as a 3D physical keyboard with knobs, sliders, textures, and labels, or a physics-modeled guitar shaped like a real guitar body and neck. In addition, a new “3D Plugin Shop” interface replaces traditional user interface menus, providing a more intuitive and immersive workflow for identifying, selecting and assembling audio components within the 3D scene. The underlying architecture has also been refactored to support script integration and dynamic code injection into plugin graphical interfaces, enabling complex behaviors such as real-time visual modulation of parameters using 3D orbiter animations or sequencer views with piano roll interaction. These improvements have been evaluated through structured user testing sessions involving developers, musicians, sound designers, and XR creators. Results indicate increased engagement, improved affordance of the provided plugins, faster learning curves, and a more exploratory approach to building audio graphs in immersive environments. This article provides insights into the design of collaborative audio applications in spatial contexts and offers design guidelines for web-based XR audio and music tools.

***Keywords***— Musical Metaverse, WebXR, WebAudio, WebMidi, Web Audio Plugins, Web standards, Web Audio Modules, CRDT.

I. WAM JAM PARTY

The convergence of Web Audio, immersive 3D environments, and real-time collaboration opens up new paradigms for musical creation and interaction. As browser-based technologies mature, most notably the Web Audio API, WebXR, and emerging standards such as Web Audio Modules (WAM, web audio plugins for the web) [5, 6, 7, 8] we are witnessing a fundamental shift in how musical experiences can be authored, shared, and performed on the web. In this context, the Musical Metaverse is not merely a speculative vision, but a tangible, evolving ecosystem in which music technologies are spatialized, embodied, and collaborative by design [1].

This paper introduces WAM Jam Party, a collaborative WebXR platform that allows users to create and interact with 3D musical installations built using WAM plugins. These installations resemble setups typically found in a music studio, consisting of MIDI note generators, virtual instruments, and audio effects connected together, each represented by a WAM plugin. To make the sound audible within the virtual environment, the audio graph ends with spatialized virtual speakers that serve as the system’s output.

This work aligns strongly with the foundational themes of the Internet of Sounds conference, specifically, networked audio systems, multi-user collaboration, web-based infrastructures, and interactive audio in distributed environments [2].

One of the distinctive aspects of WAM Jam Party is that *everything runs entirely within the web browser of the VR headset*, including full WebAssembly recreations of instruments like the Prophet V and Oberheim OB-Xd synthesizers, complex audio effects, and MIDI sequencers, all processed locally on the immersive device. Few WebXR experiments exist, like “Musical Metaverse Playgrounds” by Boem and al. [4]. In WAM Jam Party, *Multi-user collaboration is also seamless*, as participants can join simply by entering the WAM Jam Party URL in their browser or scanning a QR code with any XR-capable device.

Earlier phases of the WAM Jam Party project demonstrated the feasibility of these distributed audio graphs and their manipulation and synchronization across WebXR clients [3]. However, those early efforts mainly focused on building the core architecture needed for a stable multi-user system. The 3D interfaces were basic and automatically generated from plugin metadata, which often made it hard to tell plugins apart or interact with them in a meaningful way.
<figure><img src="image_2.png" /><figcaption>Figure 1: A 3D Graphic Editor for designing 3D GUIs for existing WAM plugins, without modifying their source code.</figcaption></figure>

This new version of WAM Jam Party introduces a 3D graphical editor that brings the focus to expressive interaction and immersive design, allowing users to create fully customized, more recognizable, plugin interfaces with unique 3D visuals, animations, and physics-driven behaviors.

Additionally, the system's architecture now supports injecting custom code directly into plugin interfaces, enabling the creation of rich, interactive 3D controls, such as animated modulators or sequencers, that significantly enhance visual feedback and interactivity around existing WAM plugins. This custom code can also interact with a 3D physics engine (such as Havok, used in many AAA video games); for example, we implemented a virtual drum kit as a MIDI note generator, where users strike drum elements with virtual sticks and see realistic physical responses on impact. This drum kit marks an initial step toward embedding more embodied instruments and interactive user interfaces into the platform.

Complementing these features, a new 3D Plugin Shop provides a spatial interface where plugins are displayed as 3D objects on virtual shelves, offering a more intuitive and engaging alternative to the floating text-based menus used in earlier versions.

<figure><img src="image_3.png" /><figcaption>Figure 2: A virtual shop with all available WAM 3D models displayed. Once picked, the WAM can be positioned in space.</figcaption></figure>

These features can be seen in action in the accompanying videos<fnref n="1" />.

## II. DEMO PROPOSAL

The application is open source<fnref n="2" /> and available online<fnref n="3" />. During the demo, multiple participants equipped with Meta Quest 3 headsets will join a shared session directly through the

browser, alongside remote users connected from our lab. Together, they will collaboratively build musical installations by assembling 3D Web Audio plugins, including note generators, virtual instruments, and audio effects, connected to spatialized audio outputs with real-time binaural rendering. Participants can modify each other's setups in real time, showcasing the system's interactive and social capabilities. The demo highlights that complex, distributed audio experiences, featuring advanced instruments like the Prophet V and Oberheim OB-Xd emulations, high quality effects, and modular graphs, can run smoothly over typical Wi-Fi or 5G networks, entirely within a web browser.

<figure><img src="image_4.png" /><figcaption>Figure 3: A typical music installation. On the left, in green, is another participant's avatar.</figcaption></figure>

## ACKNOWLEDGMENTS

This work has been supported by the French government, through the France 2030 investment plan managed by the Agence Nationale de la Recherche, as part of the "UCA DS4H" project, reference ANR-17-EURE-0004

## REFERENCES

[1] L. Turchet. "Musical Metaverse: vision, opportunities, and challenges." Personal and Ubiquitous Computing 27: pp 811-1827. 2023.

[2] L. Turchet, M. Lagrange, C. Rottondi, G. Fazekas, N. Peters, J. Østergaard, F. Font, T. Bäckström, and C. Fischione. 2023. The internet of sounds: Convergent trends, insights, and future directions. IEEE Internet of Things Journal, 10(13), pp.11264-11292.

[3] M. Buffa, A. Hofr, and D. Girard. Using Web Audio Modules for Immersive Audio Collaboration in the Musical Metaverse. In IS2 2024 - IEEE International Symposium on the Internet of Sounds 2024, Erlangen, Germany, Sept. 2024.

[4] A. Boem, and L. Turchet. "Musical Metaverse Playgrounds: exploring the design of shared virtual sonic experiences on web browsers.", 4th International Symposium on the Internet of Sounds. IEEE, 2023.

[5] J. Kleimola and O. Larkin. Web Audio Modules. In Proceedings of the Sound and Music Computing Conference 2015.

[6] M. Buffa, J. Lebrun, J. Kleimola, O. Larkin, and S. Letz. 2018. Towards an open Web Audio plugin standard. In WWW2018 – TheWebConf 2018 : The Web Conference, 27th International World Wide Web Conference. Lyon, France. 2018.

[7] M. Buffa, S. Ren, O. Campbell, J. Kleimola, O. Larkin, and T. Burns. 2022. Web Audio Modules 2.0 : An Open Web Audio Plugin Standard. In WWW '22 : The ACM Web Conference. 2022. ACM, France, pp 364-369. 2022 https://doi.org/10.1145/3487553.3524225

[8] M. Buffa, S. Ren, T. Burns, A. Vidal-Mazuy & S. Letz. Evolution of the Web Audio Modules Ecosystem. In proceedings of the Web Audio Conference 2024. https://zenodo.org/doi/10.5281/zenodo.1082564

<footnote n="1">¹ 3D editor: https://www.youtube.com/watch?v=4JOE3F579kI, VR session with multiple participants: https://www.youtube.com/watch?v=zZdZef6XB_I</footnote>

<footnote n="2">² https://github.com/doriangirard9/musical-multiverse-vr</footnote>

<footnote n="3">³ To test this application, you need to access it with the browser of a VR headset or using a browser extension for debugging WebXR applications. URL: https://wamjamparty.i3s.univ-cotedazur.fr/</footnote>
<figure><img src="image_5.png" /></figure>