DJaytopia: a hybrid intelligent DJ co-remixing system
Authors: Wu Yue , Qiu Anran , Ruan Liuxuan , Li Xuejie , Huang Jinhao , Jia Wang Stephen 
Published: July 2023

Nowadays, musical mixing platforms are featured with programmed
interventions and digitized information visualization to support DJ's
performance (Montano 2010), however, the visualization is always obscure to
the average music consumers (Beamish, Maclean, and Fels 2004). Being a
well-performed DJ requires the level of expertise and experience that most
average music consumers lack (Cliff 2000), as every audience has a
completely different taste in music (Schäfer and Sedlmeier 2010). This study
aims at developing an AI / ML-based system to lower the bar for novice DJs
and even average music consumers to create personalized music
remixes.Generally, music can be intelligently composed by analyzing harmonic
and melodic features to generate genre-specific compositional elements or to
alter the compositional structure of a song (Tan and Li 2021). Despite the
technical breakthroughs that have been made, listeners have reacted
negatively to this music due to the lack of user data to back it up and the
neglect of the user's perception of the piece (Tigre Moura and Maw 2021). In
a conventional scenario, DJs can express their attitudes towards music
preferences by listening to the music directly, which requires a well
understanding of the audience's mind. Following the recent launch and
explosion of ChatGPT, which has evidenced that an intelligent system could
help users innovate by solving their problems in textual form through
conversational interactions (Dis et al. 2023; Dwivedi et al. 2023); also
collecting the users' feedback through conversations, observing user
reactions, and inviting user reviews. Such AI-enabled systems are able to
learn about the user's preferred music style and various DJ mixing
techniques. This study adopts a typical human-in-the-loop (HITL) approach to
develop a crowd-learning music mixing system implementing AI and Virtual
Reality technologies. The proposed HITL-based co-music arrangement system
should be able to collect musical data and techniques; a VR environment is
built to provide users with a platform to record user-created music and
corresponding applied methods as well as audience ratings worldwide. After
processing the data, users can try out a compilation of songs assisted by a
robotic arm. With the help of the robotic arm, it will be easier and faster
for users to create collections with a personal touch and more specific
techniques. The essential functions include: a) Providing users with an
immersive environment to learn the basic operations of the DJ console. b)
Collecting the user's preferences for compilation techniques and the content
of different DJ's compositions for use through an “immersive online
multiplayer music compilation platform” to generate a personalized library
of methods to help the user compile songs; c) Assisting the user in creating
their preferred individual compilation style faster as they try out the DJ's
operations; d) Indicating to the user where the music needs to be equalized,
switched or arranged. Instead of showing the user the digital music signal
to assist in creating more efficiently, the system directly operates on the
DJ console.User experience experiments were conducted with both novice DJs
and experienced DJs to validate whether the proposed system could help
humans in creating more engaging music with stronger musicality. Five
participants, respectively three novice DJs and two experienced DJs, joined
two experiments of half an hour on a virtual DJ and an actual DJ console.
They started the experiment by experiencing the virtual DJ console and DJ
community in VR. They remixed independently first and then collaborated with
the robotic arm together for music production on the actual DJ console.
Three different audience also joined the experiment to evaluate the
performance of users. The result was that the music produced with the
robotic arm had better musicality. The user's attitude towards the whole
experience, reflected in whether the music was rhythmic or the system was
inspiring was recorded in the feedback. Overall, the users had a satisfying
and smooth experience, and the collaborative music remixing had a certain
level of musicality, but there is still some room for improvement in terms
of user understanding. However, the users expressed that this fresh
collaborative approach made them more interested in DJing and motivated
their desire to learn and create.