3rd Conference on AI Music Creativity
Music Session
Composition & Sound Design/Synthesis #2
Thursday, 15 September 2022
22:00 JST (UTC+9)

Phivos-Angelos Kollias
Jean-Philippe Jullin
Andrew Fyfe
Bob L. T. Sturm

Nostophiliac
Aus’ Dreams
Resembling Shakuhachi
The shadow still lay where he had been standing

Nostophiliac
We interact daily with algorithms that emulate human perception and collective memory. By trying tocommunicate with us, the algorithms sound, look and behave more and more like us by reflecting our perception and memory back to us. What if those AI tools become instruments of manipulation by tackling
the spectator’s sense of familiarity using shared cultural signs, tropes, or archetypes? We investigate andexplore the relationship between collective and individual memory reflected and manipulated through AI:
the concept of the “found object” and its algorithmic transformation of meaning. The network, fed withfamiliar audio-visual “found objects”, generates a progressive alteration, a dream-like transformation ofmeaning. On the musical side, a nonlinear feedback network that listens to familiar sound objects generates a continuous sound transformation. On the visual side, a generative adversarial network represents
a collective artificial memory and perception. Each time, the sound and image transformations create apersonal narrative, a phrase, a gesture for the spectator. The project is funded by Musikfonds, Berlin.
Phivos-Angelos Kollias is a composer-researcher with a PhD in Music inspired by Complexity Theories (University of Paris VIII). He has studied composition in Cambridge, London & Paris with the economic support of four foundations (Onassis Foundation, Megaron Athens Concert Hall, etc.). Kollias
currently lives in Berlin, composing for Interactive Music performances, Virtual Reality installations andVideo Games and working with AI Generative Visuals. He has given lectures at international conferencessuch as EMS (Paris & Leicester), SMC (Berlin), Xenakis International Colloquium (London), and EuropeChina Cultural Forum (Brussels & Beijing). His music has received seven awards and nine nominationsin international competitions. The group projects he has participated in with his music have won ten
awards and distinctions, such as FIVARS (Toronto), Forbes top XR installations, IDFA (Amsterdam), andVR NOW (Berlin), and they have been premiered in festivals like BFI (London), Nationalgalerie (Berlin), National Palace Museum (Chiayi, Taiwan) CPH:DOX (Copenhagen). His music scores are published by BabelScores (Paris). His recent projects have been supported by Musikfonds (Berlin), Deutscher Musikrat(Bonn), Neukölln Department of Culture (Berlin), Dell Computers (US), Ensemble Ipse (New York) etc.

Aus’ Dreams
Aus is the name of my computer. This work explores its anabiosis, i.e., its return to life after a period
in a suspended state. The aim was to achieve an equitable collaboration, without one of us having
more control over the performance than the other. After composing various dreams collaboratively, Ausnavigates them according to characteristics of similarity. Altering my actions, it constantly seeks toembellish them and enhance my creativity, thus co-creating a complex musical system. This work invites
the abandonment of self, attempting to engage in a state of latency to resonate with a whole.

Jean-Philippe Jullin (b.1995) is currently studying digital music at the University of Montreal. Holdinga DEC in audiovisual technique, it is through his previous work in the audiovisual field that his language
was shaped, mixing elements of art and technology. Passionate about sound experimentation, composer of experimental music, he seeks a different way of thinking about music. His performances andinstallations aim to establish a dialogue between humans and their environment by bringing the listenerinto the work. The sound meets the image and the material, approaching a multiplicity of techniques andmeanings. His main field of activity today is around sound creation in collaboration with machines, withthe objective of conceiving experiences of active contemplation, where various elements are assembled
to question our perception of space and time.

ResemblingShakuhachi
Resembling Shakuhachi is a collection of three compositions which used only the generated material
from a ‘Real-time Audio Variational Auto-encoder’ (RAVE) trained on a Shakuhachi dataset, withadditional audio effects to manipulate and augment the material for creative expression.
1. Shakuhachi Soundscape demonstrates how the Shakuhachi model can produce illusions of the natural environment through extended performance. The various sounds contained within this composition
are subject to the listener’s interpretation but the intention was to create a soundscape that alludes toan outdoor and rural environment. Thus, illustrating the sonic versatility of the model.
2. Shakuhachi Ambient explores another form of soundscape featuring long reverberations and sonicresonances. The system was driven by synthetic input material which stressed the cohesive limitationsof the Shakuhachi model. Beyond the cohesive limitations exists a trans-idiomatic terrain; sonicphenomena that transcends the familiar and recognisable sound of the Shakuhachi instrument.
3. Shakuhachi Extended consists of numerous layers of model-derived material, intricately arranged tocreate various syncopated rhythms and timbres. This is the only composition out of the collection thatincorporates multiple tracks, audio splicing, compositional arrangement as well as the use of additional
audio effects to manipulate and augment the featured material for further sonic exploration.
Andrew Fyfe is Tech Lead on the Neutone project at Qosmo, Inc. He is an AI and DSP engineer, who has
worked at Krotos Audio Ltd and Audio Imperia LLC. Andrew is also a proprietor of Otago Engineering Ltd,developing audio software and hardware for music/sound production. Andrew is currently pursuing his
PhD at the University of Glasgow, researching Neural Audio Synthesis, Neural Audio effects and developing tools for artists that utilise AI audio technologies.

The shadow still lay
where he had beenstanding
This piece is in a sense an ode to Robert Ashley’s opera for television, Perfect Lives (1984), but withdurations far reduced to be more compatible with the attention span of the general YouTube audience.
It explores a kind of stream of consciousness creation arising from interacting with generative machinelearning models of music (folk-rnn and Music Transformer), text (GPT) and images (VQGAN-CLIP). To
create the first episode, I built up a short script with GPT starting with the prompt: “deep and moving(a poem)”. GPT responded, “The shadow still lay where he had been standing”, and some other stuff.
I used VQGAN-CLIP to generate images with prompts drawn from the text. Melodic material for thisepisode comes from material generated by folk-rnn and Music Transformer. The construction of theother episodes proceeded in much the same way.
Bob L. T. Sturm is the PI of the ERC Consolidator Grant “MUSAiC: Music at the Frontiers of Artificial
Creativity and Criticism” (ERC-2019-COG No. 864189), which is exploring the implications of involving AI
technology in music practices, with a particular focus on traditional music. He plays accordion, studies
Irish traditional music, and composes. He is currently an Associate Professor of Computer Science at
KTH, Royal Institute of Technology, where he teaches courses on music informatics and machine learning. Before that he was a Lecturer in Digital Media at the Centre for Digital Music, School of Electronic
Engineering and Computer Science, Queen Mary University of London. He received a PhD in Electricaland Computer Engineering in 2009 from the University of California, Santa Barbara.