# Synthesis of Non-Linguistic Utterances for Sound Design Support Using a Genetic Algorithm
## 2 NLU Generation Using GA and Valence and Arousal Model
### Genetic Algorithm (GA) Methodology  
The process followed in this work is shown in Figure 1. Python 3.6 MIDI packages Mido and MIDIUTIL from Pypi (pypi.org, n.d.) can generate MIDI data using defined variables for pitch, duration, time, instrument, pitch-bend, and other parameters. Using this scheme, MIDI notes are strung together and played back as audio according to a set beats-per-minute tempo and instrument sound contained in the General MIDI 2.0 specification (\"General MIDI level 2\", 2023). A genotype using binary bits is used in a GA to generate NLUs. The genotype design is shown in Figure 2. In the figure, the numbers in each block are the number of bits used for that part of the genotype. Each of the four sequential sections represents a note, with the instrument contained in an additional bit at the end of the note sequence. In each note, the first 5 bits are assigned to pitch. A range of 32 pitches was selected from those available in the general MIDI 2.0 standard to ensure that sounds did not contain very low-pitched or high-pitched notes. After deciding on a minimum and maximum pitch value, pitches were spread out almost evenly. The duration of each note, measured in beats, is encoded into 3 bits, and an extra beat is added to the total so that the smallest note duration is 1 beat, and the largest 8 beats. Using a bpm of 480, 1 beat is 125 ms and 8 beats are 1 s, so a sound has a minimum duration of 500 msand a maximum of 4 s. The next 2 bits of each note are for pitch bend. Pitch bend modifies the pitch of a note over a short time in the semimetal range. There are four possible states of pitch bend, (1) 00--no pitch bend, (2) 01--pitch bend down-up, (3) 10--pitch bend up-down, (4) 11--no pitch bend. The state \"no pitch bend\" was repeated to bias it in the sound generation process. The down-up and up-down pitch bend states involve bending the pitch in the first direction, before reversing it and bending it the other way. The last bit per note is for volume, where 0 represents a volume of 0 to include the possibility of silent segments in the sounds, and 1 represents a volume of 100. All four notes are joined together to form a complete sound. The last bit of the sound (bit 45) is for the instrument, for which two are used from the general MIDI 2.0 specification, 80--square lead, and 85--synth vox, selected as they closely resemble the timbre of typical robotic voices sampled in previous work, such as R2D2, WallE, and BB8. A total of 3000 sounds were generated randomly, using the described genotype scheme, and evaluated by a single subject on a scale of 1 to 5 for their suitability as NLUs. The following criteria were used to determine the rating, with the primary guideline being that sounds that more closely resembled observed NLUs were rated higher than those that sounded more abstract:  
* Duration and syllable count--sounds that were too short or contained one syllable (discernable pitch) were rated 1. Sounds containing multiple notes and/or syllables were rated higher. * Pitch dynamic range--sounds that contained high pitch dynamic ranges, very low and very high pitches, were rated low. Observed NLUs did not contain wide pitch ranges. Sounds with very low pitch ranges were rated low. * Pitch pattern--sounds with a significant amount of variation in their pitch pattern were rated lower than sounds that followed a more consistent and harmonious pattern such as a steady increase or decrease in pitch. Sounds with less dramatic variation in pitch and sounds with pitch bend were also rated higher since they were more like the observed NLUs. One subject rated 3000 randomly generated sounds. The data were used to train a multi-layer perceptron (neural network) to infer the rating of a sound using its 45-bit geno  
Figure 1: Genetic algorithm NLU generation and valence arousal modeling process flow. Figure 2: Genetic algorithm genotype specification. type as the input data. The input data are strings of length 45 that were converted to arrays of integers with 45 elements.