Automatic music creation without artificial intelligence
Tags: javascript, music, midi, android
Published: 2024-04-02 07:03:37

Automatic creation of music according to specified parameters.

- Open in browser - https://surikov.github.io/rockdice/main.html.
- Sources - https://github.com/surikov/rockdice
- Android version - https://play.google.com/store/apps/details?id=rockdice.chord.progression

Uses Web Audio API, TypeScript, Android WebView, wavetable synthesis, and basic music theory.

## Description of application

In the main window, you can use the slider to select the progression. The sequence of chords determines the mood of the melody (minor on the left, major on the right, jazz seventh chord in the middle). 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jaioe7z0xb9rzh29u8zz.png)

Round switches allow you to select riffs for the standard 4 layers of melody.

- Drums - set the rhythm
- Bass – harmonic basis and rhythm
- The leader creates a melodic pattern.
- Pad – counterpoint, long notes that show the function of a chord.

The blue button with a die selects random values.

The application modulates selected fragments to the specified chords. If you turn the switches and listen to a few melodies, you can be convinced that with any combination of chords and instruments, the sound of the result is quite “human”.

### Similar projects

The first synthesizers had built-in functions for auto-generating arrangements. An example of performance on a Cassio MT-100 from the 1980s - https://www.youtube.com/watch?v=Aepm6V4yvhw

Modern Captain Plugins Epic app, suitable only for professionals - 
https://www.youtube.com/watch?v=ZuB_t1DBwp8

The BandLab SongStarter web application does not support any settings, it simply generates melodies in one style - 
https://www.youtube.com/watch?v=EDRPy8KtY0c

### Settings

When you click on the gear, the settings window opens. You can edit volume by layers, speed, change chords, transpose:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zweo11e4bdau3nxdwwbe.png)

For each track you can choose a riff from the list:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zzsydddih6d5t61tn6sm.png)

You can also choose a chord progression:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/msm402up3h7fewoe4e9q.png)

### History/Undo

The ability to undo a previous action and return to a previous state is a typical requirement for any application. When you click on the button with the Cancel icon (top right), a window opens with the history of selecting melodies:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x66h9f06vqsp48ogce7n.png)

### Publishing and exporting

Like any music application, RockDice allows you to export the created melody to a MIDI file or Wav file (buttons on the bottom panel). The resulting files can be edited in other music applications. Example of tracks imported into BandLab MixEditor (any track can be forked and opened in the editor):

https://www.bandlab.com/sss1024/tracks

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k8dqsbjazpqs8g3dswwo.png)

### Collaboration

The link to the melody opens in the application exactly as the author sent it.

Clicking the button with the standard Share icon opens a preview window. In it you can send a link to the melody by email, instant messenger or social networks:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/stdor3m0o7zbwmns54c7.png)

Most social networks correctly recognize a link with a description and image of the selected tools:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7kr0f8mnovwg9j9ofx7i.png)

## Implementation Description

How it's done.

### Playing sound

The WebAudioFont library is used to play sound:

https://github.com/surikov/webaudiofont

The library contains more than 2000 digitized instruments and basic filters for sound processing:

- 10-band equalizer to customize tone
- echo for surround sound
- dynamic compressor to highlight voices

Documentation:

https://surikov.github.io/webaudiofont/npm/src/docs/index.html

### Modulation of fragments

For selected (or manually entered) chords, extractMode calculates the root note and scale, see below.

https://github.com/surikov/rockdice/blob/main/ts/code/zvoogharmonizer.ts#L1156

For music playing, about two hundred riffs with different instruments were added to the project; you can see them in the folder

https://github.com/surikov/rockdice/tree/main/patterns

Each pattern, in addition to notes, also contains a description of the mode and tonic.

The selected riffs in the morphSchedule function are transposed to the root of the selected chords and modulated according to the scale of the chord progression, see below.

https://github.com/surikov/rockdice/blob/main/ts/code/zvoogharmonizer.ts#L377

- Transposing a melody involves moving all notes the same number of semitones. Simply put, make the sound higher or lower.
- Modulation of a melody is a shift of the scale by certain steps. In a simple way - from minor to major, etc.

### Music instrument's Features

When playing musical fragments, the specifics of performance on specific instruments are taken into account. For example:

- When playing guitar with distortion, you typically use both open string strumming and palm muted strumming. Those. There is only one instrument, but for a realistic sound you need two separate sets of samples for [open](https://surikov.github.io/webaudiofontdata/sound/0300_LesPaul_sf2.html) and [muted](https://surikov.github.io/webaudiofontdata/sound/0290_LesPaul_sf2.html) strings
- on an acoustic guitar, hitting the strings down and up is noticeably different, see [example](https://surikov.github.io/webaudiofont/examples/strum.html)
- sets of notes of similar chords (for example G and Gm) can be pressed on completely different frets, this must be taken into account when modulating fragments
- and so on.

As a result, the generated music sounds less monotonous.

### Application State's storage

Selected chords, fragments and other settings are saved in the browser's localStorage, see [readObjectFromlocalStorage](https://github.com/surikov/rockdice/blob/main/ts/code/zvoogapp.ts#L2376). As a result, when you reopen the application, its state is restored to the same state as in the last session.

When creating a link for publication, all data is encoded into a long, very long URL, example

https://mzxbox.ru/RockDice/share.php?seed=%7B%22drumsSeed%22%3A21%2C%22bassSeed%22%3A12%2C%22leadSeed%22%3A6%2C%22padSeed%22%3A12%2C%22drumsVolume%22%3A111%2C%22bassVolume%22%3A99%2C%22leadVolume%22%3A66%2C%22padVolume%22%3A77%2C%22chords%22%3A%5B%22Cm%22%2C%222%2F1%22%2C%22Ebm%22%2C%222%2F1%22%5D%2C%22tempo%22%3A130%2C%22mode%22%3A%22Ionian%22%2C%22tone%22%3A%22D%23%22%2C%22version%22%3A%22v2.83%22%2C%22comment%22%3A%22%22%2C%22ui%22%3A%22web%22%7D

### Link markup

Published links comply with the Open Graph and Twitter Cards protocols.

These protocols are supported by most social networking systems. networks. Essentially, this is a requirement that the page being linked to include a description of the page and a preview image.

The image and page for publication are generated dynamically by a regular PHP script, see.

https://github.com/surikov/rockdice/blob/main/server/share.php

All information for the preview is in the tags

`<meta name="twitter:card" content="summary" />
<meta property="og:title" content="<?php echo $line; ?>" />
<meta property="og:url" content="https://mzxbox.ru/RockDice/share.php?seed=<?php echo $encoded; ?>" />
<meta property="og:image" content="https://mzxbox.ru/RockDice/picture.php?drums=<?php echo $drums; ?>&prog=<?php echo urlencode($line); ?>&bass=<?php echo $bass; ?>&lead=<?php echo $lead; ?>&pad=<?php echo $pad; ?>" />`

Scripts are hosted on regular hosting at a price of approximately 200 rubles per month.

### Android

To create a mobile version, you can use the WebView component.

https://developer.android.com/reference/android/webkit/WebView

Essentially, it's regular Chrome built into an Activity. Web pages are opened locally from application resources, files and digitized tools are also downloaded locally.

In terms of functionality, the mobile version is no different from the web version. Here is the compiled application (not updated for a long time):

https://play.google.com/store/apps/details?id=rockdice.chord.progression

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vsrd2k5i1dj052leqlfy.png)

WebView has some minor limitations that should be taken into account.

For example, access to the file system for saving melodies in MIDI and Wav files. In a large browser, it is enough to create a blob with binary data of the correct MIME type and open a link to it, for example

`let blob: Blob = new Blob([dataview], { type: 'audio/wav' });
let ourl = URL.createObjectURL(blob);
let a = document.createElement("a");
document.body.appendChild(a);
a.href = ourl;
a.download = 'rockdice';
a.click();`

The browser correctly recognizes the file type and displays the normal save dialog with the correct extension.

For an embedded WebView, you will need to create a save function using Android, embed it in the application's JavaScript via [addJavascriptInterface](https://developer.android.com/reference/android/webkit/WebView#addJavascriptInterface(java.lang.Object,%20java.lang.String)), and access it from the web page.

It is also necessary that published links to melodies open not in the phone’s browser, but in the mobile version of the application. To do this, you need to configure a filter in the AndroidManifest.xml of the application:

`<intent-filter>
    <action android:name="android.intent.action.VIEW" />
    <category android:name="android.intent.category.DEFAULT" />
    <category android:name="android.intent.category.BROWSABLE" />
    <data
        android:host="surikov.github.io"
        android:pathPrefix="/rockdice/main.html"
        android:scheme="https"
        />
</intent-filter>`

You can do without the mobile version, because Chrome on phones is no different from Chrome on a computer.