Audio visualization with Rust
Tags: rust, audio
Published: 2019-06-22 17:00:40

---
title: Audio visualization with Rust
published: true
description: An audio visualization build with Rust using Portaudio and Three.
tags: Rust, audio
cover_image: https://github.com/maniflames/MicViz/blob/master/assets/mic.gif?raw=true
---

Since the start of this year I have been very determined to learn more about rust. I knew I wanted to do something with audio but had no idea where to start. Thankfully Justin Wernick created [Rusty Mic](https://github.com/JWorthe/rusty_microphone) and wrote a [blogpost](https://www.worthe-it.co.za/programming/2017/08/14/rusty-microphone.html) about it. I decided to build my own simple 'microphone visualization' with a different graphics library. If you catch something that could be improved let me know, I'd love to learn!

If you just want to check out the code you can check out the repo I made for this small project:
{% github maniflames/MicViz %}


# Dependencies
For this project I'll be using two crates: [portaudio](https://crates.io/crates/portaudio) for working with audio and [three](https://crates.io/crates/three). So go ahead and put the following in your `Cargo.toml`:

```toml
[dependencies]
portaudio = "0.7.0"
three = "0.4.0"
```

Usually that would be all you'd have to do but in this case things are a little different. The portaudio crate just consists of Rust bindings to an actual installation of portaudio so you'll have to install that yourself. You can [download](http://www.portaudio.com/download.html) and [compile](http://portaudio.com/docs/v19-doxydocs/tutorial_start.html) portaudio for Windows, Linux and MacOS. However if you happen to dev on a mac it's probably easier to install it through homebrew with pkg-config.

```bash
brew install pkg-config
brew install portaudio
```

# Reading audio input 

Let's read input from the default mic of your computer first. To accomplish this with portaudio's non-blocking inputstream. I'll try to be clear as possible by just commenting code line by line. 

```rust
use portaudio;
use std::sync::mpsc::*;

fn main() {

    // Construct a portaudio instance that will connect to a native audio API
    let pa = portaudio::PortAudio::new().expect("Unable to init PortAudio"); 
    // Collect information about the default microphone
    let mic_index = pa.default_input_device().expect("Unable to get default device");
    let mic = pa.device_info(mic_index).expect("unable to get mic info");

    // Set parameters for the stream settings.
    // We pass which mic should be used, how many channels are used,
    // whether all the values of all the channels should be passed in a 
    // single audiobuffer and the latency that should be considered 
    let input_params = portaudio::StreamParameters::<f32>::new( mic_index, 1, true, mic.default_low_input_latency);

    // Settings for an inputstream.
    // Here we pass the stream parameters we set before,
    // the sample rate of the mic and the amount values we want to receive
    let input_settings = portaudio::InputStreamSettings::new(input_params, mic.default_sample_rate, 256);

    // Creating a channel so we can receive audio values asynchronously
    let (sender, receiver) = channel(); 

    // A callback function that should be as short as possible so we send all the info to a different thread
    let callback = move |portaudio::InputStreamCallbackArgs {buffer, .. }| {
        match sender.send(buffer) {
            Ok(_) => portaudio::Continue, 
            Err(_) => portaudio::Complete
        }
    };

    // Creating & starting the input stream with our settings & callback
    let mut stream = pa.open_non_blocking_stream(input_settings, callback).expect("Unable to create stream"); 
    stream.start().expect("Unable to start stream");

    //Printing values every time we receive new ones while the stream is active
    while stream.is_active().unwrap() {
       while let Ok(buffer) = receiver.try_recv() {
            println!("{:?}", buffer); 
       }
    }
}

```

Running this should look something like this:   
![](http://assets.imanidap.nl/dev/micviz-terminal.gif)


# Visualizing the audio

If you look closely you'll notice that all you get back from your microphone is a `slice` containing numbers between -1.0 and 1.0. These numbers are a representation of how sound waves have manipulated a moving part within your microphone over time. There are different ways to visualise this change but to keep it simple let's draw a line for every change that occurs in time. We'll animate the lines by simply removing them and drawing new lines when we receive new changes. 

```rust
use portaudio;
use std::sync::mpsc::*;
use three;

//struct for storing the application state
#[derive(Debug)]
struct State {
    sound_values: Vec<f32>,
    scene_meshes: Vec<three::Mesh>
}
                                                                                                                                                                        
fn main() {
    // Receiving audio input 
    let pa = portaudio::PortAudio::new().expect("Unable to init PortAudio"); 
    let mic_index = pa.default_input_device().expect("Unable to get default device");
    let mic = pa.device_info(mic_index).expect("unable to get mic info");

    let input_params = portaudio::StreamParameters::<f32>::new(mic_index, 1, true, mic.default_low_input_latency);
    let input_settings = portaudio::InputStreamSettings::new(input_params, mic.default_sample_rate, 256);

    let (sender, receiver) = channel();

    let callback = move |portaudio::InputStreamCallbackArgs {buffer, .. }| {
        match sender.send(buffer) {
            Ok(_) => portaudio::Continue, 
            Err(_) => portaudio::Complete
        }
    };

    let mut stream = pa.open_non_blocking_stream(input_settings, callback).expect("Unable to create stream"); 
    stream.start().expect("Unable to start stream"); 

    // Create a full screen window with a black background
    let mut builder = three::Window::builder("My Mic"); 
    builder.fullscreen(true); 
    let mut win = builder.build(); 
    win.scene.background = three::Background::Color(0x000000);

    // Create a variable that will contain the state off the app
    let mut state = State {
        sound_values: Vec::new(),
        scene_meshes: Vec::new()
    };

    // Create a camera that will be put in the scene on location 0.0, 0.0
    let camera = win.factory.orthographic_camera([0.0, 0.0], 1.0, -1.0 .. 1.0); 

    //Animation loop that will run until you press ESC or exit the program
    while win.update() && !win.input.hit(three::KEY_ESCAPE) {
        // Put new lines in the scene temporarily save them in the state
        update_lines(&mut win, &mut state);
        // Show the lines
        win.render(&camera);
        // Remove all lines from the scene and the state
        remove_lines(&mut win, &mut state);

        //Update state
        while let Ok(buffer) = receiver.try_recv() {
            update_sound_values(&buffer, &mut state); 
       }
    }
}

// Pass new samples into the state by overriding the vector
fn update_sound_values(samples: &[f32], state: &mut State) {
   state.sound_values = samples.to_vec(); 
}

// Put new lines in the scene temporarily save them in the state
fn update_lines(win: &mut three::window::Window, state: &mut State) {
    for (index, y_position) in state.sound_values.iter().enumerate() {

        // calculate the x position of the line by calculating a normalized x position between 0.0 and 1.0 (i / num_samples)
        // With the scale variable the size of the visualization can be changed. 
        let i = index as f32; 
        let num_samples = state.sound_values.len() as f32; 
        let scale = 3.0; 
        let x_position = (i / (num_samples / scale)) - (0.5 * scale);

        // create the geometry for a line with the calculated positions
        // three is a 3D graphics library so we pass the x, y, z values
        let geometry = three::Geometry::with_vertices(vec![
            [x_position, y_position.clone(), 0.0].into(),
            [x_position, -y_position.clone(), 0.0].into()
        ]);

        // create material so for our line, in this case white line material
        let material = three::material::Line {
            color: 0xFFFFFF,
        };

        // create a 3D object from the geometry and the material
        let mesh = win.factory.mesh(geometry, material);

        // Put the line in the scene and store it in the state
        win.scene.add(&mesh); 
        state.scene_meshes.push(mesh); 
    }
}

// Remove all lines from the scene and the state
fn remove_lines(win: &mut three::window::Window, state: &mut State) {
    for mesh in &state.scene_meshes {
        win.scene.remove(&mesh); 
    }

    state.scene_meshes.clear(); 
}
```

If you run the script it should look something like this: 

![](https://github.com/maniflames/MicViz/blob/master/assets/mic.gif?raw=true) 


Enjoy your MicViz, if you have some time on your hands show off your version of it 😄👀

And again, let me know any way to improve presented code if you happen to know a thing or two about Rust!