Repository: marimo-team/marimo
Pull request: improvement : Adding functionality to handle numpy array as src in mo.audio simliar to ipython audio
Added functionality to handle `numpy` array in mo.audio 

fixes #4197

- [x] I have read the [contributor guidelines](https://github.com/marimo-team/marimo/blob/main/CONTRIBUTING.md).
- [x] I have run the code and verified that it works as expected.

@akshayka


Problem statement:
`IPython.display.Audio` is not being updated
### Describe the bug

I have created a slider for start offset of an audio file. Moving it updates the value and re-runs the affected cells, but `Audio` is not updated, i.e. it still plays the same section. If I comment out that cell, run it, uncomment, and run it again, it is updated. I looked around if there is anything special I had to do for using `IPython.display` objects but couldn't find anything.

### Environment

<details>

```
{
  "marimo": "0.11.21",
  "OS": "Linux",
  "OS Version": "6.13.7-arch1-1",
  "Processor": "",
  "Python Version": "3.9.21",
  "Binaries": {
    "Browser": "--",
    "Node": "v23.9.0"
  },
  "Dependencies": {
    "click": "8.1.8",
    "docutils": "0.21.2",
    "itsdangerous": "2.2.0",
    "jedi": "0.19.2",
    "markdown": "3.7",
    "narwhals": "1.31.0",
    "packaging": "24.2",
    "psutil": "7.0.0",
    "pygments": "2.19.1",
    "pymdown-extensions": "10.14.3",
    "pyyaml": "6.0.2",
    "ruff": "0.11.0",
    "starlette": "0.46.1",
    "tomlkit": "0.13.2",
    "typing-extensions": "4.12.2",
    "uvicorn": "0.34.0",
    "websockets": "15.0.1"
  },
  "Optional Dependencies": {
    "polars": "1.25.2"
  },
  "Experimental Flags": {}
}

```

</details>

### Code to reproduce

```python
import marimo

__generated_with = "0.11.21"
app = marimo.App(width="medium")


@app.cell
def _():
    import marimo as mo
    import torchaudio
    from IPython.display import Audio
    return Audio, mo, torchaudio


@app.cell
def _(torchaudio):
    SAMPLE_WAV = "/home/kureta/Music/Flute Samples/01. Air.wav"
    waveform, SAMPLE_RATE = torchaudio.load(SAMPLE_WAV)
    return SAMPLE_RATE, SAMPLE_WAV, waveform


@app.cell
def _(mo):
    offset = mo.ui.slider(0.0, 60.0, 0.5, label="Start offset in seconds.")
    offset
    return (offset,)


@app.cell
def _(Audio, SAMPLE_RATE, offset, waveform):
    start, end = (
        int(offset.value * SAMPLE_RATE),
        int((offset.value + 10.0) * SAMPLE_RATE),
    )
    Audio(data=waveform[:, start:end], rate=SAMPLE_RATE)
    return end, start


@app.cell
def _():
    return


if __name__ == "__main__":
    app.run()
```

Hints from issue discussion:
Thanks for reporting. Can you use our built in audio component in the meantime?

https://docs.marimo.io/api/media/audio/

Edit: I suppose we don't expose enough options, compared to IPython's audio component?
Thanks for the light-speed response :)
I process the loaded file and generate new audio data as `numpy` arrays or `torch` tensors. I need to be able to play them back. `marimo`'s native `audio` component seems to accept `BytesIO` but I don't know how I can specify the sample rate. So in short, I need to be able to play back `numpy` arrays.

Commit message: Adding functionality to handle numpy array as src in mo.audio simliar to ipython

Patch to review:
diff --git a/marimo/_plugins/stateless/audio.py b/marimo/_plugins/stateless/audio.py
index 347063182fe..8b5e39cc8f7 100644
--- a/marimo/_plugins/stateless/audio.py
+++ b/marimo/_plugins/stateless/audio.py
@@ -3,6 +3,7 @@
 
 import io
 import os
+import wave
 from typing import Optional, Union
 
 import marimo._output.data.data as mo_data
@@ -12,15 +13,66 @@
 from marimo._plugins.core.media import io_to_data_url
 
 
+def convert_numpy_to_wav(data, rate: int, normalize: bool) -> bytes:
+    import numpy as np
+
+    def get_normalization_factor(
+        max_abs_value, normalize
+    ) -> Union[float, int]:
+        if not normalize and max_abs_value > 1:
+            raise ValueError(
+                "Audio data must be between -1 and 1 when normalize=False."
+            )
+        return max_abs_value if normalize else 1
+
+    data = np.array(data, dtype=float)
+    if len(data.shape) == 1:
+        nchannels = 1
+    elif len(data.shape) == 2:
+        nchannels = data.shape[0]
+        data = data.T.ravel()
+    else:
+        raise ValueError("Array audio input must be a 1D or 2D array")
+
+    max_abs_value = np.max(np.abs(data))
+    normalization_factor = get_normalization_factor(max_abs_value, normalize)
+    scaled = data / normalization_factor * 32767
+    scaled = scaled.astype("<h").tobytes()
+
+    buffer = io.BytesIO()
+    waveobj = wave.open(buffer, mode="wb")
+    waveobj.setnchannels(nchannels)
+    waveobj.setframerate(rate)
+    waveobj.setsampwidth(2)
+    waveobj.setcomptype("NONE", "NONE")
+    waveobj.writeframes(scaled)
+    val = buffer.getvalue()
+    waveobj.close()
+    return val
+
+
 @mddoc
 def audio(
     src: Union[str, io.BytesIO],
+    rate: int,
+    normalize: bool = True,
 ) -> Html:
     """Render an audio file as HTML.
 
     Args:
         src: a path or URL to an audio file, bytes,
             or a file-like object opened in binary mode
+            Numpy 1d array containing the desired waveform (mono)
+            Numpy 2d array containing waveforms for each channel.Shape=(NCHAN, NSAMPLES).
+        rate : integer
+            The sampling rate of the raw data.
+            Only required when data parameter is being used as an array
+        normalize : bool
+            Whether audio should be normalized (rescaled) to the maximum possible
+            range. Default is `True`. When set to `False`, `data` must be between
+            -1 and 1 (inclusive), otherwise an error is raised.
+            Applies only when `data` is a list or array of samples; other types of
+            audio are never normalized.
 
     Returns:
         An audio player as an `Html` object.
@@ -35,6 +87,7 @@ def audio(
         ```
     """
     resolved_src: Optional[str]
+    import numpy as np
 
     if isinstance(src, (io.BufferedReader, io.BytesIO)):
         pos = src.tell()
@@ -49,6 +102,13 @@ def audio(
             resolved_src = mo_data.audio(
                 f.read(), ext=os.path.splitext(src)[1]
             ).url
+    elif isinstance(src, np.ndarray):
+        if rate is None:
+            raise ValueError(
+                "rate must be specified when data is a numpy array of audio samples."
+            )
+        wav_data = convert_numpy_to_wav(src, rate, normalize)
+        resolved_src = mo_data.audio(wav_data).url
     else:
         resolved_src = io_to_data_url(src, fallback_mime_type="audio/wav")
 


Code review comments:
diff_hunk: @@ -49,6 +102,13 @@ def audio(
             resolved_src = mo_data.audio(
                 f.read(), ext=os.path.splitext(src)[1]
             ).url
+    elif isinstance(src, np.ndarray):
path: marimo/_plugins/stateless/audio.py
text: @user1:
we should first check if numpy is imported with `DependencyManager.numpy.imported()`. 

i guess we don't have any existing tests. i can follow-up to create some tests for this file afterwards (unless you'd like to add them)