Repository: livekit/agents
Pull request #1771: add BackgroundAudio utility & default sounds 
Author: login: theomonnom
Created: 2025-03-28T01:07:35Z | Merged: 2025-03-28T15:40:14Z

- this also fix the thinking state not happening for tools
- each audio file has a volume & probability to happen
- play() method to play another sound at at any time (this returns a PlayHandle, with a stop() method) 
   - play() has an option to loop 
- this doesn't work in the console subcommand (as expected) 


Diff:
diff --git a/.github/next-release/changeset-41eff283.md b/.github/next-release/changeset-41eff283.md
new file mode 100644
index 0000000000..55f1ed5226
--- /dev/null
+++ b/.github/next-release/changeset-41eff283.md
@@ -0,0 +1,5 @@
+---
+"livekit-agents": patch
+---
+
+add BackgroundAudio utility & default sounds  (#1771)
diff --git a/examples/voice_agents/background_audio.py b/examples/voice_agents/background_audio.py
new file mode 100644
index 0000000000..5c44e02362
--- /dev/null
+++ b/examples/voice_agents/background_audio.py
@@ -0,0 +1,63 @@
+import asyncio
+import logging
+
+from dotenv import load_dotenv
+
+from livekit.agents import (
+    Agent,
+    AgentSession,
+    BackgroundAudio,
+    JobContext,
+    WorkerOptions,
+    cli,
+    function_tool,
+)
+from livekit.plugins import openai
+
+logger = logging.getLogger("background-audio")
+
+load_dotenv()
+
+
+class FakeWebSearchAgent(Agent):
+    def __init__(self):
+        super().__init__(instructions="You are a helpful assistant")
+
+    @function_tool
+    async def search_web(self, query: str) -> str:
+        """
+        Search the web for information based on the given query.
+
+        Always use this function whenever the user requests a web search
+
+        Args:
+            query: The search query to look up on the web.
+        """
+
+        # simulate a long web search to demonstrate the background "thinking" audio
+        logger.info("FakeWebSearchAgent thinking...")
+        await asyncio.sleep(5)
+        return "The request failed, give the users some information based on your knowledge"
+
+
+async def entrypoint(ctx: JobContext):
+    await ctx.connect()
+
+    session = AgentSession(llm=openai.realtime.RealtimeModel())
+    await session.start(FakeWebSearchAgent(), room=ctx.room)
+
+    # by default BackgroundAudio will play a office ambience sound and a keyboard typing sound
+    # when the agent is thinking.
+    background_audio = BackgroundAudio(
+        # ambient_sound="background-sound.ogg",
+        # thinking_sound="beep.ogg",
+    )
+
+    await background_audio.start(room=ctx.room, agent_session=session)
+
+    # Play another audio file at any time using the play method:
+    # background_audio.play("filepath.ogg")
+
+
+if __name__ == "__main__":
+    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
diff --git a/livekit-agents/livekit/agents/__init__.py b/livekit-agents/livekit/agents/__init__.py
index 9d8eb3faca..bd8040ac5c 100644
--- a/livekit-agents/livekit/agents/__init__.py
+++ b/livekit-agents/livekit/agents/__init__.py
@@ -42,6 +42,7 @@
 )
 from .version import __version__
 from .voice import Agent, AgentEvent, AgentSession, RunContext, io
+from .voice.background_audio import BackgroundAudio
 from .voice.room_io import RoomInputOptions, RoomIO, RoomOutputOptions
 from .worker import Worker, WorkerOptions, WorkerPermissions, WorkerType
 
@@ -86,6 +87,7 @@
     "NOT_GIVEN",
     "NotGivenOr",
     "DEFAULT_API_CONNECT_OPTIONS",
+    "BackgroundAudio",
 ]
 
 # Cleanup docs of unexported modules
diff --git a/livekit-agents/livekit/agents/resources/.gitattributes  b/livekit-agents/livekit/agents/resources/.gitattributes 
new file mode 100644
index 0000000000..09219da010
--- /dev/null
+++ b/livekit-agents/livekit/agents/resources/.gitattributes 	
@@ -0,0 +1 @@
+*.ogg filter=lfs diff=lfs merge=lfs -text
\ No newline at end of file
diff --git a/livekit-agents/livekit/agents/resources/LICENSE b/livekit-agents/livekit/agents/resources/LICENSE
new file mode 100644
index 0000000000..3dec12b35f
--- /dev/null
+++ b/livekit-agents/livekit/agents/resources/LICENSE
@@ -0,0 +1,3 @@
+keyboard-typing.ogg by Anton -- https://freesound.org/s/137/ -- License: Attribution 4.0
+keyboard-typing2.opg by Anton -- https://freesound.org/s/137/ -- License: Attribution 4.0
+office-ambience.ogg by dnlburnett -- https://freesound.org/s/335711/ -- License: Attribution 3.0
\ No newline at end of file
diff --git a/livekit-agents/livekit/agents/resources/keyboard-typing.ogg b/livekit-agents/livekit/agents/resources/keyboard-typing.ogg
new file mode 100644
index 0000000000..a0ea4bb32f
Binary files /dev/null and b/livekit-agents/livekit/agents/resources/keyboard-typing.ogg differ
diff --git a/livekit-agents/livekit/agents/resources/keyboard-typing2.ogg b/livekit-agents/livekit/agents/resources/keyboard-typing2.ogg
new file mode 100644
index 0000000000..a0b69969d9
Binary files /dev/null and b/livekit-agents/livekit/agents/resources/keyboard-typing2.ogg differ
diff --git a/livekit-agents/livekit/agents/resources/office-ambience.ogg b/livekit-agents/livekit/agents/resources/office-ambience.ogg
new file mode 100644
index 0000000000..f02790f8f2
Binary files /dev/null and b/livekit-agents/livekit/agents/resources/office-ambience.ogg differ
diff --git a/livekit-agents/livekit/agents/utils/audio.py b/livekit-agents/livekit/agents/utils/audio.py
index 8362ae2319..db7c89a21f 100644
--- a/livekit-agents/livekit/agents/utils/audio.py
+++ b/livekit-agents/livekit/agents/utils/audio.py
@@ -1,11 +1,17 @@
 from __future__ import annotations
 
+import asyncio
 import ctypes
+from collections.abc import AsyncGenerator
 from typing import Union
 
+import aiofiles
+
 from livekit import rtc
 
 from ..log import logger
+from .aio.utils import cancel_and_wait
+from .codecs import AudioStreamDecoder
 
 # deprecated aliases
 AudioBuffer = Union[list[rtc.AudioFrame], rtc.AudioFrame]
@@ -144,3 +150,38 @@ def flush(self) -> list[rtc.AudioFrame]:
                 samples_per_channel=len(self._buf) // 2,
             )
         ]
+
+
+async def audio_frames_from_file(
+    file_path: str, sample_rate: int = 48000, num_channels: int = 1
+) -> AsyncGenerator[rtc.AudioFrame, None]:
+    """
+    Decode the audio file into rtc.AudioFrame instances and yield them as an async iterable.
+    Args:
+        file_path (str): The path to the audio file.
+        sample_rate (int, optional): Desired sample rate. Defaults to 48000.
+        num_channels (int, optional): Number of channels (1 for mono, 2 for stereo). Defaults to 1.
+    Returns:
+        AsyncIterable[rtc.AudioFrame]: An async iterable that yields decoded AudioFrame
+    """
+    decoder = AudioStreamDecoder(sample_rate=sample_rate, num_channels=num_channels)
+
+    async def file_reader():
+        async with aiofiles.open(file_path, mode="rb") as f:
+            while True:
+                chunk = await f.read(4096)
+                if not chunk:
+                    break
+
+                decoder.push(chunk)
+
+        decoder.end_input()
+
+    reader_task = asyncio.create_task(file_reader())
+
+    try:
+        async for frame in decoder:
+            yield frame
+
+    finally:
+        await cancel_and_wait(reader_task)
diff --git a/livekit-agents/livekit/agents/utils/codecs/decoder.py b/livekit-agents/livekit/agents/utils/codecs/decoder.py
index e26312eed5..1e00cc1f2d 100644
--- a/livekit-agents/livekit/agents/utils/codecs/decoder.py
+++ b/livekit-agents/livekit/agents/utils/codecs/decoder.py
@@ -14,21 +14,16 @@
 
 import asyncio
 import io
+import threading
 from collections.abc import AsyncIterator
 from concurrent.futures import ThreadPoolExecutor
 from typing import Optional
 
-from livekit.agents.log import logger
-from livekit.agents.utils import aio
-
-try:
-    # preload to ensure faster startup
-    import av  # noqa
-except ImportError:
-    pass
-import threading
+import av
 
 from livekit import rtc
+from livekit.agents.log import logger
+from livekit.agents.utils import aio
 
 
 class StreamBuffer:
@@ -96,13 +91,6 @@ class AudioStreamDecoder:
     _executor: Optional[ThreadPoolExecutor] = None
 
     def __init__(self, *, sample_rate: int = 48000, num_channels: int = 1):
-        try:
-            import av  # noqa
-        except ImportError:
-            raise ImportError(  # noqa: B904
-                "You haven't included the 'codecs' optional dependencies. Please install the 'codecs' extra by running `pip install livekit-agents[codecs]`"  # noqa: E501
-            )
-
         self._sample_rate = sample_rate
         self._layout = "mono"
         if num_channels == 2:
diff --git a/livekit-agents/livekit/agents/voice/agent_activity.py b/livekit-agents/livekit/agents/voice/agent_activity.py
index 92209618cf..fc131dab46 100644
--- a/livekit-agents/livekit/agents/voice/agent_activity.py
+++ b/livekit-agents/livekit/agents/voice/agent_activity.py
@@ -907,7 +907,7 @@ def _on_first_frame(_: asyncio.Future) -> None:
             text_out.first_text_fut.add_done_callback(_on_first_frame)
 
         # start to execute tools (only after play())
-        exe_task, fnc_outputs = perform_tool_executions(
+        exe_task, tool_output = perform_tool_executions(
             session=self._session,
             speech_handle=speech_handle,
             tool_ctx=tool_ctx,
@@ -968,11 +968,15 @@ def _on_first_frame(_: asyncio.Future) -> None:
         log_event("playout completed", speech_id=speech_handle.id)
 
         speech_handle._mark_playout_done()  # mark the playout done before waiting for the tool execution  # noqa: E501
+
+        tool_output.first_tool_fut.add_done_callback(
+            lambda _: self._session._update_agent_state(AgentState.THINKING)
+        )
         await exe_task
 
         # important: no agent output should be used after this point
 
-        if len(fnc_outputs) > 0:
+        if len(tool_output.output) > 0:
             if speech_handle.step_index >= self._session.options.max_tool_steps:
                 logger.warning(
                     "maximum number of function calls steps reached",
@@ -992,7 +996,7 @@ def _on_first_frame(_: asyncio.Future) -> None:
                 function_calls=[],
                 function_call_outputs=[],
             )
-            for py_out in fnc_outputs:
+            for py_out in tool_output.output:
                 sanitized_out = py_out.sanitize()
 
                 if sanitized_out.fnc_call_out is not None:
@@ -1171,7 +1175,7 @@ async def _read_messages(
             )
         ]
 
-        exe_task, fnc_outputs = perform_tool_executions(
+        exe_task, tool_output = perform_tool_executions(
             session=self._session,
             speech_handle=speech_handle,
             tool_ctx=tool_ctx,
@@ -1209,10 +1213,7 @@ async def _read_messages(
 
             for msg_id, text_out, _ in message_outputs:
                 msg = llm.ChatMessage(
-                    role="assistant",
-                    content=[text_out.text],
-                    id=msg_id,
-                    interrupted=True,
+                    role="assistant", content=[text_out.text], id=msg_id, interrupted=True
                 )
                 self._session._conversation_item_added(msg)
             return
@@ -1222,23 +1223,23 @@ async def _read_messages(
 
         for msg_id, text_out, _ in message_outputs:
             msg = llm.ChatMessage(
-                role="assistant",
-                content=[text_out.text],
-                id=msg_id,
-                interrupted=False,
+                role="assistant", content=[text_out.text], id=msg_id, interrupted=False
             )
             self._session._conversation_item_added(msg)
 
+        tool_output.first_tool_fut.add_done_callback(
+            lambda _: self._session._update_agent_state(AgentState.THINKING)
+        )
         await exe_task
 
         # important: no agent ouput should be used after this point
 
-        if len(fnc_outputs) > 0:
+        if len(tool_output.output) > 0:
             new_fnc_outputs: list[llm.FunctionCallOutput] = []
             new_agent_task: Agent | None = None
             ignore_task_switch = False
 
-            for py_out in fnc_outputs:
+            for py_out in tool_output.output:
                 sanitized_out = py_out.sanitize()
 
                 if sanitized_out.fnc_call_out is not None:
diff --git a/livekit-agents/livekit/agents/voice/background_audio.py b/livekit-agents/livekit/agents/voice/background_audio.py
new file mode 100644
index 0000000000..5860b21e2d
--- /dev/null
+++ b/livekit-agents/livekit/agents/voice/background_audio.py
@@ -0,0 +1,410 @@
+import asyncio
+import atexit
+import contextlib
+import random
+from collections.abc import AsyncGenerator, AsyncIterator
+from importlib.resources import as_file, files
+from typing import NamedTuple, Union, cast
+
+import numpy as np
+
+from livekit import rtc
+
+from ..log import logger
+from ..types import NOT_GIVEN, NotGivenOr
+from ..utils import is_given, log_exceptions
+from ..utils.aio import cancel_and_wait
+from ..utils.audio import audio_frames_from_file
+from .agent_session import AgentSession
+from .events import AgentStateChangedEvent
+
+SoundSource = Union[AsyncIterator[rtc.AudioFrame], str]
+
+
+class BackgroundSound(NamedTuple):
+    sound: SoundSource
+    volume: float = 1.0
+    probability: float = 1.0
+
+
+# The queue size is set to 400ms, which determines how much audio Rust will buffer.
+# We intentionally keep this small within BackgroundAudio because calling
+# AudioSource.clear_queue() would abruptly cut off ambient sounds.
+# Instead, we remove the sound from the mixer, and it will get removed 400ms later.
+_AUDIO_SOURCE_BUFFER_MS = 400
+
+_resource_stack = contextlib.ExitStack()
+atexit.register(_resource_stack.close)
+
+
+class BackgroundAudio:
+    def __init__(
+        self,
+        *,
+        ambient_sound: NotGivenOr[
+            Union[SoundSource, BackgroundSound, list[BackgroundSound], None]
+        ] = NOT_GIVEN,
+        thinking_sound: NotGivenOr[
+            Union[SoundSource, BackgroundSound, list[BackgroundSound], None]
+        ] = NOT_GIVEN,
+    ) -> None:
+        """
+        Initializes the BackgroundAudio component with optional ambient and thinking sounds.
+
+        This component creates and publishes a continuous audio track to a LiveKit room while managing
+        the playback of ambient and agent “thinking” sounds. It supports two types of audio sources:
+        - A file path (string) pointing to an audio file, which can be looped.
+        - An AsyncIterator that yields rtc.AudioFrame
+
+        If no ambient sound is provided, a default low‑volume office ambience sound
+        (from "office-ambience.ogg" in the package resources) is used. Similarly, if no thinking sound is provided,
+        a default list of two keyboard typing sounds (from "keyboard-typing.ogg" and "keyboard-typing2.ogg")
+        with higher volume and associated selection probabilities is used.
+
+        When a list (or BackgroundSound) is supplied, the component considers each sound’s volume and probability:
+        - The probability value determines the chance that a particular sound is selected for playback.
+        - A total probability below 1.0 means there is a chance no sound will be selected (resulting in silence).
+
+        Args:
+            ambient_sound (NotGivenOr[Union[SoundSource, BackgroundSound, List[BackgroundSound], None]], optional):
+                The ambient sound to be played continuously. For file paths, the sound will be looped.
+                For AsyncIterator sources, ensure the iterator is infinite or looped. Defaults to a low‑volume
+                office ambience sound.
+            thinking_sound (NotGivenOr[Union[SoundSource, BackgroundSound, List[BackgroundSound], None]], optional):
+                The sound to be played when the associated agent enters a “thinking” state. This can be a single
+                sound source or a list of BackgroundSound objects (with volume and probability settings). Defaults
+                to a set of keyboard typing sounds.
+        """  # noqa: E501
+        default_office = files("livekit.agents.resources") / "office-ambience.ogg"
+        default_keyboard = files("livekit.agents.resources") / "keyboard-typing.ogg"
+        default_keyboard2 = files("livekit.agents.resources") / "keyboard-typing2.ogg"
+
+        office_path = _resource_stack.enter_context(as_file(default_office))
+        keyboard_path = _resource_stack.enter_context(as_file(default_keyboard))
+        keyboard_path2 = _resource_stack.enter_context(as_file(default_keyboard2))
+
+        self._ambient_sound = (
+            ambient_sound
+            if is_given(ambient_sound)
+            else BackgroundSound(str(office_path), volume=0.2)
+        )
+        self._thinking_sound = (
+            thinking_sound
+            if is_given(thinking_sound)
+            else [
+                BackgroundSound(str(keyboard_path), volume=0.8, probability=0.4),
+                BackgroundSound(str(keyboard_path2), volume=0.8, probability=0.6),
+            ]
+        )
+
+        self._audio_source = rtc.AudioSource(48000, 1, queue_size_ms=_AUDIO_SOURCE_BUFFER_MS)
+        self._audio_mixer = rtc.AudioMixer(48000, 1, blocksize=4800, capacity=1)
+        self._publication: rtc.LocalTrackPublication | None = None
+        self._lock = asyncio.Lock()
+
+        self._republish_task: asyncio.Task | None = None  # republish the task on reconnect
+        self._mixer_atask: asyncio.Task | None = None
+
+        self._play_tasks: list[asyncio.Task] = []
+
+        self._ambient_handle: PlayHandle | None = None
+        self._thinking_handle: PlayHandle | None = None
+
+    def _select_sound_from_list(
+        self, sounds: list[BackgroundSound]
+    ) -> Union[BackgroundSound, None]:
+        """
+        Selects a sound from a list of BackgroundSound based on their probabilities.
+        Returns None if no sound is selected (when sum of probabilities < 1.0).
+        """
+        total_probability = sum(sound.probability for sound in sounds)
+        if total_probability <= 0:
+            return None
+
+        if total_probability < 1.0 and random.random() > total_probability:
+            return None
+
+        normalize_factor = 1.0 if total_probability <= 1.0 else total_probability
+        r = random.random() * min(total_probability, 1.0)
+        cumulative = 0.0
+
+        for sound in sounds:
+            if sound.probability <= 0:
+                continue
+
+            norm_prob = sound.probability / normalize_factor
+            cumulative += norm_prob
+
+            if r <= cumulative:
+                return sound
+
+        return sounds[-1]
+
+    def _normalize_sound_source(
+        self, source: Union[SoundSource, BackgroundSound, list[BackgroundSound], None]
+    ) -> Union[tuple[SoundSource, float], None]:
+        if source is None:
+            return None
+
+        if isinstance(source, list):
+            selected = self._select_sound_from_list(cast(list[BackgroundSound], source))
+            if selected is None:
+                return None
+            return selected.sound, selected.volume
+
+        if isinstance(source, BackgroundSound):
+            return source.sound, source.volume
+
+        return source, 1.0
+
+    def play(
+        self,
+        sound: Union[SoundSource, BackgroundSound, list[BackgroundSound]],
+        *,
+        loop: bool = False,
+    ) -> "PlayHandle":
+        """
+        Plays a sound once or in a loop.
+
+        Args:
+            sound (Union[SoundSource, BackgroundSound, List[BackgroundSound]]):
+                The sound to play. Can be:
+                - A string pointing to a file path
+                - An AsyncIterator that yields `rtc.AudioFrame`
+                - A BackgroundSound object with volume and probability
+                - A list of BackgroundSound objects, where one will be selected based on probability
+
+                If a string is provided and `loop` is True, the sound will be looped.
+                If an AsyncIterator is provided, it is played until exhaustion (and cannot be looped
+                automatically).
+            loop (bool, optional):
+                Whether to loop the sound. Only applicable if `sound` is a string or contains strings.
+                Defaults to False.
+
+        Returns:
+            PlayHandle: An object representing the playback handle. This can be
+            awaited or stopped manually.
+        """  # noqa: E501
+        if not self._mixer_atask:
+            raise RuntimeError("BackgroundAudio is not started")
+
+        normalized = self._normalize_sound_source(sound)
+        if normalized is None:
+            play_handle = PlayHandle()
+            play_handle._mark_playout_done()
+            return play_handle
+
+        sound_source, volume = normalized
+
+        if loop and isinstance(sound_source, AsyncIterator):
+            raise ValueError(
+                "Looping sound via AsyncIterator is not supported. Use a string file path or your own 'infinite' AsyncIterator with loop=False"  # noqa: E501
+            )
+
+        play_handle = PlayHandle()
+        task = asyncio.create_task(self._play_task(play_handle, sound_source, volume, loop))
+        task.add_done_callback(lambda _: self._play_tasks.remove(task))
+        task.add_done_callback(lambda _: play_handle._mark_playout_done())
+        self._play_tasks.append(task)
+        return play_handle
+
+    async def start(
+        self,
+        *,
+        room: rtc.Room,
+        agent_session: NotGivenOr[AgentSession] = NOT_GIVEN,
+        track_publish_options: NotGivenOr[rtc.TrackPublishOptions] = NOT_GIVEN,
+    ) -> None:
+        """
+        Starts the background audio system, publishing the audio track
+        and beginning playback of any configured ambient sound.
+
+        If `ambient_sound` is provided (and contains file paths), they will loop
+        automatically. If `ambient_sound` contains AsyncIterators, they are assumed
+        to be already infinite or looped.
+
+        Args:
+            room (rtc.Room):
+                The LiveKit Room object where the audio track will be published.
+            agent_session (NotGivenOr[AgentSession], optional):
+                The session object used to track the agent's state (e.g., "thinking").
+                Required if `thinking_sound` is provided.
+            track_publish_options (NotGivenOr[rtc.TrackPublishOptions], optional):
+                Options used when publishing the audio track. If not given, defaults will
+                be used.
+        """
+        async with self._lock:
+            self._room = room
+            self._agent_session = agent_session or None
+            self._track_publish_options = track_publish_options or None
+
+            await self._publish_track()
+
+            self._mixer_atask = asyncio.create_task(self._run_mixer_task())
+            self._room.on("reconnected", self._on_reconnected)
+
+            if self._agent_session:
+                self._agent_session.on("agent_state_changed", self._agent_state_changed)
+
+            if self._ambient_sound:
+                normalized = self._normalize_sound_source(self._ambient_sound)
+                if normalized:
+                    sound_source, volume = normalized
+                    selected_sound = BackgroundSound(sound_source, volume)
+                    if isinstance(sound_source, str):
+                        self._ambient_handle = self.play(selected_sound, loop=True)
+                    else:
+                        self._ambient_handle = self.play(selected_sound)
+
+    async def aclose(self) -> None:
+        """
+        Gracefully closes the background audio system, canceling all ongoing
+        playback tasks and unpublishing the audio track.
+        """
+        async with self._lock:
+            if not self._mixer_atask:
+                return  # not started
+
+            await cancel_and_wait(*self._play_tasks)
+
+            if self._republish_task:
+                await cancel_and_wait(self._republish_task)
+
+            await cancel_and_wait(self._mixer_atask)
+
+            await self._audio_source.aclose()
+            await self._audio_mixer.aclose()
+
+            if self._agent_session:
+                self._agent_session.off("agent_state_changed", self._agent_state_changed)
+
+            self._room.off("reconnected", self._on_reconnected)
+
+            with contextlib.suppress(Exception):
+                if self._publication is not None:
+                    await self._room.local_participant.unpublish_track(self._publication.sid)
+
+    def _on_reconnected(self) -> None:
+        if self._republish_task:
+            self._republish_task.cancel()
+
+        self._publication = None
+        self._republish_task = asyncio.create_task(self._republish_track_task())
+
+    def _agent_state_changed(self, ev: AgentStateChangedEvent) -> None:
+        if not self._thinking_sound:
+            return
+
+        if ev.state == "thinking":
+            if self._thinking_handle and not self._thinking_handle.done():
+                return
+
+            self._thinking_handle = self.play(self._thinking_sound)
+
+        elif self._thinking_handle:
+            self._thinking_handle.stop()
+
+    @log_exceptions(logger=logger)
+    async def _play_task(
+        self, play_handle: "PlayHandle", sound: SoundSource, volume: float, loop: bool
+    ) -> None:
+        if isinstance(sound, str):
+            if loop:
+                sound = _loop_audio_frames(sound)
+            else:
+                sound = audio_frames_from_file(sound)
+
+        async def _gen_wrapper() -> AsyncGenerator[rtc.AudioFrame, None]:
+            async for frame in sound:
+                if volume != 1.0:
+                    data = np.frombuffer(frame.data, dtype=np.int16).astype(np.float32)
+                    data *= 10 ** (np.log10(volume))
+                    np.clip(data, -32768, 32767, out=data)
+                    yield rtc.AudioFrame(
+                        data=data.astype(np.int16).tobytes(),
+                        sample_rate=frame.sample_rate,
+                        num_channels=frame.num_channels,
+                        samples_per_channel=frame.samples_per_channel,
+                    )
+                else:
+                    yield frame
+
+            # TODO(theomonnom): the wait_for_playout() may be innaccurate by 400ms
+            play_handle._mark_playout_done()
+
+        gen = _gen_wrapper()
+        try:
+            self._audio_mixer.add_stream(gen)
+            await play_handle.wait_for_playout()  # wait for playout or interruption
+        finally:
+            if play_handle._stop_fut.done():
+                self._audio_mixer.remove_stream(gen)
+                await gen.aclose()
+
+            play_handle._mark_playout_done()  # the task could be cancelled
+
+    @log_exceptions(logger=logger)
+    async def _run_mixer_task(self) -> None:
+        async for frame in self._audio_mixer:
+            await self._audio_source.capture_frame(frame)
+
+    async def _publish_track(self) -> None:
+        if self._publication is not None:
+            return
+
+        track = rtc.LocalAudioTrack.create_audio_track("background_audio", self._audio_source)
+        self._publication = await self._room.local_participant.publish_track(
+            track, self._track_publish_options or rtc.TrackPublishOptions()
+        )
+
+    @log_exceptions(logger=logger)
+    async def _republish_track_task(self) -> None:
+        # used to republish the track on agent reconnect
+        async with self._lock:
+            await self._publish_track()
+
+
+class PlayHandle:
+    def __init__(self) -> None:
+        self._done_fut = asyncio.Future()
+        self._stop_fut = asyncio.Future()
+
+    def done(self) -> bool:
+        """
+        Returns True if the sound has finished playing.
+        """
+        return self._done_fut.done()
+
+    def stop(self) -> None:
+        """
+        Stops the sound from playing.
+        """
+        if self.done():
+            return
+
+        with contextlib.suppress(asyncio.InvalidStateError):
+            self._stop_fut.set_result(None)
+            self._mark_playout_done()  # TODO(theomonnom): move this to _play_task
+
+    async def wait_for_playout(self) -> None:
+        """
+        Waits for the sound to finish playing.
+        """
+        await asyncio.shield(self._done_fut)
+
+    def __await__(self):
+        async def _await_impl() -> PlayHandle:
+            await self.wait_for_playout()
+            return self
+
+        return _await_impl().__await__()
+
+    def _mark_playout_done(self) -> None:
+        with contextlib.suppress(asyncio.InvalidStateError):
+            self._done_fut.set_result(None)
+
+
+async def _loop_audio_frames(file_path: str) -> AsyncGenerator[rtc.AudioFrame, None]:
+    while True:
+        async for frame in audio_frames_from_file(file_path):
+            yield frame
diff --git a/livekit-agents/livekit/agents/voice/generation.py b/livekit-agents/livekit/agents/voice/generation.py
index 17b9cc7dca..a79cda0f46 100644
--- a/livekit-agents/livekit/agents/voice/generation.py
+++ b/livekit-agents/livekit/agents/voice/generation.py
@@ -242,6 +242,12 @@ async def _audio_forwarding_task(
         audio_output.flush()
 
 
+@dataclass
+class _ToolOutput:
+    output: list[_PythonOutput]
+    first_tool_fut: asyncio.Future
+
+
 def perform_tool_executions(
     *,
     session: AgentSession,
@@ -249,11 +255,8 @@ def perform_tool_executions(
     tool_ctx: ToolContext,
     tool_choice: NotGivenOr[llm.ToolChoice],
     function_stream: AsyncIterable[llm.FunctionCall],
-) -> tuple[
-    asyncio.Task,
-    list[_PythonOutput],
-]:
-    out: list[_PythonOutput] = []
+) -> tuple[asyncio.Task, _ToolOutput]:
+    tool_output = _ToolOutput(output=[], first_tool_fut=asyncio.Future())
     task = asyncio.create_task(
         _execute_tools_task(
             session=session,
@@ -261,11 +264,11 @@ def perform_tool_executions(
             tool_ctx=tool_ctx,
             tool_choice=tool_choice,
             function_stream=function_stream,
-            out=out,
+            tool_output=tool_output,
         ),
         name="execute_tools_task",
     )
-    return task, out
+    return task, tool_output
 
 
 @utils.log_exceptions(logger=logger)
@@ -276,7 +279,7 @@ async def _execute_tools_task(
     tool_ctx: ToolContext,
     tool_choice: NotGivenOr[llm.ToolChoice],
     function_stream: AsyncIterable[llm.FunctionCall],
-    out: list[_PythonOutput],
+    tool_output: _ToolOutput,
 ) -> None:
     """execute tools, when cancelled, stop executing new tools but wait for the pending ones"""
 
@@ -324,6 +327,9 @@ async def _execute_tools_task(
                 )
                 continue
 
+            if not tool_output.first_tool_fut.done():
+                tool_output.first_tool_fut.set_result(None)
+
             logger.debug(
                 "executing tool",
                 extra={
@@ -365,11 +371,11 @@ def _log_exceptions(task: asyncio.Task) -> None:
                         exc_info=task.exception(),
                     )
                     py_out.exception = task.exception()  # noqa: B023
-                    out.append(py_out)  # noqa: B023
+                    tool_output.output.append(py_out)  # noqa: B023
                     return
 
                 py_out.output = task.result()  # noqa: B023
-                out.append(py_out)  # noqa: B023
+                tool_output.output.append(py_out)  # noqa: B023
                 tasks.remove(task)
 
             task.add_done_callback(_log_exceptions)
@@ -397,7 +403,7 @@ def _log_exceptions(task: asyncio.Task) -> None:
     finally:
         await utils.aio.cancel_and_wait(*tasks)
 
-        if len(out) > 0:
+        if len(tool_output.output) > 0:
             logger.debug(
                 "tools execution completed",
                 extra={"speech_id": speech_handle.id},
diff --git a/livekit-agents/livekit/agents/voice/room_io/_output.py b/livekit-agents/livekit/agents/voice/room_io/_output.py
index 47b6ae92df..a7d66da1cc 100644
--- a/livekit-agents/livekit/agents/voice/room_io/_output.py
+++ b/livekit-agents/livekit/agents/voice/room_io/_output.py
@@ -23,7 +23,7 @@ def __init__(
         sample_rate: int,
         num_channels: int,
         track_publish_options: rtc.TrackPublishOptions,
-        queue_size_ms: int = 300_000,  # TODO(long): move buffer to python
+        queue_size_ms: int = 100_000,  # TODO(long): move buffer to python
     ) -> None:
         super().__init__(next_in_chain=None, sample_rate=sample_rate)
         self._room = room
diff --git a/livekit-agents/pyproject.toml b/livekit-agents/pyproject.toml
index d3447c6bff..4bd2114ba3 100644
--- a/livekit-agents/pyproject.toml
+++ b/livekit-agents/pyproject.toml
@@ -24,7 +24,7 @@ classifiers = [
 ]
 dependencies = [
     "click~=8.1",
-    "livekit>=0.21.1",
+    "livekit>=1.0.0",
     "livekit-api>=0.8.2",
     "livekit-protocol>=0.8.2",
     "protobuf>=3",
@@ -38,6 +38,8 @@ dependencies = [
     "docstring_parser>=0.16",
     "eval-type-backport",
     "colorama>=0.4.6",
+    "av>=12.0.0",
+    "numpy>=1.26.0",
 ]
 
 [project.optional-dependencies]
diff --git a/uv.lock b/uv.lock
index 7c0275611e..d4af1006cf 100644
--- a/uv.lock
+++ b/uv.lock
@@ -1130,20 +1130,22 @@ wheels = [
 
 [[package]]
 name = "livekit"
-version = "0.22.0"
+version = "1.0.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
     { name = "aiofiles" },
+    { name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
+    { name = "numpy", version = "2.2.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" },
     { name = "protobuf" },
     { name = "types-protobuf" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/73/4f/8caf991b2e88d8c748c214793463586eddf674a36d3dda723ceec615daf9/livekit-0.22.0.tar.gz", hash = "sha256:273fd75a164fb1be6745ddd624361fba86131be397e629597719f303f2c29730", size = 107242 }
+sdist = { url = "https://files.pythonhosted.org/packages/a5/4f/b851ea99e75c04505ff000cf2f512456271cb47e0192266442f7db3b60d0/livekit-1.0.0.tar.gz", hash = "sha256:550cbbf969c1f3985133fa9dbe899376611b0882684bd163fd4ed9d782dbc9b4", size = 110214 }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/c8/5d/8931c51c070242ac358a983c6f3ad0c17053b0a8ec9d241de6f6bc8aef86/livekit-0.22.0-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:0f747fabcef48390369674508ccdb809bcdaed5f835bec104101fd086891a1e9", size = 10422577 },
-    { url = "https://files.pythonhosted.org/packages/35/1c/8ea960e33b56ecc1fc05d3685120c1ce61018c5739154de18bd0bf4cfa70/livekit-0.22.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:d05ada7631c7598c309aba76d45a2e2e28d02290877724f470b558d5ca38c11a", size = 9100730 },
-    { url = "https://files.pythonhosted.org/packages/cd/02/a3b44843c38d2272a9cad2e7a3bfc9195a89e15f98d4f0e387c668c804a9/livekit-0.22.0-py3-none-manylinux_2_28_aarch64.whl", hash = "sha256:727efd7d895df3cb533869fbc1db5c97bac7b411cb1056b22d8949b75115d824", size = 10156999 },
-    { url = "https://files.pythonhosted.org/packages/be/16/3daaaef93a348e33d9e7b6c6c66005dba2f3b384ffa858aa04d177848e2e/livekit-0.22.0-py3-none-manylinux_2_28_x86_64.whl", hash = "sha256:c37da83a9f94c3f582d44d574870dfd21d477e72fe938d4a814476d600eb7bb2", size = 11656537 },
-    { url = "https://files.pythonhosted.org/packages/b3/de/7db07aaa60d5c0d04490510e8e13dfac07e2d32e7b8631cc5e8c04284bdf/livekit-0.22.0-py3-none-win_amd64.whl", hash = "sha256:c3bbc54d187363cc2a654945f7917c50591759790fcc3092d3ac7d233e9dac63", size = 11005658 },
+    { url = "https://files.pythonhosted.org/packages/be/2f/25ce6f400fd7eacdcf8fa10afa30b3aeabc17a632ddac5c3c350d8f0d0bd/livekit-1.0.0-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:aaec101382b3e4063298f86ec00ddcd9dd90be76f8447f546d3b09e55dbf6870", size = 10425048 },
+    { url = "https://files.pythonhosted.org/packages/e8/52/1e38d85c3f101a9c1785e3a8bda4560653c3d967207919cd0c273bce6eff/livekit-1.0.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:68d374ba0316e56a6b9ee14c4e38d9e8e4087e3b7d3e92ab3989dd5e64e12f59", size = 9103201 },
+    { url = "https://files.pythonhosted.org/packages/a0/b7/afce00128683fb0b9d2b58b32e3d18b275a16223f9ad0b506ad7bcba0cb3/livekit-1.0.0-py3-none-manylinux_2_28_aarch64.whl", hash = "sha256:12c3b22a8045d78c468521b128b8e09d352bcff9ed89568c9afb22203d2af9c3", size = 10159472 },
+    { url = "https://files.pythonhosted.org/packages/cb/e9/85974b9c0d56d8166e8881bc38cf45d86903f68d0e88f08828edc5cb8139/livekit-1.0.0-py3-none-manylinux_2_28_x86_64.whl", hash = "sha256:dcfdd567f12625200fb4b8f603af51f93712aa9b3260a5231bfd9538c72144af", size = 11659007 },
+    { url = "https://files.pythonhosted.org/packages/12/96/16180feccc50b90eff00be63fb81262b3ad4d2a5bcb093c7956a3265804c/livekit-1.0.0-py3-none-win_amd64.whl", hash = "sha256:d4391c81fae3f7b078507d01505ffe7c755378390e611adf6bd9bd7cef4b0480", size = 11008157 },
 ]
 
 [[package]]
@@ -1151,6 +1153,7 @@ name = "livekit-agents"
 source = { editable = "livekit-agents" }
 dependencies = [
     { name = "aiohttp" },
+    { name = "av" },
     { name = "click" },
     { name = "colorama" },
     { name = "docstring-parser" },
@@ -1158,6 +1161,8 @@ dependencies = [
     { name = "livekit" },
     { name = "livekit-api" },
     { name = "livekit-protocol" },
+    { name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
+    { name = "numpy", version = "2.2.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" },
     { name = "protobuf" },
     { name = "psutil" },
     { name = "pyjwt" },
@@ -1243,12 +1248,13 @@ turn-detector = [
 [package.metadata]
 requires-dist = [
     { name = "aiohttp", specifier = "~=3.10" },
+    { name = "av", specifier = ">=12.0.0" },
     { name = "av", marker = "extra == 'codecs'", specifier = ">=12.0.0" },
     { name = "click", specifier = "~=8.1" },
     { name = "colorama", specifier = ">=0.4.6" },
     { name = "docstring-parser", specifier = ">=0.16" },
     { name = "eval-type-backport" },
-    { name = "livekit", specifier = ">=0.21.1" },
+    { name = "livekit", specifier = ">=1.0.0" },
     { name = "livekit-api", specifier = ">=0.8.2" },
     { name = "livekit-plugins-anthropic", marker = "extra == 'anthropic'", editable = "livekit-plugins/livekit-plugins-anthropic" },
     { name = "livekit-plugins-assemblyai", marker = "extra == 'assemblyai'", editable = "livekit-plugins/livekit-plugins-assemblyai" },
@@ -1272,6 +1278,7 @@ requires-dist = [
     { name = "livekit-plugins-speechmatics", marker = "extra == 'speechmatics'", editable = "livekit-plugins/livekit-plugins-speechmatics" },
     { name = "livekit-plugins-turn-detector", marker = "extra == 'turn-detector'", editable = "livekit-plugins/livekit-plugins-turn-detector" },
     { name = "livekit-protocol", specifier = ">=0.8.2" },
+    { name = "numpy", specifier = ">=1.26.0" },
     { name = "numpy", marker = "extra == 'codecs'", specifier = ">=1.26.0" },
     { name = "pillow", marker = "extra == 'images'", specifier = ">=10.3.0" },
     { name = "protobuf", specifier = ">=3" },


Reviews:
user: login: theomonnom
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-28T01:08:08Z
user: login: theomonnom
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-28T01:10:20Z
user: login: longcw
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T03:50:26Z
user: login: Shubhrakanti
body: Awesome!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T04:24:38Z

Review comments:
diff_hunk: @@ -23,7 +23,7 @@ def __init__(
         sample_rate: int,
         num_channels: int,
         track_publish_options: rtc.TrackPublishOptions,
-        queue_size_ms: int = 300_000,  # TODO(long): move buffer to python
+        queue_size_ms: int = 100_000,  # TODO(long): move buffer to python
path: livekit-agents/livekit/agents/voice/room_io/_output.py
user: login: theomonnom
body: I was getting a panic on the Rust side due to this buffer (int overflow?)
created_at: 2025-03-28T01:08:08Z
author_association: MEMBER
diff_hunk: @@ -23,7 +23,7 @@ def __init__(
         sample_rate: int,
         num_channels: int,
         track_publish_options: rtc.TrackPublishOptions,
-        queue_size_ms: int = 300_000,  # TODO(long): move buffer to python
+        queue_size_ms: int = 100_000,  # TODO(long): move buffer to python
path: livekit-agents/livekit/agents/voice/room_io/_output.py
user: login: theomonnom
body: 300000*48000/1000 (4294967295)
The mul op is before the 1000 div, so I got a uint overflow


created_at: 2025-03-28T01:10:19Z
author_association: MEMBER
diff_hunk: @@ -23,7 +23,7 @@ def __init__(
         sample_rate: int,
         num_channels: int,
         track_publish_options: rtc.TrackPublishOptions,
-        queue_size_ms: int = 300_000,  # TODO(long): move buffer to python
+        queue_size_ms: int = 100_000,  # TODO(long): move buffer to python
path: livekit-agents/livekit/agents/voice/room_io/_output.py
user: login: longcw
body: It was increased for long agent speech for transcription sync, but now it's okay to reduce it back as there is start_time annotation.

created_at: 2025-03-28T03:50:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -907,7 +907,7 @@ def _on_first_frame(_: asyncio.Future) -> None:
             text_out.first_text_fut.add_done_callback(_on_first_frame)
 
         # start to execute tools (only after play())
-        exe_task, fnc_outputs = perform_tool_executions(
+        exe_task, tool_output = perform_tool_executions(
path: livekit-agents/livekit/agents/voice/agent_activity.py
user: login: Shubhrakanti
body: Nice I like keeping the naming consistent! If possible in separate PR to make this one easier to read.
created_at: 2025-03-28T04:18:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -968,11 +968,15 @@ def _on_first_frame(_: asyncio.Future) -> None:
         log_event("playout completed", speech_id=speech_handle.id)
 
         speech_handle._mark_playout_done()  # mark the playout done before waiting for the tool execution  # noqa: E501
+
+        tool_output.first_tool_fut.add_done_callback(
+            lambda _: self._session._update_agent_state(AgentState.THINKING)
+        )
path: livekit-agents/livekit/agents/voice/agent_activity.py
user: login: Shubhrakanti
body: What does this have to do with background audio? 
created_at: 2025-03-28T04:21:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -968,11 +968,15 @@ def _on_first_frame(_: asyncio.Future) -> None:
         log_event("playout completed", speech_id=speech_handle.id)
 
         speech_handle._mark_playout_done()  # mark the playout done before waiting for the tool execution  # noqa: E501
+
+        tool_output.first_tool_fut.add_done_callback(
+            lambda _: self._session._update_agent_state(AgentState.THINKING)
+        )
path: livekit-agents/livekit/agents/voice/agent_activity.py
user: login: Shubhrakanti
body: Ah I see this fixes a separate issue of the AgentState not being populated correctly with tool calls.
created_at: 2025-03-28T04:22:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -144,3 +150,38 @@ def flush(self) -> list[rtc.AudioFrame]:
                 samples_per_channel=len(self._buf) // 2,
             )
         ]
+
+
+async def audio_frames_from_file(
+    file_path: str, sample_rate: int = 48000, num_channels: int = 1
+) -> AsyncGenerator[rtc.AudioFrame, None]:
+    """
+    Decode the audio file into rtc.AudioFrame instances and yield them as an async iterable.
+    Args:
+        file_path (str): The path to the audio file.
+        sample_rate (int, optional): Desired sample rate. Defaults to 48000.
+        num_channels (int, optional): Number of channels (1 for mono, 2 for stereo). Defaults to 1.
+    Returns:
+        AsyncIterable[rtc.AudioFrame]: An async iterable that yields decoded AudioFrame
+    """
+    decoder = AudioStreamDecoder(sample_rate=sample_rate, num_channels=num_channels)
+
+    async def file_reader():
+        async with aiofiles.open(file_path, mode="rb") as f:
+            while True:
+                chunk = await f.read(4096)
+                if not chunk:
+                    break
+
+                decoder.push(chunk)
+
+        decoder.end_input()
+        await decoder.aclose()
+
+    reader_task = asyncio.create_task(file_reader())
+
+    try:
+        async for frame in decoder:
+            yield frame
+    finally:
+        await cancel_and_wait(reader_task)
path: livekit-agents/livekit/agents/utils/audio.py
user: login: Shubhrakanti
body: This is so beautiful. You're so cracked.
created_at: 2025-03-28T04:24:11Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-28T01:07:56Z
body: <!-- changeset-checker -->
### ✅ Changeset File Detected

The following changeset entries were found:

- `patch` - `livekit-agents`

**Change description:**
add BackgroundAudio utility & default sounds  (#1771)
author_association: CONTRIBUTOR
user: login: theomonnom
created_at: 2025-03-28T01:11:46Z
body: I'm still chasing a bug where the thinking audio isn't being fully played
author_association: MEMBER
user: login: toddsor
created_at: 2025-04-01T08:56:05Z
body: After this commit I am getting the following when checking out the project on Win 11. There is a space after the '.gitattributes' file name. Is this because of a os issue? Something on my end? I have win32 long file names enabled.

```
Cloning into 'C:\Projects\playground\livekit\agents'...
remote: Enumerating objects: 29137, done.        
remote: Counting objects: 100% (1749/1749), done.        
remote: Compressing objects: 100% (653/653), done.        
remote: Total 29137 (delta 1455), reused 1105 (delta 1090), pack-reused 27388 (from 3)        
Receiving objects: 100% (29137/29137), 10.72 MiB | 6.42 MiB/s, done.
Resolving deltas: 100% (17531/17531), done.
error: invalid path 'livekit-agents/livekit/agents/resources/.gitattributes '
fatal: unable to checkout working tree
```
author_association: NONE