Repository: home-assistant/core
Pull request #141387: Add start conversation support to ESPHome
Author: login: synesthesiam
Labels: name: cla-signed
name: small-pr
description: PRs with less than 30 lines.
name: new-feature
name: integration: esphome
name: has-tests
name: by-code-owner
name: Quality Scale: No score
name: Quality Scale: internal
name: integration: assist_satellite
Created: 2025-03-25T15:40:21Z | Merged: 2025-03-25T18:38:52Z

<!--
  You are amazing! Thanks for contributing to our project!
  Please, DO NOT DELETE ANY TEXT from this template! (unless instructed).
-->
## Breaking change
<!--
  If your PR contains a breaking change for existing users, it is important
  to tell them what breaks, how to make it work again and why we did this.
  This piece of text is published with the release notes, so it helps if you
  write it towards our users, not us.
  Note: Remove this section if this PR is NOT a breaking change.
-->


## Proposed change
<!--
  Describe the big picture of your changes here to communicate to the
  maintainers why we should accept this pull request. If it fixes a bug
  or resolves a feature request, be sure to link to that issue in the
  additional information section.
-->
Adds support for `start_conversation` to ESPHome Assist satellite entities (architecture discussion: https://github.com/home-assistant/architecture/discussions/1143)

## Type of change
<!--
  What type of change does your PR introduce to Home Assistant?
  NOTE: Please, check only 1! box!
  If your PR requires multiple boxes to be checked, you'll most likely need to
  split it into multiple PRs. This makes things easier and faster to code review.
-->

- [ ] Dependency upgrade
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New integration (thank you!)
- [x] New feature (which adds functionality to an existing integration)
- [ ] Deprecation (breaking change to happen in the future)
- [ ] Breaking change (fix/feature causing existing functionality to break)
- [ ] Code quality improvements to existing code or addition of tests

## Additional information
<!--
  Details are important, and help maintainers processing your PR.
  Please be sure to fill out additional details, if applicable.
-->

- This PR fixes or closes issue: fixes #
- This PR is related to issue: 
- Link to documentation pull request: 
- Link to developer documentation pull request: 
- Link to frontend pull request: 

## Checklist
<!--
  Put an `x` in the boxes that apply. You can also fill these out after
  creating the PR. If you're unsure about any of them, don't hesitate to ask.
  We're here to help! This is simply a reminder of what we are going to look
  for before merging your code.
-->

- [ ] The code change is tested and works locally.
- [ ] Local tests pass. **Your PR cannot be merged unless tests pass**
- [ ] There is no commented out code in this PR.
- [ ] I have followed the [development checklist][dev-checklist]
- [ ] I have followed the [perfect PR recommendations][perfect-pr]
- [ ] The code has been formatted using Ruff (`ruff format homeassistant tests`)
- [ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

- [ ] Documentation added/updated for [www.home-assistant.io][docs-repository]

If the code communicates with devices, web services, or third-party tools:

- [ ] The [manifest file][manifest-docs] has all fields filled out correctly.  
      Updated and included derived files by running: `python3 -m script.hassfest`.
- [ ] New or updated dependencies have been added to `requirements_all.txt`.  
      Updated by running `python3 -m script.gen_requirements_all`.
- [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

<!--
  This project is very active and we have a high turnover of pull requests.

  Unfortunately, the number of incoming pull requests is higher than what our
  reviewers can review and merge so there is a long backlog of pull requests
  waiting for review. You can help here!
  
  By reviewing another pull request, you will help raise the code quality of
  that pull request and the final review will be faster. This way the general
  pace of pull request reviews will go up and your wait time will go down.
  
  When picking a pull request to review, try to choose one that hasn't yet
  been reviewed.

  Thanks for helping out!
-->

To help with the load of incoming pull requests:

- [ ] I have reviewed two other [open pull requests][prs] in this repository.

[prs]: https://github.com/home-assistant/core/pulls?q=is%3Aopen+is%3Apr+-author%3A%40me+-draft%3Atrue+-label%3Awaiting-for-upstream+sort%3Acreated-desc+review%3Anone+-status%3Afailure

<!--
  Thank you for contributing <3

  Below, some useful links you could explore:
-->
[dev-checklist]: https://developers.home-assistant.io/docs/development_checklist/
[manifest-docs]: https://developers.home-assistant.io/docs/creating_integration_manifest/
[quality-scale]: https://developers.home-assistant.io/docs/integration_quality_scale_index/
[docs-repository]: https://github.com/home-assistant/home-assistant.io
[perfect-pr]: https://developers.home-assistant.io/docs/review-process/#creating-the-perfect-pr



Diff:
diff --git a/homeassistant/components/esphome/assist_satellite.py b/homeassistant/components/esphome/assist_satellite.py
index fdd16d20d77cd4..4206b545588f2f 100644
--- a/homeassistant/components/esphome/assist_satellite.py
+++ b/homeassistant/components/esphome/assist_satellite.py
@@ -253,6 +253,11 @@ async def async_added_to_hass(self) -> None:
             # Will use media player for TTS/announcements
             self._update_tts_format()
 
+        if feature_flags & VoiceAssistantFeature.START_CONVERSATION:
+            self._attr_supported_features |= (
+                assist_satellite.AssistSatelliteEntityFeature.START_CONVERSATION
+            )
+
         # Update wake word select when config is updated
         self.async_on_remove(
             self.entry_data.async_register_assist_satellite_set_wake_word_callback(
@@ -342,6 +347,23 @@ async def async_announce(
 
         Should block until the announcement is done playing.
         """
+        await self._do_announce(announcement, run_pipeline_after=False)
+
+    async def async_start_conversation(
+        self, start_announcement: assist_satellite.AssistSatelliteAnnouncement
+    ) -> None:
+        """Start a conversation from the satellite."""
+        await self._do_announce(start_announcement, run_pipeline_after=True)
+
+    async def _do_announce(
+        self,
+        announcement: assist_satellite.AssistSatelliteAnnouncement,
+        run_pipeline_after: bool,
+    ) -> None:
+        """Announce media on the satellite.
+
+        Optionally run a voice pipeline after the announcement has finished.
+        """
         _LOGGER.debug(
             "Waiting for announcement to finished (message=%s, media_id=%s)",
             announcement.message,
@@ -374,7 +396,10 @@ async def async_announce(
                 media_id = async_process_play_media_url(self.hass, proxy_url)
 
         await self.cli.send_voice_assistant_announcement_await_response(
-            media_id, _ANNOUNCEMENT_TIMEOUT_SEC, announcement.message
+            media_id,
+            _ANNOUNCEMENT_TIMEOUT_SEC,
+            announcement.message,
+            start_conversation=run_pipeline_after,
         )
 
     async def handle_pipeline_start(
diff --git a/tests/components/esphome/test_assist_satellite.py b/tests/components/esphome/test_assist_satellite.py
index 329a7b5179a346..081070b23f1c31 100644
--- a/tests/components/esphome/test_assist_satellite.py
+++ b/tests/components/esphome/test_assist_satellite.py
@@ -25,7 +25,12 @@
 )
 import pytest
 
-from homeassistant.components import assist_satellite, conversation, tts
+from homeassistant.components import (
+    assist_pipeline,
+    assist_satellite,
+    conversation,
+    tts,
+)
 from homeassistant.components.assist_pipeline import PipelineEvent, PipelineEventType
 from homeassistant.components.assist_satellite import (
     AssistSatelliteConfiguration,
@@ -1160,7 +1165,7 @@ async def test_announce_supported_features(
         Awaitable[MockESPHomeDevice],
     ],
 ) -> None:
-    """Test that the announce supported feature is set by flags."""
+    """Test that the announce supported feature is not set by default."""
     mock_device: MockESPHomeDevice = await mock_esphome_device(
         mock_client=mock_client,
         entity_info=[],
@@ -1207,11 +1212,12 @@ async def test_announce_message(
     done = asyncio.Event()
 
     async def send_voice_assistant_announcement_await_response(
-        media_id: str, timeout: float, text: str
+        media_id: str, timeout: float, text: str, start_conversation: bool
     ):
         assert satellite.state == AssistSatelliteState.RESPONDING
         assert media_id == "http://10.10.10.10:8123/api/tts_proxy/test-token"
         assert text == "test-text"
+        assert not start_conversation
 
         done.set()
 
@@ -1296,10 +1302,11 @@ async def test_announce_media_id(
     done = asyncio.Event()
 
     async def send_voice_assistant_announcement_await_response(
-        media_id: str, timeout: float, text: str
+        media_id: str, timeout: float, text: str, start_conversation: bool
     ):
         assert satellite.state == AssistSatelliteState.RESPONDING
         assert media_id == "https://www.home-assistant.io/proxied.flac"
+        assert not start_conversation
 
         done.set()
 
@@ -1338,6 +1345,234 @@ async def send_voice_assistant_announcement_await_response(
         )
 
 
+async def test_start_conversation_supported_features(
+    hass: HomeAssistant,
+    mock_client: APIClient,
+    mock_esphome_device: Callable[
+        [APIClient, list[EntityInfo], list[UserService], list[EntityState]],
+        Awaitable[MockESPHomeDevice],
+    ],
+) -> None:
+    """Test that the start conversation supported feature is not set by default."""
+    mock_device: MockESPHomeDevice = await mock_esphome_device(
+        mock_client=mock_client,
+        entity_info=[],
+        user_service=[],
+        states=[],
+        device_info={
+            "voice_assistant_feature_flags": VoiceAssistantFeature.VOICE_ASSISTANT
+        },
+    )
+    await hass.async_block_till_done()
+
+    satellite = get_satellite_entity(hass, mock_device.device_info.mac_address)
+    assert satellite is not None
+
+    assert not (
+        satellite.supported_features & AssistSatelliteEntityFeature.START_CONVERSATION
+    )
+
+
+async def test_start_conversation_message(
+    hass: HomeAssistant,
+    mock_client: APIClient,
+    mock_esphome_device: Callable[
+        [APIClient, list[EntityInfo], list[UserService], list[EntityState]],
+        Awaitable[MockESPHomeDevice],
+    ],
+) -> None:
+    """Test start conversation with message."""
+    mock_device: MockESPHomeDevice = await mock_esphome_device(
+        mock_client=mock_client,
+        entity_info=[],
+        user_service=[],
+        states=[],
+        device_info={
+            "voice_assistant_feature_flags": VoiceAssistantFeature.VOICE_ASSISTANT
+            | VoiceAssistantFeature.SPEAKER
+            | VoiceAssistantFeature.API_AUDIO
+            | VoiceAssistantFeature.ANNOUNCE
+            | VoiceAssistantFeature.START_CONVERSATION
+        },
+    )
+    await hass.async_block_till_done()
+
+    satellite = get_satellite_entity(hass, mock_device.device_info.mac_address)
+    assert satellite is not None
+
+    pipeline = assist_pipeline.Pipeline(
+        conversation_engine="test engine",
+        conversation_language="en",
+        language="en",
+        name="test pipeline",
+        stt_engine="test stt",
+        stt_language="en",
+        tts_engine="test tts",
+        tts_language="en",
+        tts_voice=None,
+        wake_word_entity=None,
+        wake_word_id=None,
+    )
+
+    done = asyncio.Event()
+
+    async def send_voice_assistant_announcement_await_response(
+        media_id: str, timeout: float, text: str, start_conversation: bool
+    ):
+        assert satellite.state == AssistSatelliteState.RESPONDING
+        assert media_id == "http://10.10.10.10:8123/api/tts_proxy/test-token"
+        assert text == "test-text"
+        assert start_conversation
+
+        done.set()
+
+    with (
+        patch(
+            "homeassistant.components.tts.generate_media_source_id",
+            return_value="media-source://bla",
+        ),
+        patch(
+            "homeassistant.components.tts.async_resolve_engine",
+            return_value="tts.cloud_tts",
+        ),
+        patch(
+            "homeassistant.components.tts.async_create_stream",
+            return_value=MockResultStream(hass, "wav", b""),
+        ),
+        patch.object(
+            mock_client,
+            "send_voice_assistant_announcement_await_response",
+            new=send_voice_assistant_announcement_await_response,
+        ),
+        patch(
+            "homeassistant.components.assist_satellite.entity.async_get_pipeline",
+            return_value=pipeline,
+        ),
+    ):
+        async with asyncio.timeout(1):
+            await hass.services.async_call(
+                assist_satellite.DOMAIN,
+                "start_conversation",
+                {"entity_id": satellite.entity_id, "start_message": "test-text"},
+                blocking=True,
+            )
+            await done.wait()
+            assert satellite.state == AssistSatelliteState.IDLE
+
+
+async def test_start_conversation_media_id(
+    hass: HomeAssistant,
+    mock_client: APIClient,
+    mock_esphome_device: Callable[
+        [APIClient, list[EntityInfo], list[UserService], list[EntityState]],
+        Awaitable[MockESPHomeDevice],
+    ],
+    device_registry: dr.DeviceRegistry,
+) -> None:
+    """Test start conversation with media id."""
+    mock_device: MockESPHomeDevice = await mock_esphome_device(
+        mock_client=mock_client,
+        entity_info=[
+            MediaPlayerInfo(
+                object_id="mymedia_player",
+                key=1,
+                name="my media_player",
+                unique_id="my_media_player",
+                supports_pause=True,
+                supported_formats=[
+                    MediaPlayerSupportedFormat(
+                        format="flac",
+                        sample_rate=48000,
+                        num_channels=2,
+                        purpose=MediaPlayerFormatPurpose.ANNOUNCEMENT,
+                        sample_bytes=2,
+                    ),
+                ],
+            )
+        ],
+        user_service=[],
+        states=[],
+        device_info={
+            "voice_assistant_feature_flags": VoiceAssistantFeature.VOICE_ASSISTANT
+            | VoiceAssistantFeature.SPEAKER
+            | VoiceAssistantFeature.API_AUDIO
+            | VoiceAssistantFeature.ANNOUNCE
+            | VoiceAssistantFeature.START_CONVERSATION
+        },
+    )
+    await hass.async_block_till_done()
+
+    dev = device_registry.async_get_device(
+        connections={(dr.CONNECTION_NETWORK_MAC, mock_device.entry.unique_id)}
+    )
+
+    satellite = get_satellite_entity(hass, mock_device.device_info.mac_address)
+    assert satellite is not None
+
+    pipeline = assist_pipeline.Pipeline(
+        conversation_engine="test engine",
+        conversation_language="en",
+        language="en",
+        name="test pipeline",
+        stt_engine="test stt",
+        stt_language="en",
+        tts_engine="test tts",
+        tts_language="en",
+        tts_voice=None,
+        wake_word_entity=None,
+        wake_word_id=None,
+    )
+
+    done = asyncio.Event()
+
+    async def send_voice_assistant_announcement_await_response(
+        media_id: str, timeout: float, text: str, start_conversation: bool
+    ):
+        assert satellite.state == AssistSatelliteState.RESPONDING
+        assert media_id == "https://www.home-assistant.io/proxied.flac"
+        assert start_conversation
+
+        done.set()
+
+    with (
+        patch.object(
+            mock_client,
+            "send_voice_assistant_announcement_await_response",
+            new=send_voice_assistant_announcement_await_response,
+        ),
+        patch(
+            "homeassistant.components.esphome.assist_satellite.async_create_proxy_url",
+            return_value="https://www.home-assistant.io/proxied.flac",
+        ) as mock_async_create_proxy_url,
+        patch(
+            "homeassistant.components.assist_satellite.entity.async_get_pipeline",
+            return_value=pipeline,
+        ),
+    ):
+        async with asyncio.timeout(1):
+            await hass.services.async_call(
+                assist_satellite.DOMAIN,
+                "start_conversation",
+                {
+                    "entity_id": satellite.entity_id,
+                    "start_media_id": "https://www.home-assistant.io/resolved.mp3",
+                },
+                blocking=True,
+            )
+            await done.wait()
+            assert satellite.state == AssistSatelliteState.IDLE
+
+        mock_async_create_proxy_url.assert_called_once_with(
+            hass,
+            dev.id,
+            "https://www.home-assistant.io/resolved.mp3",
+            media_format="flac",
+            rate=48000,
+            channels=2,
+            width=2,
+        )
+
+
 async def test_satellite_unloaded_on_disconnect(
     hass: HomeAssistant,
     mock_client: APIClient,


Reviews:
user: login: MartinHjelmare
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T15:47:41Z
user: login: synesthesiam
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T15:57:14Z
user: login: synesthesiam
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T16:04:48Z
user: login: balloob
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-25T18:35:44Z

Review comments:
diff_hunk: @@ -262,6 +262,8 @@ async def async_internal_start_conversation(
             raise SatelliteBusyError
 
         self._is_announcing = True
+        self._set_state(AssistSatelliteState.RESPONDING)
path: homeassistant/components/assist_satellite/entity.py
user: login: MartinHjelmare
body: Please make a separate PR for the base entity changes.
created_at: 2025-03-25T15:47:21Z
author_association: MEMBER
diff_hunk: @@ -262,6 +262,8 @@ async def async_internal_start_conversation(
             raise SatelliteBusyError
 
         self._is_announcing = True
+        self._set_state(AssistSatelliteState.RESPONDING)
path: homeassistant/components/assist_satellite/entity.py
user: login: MartinHjelmare
body: Is this a fix?
created_at: 2025-03-25T15:47:37Z
author_association: MEMBER
diff_hunk: @@ -262,6 +262,8 @@ async def async_internal_start_conversation(
             raise SatelliteBusyError
 
         self._is_announcing = True
+        self._set_state(AssistSatelliteState.RESPONDING)
path: homeassistant/components/assist_satellite/entity.py
user: login: synesthesiam
body: Yes, forgot to break that out into a separate PR :+1: 
created_at: 2025-03-25T15:57:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -262,6 +262,8 @@ async def async_internal_start_conversation(
             raise SatelliteBusyError
 
         self._is_announcing = True
+        self._set_state(AssistSatelliteState.RESPONDING)
path: homeassistant/components/assist_satellite/entity.py
user: login: synesthesiam
body: Split out here: https://github.com/home-assistant/core/pull/141388
created_at: 2025-03-25T16:04:48Z
author_association: CONTRIBUTOR

PR comments:
user: login: home-assistant[bot]
created_at: 2025-03-25T15:40:29Z
body: 
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (`assist_satellite`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L149) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `assist_satellite` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign assist_satellite` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: home-assistant[bot]
created_at: 2025-03-25T15:40:32Z
body: 
Hey there @ottowinter, @jesserockz, @kbx81, @bdraco, mind taking a look at this pull request as it has been labeled with an integration (`esphome`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L444) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `esphome` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign esphome` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: synesthesiam
created_at: 2025-03-25T16:39:07Z
body: Waiting on: https://github.com/home-assistant/core/pull/141388
author_association: CONTRIBUTOR