Repository: home-assistant/core
Pull request #139810: Track when an LLM expects to continue a conversation
Author: login: balloob
Labels: name: core
name: cla-signed
name: integration: conversation
name: small-pr
description: PRs with less than 30 lines.
name: new-feature
name: has-tests
name: by-code-owner
name: Quality Scale: No score
name: Quality Scale: internal
name: integration: openai_conversation
name: integration: google_generative_ai_conversation
name: integration: ollama
name: integration: anthropic
Created: 2025-03-05T02:53:26Z | Merged: 2025-03-07T03:52:29Z

<!--
  You are amazing! Thanks for contributing to our project!
  Please, DO NOT DELETE ANY TEXT from this template! (unless instructed).
-->
## Breaking change
<!--
  If your PR contains a breaking change for existing users, it is important
  to tell them what breaks, how to make it work again and why we did this.
  This piece of text is published with the release notes, so it helps if you
  write it towards our users, not us.
  Note: Remove this section if this PR is NOT a breaking change.
-->


## Proposed change
<!--
  Describe the big picture of your changes here to communicate to the
  maintainers why we should accept this pull request. If it fixes a bug
  or resolves a feature request, be sure to link to that issue in the
  additional information section.
-->
Chat log can now track if the LLM has requested the conversation to be continued. This data is passed into the ConversationResult.

## Type of change
<!--
  What type of change does your PR introduce to Home Assistant?
  NOTE: Please, check only 1! box!
  If your PR requires multiple boxes to be checked, you'll most likely need to
  split it into multiple PRs. This makes things easier and faster to code review.
-->

- [ ] Dependency upgrade
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New integration (thank you!)
- [x] New feature (which adds functionality to an existing integration)
- [ ] Deprecation (breaking change to happen in the future)
- [ ] Breaking change (fix/feature causing existing functionality to break)
- [ ] Code quality improvements to existing code or addition of tests

## Additional information
<!--
  Details are important, and help maintainers processing your PR.
  Please be sure to fill out additional details, if applicable.
-->

- This PR fixes or closes issue: fixes #
- This PR is related to issue: 
- Link to documentation pull request: 
- Link to developer documentation pull request: 
- Link to frontend pull request: 

## Checklist
<!--
  Put an `x` in the boxes that apply. You can also fill these out after
  creating the PR. If you're unsure about any of them, don't hesitate to ask.
  We're here to help! This is simply a reminder of what we are going to look
  for before merging your code.
-->

- [ ] The code change is tested and works locally.
- [ ] Local tests pass. **Your PR cannot be merged unless tests pass**
- [ ] There is no commented out code in this PR.
- [ ] I have followed the [development checklist][dev-checklist]
- [ ] I have followed the [perfect PR recommendations][perfect-pr]
- [ ] The code has been formatted using Ruff (`ruff format homeassistant tests`)
- [ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

- [ ] Documentation added/updated for [www.home-assistant.io][docs-repository]

If the code communicates with devices, web services, or third-party tools:

- [ ] The [manifest file][manifest-docs] has all fields filled out correctly.  
      Updated and included derived files by running: `python3 -m script.hassfest`.
- [ ] New or updated dependencies have been added to `requirements_all.txt`.  
      Updated by running `python3 -m script.gen_requirements_all`.
- [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

<!--
  This project is very active and we have a high turnover of pull requests.

  Unfortunately, the number of incoming pull requests is higher than what our
  reviewers can review and merge so there is a long backlog of pull requests
  waiting for review. You can help here!
  
  By reviewing another pull request, you will help raise the code quality of
  that pull request and the final review will be faster. This way the general
  pace of pull request reviews will go up and your wait time will go down.
  
  When picking a pull request to review, try to choose one that hasn't yet
  been reviewed.

  Thanks for helping out!
-->

To help with the load of incoming pull requests:

- [ ] I have reviewed two other [open pull requests][prs] in this repository.

[prs]: https://github.com/home-assistant/core/pulls?q=is%3Aopen+is%3Apr+-author%3A%40me+-draft%3Atrue+-label%3Awaiting-for-upstream+sort%3Acreated-desc+review%3Anone+-status%3Afailure

<!--
  Thank you for contributing <3

  Below, some useful links you could explore:
-->
[dev-checklist]: https://developers.home-assistant.io/docs/development_checklist/
[manifest-docs]: https://developers.home-assistant.io/docs/creating_integration_manifest/
[quality-scale]: https://developers.home-assistant.io/docs/integration_quality_scale_index/
[docs-repository]: https://github.com/home-assistant/home-assistant.io
[perfect-pr]: https://developers.home-assistant.io/docs/review-process/#creating-the-perfect-pr



Diff:
diff --git a/homeassistant/components/anthropic/conversation.py b/homeassistant/components/anthropic/conversation.py
index 5511119d377b08..8d3ba5085ee1ac 100644
--- a/homeassistant/components/anthropic/conversation.py
+++ b/homeassistant/components/anthropic/conversation.py
@@ -305,7 +305,9 @@ async def _async_handle_message(
         intent_response = intent.IntentResponse(language=user_input.language)
         intent_response.async_set_speech(response_content.content or "")
         return conversation.ConversationResult(
-            response=intent_response, conversation_id=chat_log.conversation_id
+            response=intent_response,
+            conversation_id=chat_log.conversation_id,
+            continue_conversation=chat_log.continue_conversation,
         )
 
     async def _async_entry_update_listener(
diff --git a/homeassistant/components/conversation/chat_log.py b/homeassistant/components/conversation/chat_log.py
index 19482af1983250..355f423dbb6b46 100644
--- a/homeassistant/components/conversation/chat_log.py
+++ b/homeassistant/components/conversation/chat_log.py
@@ -183,6 +183,25 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return whether the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content.strip().endswith(  # type: ignore[union-attr]
+                (
+                    "?",
+                    ";",  # Greek question mark
+                )
+            )
+        )
+
     @property
     def unresponded_tool_results(self) -> bool:
         """Return if there are unresponded tool results."""
diff --git a/homeassistant/components/google_generative_ai_conversation/conversation.py b/homeassistant/components/google_generative_ai_conversation/conversation.py
index 2c84249dcb3041..f17b3ffd8fcc39 100644
--- a/homeassistant/components/google_generative_ai_conversation/conversation.py
+++ b/homeassistant/components/google_generative_ai_conversation/conversation.py
@@ -471,7 +471,9 @@ async def _async_handle_message(
             " ".join([part.text.strip() for part in response_parts if part.text])
         )
         return conversation.ConversationResult(
-            response=response, conversation_id=chat_log.conversation_id
+            response=response,
+            conversation_id=chat_log.conversation_id,
+            continue_conversation=chat_log.continue_conversation,
         )
 
     async def _async_entry_update_listener(
diff --git a/homeassistant/components/ollama/conversation.py b/homeassistant/components/ollama/conversation.py
index 90e81544f666e0..85daf7420355d9 100644
--- a/homeassistant/components/ollama/conversation.py
+++ b/homeassistant/components/ollama/conversation.py
@@ -292,7 +292,9 @@ async def _async_handle_message(
             )
         intent_response.async_set_speech(chat_log.content[-1].content or "")
         return conversation.ConversationResult(
-            response=intent_response, conversation_id=chat_log.conversation_id
+            response=intent_response,
+            conversation_id=chat_log.conversation_id,
+            continue_conversation=chat_log.continue_conversation,
         )
 
     def _trim_history(self, message_history: MessageHistory, max_messages: int) -> None:
diff --git a/homeassistant/components/openai_conversation/conversation.py b/homeassistant/components/openai_conversation/conversation.py
index cc09ec77c0e3e5..37be41947f7773 100644
--- a/homeassistant/components/openai_conversation/conversation.py
+++ b/homeassistant/components/openai_conversation/conversation.py
@@ -310,7 +310,9 @@ async def _async_handle_message(
         assert type(chat_log.content[-1]) is conversation.AssistantContent
         intent_response.async_set_speech(chat_log.content[-1].content or "")
         return conversation.ConversationResult(
-            response=intent_response, conversation_id=chat_log.conversation_id
+            response=intent_response,
+            conversation_id=chat_log.conversation_id,
+            continue_conversation=chat_log.continue_conversation,
         )
 
     async def _async_entry_update_listener(
diff --git a/tests/components/conversation/test_chat_log.py b/tests/components/conversation/test_chat_log.py
index c0687ebecfb79d..97094740af0df5 100644
--- a/tests/components/conversation/test_chat_log.py
+++ b/tests/components/conversation/test_chat_log.py
@@ -14,6 +14,7 @@
     ConversationInput,
     ConverseError,
     ToolResultContent,
+    UserContent,
     async_get_chat_log,
 )
 from homeassistant.components.conversation.chat_log import DATA_CHAT_LOGS
@@ -643,3 +644,30 @@ async def test_chat_log_reuse(
             assert len(chat_log.content) == 2
             assert chat_log.content[1].role == "user"
             assert chat_log.content[1].content == mock_conversation_input.text
+
+
+async def test_chat_log_continue_conversation(
+    hass: HomeAssistant,
+    mock_conversation_input: ConversationInput,
+) -> None:
+    """Test continue conversation."""
+    with (
+        chat_session.async_get_chat_session(hass) as session,
+        async_get_chat_log(hass, session) as chat_log,
+    ):
+        assert chat_log.continue_conversation is False
+        chat_log.async_add_user_content(UserContent(mock_conversation_input.text))
+        assert chat_log.continue_conversation is False
+        chat_log.async_add_assistant_content_without_tools(
+            AssistantContent(
+                agent_id="mock-agent-id",
+                content="Hey? ",
+            )
+        )
+        chat_log.async_add_assistant_content_without_tools(
+            AssistantContent(
+                agent_id="mock-agent-id",
+                content="Ποιο είναι το αγαπημένο σου χρώμα στα ελληνικά;",
+            )
+        )
+        assert chat_log.continue_conversation is True


Reviews:
user: login: synesthesiam
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T04:12:04Z
user: login: tronikos
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-05T04:59:57Z
user: login: balloob
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T05:06:45Z
user: login: tronikos
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-07T00:40:02Z

Review comments:
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content[-1] == "?"  # type: ignore[union-attr]
path: homeassistant/components/conversation/chat_log.py
user: login: synesthesiam
body: Should you do a `strip()` here in case the LLM adds a newline?
created_at: 2025-03-05T04:12:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
path: homeassistant/components/conversation/chat_log.py
user: login: tronikos
body: s/if/whether/
created_at: 2025-03-05T04:41:56Z
author_association: MEMBER
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content.strip()[-1] == "?"  # type: ignore[union-attr]
path: homeassistant/components/conversation/chat_log.py
user: login: tronikos
body: In Greek the question mark is ";"
created_at: 2025-03-05T04:54:49Z
author_association: MEMBER
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content.strip()[-1] == "?"  # type: ignore[union-attr]
path: homeassistant/components/conversation/chat_log.py
user: login: tronikos
body: This will raise IndexError if the content is empty. Maybe use endswith?
created_at: 2025-03-05T04:56:28Z
author_association: MEMBER
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content.strip()[-1] == "?"  # type: ignore[union-attr]
path: homeassistant/components/conversation/chat_log.py
user: login: tronikos
body: If you extract it to a local variable maybe you can use the  ignore type once?
created_at: 2025-03-05T04:58:29Z
author_association: MEMBER
diff_hunk: @@ -183,6 +183,20 @@ class ChatLog:
     llm_api: llm.APIInstance | None = None
     delta_listener: Callable[[ChatLog, dict], None] | None = None
 
+    @property
+    def continue_conversation(self) -> bool:
+        """Return if the conversation should continue."""
+        if not self.content:
+            return False
+
+        last_msg = self.content[-1]
+
+        return (
+            last_msg.role == "assistant"
+            and last_msg.content is not None  # type: ignore[union-attr]
+            and last_msg.content.strip()[-1] == "?"  # type: ignore[union-attr]
path: homeassistant/components/conversation/chat_log.py
user: login: balloob
body: The problem is that `last_msg.role == "assistant"` is not picked up by MyPy to mean that it can only be an AssistantContent object. So any interaction needs to be guarded.
created_at: 2025-03-05T05:06:45Z
author_association: MEMBER

PR comments:
user: login: home-assistant[bot]
created_at: 2025-03-05T02:53:32Z
body: 
Hey there @shulyaka, mind taking a look at this pull request as it has been labeled with an integration (`anthropic`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L114) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `anthropic` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign anthropic` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: home-assistant[bot]
created_at: 2025-03-05T02:53:33Z
body: 
Hey there @synesthesiam, mind taking a look at this pull request as it has been labeled with an integration (`ollama`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L1071) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `ollama` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign ollama` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: home-assistant[bot]
created_at: 2025-03-05T02:53:33Z
body: 
Hey there @tronikos, mind taking a look at this pull request as it has been labeled with an integration (`google_generative_ai_conversation`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L573) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `google_generative_ai_conversation` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign google_generative_ai_conversation` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: home-assistant[bot]
created_at: 2025-03-05T02:53:33Z
body: 
Hey there @home-assistant/core, @synesthesiam, mind taking a look at this pull request as it has been labeled with an integration (`conversation`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L287) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `conversation` can trigger bot actions by commenting:

- `@home-assistant close` Closes the pull request.
- `@home-assistant rename Awesome new title` Renames the pull request.
- `@home-assistant reopen` Reopen the pull request.
- `@home-assistant unassign conversation` Removes the current integration label and assignees on the pull request, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

</details>


author_association: CONTRIBUTOR
user: login: home-assistant[bot]
created_at: 2025-03-05T04:59:59Z
body: <!-- ReviewDrafterComment -->
Please take a look at the requested changes, and use the **Ready for review** button when you are done, thanks :+1:

[_Learn more about our pull request process._](https://developers.home-assistant.io/docs/review-process#prs-are-being-drafted-when-changes-are-needed)

author_association: CONTRIBUTOR
user: login: arturpragacz
created_at: 2025-03-06T00:28:21Z
body: While cool, this feels very hacky. I don't think we should try to guess whether LLM wants to continue conversation by trying to detect some symbols. This intention should instead be communicated out of band by the LLM itself.

Aside from possible false positives and false negatives, the approach here is going to have some localization challenges. Greek was already mentioned, but there are more: https://en.wikipedia.org/wiki/Question_mark#In_other_languages_and_scripts
author_association: MEMBER
user: login: balloob
created_at: 2025-03-07T03:52:24Z
body: @arturpragacz this is going to already be an improvement (versus nothing). We can fine tune it later by exploring adding a tool call. However, I'm afraid that this might confuse smaller models so will require testing.
author_association: MEMBER