Repository: pipecat-ai/pipecat

Issue:
Improvements to MarkdownTextFilter
### Problem Statement

I was having a couple of problems with how things were pronounced, so used the `MarkdownTextFilter` to solve:
* A 10 x 10 unit should be "10 by 10" not "10 x 10" when pronounced 
* Do not include "https://" when reading off websites, e.g. github.com vs https://github.com/

### Proposed Solution

Just added 
```
# Make dimensions more readable
filtered_text = re.sub(r'(\d+)x(\d+)', r'\1 by \2', filtered_text)

# Make links more readable
filtered_text = re.sub(r'https?://', '', filtered_text)
```
I don't know if they should be optional, or always stay. Happy to create the PR for this, if approved

### Alternative Solutions

_No response_

### Additional Context

_No response_

### Would you be willing to help implement this feature?

- [x] Yes, I'd like to contribute
- [ ] No, I'm just suggesting

Discussion:
@markbackman: There are endless cases to handle like this. The MarkdownTextFilter is to remove the most common cases that also break the TTS service. For example, reading tables or repeat characters can cause noticeable audio issues; this varies by TTS provider.

We would be open to a change for links, as they're common enough. But the dimensions fall into the endless use cases category. It might be better to write your own custom filter for cases like this.@marctorsoc: > There are endless cases to handle like this. The MarkdownTextFilter is to remove the most common cases that also break the TTS service. For example, reading tables or repeat characters can cause noticeable audio issues; this varies by TTS provider.
> 
> We would be open to a change for links, as they're common enough. But the dimensions fall into the endless use cases category. It might be better to write your own custom filter for cases like this.

I agree with you on the dimensions. Will add a PR next week for the links, when I have the time. Would you be open to make email addresses always lowercase? Right now I have this
```
        ## Make links more readable
        filtered_text = re.sub(r"https?://", "", filtered_text)

        ## Make email addresses lowercase
        filtered_text = re.sub(
            r"([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})",
            lambda m: m.group(1).lower(),
            filtered_text,
        )
```@markbackman: Regarding links, that'd be great.

Regarding emails, is there an issue with TTS providers for that? The goal of the MarkdownTextFilter is to make text more readable. Ideally, it would only process text when needed to improve TTS performance.@marctorsoc: yeah so if I tell an email address, sometimes the STT will put it in uppercase, and then the LLM will keep it like that when calling a tool or answering to the user. But yeah, not really affecting TTS performance. So won't do in the PR@marctorsoc: @markbackman please see https://github.com/pipecat-ai/pipecat/pull/1862@markbackman: Thanks for the submission! 🙌 

Solution patch:
diff --git a/src/pipecat/services/elevenlabs/tts.py b/src/pipecat/services/elevenlabs/tts.py
index 4b48a3d3..6f658489 100644
--- a/src/pipecat/services/elevenlabs/tts.py
+++ b/src/pipecat/services/elevenlabs/tts.py
@@ -388,7 +388,7 @@ class ElevenLabsTTSService(AudioContextWordTTSService):
             msg = json.loads(message)
             # Check if this message belongs to the current context
             # The default context may return null/None for context_id
-            received_ctx_id = msg.get("context_id")
+            received_ctx_id = msg.get("contextId")
             if (
                 self._context_id is not None
                 and received_ctx_id is not None
@@ -408,7 +408,7 @@ class ElevenLabsTTSService(AudioContextWordTTSService):
                 word_times = calculate_word_times(msg["alignment"], self._cumulative_time)
                 await self.add_word_timestamps(word_times)
                 self._cumulative_time = word_times[-1][1]
-            if msg.get("is_final"):
+            if msg.get("isFinal"):
                 logger.trace(f"Received final message for context {received_ctx_id}")
                 # Context has finished
                 if self._context_id == received_ctx_id:
diff --git a/src/pipecat/utils/text/markdown_text_filter.py b/src/pipecat/utils/text/markdown_text_filter.py
index 6f5e16bd..5ec960ad 100644
--- a/src/pipecat/utils/text/markdown_text_filter.py
+++ b/src/pipecat/utils/text/markdown_text_filter.py
@@ -100,6 +100,9 @@ class MarkdownTextFilter(BaseTextFilter):
             # Restore leading and trailing spaces
             filtered_text = re.sub("§", " ", filtered_text)
 
+            ## Make links more readable
+            filtered_text = re.sub(r"https?://", "", filtered_text)
+
             return filtered_text
         else:
             return text


Test patch:
diff --git a/tests/test_markdown_text_filter.py b/tests/test_markdown_text_filter.py
index a82a8581..d1cd79a4 100644
--- a/tests/test_markdown_text_filter.py
+++ b/tests/test_markdown_text_filter.py
@@ -137,6 +137,18 @@ class TestMarkdownTextFilter(unittest.IsolatedAsyncioTestCase):
                 result, expected, f"Newline handling failed for:\n{input_text}\nGot:\n{result}"
             )
 
+    async def test_links_cleaning(self):
+        """Test cleaning of links and URLs, i.e. https?:// is removed."""
+        test_cases = {
+            "Please check http://example.com": "Please check example.com",
+            "Visit https://www.google.com for more": "Visit www.google.com for more",
+            "No link here": "No link here",  # No link to clean
+        }
+
+        for input_text, expected in test_cases.items():
+            result = await self.filter.filter(input_text)
+            self.assertEqual(result, expected, f"Link cleaning failed for: '{input_text}'")
+
     async def test_numbered_list_marker_handling(self):
         """Test handling of numbered lists with the special §NUM§ marker."""
         test_cases = {