Repository: langchain-ai/langchain (Python)
Date: 2023-05-09 21:23:12+00:00
Task type: Bug Fix

Issue:
WhatsAppChatLoader doesn't work on chats exported from WhatsApp
### System Info

langchain               0.0.158
Mac OS M1
Python 3.11

### Who can help?

@ey

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Use 'Export Chat' feature on WhatsApp.
2. Observe this format for the txt file
```
[11/8/21, 9:41:32 AM] User name: Message text
```

The regular expression used by WhatsAppChatLoader doesn't parse this format successfully

### Expected behavior

Parsing fails


Issue discussion:
it also doesn't work on Ukrainian date format, e.g.

```
[05.05.23, 15:45:46] User: text
```

---

I used the following input formats:
```
[05.05.23, 15:48:11] James: Hi here
[11/8/21, 9:41:32 AM] User name: Message 123
1/23/23, 3:19 AM - User 2: Bye!
1/23/23, 3:22_AM - User 1: And let me know if anything changes
```

New regex that seems to work with all three:
```python

message_line_regex = r"""
    \[?                      # Optional opening square bracket
    (                        # Start of group 1
        \d{1,2}              # Match 1-2 digits for the day
        [\/.]                # Match a forward slash or period as the date separator
        \d{1,2}              # Match 1-2 digits for the month
        [\/.]                # Match a forward slash or period as the date separator
        \d{2,4}              # Match 2-4 digits for the year
        ,\s                  # Match a comma and a space
        \d{1,2}              # Match 1-2 digits for the hour
        :\d{2}               # Match 2 digits for the minutes
        (?:                  # Optional group for seconds
            :\d{2}           # Match 2 digits for the seconds
        )?                   # Make seconds group optional
        (?:[ _](?:AM|PM))?   # Optional space or underscore and AM/PM suffix for 12-hour format
    )                        # End of group 1
    \]?                      # Optional closing square bracket
    [\s-]*                   # Match any number of spaces or hyphens
    ([\w\s]+)                # Match and capture one or more word characters or spaces as group 2 (the sender)
    [:]+                     # Match one or more colons
    \s                       # Match a single space
    (.+)                     # Match and capture one or more of any character as group 3 (the message content)
"""
```

I can make a PR, but should I test any other formats before?

Solution patch:
diff --git a/langchain/document_loaders/whatsapp_chat.py b/langchain/document_loaders/whatsapp_chat.py
--- a/langchain/document_loaders/whatsapp_chat.py
+++ b/langchain/document_loaders/whatsapp_chat.py
@@ -44,7 +44,7 @@ def load(self) -> List[Document]:
             )
             \]?
             [\s-]*
-            ([\w\s]+)
+            ([~\w\s]+)
             [:]+
             \s
             (.+)


Test patch:
diff --git a/tests/integration_tests/document_loaders/test_whatsapp_chat.py b/tests/integration_tests/document_loaders/test_whatsapp_chat.py
--- a/tests/integration_tests/document_loaders/test_whatsapp_chat.py
+++ b/tests/integration_tests/document_loaders/test_whatsapp_chat.py
@@ -16,4 +16,5 @@ def test_whatsapp_chat_loader() -> None:
         "User name on 11/8/21, 9:41:32 AM: Message 123\n\n"
         "User 2 on 1/23/23, 3:19 AM: Bye!\n\n"
         "User 1 on 1/23/23, 3:22_AM: And let me know if anything changes\n\n"
+        "~ User name 2 on 1/24/21, 12:41:03 PM: Of course!\n\n"
     )
diff --git a/tests/integration_tests/examples/whatsapp_chat.txt b/tests/integration_tests/examples/whatsapp_chat.txt
--- a/tests/integration_tests/examples/whatsapp_chat.txt
+++ b/tests/integration_tests/examples/whatsapp_chat.txt
@@ -1,4 +1,5 @@
 [05.05.23, 15:48:11] James: Hi here
 [11/8/21, 9:41:32 AM] User name: Message 123
 1/23/23, 3:19 AM - User 2: Bye!
-1/23/23, 3:22_AM - User 1: And let me know if anything changes
\ No newline at end of file
+1/23/23, 3:22_AM - User 1: And let me know if anything changes
+[1/24/21, 12:41:03 PM] ~ User name 2: Of course!
\ No newline at end of file