Repository: zulip/zulip

Issue:
Youtube preview overrides markdown link titles
![image](https://user-images.githubusercontent.com/229422/89907165-98c47c80-dbec-11ea-8c23-fd89960ddda6.png)

Doesn't seem to happen for previews of sites.


Discussion:
Hello @zulip/server-markdown members, this issue was labeled with the "area: markdown" label, so you may want to check it out!

<!-- areaLabelAddition -->
I'd like to work on this. Can someone guide me as to where I can get started?
Check out the [Zulip developer documentation](https://zulip.readthedocs.io/en/latest/development/index.html).

The relevant code seems to be:

https://github.com/zulip/zulip/blob/0d6047840b28efbd9d51397958bc410e1f4399c7/zerver/lib/markdown/__init__.py#L1231-L1233
@aero31aero can likely help investigate.  
As @Gittenburg said, the bug is in L1233 above. The easiest fix here would be something like:

```py
# replace title of link if 
def maybe_replace_link_title(found_url, title):
    (url, text) = found_url.result
    if text == url:
        found_url.family.child.text = title
```

This same bug should be present with vimeo links too.

For easier manual testing, just comment this check:

```py

            if not self.md.url_embed_preview_enabled:
                continue
```
Slightly long term, do we really want to modify the original link text and not add the title to the preview like we do for our Twitter previews?
Yeah I think having the title above the preview would be better.
@zulipbot claim
Welcome to Zulip, @akshatdalton! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:
* Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
* [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!
I'd like to work on this. Could someone guide me to get started?
@zulipbot claim
@zulipbot claim
Hello @dynamoh, it looks like someone has already claimed this issue! Since we believe multiple assignments to the same issue may cause some confusion, we encourage you to search for other unclaimed issues to work on. However, you can always reclaim this issue if no one is working on it.

We look forward to your valuable contributions!
I see that this is already being worked on. Is there a slack channel I can join?
No, we use Zulip: https://zulip.readthedocs.io/en/latest/contributing/chat-zulip-org.html

Resolving patch:
diff --git a/zerver/lib/markdown/__init__.py b/zerver/lib/markdown/__init__.py
--- a/zerver/lib/markdown/__init__.py
+++ b/zerver/lib/markdown/__init__.py
@@ -1234,13 +1234,19 @@ def run(self, root: Element) -> None:
                 if youtube is not None:
                     title = self.youtube_title(extracted_data)
                     if title is not None:
-                        found_url.family.child.text = title
+                        if url == text:
+                            found_url.family.child.text = title
+                        else:
+                            found_url.family.child.text = text
                     continue
                 self.add_embed(root, url, extracted_data)
                 if self.vimeo_id(url):
                     title = self.vimeo_title(extracted_data)
                     if title:
-                        found_url.family.child.text = title
+                        if url == text:
+                            found_url.family.child.text = title
+                        else:
+                            found_url.family.child.text = text
 
 class Timestamp(markdown.inlinepatterns.Pattern):
     def handleMatch(self, match: Match[str]) -> Optional[Element]:


Test patch:
diff --git a/zerver/tests/test_link_embed.py b/zerver/tests/test_link_embed.py
--- a/zerver/tests/test_link_embed.py
+++ b/zerver/tests/test_link_embed.py
@@ -741,3 +741,34 @@ def test_youtube_url_title_replaces_url(self) -> None:
         msg.refresh_from_db()
         expected_content = '<p><a href="https://www.youtube.com/watch?v=eSJTXC7Ixgg">YouTube - Clearer Code at Scale - Static Types at Zulip and Dropbox</a></p>\n<div class="youtube-video message_inline_image"><a data-id="eSJTXC7Ixgg" href="https://www.youtube.com/watch?v=eSJTXC7Ixgg"><img src="https://i.ytimg.com/vi/eSJTXC7Ixgg/default.jpg"></a></div>'
         self.assertEqual(expected_content, msg.rendered_content)
+
+    @override_settings(INLINE_URL_EMBED_PREVIEW=True)
+    def test_custom_title_replaces_youtube_url_title(self) -> None:
+        url = '[Youtube link](https://www.youtube.com/watch?v=eSJTXC7Ixgg)'
+        with mock_queue_publish('zerver.lib.actions.queue_json_publish'):
+            msg_id = self.send_personal_message(
+                self.example_user('hamlet'),
+                self.example_user('cordelia'),
+                content=url,
+            )
+        msg = Message.objects.select_related("sender").get(id=msg_id)
+        event = {
+            'message_id': msg_id,
+            'urls': [url],
+            'message_realm_id': msg.sender.realm_id,
+            'message_content': url}
+
+        mocked_data = {'title': 'Clearer Code at Scale - Static Types at Zulip and Dropbox'}
+        mocked_response = mock.Mock(side_effect=self.create_mock_response(url))
+        with self.settings(TEST_SUITE=False, CACHES=TEST_CACHES):
+            with mock.patch('requests.get', mocked_response), self.assertLogs(level='INFO') as info_logs:
+                with mock.patch('zerver.lib.markdown.link_preview.link_embed_data_from_cache',
+                                lambda *args, **kwargs: mocked_data):
+                    FetchLinksEmbedData().consume(event)
+            self.assertTrue(
+                'INFO:root:Time spent on get_link_embed_data for [Youtube link](https://www.youtube.com/watch?v=eSJTXC7Ixgg):' in info_logs.output[0]
+            )
+
+        msg.refresh_from_db()
+        expected_content = '<p><a href="https://www.youtube.com/watch?v=eSJTXC7Ixgg">Youtube link</a></p>\n<div class="youtube-video message_inline_image"><a data-id="eSJTXC7Ixgg" href="https://www.youtube.com/watch?v=eSJTXC7Ixgg"><img src="https://i.ytimg.com/vi/eSJTXC7Ixgg/default.jpg"></a></div>'
+        self.assertEqual(expected_content, msg.rendered_content)