Repository: zulip/zulip

Issue:
Support for Youtube playlist embeds
Some youtube playlist urls are not recognized by zulip for embedding.

Works:
https://www.youtube.com/watch?v=OPf0YbXqDm0&list=PLMC9KNkIncKtPzgY-5rmhvj7fax8fdxoj

Doesn't work:
https://www.youtube.com/playlist?list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo
https://youtu.be/fJXwerDHg18?list=PL30nYyn47z1Clxr8tSpbRagFu66nrk32x


Discussion:
Hello @zulip/server-markdown members, this issue was labeled with the **area: markdown** label, so you may want to check it out!

<!-- areaLabelNotification -->
I would like to also add that youtube's shortened versions do not get recognized by zulip.
Ex:
https://youtu.be/gjDrEdEzfQc vs
https://www.youtube.com/watch?v=gjDrEdEzfQc&feature=youtu.be
@zulipbot claim
Not supporting the shortened code is in particular, annoying, as most youtube links will come in this form now if you take straight from youtube...
@Yoshi8765 hmm, I tried every single link above and it seemed to be embedded in Zulip every time. Maybe just a settings issue on your realm? I attached the screenshot below of all the links + their thumbnails which seemed to work fine.

![screen shot 2018-03-10 at 12 13 31 pm](https://user-images.githubusercontent.com/12992571/37246345-98bbdfb6-245c-11e8-87f1-c29137ca6153.png)

@lonerz I just retried it in my realm. Each are being posted as separate posts. 
![](https://i.imgur.com/tp2IaGR.png)

I've look at my realm (organization) settings, but I can't see anything that might be affecting this. Is there anything I can provide? If you tried to test that in chat.zulip.org I believe the platform there is a bleeding edge version which is different from what other realms are on.
@Yoshi8765 I tried this on my dev server off the current master. It seems like chat.zulip.org has the same issues you are facing. @timabbott is there a difference between the markdown link parsing on chat.zulip.org and the dev server?
@lonerz yes, in the development environment, we have the `INLINE_URL_EMBED_PREVIEW` setting on; it's off by default in production (but can be toggled).  
Hello @lonerz, you have been unassigned from this issue because you have not updated this issue or any referenced pull requests for over 14 days.

You can reclaim this issue or claim any other issue by commenting `@zulipbot claim` on that issue.

Thanks for your contributions, and hope to see you again soon!

Resolving patch:
diff --git a/zerver/lib/bugdown/__init__.py b/zerver/lib/bugdown/__init__.py
--- a/zerver/lib/bugdown/__init__.py
+++ b/zerver/lib/bugdown/__init__.py
@@ -677,16 +677,21 @@ def youtube_id(self, url: str) -> Optional[str]:
         if not self.markdown.image_preview_enabled:
             return None
         # Youtube video id extraction regular expression from http://pastebin.com/KyKAFv1s
-        # Slightly modified to support URLs of the form youtu.be/<id>
+        # Slightly modified to support URLs of the forms
+        #   - youtu.be/<id>
+        #   - youtube.com/playlist?v=<id>&list=<list-id>
+        #   - youtube.com/watch_videos?video_ids=<id1>,<id2>,<id3>
         # If it matches, match.group(2) is the video id.
         schema_re = r'(?:https?://)'
         host_re = r'(?:youtu\.be/|(?:\w+\.)?youtube(?:-nocookie)?\.com/)'
-        param_re = r'(?:(?:(?:v|embed)/)|(?:(?:watch(?:_popup)?(?:\.php)?)?(?:\?|#!?)(?:.+&)?v=))'
+        param_re = r'(?:(?:(?:v|embed)/)|' + \
+            r'(?:(?:(?:watch|playlist)(?:_popup|_videos)?(?:\.php)?)?(?:\?|#!?)(?:.+&)?v(?:ideo_ids)?=))'
         id_re = r'([0-9A-Za-z_-]+)'
         youtube_re = r'^({schema_re}?{host_re}{param_re}?)?{id_re}(?(1).+)?$'
         youtube_re = youtube_re.format(schema_re=schema_re, host_re=host_re, id_re=id_re, param_re=param_re)
         match = re.match(youtube_re, url)
-        if match is None:
+        # URLs of the form youtube.com/playlist?list=<list-id> are incorrectly matched
+        if match is None or match.group(2) == 'playlist':
             return None
         return match.group(2)
 


Test patch:
diff --git a/zerver/tests/test_bugdown.py b/zerver/tests/test_bugdown.py
--- a/zerver/tests/test_bugdown.py
+++ b/zerver/tests/test_bugdown.py
@@ -360,6 +360,21 @@ def test_inline_youtube(self) -> None:
 
         self.assertEqual(converted, '<p><a href="https://youtu.be/hx1mjT73xYE" target="_blank" title="https://youtu.be/hx1mjT73xYE">https://youtu.be/hx1mjT73xYE</a></p>\n<div class="youtube-video message_inline_image"><a data-id="hx1mjT73xYE" href="https://youtu.be/hx1mjT73xYE" target="_blank" title="https://youtu.be/hx1mjT73xYE"><img src="https://i.ytimg.com/vi/hx1mjT73xYE/default.jpg"></a></div>')
 
+        msg = 'https://www.youtube.com/playlist?list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo'
+        not_converted = bugdown_convert(msg)
+
+        self.assertEqual(not_converted, '<p><a href="https://www.youtube.com/playlist?list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo" target="_blank" title="https://www.youtube.com/playlist?list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo">https://www.youtube.com/playlist?list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo</a></p>')
+
+        msg = 'https://www.youtube.com/playlist?v=O5nskjZ_GoI&list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo'
+        converted = bugdown_convert(msg)
+
+        self.assertEqual(converted, '<p><a href="https://www.youtube.com/playlist?v=O5nskjZ_GoI&amp;list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo" target="_blank" title="https://www.youtube.com/playlist?v=O5nskjZ_GoI&amp;list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo">https://www.youtube.com/playlist?v=O5nskjZ_GoI&amp;list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo</a></p>\n<div class="youtube-video message_inline_image"><a data-id="O5nskjZ_GoI" href="https://www.youtube.com/playlist?v=O5nskjZ_GoI&amp;list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo" target="_blank" title="https://www.youtube.com/playlist?v=O5nskjZ_GoI&amp;list=PL8dPuuaLjXtNlUrzyH5r6jN9ulIgZBpdo"><img src="https://i.ytimg.com/vi/O5nskjZ_GoI/default.jpg"></a></div>')
+
+        msg = 'http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw'
+        converted = bugdown_convert(msg)
+
+        self.assertEqual(converted, '<p><a href="http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw" target="_blank" title="http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw">http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw</a></p>\n<div class="youtube-video message_inline_image"><a data-id="nOJgD4fcZhI" href="http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw" target="_blank" title="http://www.youtube.com/watch_videos?video_ids=nOJgD4fcZhI,i96UO8-GFvw"><img src="https://i.ytimg.com/vi/nOJgD4fcZhI/default.jpg"></a></div>')
+
     def test_inline_vimeo(self) -> None:
         msg = 'Check out the debate: https://vimeo.com/246979354'
         converted = bugdown_convert(msg)