Repository: gohugoio/hugo (language: go)

Issue:
Youtube shortcode fails on 0.125.0+ on an unlisted video (media type not resolved)
The following shortcode:

```plaintext
{{< youtube ZVs79y0gjCA >}}
```

leads to this error:

```plaintext
ERROR The "youtube" shortcode was unable to get remote resource 
"https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZVs79y0gjCA". 
error calling resources.GetRemote: 
failed to resolve media type for remote resource 
"https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZVs79y0gjCA". 
See "/home/patrick/gitlab.com/wonderland/wonderland-engine-website/content/en/news/vertex-color-baked-lighting/index.md:179:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
```

and the server does not start. This has happened since 0.125.0 (0.125.1 also shows the error; it seems unconnected to #12383).

In 0.124.1 it is parsed and shows the video.

hugo v0.124.1-db083b05f16c945fec04f745f0ca8640560cf1ec+extended linux/amd64 BuildDate=2024-03-20T11:40:10Z VendorInfo=gohugoio
hugo v0.125.1-68c5ad638c2072969e47262926b912e80fd71a77+extended linux/amd64 BuildDate=2024-04-18T08:21:19Z VendorInfo=gohugoio

This video is unlisted and other videos on the side do not run into this error.



Discussion / hints:
I am unable to reproduce the problem with v0.125.1 using:

```text
{{< youtube ZVs79y0gjCA >}}
```

I tested with another unlisted video as well:

![image](https://github.com/gohugoio/hugo/assets/335669/5ce6abd3-2a5a-42e0-ad5a-f861f3e92a1c)

This seems to work fine:

```text
{{< youtube 5I0gZlBdY8M >}}
```
I can confirm this behavior,

I am examining the problem closely.

ERROR The "youtube" shortcode was unable to get remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYCaotPnKU5w". error calling resources.GetRemote: failed to resolve media type for remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYCaotPnKU5w". See "<path>/20220612002812-exploring_my_exo_brain.md:44:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
ERROR The "youtube" shortcode was unable to get remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKkF9Sl_NgVc". error calling resources.GetRemote: failed to resolve media type for remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKkF9Sl_NgVc". See "/<path>/20220610163147-how_to_install_intersubs_on_mpv_to_learn_hebrew.md:38:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
ERROR The "youtube" shortcode was unable to get remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Ds7db59pQruY". error calling resources.GetRemote: failed to resolve media type for remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Ds7db59pQruY". See "<path>/20230822214616-how_to_generate_a_graph_of_a_repo_s_git_history.md:42:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
ERROR The "youtube" shortcode was unable to get remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DnFAIp6z2YdA". error calling resources.GetRemote: failed to resolve media type for remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DnFAIp6z2YdA". See "<path>/20230822214616-how_to_generate_a_graph_of_a_repo_s_git_history.md:47:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
@vonpupp What's the video id in your shortcode call? Please show the markdown.

Also:

- Are these "unlisted" per the OP's report above?
- Which operating system and version?

The YouTube response header has:

```text
Content-Type: application/json
```

So I'm wondering if your system doesn't understand that content type? I have no problem adding _any_ of the videos above with Ubuntu 22.04.4 LTS.

---

EDIT Sat 20 Apr 2024 08:35:04-08:00

I just tested again with a virgin install (plus Git, Go, and VSCode) of Ubuntu 22.04.2 LTS.

```text
{{< youtube ZVs79y0gjCA >}}

{{< youtube 5I0gZlBdY8M >}}

{{< youtube nFAIp6z2YdA >}}
```

No problems, so I am still suspicious of media types (AKA MIME types) on each of your systems. I'm not saying we shouldn't address this, but the underlying cause may determine _how_ we address this.

I am disinclined to remove error checking, though maybe we downgrade to a warning. However, if this turns out to be a media type issue on your systems, we may want to look at what `resources.GetRemote` is doing.


Thanks you for your answer @jmooring.

I am using Arch Linux. The content is an org-mode file converted to md using ox-hugo (an exporter for org-mode). Here is a snippet of the code that I use on the org-mode file.
```
#+html: {{< youtube YCaotPnKU5w >}}
```
The [video](https://www.youtube.com/watch?v=YCaotPnKU5w) is unlisted. It used to work previously. My version of hugo is: 
```
hugo version
hugo` v0.125.1+extended linux/amd64 BuildDate=unknown
```

Here is the rendering of the md file using ox-hugo:
```
+++                                                                                                                                                                                                                                                                               
title = "Exploring my [exo]brain"                                                                                                                                                                                                                                                 
author = ["vonpupp"]                                                                                                                                                                                                                                        
date = 2022-06-12                                                                                                                                                                                                                                                                 
lastmod = 2024-04-20T20:35:29-03:00                                                                                                                                                                                                                                               
tags = ["exobrain", "digitalgardens", "research", "graphs", "communities", "clustering"]                                                                                                                                                                                          
categories = ["emacs", "visualization"]                                                                                                                                                                                                                                           
draft = false                                                                                                                                                                                                                                                                     
image = "/images/fm_exploring_exobrain.png"                                                                                                                                                                                                                                       
featured_image = "/images/exploring-exobrain.png"                                                                                                                                                                                                                                
+++                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                  
I really like working with graphs. I remember a project that I worked a while                                                                                                                                                                                                     
ago where I parsed the client requirements relationships into a graph and I                                                                                                                                                                                                       
could determine the priorities of modules development based on that. Good                                                                                                                                                                                                         
souvenirs. This original idea came from an analysis I did on my social                                                                                                                                                                                                            
networks out of curiosity and I could identify and [find communities on graphs]({{< relref "20220109211521-finding_communities_on_graphs.md" >}}).                                                                                                                                
                                                                                                                                                                                                                                                                                  
... text shortened ...

Now lets preview this thing... Most screenshots have been anonimized.  At the                                                                                                                                                                                                     
time of this writing this is how my notes looks like. Keep in mind that out of                                                                                                                                                                                                    
all of my notes, only a few selected ones are shared here on my website.                                                                                                                                                                                                          
                                                                                                                                                                                                                                                                                  
I am going to show first a little video sample                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                  
{{< youtube YCaotPnKU5w >}}                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                  
Figure [1](#figure--fig:full-headings) is how the exobrain looks considering all notes                                                                                                                                                                                            
```

Thank you very much!                                                                                                                                                                                                   
```
```
{{< youtube YCaotPnKU5w >}}  
```
The above renders for me without error. 

On your system, do this:

```text
touch foo.json
mimetype foo.json 
```

What's the result?
```plaintext
[10:51:17][patrick@Behemoth]~/gitlab.com/wonderland/wonderland-engine-website(+1|%0|ok)
❯ hugo version
hugo v0.125.2-4e483f5d4abae136c4312d397a55e9e1d39148df+extended linux/amd64 BuildDate=2024-04-20T15:29:44Z VendorInfo=gohugoio
[10:51:21][patrick@Behemoth]~/gitlab.com/wonderland/wonderland-engine-website(+1|%0|ok)
❯ touch foo.json
[10:51:24][patrick@Behemoth]~/gitlab.com/wonderland/wonderland-engine-website(+1|%0|ok)
❯ mimetype foo.json 
foo.json: application/json
[10:51:28][patrick@Behemoth]~/gitlab.com/wonderland/wonderland-engine-website(+1|%0|ok)
❯ 
```

I will investigate if there are any other pointers surrounding the mystery.
Going back to [the old shortcode](https://github.com/gohugoio/hugo/commit/8a0ea12d8a906bd41b4cd7108651efc4fb9be237#diff-ec6f8f4278a55ee3bcd975bc37ff67a5bdac3ca121a11754af3292d1f48ea112) helps for now. 

I think it might be an issue with `resources.GetRemote` because I recently tried to use that function with other services (Github REST API, a simple json edge function on Netlify, etc.) and it also resulted in mime type issues. Maybe that goes lost somewhere in this set up. 
Sure,

```
mimetype foo.json
foo.json: application/json
```

I have an older static binary version that works fine:
```
Hugo Static Site Generator v0.59.0-DEV-5276F1E9:CHROMA-26F03CB linux/amd64 BuildDate: 2019-09-11T08:16:28-0400
```
@vonpupp Instead of using an earlier version, for now I suggest you do as the message suggests:

> You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']


Same issue here with v0.125.1 & v0.125.2, but with any YouTube video.. for example:

```
{{< youtube id="WmR9IMUD_CY" >}}
```

which is a public video, results this error:

```
Start building sites …
hugo v0.125.2+extended linux/amd64 BuildDate=unknown

ERROR The "youtube" shortcode was unable to get remote resource "https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWmR9IMUD_CY". failed to fetch remote resource: Forbidden. See "/home/sudocss/path/to/my/hugo/project/content/articles/article-name/index.md:52:1"
You can suppress this error by adding the following to your site configuration:
ignoreLogs = ['err-youtube-remote']
Total in 2078 ms
Error: error building site: logged 1 error(s)
```

I'm on EndeavourOS.
This is frustrating. I cannot reproduce the problem with any of the video id's referenced in this issue. 
```text
{{< youtube ZVs79y0gjCA >}}

{{< youtube 5I0gZlBdY8M >}}

{{< youtube YCaotPnKU5w >}}

{{< youtube KkF9Sl_NgVc >}}

{{< youtube s7db59pQruY >}}

{{< youtube nFAIp6z2YdA >}}

{{< youtube WmR9IMUD_CY >}}
```

Tested with Ubuntu 22.04 (virgin install plus Git, Go, and VSCode) and Windows 11.

@sudoCSS As suggested in the error message, add this to your site config until we can figure out what's causing this:

```text
ignoreLogs = ['err-youtube-remote']
```


@davidsneighbour @sudoCss @vonpupp 

I would appreciate it if you could post the result of:

```text
curl -I https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZVs79y0gjCA
```
Sure

![Screenshot_2024-04-21_20-08-26](https://github.com/gohugoio/hugo/assets/84265453/002fd49b-09aa-4070-bfa3-f9e343a499e7)

@sudoCss And what do you see in the browser if you click this:

https://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZVs79y0gjCA

This:

![Screenshot_2024-04-21_20-23-41](https://github.com/gohugoio/hugo/assets/84265453/6a9e0051-fd73-4769-9d26-02f937130db5)

```
❯ HTTP/2 404 
vary: X-Origin
vary: Referer
vary: Origin,Accept-Encoding
content-type: application/json; charset=UTF-8
date: Mon, 22 Apr 2024 08:27:07 GMT
server: scaffolding on HTTPServer2
cache-control: private
x-xss-protection: 0
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
accept-ranges: none

[1]+  Done                    curl -I https://www.youtube.com/oembed?format=json
```

and the direct link in the browser shows the JSON response for the oembed.
I added the config option and the error goes away, naturally. The interesting thing is that the video is shown in the resulting page. So either a fallback comes into play or the error message is somehow false?
@jmooring looking at https://github.com/gohugoio/hugo/blob/f40f50ead0c46db984e9b970e0361e85c396c200/tpl/tplimpl/embedded/templates/shortcodes/youtube.html#L30 I see that the only data item used from the returned JSON is the `title`. I briefly tested removing that remote call and I didn't see any difference in the YouTube videos I saw. 

I suggest that we make the life a little easier for ourselves and either:

1. Remove that GetRemote block if it doesn't add any obvious value
2. Or silence any error/404.



Fix patch:
diff --git a/tpl/tplimpl/embedded/templates/_default/rss.xml b/tpl/tplimpl/embedded/templates/_default/rss.xml
index a3cade65717..2e505f1bc56 100644
--- a/tpl/tplimpl/embedded/templates/_default/rss.xml
+++ b/tpl/tplimpl/embedded/templates/_default/rss.xml
@@ -48,7 +48,7 @@
     <title>{{ if eq .Title .Site.Title }}{{ .Site.Title }}{{ else }}{{ with .Title }}{{ . }} on {{ end }}{{ .Site.Title }}{{ end }}</title>
     <link>{{ .Permalink }}</link>
     <description>Recent content {{ if ne .Title .Site.Title }}{{ with .Title }}in {{ . }} {{ end }}{{ end }}on {{ .Site.Title }}</description>
-    <generator>Hugo {{ hugo.Version }}</generator>
+    <generator>Hugo</generator>
     <language>{{ site.Language.LanguageCode }}</language>{{ with $authorEmail }}
     <managingEditor>{{.}}{{ with $authorName }} ({{ . }}){{ end }}</managingEditor>{{ end }}{{ with $authorEmail }}
     <webMaster>{{ . }}{{ with $authorName }} ({{ . }}){{ end }}</webMaster>{{ end }}{{ with .Site.Copyright }}
diff --git a/tpl/tplimpl/embedded/templates/shortcodes/youtube.html b/tpl/tplimpl/embedded/templates/shortcodes/youtube.html
index bf72be45be8..93fa1819707 100644
--- a/tpl/tplimpl/embedded/templates/shortcodes/youtube.html
+++ b/tpl/tplimpl/embedded/templates/shortcodes/youtube.html
@@ -8,10 +8,10 @@
 @param {int} [end] The time, measured in seconds from the start of the video, when the player should stop playing the video.
 @param {string} [id] The video id. Optional if the id is provided as first positional argument.
 @param {string} [loading=eager] The loading attribute of the iframe element.
-@param {bool} [loop=false] Whether to indefinitely repeat the video.
+@param {bool} [loop=false] Whether to indefinitely repeat the video. Ignores the start and end arguments after the first play.
 @param {bool} [mute=false] Whether to mute the video. Always true when autoplay is true.
 @param {int} [start] The time, measured in seconds from the start of the video, when the player should start playing the video.
-@param {string} [title] The title attribute of the iframe element. Defaults to the title returned by YouTube oEmbed API.
+@param {string} [title] The title attribute of the iframe element. Defaults to "YouTube video".
 
 @returns {template.HTML}
 
@@ -26,20 +26,6 @@
 {{- if not $pc.Disable }}
   {{- with $id := or (.Get "id") (.Get 0) }}
 
-    {{- /* Get data from the YouTube oEmbed API. */}}
-    {{- $q := querify "url" (printf "https://www.youtube.com/watch?v=%s" $id) "format" "json" }}
-    {{- $url := printf "https://www.youtube.com/oembed?%s" $q }}
-    {{- $data := dict }}
-    {{- with resources.GetRemote $url }}
-      {{- with .Err }}
-        {{- erroridf $remoteErrID "The %q shortcode was unable to get remote resource %q. %s. See %s" $.Name $url . $.Position }}
-      {{- else }}
-        {{- $data = .Content | transform.Unmarshal }}
-      {{- end }}
-    {{- else }}
-      {{- erroridf $remoteErrID "The %q shortcode was unable to get remote resource %q. See %s" $.Name $url $.Position }}
-    {{- end }}
-
     {{/* Set defaults. */}}
     {{- $allowFullScreen := "allowfullscreen" }}
     {{- $autoplay := 0 }}
@@ -50,7 +36,7 @@
     {{- $loop := 0 }}
     {{- $mute := 0 }}
     {{- $start := 0 }}
-    {{- $title := $data.title }}
+    {{- $title := "YouTube video" }}
 
     {{- /* Get arguments. */}}
     {{- if in (slice "false" false 0) ($.Get "allowFullScreen") }}


Test patch: