Repository: gohugoio/hugo (language: go)

Issue:
nil pointer outputformat shortcode bug
### What version of Hugo are you using (hugo version)?

This is confirmed an issue with hugo 101.0 which we are currently using and also with the latest hugo 104.3. 
(both extended)

### Does this issue reproduce with the latest release?

Yes

### What is the issue?

Under a specific set of circumstances a `runtime error: invalid memory address or nil pointer dereference` occurs

#### Full Error

<details>
  <summary>View Stack Trace</summary>
  
```bash
ERROR 2022/10/25 09:49:02 [BUG] Got panic:
runtime error: invalid memory address or nil pointer dereference
goroutine 269 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func1.1()
        /root/project/hugo/hugolib/page__per_output.go:101 +0xaa
panic({0x25f2040, 0x23f09f0})
        /usr/local/go/src/runtime/panic.go:884 +0x212
github.com/gohugoio/hugo/hugolib.(*pageContentOutput).renderContent(0x0, {0xc000a83140, 0x11, 0x18}, 0x28?)
        /root/project/hugo/hugolib/page__per_output.go:636 +0x54
github.com/gohugoio/hugo/hugolib.renderShortcode(0x1, 0xc000194fc0, {{0xc00034253a, 0x2}, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, ...}, ...}}, ...)
        /root/project/hugo/hugolib/shortcode.go:385 +0xd4e
github.com/gohugoio/hugo/hugolib.renderShortcode(0x0, 0xc000194fc0, {{0xc00034253a, 0x2}, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, ...}, ...}}, ...)
        /root/project/hugo/hugolib/shortcode.go:368 +0xba5
github.com/gohugoio/hugo/hugolib.(*shortcodeHandler).renderShortcodesForPage(0xc00070bc70, 0xc000b57440, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, {0x283d1ef, 0x1}, ...}, ...})
        /root/project/hugo/hugolib/shortcode.go:487 +0x18a
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func1()
        /root/project/hugo/hugolib/page__per_output.go:112 +0x178
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func2({0x837aa9?, 0x3?})
        /root/project/hugo/hugolib/page__per_output.go:201 +0x1b
github.com/gohugoio/hugo/lazy.(*Init).withTimeout.func1()
        /root/project/hugo/lazy/init.go:188 +0x3c
created by github.com/gohugoio/hugo/lazy.(*Init).withTimeout
        /root/project/hugo/lazy/init.go:187 +0x11e
Error: Error building site: "C:\Users\davidejones\Desktop\HUGOTEST\content\fr\tracing\visualization\trace.md:1:1": runtime error: invalid memory address or nil pointer dereference
Total in 402 ms
```
</details>

### What specific conditions combine to lead to this error?

- Looping over pages in a custom output format that accesses the page `.Content` or `.Plain` 
- Page markdown content containing a nested shortcode
- Multilanguage site

I'm certain these conditions apply because doing any one of these individually causes the problem to be resolved
- Removing the custom output format template or not calling `.Content` or `.Plain` in it causes it to work
- Removing the nested shortcode causes it to work
- `disableLanguages = ['fr']` causes it to work

#### Recreated Example

<details>
  <summary>TestNestedShortcodeCustomOutputFormat()</summary>
  
  Add the following test to hugolib/shortcode_test.go to recreate
  
  ```go
  func TestNestedShortcodeCustomOutputFormat(t *testing.T) {
	t.Parallel()

	files := `
-- config.toml --

[outputFormats.Foobar]
baseName = "foobar"
isPlainText = true
mediaType = "application/json"
notAlternative = true

[languages.en]
languageName = "English"

[languages.en.outputs]
home = [ "HTML", "RSS", "Foobar" ]

[languages.fr]
languageName = "Français"

[[module.mounts]]
source = "content/en"
target = "content"
lang = "en"

[[module.mounts]]
source = "content/fr"
target = "content"
lang = "fr"

-- layouts/_default/list.foobar.json --
{{- $.Scratch.Add "data" slice -}}
{{- range (where .Site.AllPages "Kind" "!=" "home") -}}
	{{- $.Scratch.Add "data" (dict "content" (.Plain | truncate 10000) "type" .Type "full_url" .Permalink) -}}
{{- end -}}
{{- $.Scratch.Get "data" | jsonify -}}
-- content/en/p1.md --
---
title: "p1"
---

### More information

{{< tabs >}}
{{% tab "Test" %}}

It's a test

{{% /tab %}}
{{< /tabs >}}

-- content/fr/p2.md --
---
title: Test
---

### Plus d'informations

{{< tabs >}}
{{% tab "Test" %}}

C'est un test

{{% /tab %}}
{{< /tabs >}}

-- layouts/shortcodes/tabs.html --
<div>
  <div class="tab-content">{{ .Inner }}</div>
</div>

-- layouts/shortcodes/tab.html --
<div>{{ .Inner }}</div>

-- layouts/_default/single.html --
{{ .Content }}
`

	b := NewIntegrationTestBuilder(
		IntegrationTestConfig{
			T:           t,
			TxtarString: files,
			Running:     true,
			Verbose:     true,
		},
	).Build()

	b.AssertFileContent("public/fr/p2/index.html", `plus-dinformations`)

}
  ```
</details>

### Error details

Debugging it seems to be that `cp` on `p.pageOutput` is nil at times

**hugolib/shortcode.go**
```go
...

// Pre Hugo 0.55 this was the behaviour even for the outer-most
// shortcode.
if sc.doMarkup && (level > 0 || sc.configVersion() == 1) {
    var err error
    b, err := p.pageOutput.cp.renderContent([]byte(inner), false)
    if err != nil {
        return "", false, err
    }
    
...
```


For further context our site was working great and we discovered this bug when adding the output format code (which works successfully on other sites of ours)
nil pointer outputformat shortcode bug
### What version of Hugo are you using (hugo version)?

This is confirmed an issue with hugo 101.0 which we are currently using and also with the latest hugo 104.3. 
(both extended)

### Does this issue reproduce with the latest release?

Yes

### What is the issue?

Under a specific set of circumstances a `runtime error: invalid memory address or nil pointer dereference` occurs

#### Full Error

<details>
  <summary>View Stack Trace</summary>
  
```bash
ERROR 2022/10/25 09:49:02 [BUG] Got panic:
runtime error: invalid memory address or nil pointer dereference
goroutine 269 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func1.1()
        /root/project/hugo/hugolib/page__per_output.go:101 +0xaa
panic({0x25f2040, 0x23f09f0})
        /usr/local/go/src/runtime/panic.go:884 +0x212
github.com/gohugoio/hugo/hugolib.(*pageContentOutput).renderContent(0x0, {0xc000a83140, 0x11, 0x18}, 0x28?)
        /root/project/hugo/hugolib/page__per_output.go:636 +0x54
github.com/gohugoio/hugo/hugolib.renderShortcode(0x1, 0xc000194fc0, {{0xc00034253a, 0x2}, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, ...}, ...}}, ...)
        /root/project/hugo/hugolib/shortcode.go:385 +0xd4e
github.com/gohugoio/hugo/hugolib.renderShortcode(0x0, 0xc000194fc0, {{0xc00034253a, 0x2}, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, ...}, ...}}, ...)
        /root/project/hugo/hugolib/shortcode.go:368 +0xba5
github.com/gohugoio/hugo/hugolib.(*shortcodeHandler).renderShortcodesForPage(0xc00070bc70, 0xc000b57440, {{0xc0003432a0, 0x6}, {{0x2868128, 0xb}, {0x2841e13, 0x4}, {0x283d1ef, 0x1}, ...}, ...})
        /root/project/hugo/hugolib/shortcode.go:487 +0x18a
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func1()
        /root/project/hugo/hugolib/page__per_output.go:112 +0x178
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func2({0x837aa9?, 0x3?})
        /root/project/hugo/hugolib/page__per_output.go:201 +0x1b
github.com/gohugoio/hugo/lazy.(*Init).withTimeout.func1()
        /root/project/hugo/lazy/init.go:188 +0x3c
created by github.com/gohugoio/hugo/lazy.(*Init).withTimeout
        /root/project/hugo/lazy/init.go:187 +0x11e
Error: Error building site: "C:\Users\davidejones\Desktop\HUGOTEST\content\fr\tracing\visualization\trace.md:1:1": runtime error: invalid memory address or nil pointer dereference
Total in 402 ms
```
</details>

### What specific conditions combine to lead to this error?

- Looping over pages in a custom output format that accesses the page `.Content` or `.Plain` 
- Page markdown content containing a nested shortcode
- Multilanguage site

I'm certain these conditions apply because doing any one of these individually causes the problem to be resolved
- Removing the custom output format template or not calling `.Content` or `.Plain` in it causes it to work
- Removing the nested shortcode causes it to work
- `disableLanguages = ['fr']` causes it to work

#### Recreated Example

<details>
  <summary>TestNestedShortcodeCustomOutputFormat()</summary>
  
  Add the following test to hugolib/shortcode_test.go to recreate
  
  ```go
  func TestNestedShortcodeCustomOutputFormat(t *testing.T) {
	t.Parallel()

	files := `
-- config.toml --

[outputFormats.Foobar]
baseName = "foobar"
isPlainText = true
mediaType = "application/json"
notAlternative = true

[languages.en]
languageName = "English"

[languages.en.outputs]
home = [ "HTML", "RSS", "Foobar" ]

[languages.fr]
languageName = "Français"

[[module.mounts]]
source = "content/en"
target = "content"
lang = "en"

[[module.mounts]]
source = "content/fr"
target = "content"
lang = "fr"

-- layouts/_default/list.foobar.json --
{{- $.Scratch.Add "data" slice -}}
{{- range (where .Site.AllPages "Kind" "!=" "home") -}}
	{{- $.Scratch.Add "data" (dict "content" (.Plain | truncate 10000) "type" .Type "full_url" .Permalink) -}}
{{- end -}}
{{- $.Scratch.Get "data" | jsonify -}}
-- content/en/p1.md --
---
title: "p1"
---

### More information

{{< tabs >}}
{{% tab "Test" %}}

It's a test

{{% /tab %}}
{{< /tabs >}}

-- content/fr/p2.md --
---
title: Test
---

### Plus d'informations

{{< tabs >}}
{{% tab "Test" %}}

C'est un test

{{% /tab %}}
{{< /tabs >}}

-- layouts/shortcodes/tabs.html --
<div>
  <div class="tab-content">{{ .Inner }}</div>
</div>

-- layouts/shortcodes/tab.html --
<div>{{ .Inner }}</div>

-- layouts/_default/single.html --
{{ .Content }}
`

	b := NewIntegrationTestBuilder(
		IntegrationTestConfig{
			T:           t,
			TxtarString: files,
			Running:     true,
			Verbose:     true,
		},
	).Build()

	b.AssertFileContent("public/fr/p2/index.html", `plus-dinformations`)

}
  ```
</details>

### Error details

Debugging it seems to be that `cp` on `p.pageOutput` is nil at times

**hugolib/shortcode.go**
```go
...

// Pre Hugo 0.55 this was the behaviour even for the outer-most
// shortcode.
if sc.doMarkup && (level > 0 || sc.configVersion() == 1) {
    var err error
    b, err := p.pageOutput.cp.renderContent([]byte(inner), false)
    if err != nil {
        return "", false, err
    }
    
...
```


For further context our site was working great and we discovered this bug when adding the output format code (which works successfully on other sites of ours)


Discussion / hints:

Fix patch:
diff --git a/hugolib/page.go b/hugolib/page.go
index 37bf528c759..9f0d08ca351 100644
--- a/hugolib/page.go
+++ b/hugolib/page.go
@@ -909,6 +909,7 @@ func (p *pageState) shiftToOutputFormat(isRenderingSite bool, idx int) error {
 				}
 				return cp, nil
 			})
+			p.pageOutput.contentRenderer = lcp
 			p.pageOutput.ContentProvider = lcp
 			p.pageOutput.TableOfContentsProvider = lcp
 			p.pageOutput.PageRenderProvider = lcp
diff --git a/hugolib/page__output.go b/hugolib/page__output.go
index 41332347797..4ef78b61262 100644
--- a/hugolib/page__output.go
+++ b/hugolib/page__output.go
@@ -81,6 +81,7 @@ type pageOutput struct {
 
 	// These interface provides the functionality that is specific for this
 	// output format.
+	contentRenderer page.ContentRenderer
 	pagePerOutputProviders
 	page.ContentProvider
 	page.TableOfContentsProvider
@@ -94,10 +95,12 @@ func (p *pageOutput) initContentProvider(cp *pageContentOutput) {
 	if cp == nil {
 		return
 	}
+	p.contentRenderer = cp
 	p.ContentProvider = cp
 	p.TableOfContentsProvider = cp
 	p.PageRenderProvider = cp
 	p.cp = cp
+
 }
 
 func (p *pageOutput) enablePlaceholders() {
diff --git a/hugolib/page__per_output.go b/hugolib/page__per_output.go
index de70047c9f7..e8dc6950766 100644
--- a/hugolib/page__per_output.go
+++ b/hugolib/page__per_output.go
@@ -123,7 +123,7 @@ func newPageContentOutput(p *pageState, po *pageOutput) (*pageContentOutput, err
 		isHTML := cp.p.m.markup == "html"
 
 		if !isHTML {
-			r, err := cp.renderContent(cp.workContent, true)
+			r, err := po.contentRenderer.RenderContent(cp.workContent, true)
 			if err != nil {
 				return err
 			}
@@ -183,7 +183,7 @@ func newPageContentOutput(p *pageState, po *pageOutput) (*pageContentOutput, err
 				}
 			}
 		} else if cp.p.m.summary != "" {
-			b, err := cp.renderContent([]byte(cp.p.m.summary), false)
+			b, err := po.contentRenderer.RenderContent([]byte(cp.p.m.summary), false)
 			if err != nil {
 				return err
 			}
@@ -629,7 +629,7 @@ func (p *pageContentOutput) setAutoSummary() error {
 	return nil
 }
 
-func (cp *pageContentOutput) renderContent(content []byte, renderTOC bool) (converter.Result, error) {
+func (cp *pageContentOutput) RenderContent(content []byte, renderTOC bool) (converter.Result, error) {
 	if err := cp.initRenderHooks(); err != nil {
 		return nil, err
 	}
diff --git a/hugolib/shortcode.go b/hugolib/shortcode.go
index b2f42ff1d99..2951a143608 100644
--- a/hugolib/shortcode.go
+++ b/hugolib/shortcode.go
@@ -382,7 +382,7 @@ func renderShortcode(
 		// shortcode.
 		if sc.doMarkup && (level > 0 || sc.configVersion() == 1) {
 			var err error
-			b, err := p.pageOutput.cp.renderContent([]byte(inner), false)
+			b, err := p.pageOutput.contentRenderer.RenderContent([]byte(inner), false)
 			if err != nil {
 				return "", false, err
 			}
diff --git a/resources/page/page.go b/resources/page/page.go
index 50459c4655f..f9b8fdcec70 100644
--- a/resources/page/page.go
+++ b/resources/page/page.go
@@ -19,6 +19,7 @@ import (
 	"html/template"
 
 	"github.com/gohugoio/hugo/identity"
+	"github.com/gohugoio/hugo/markup/converter"
 
 	"github.com/bep/gitmap"
 	"github.com/gohugoio/hugo/config"
@@ -105,6 +106,11 @@ type ContentProvider interface {
 	Len() int
 }
 
+// ContentRenderer provides the content rendering methods for some content.
+type ContentRenderer interface {
+	RenderContent(content []byte, renderTOC bool) (converter.Result, error)
+}
+
 // FileProvider provides the source file.
 type FileProvider interface {
 	File() source.File
diff --git a/resources/page/page_lazy_contentprovider.go b/resources/page/page_lazy_contentprovider.go
index ba4f8f8ef77..2e4ddc35272 100644
--- a/resources/page/page_lazy_contentprovider.go
+++ b/resources/page/page_lazy_contentprovider.go
@@ -17,6 +17,7 @@ import (
 	"html/template"
 
 	"github.com/gohugoio/hugo/lazy"
+	"github.com/gohugoio/hugo/markup/converter"
 )
 
 // OutputFormatContentProvider represents the method set that is "outputFormat aware" and that we
@@ -24,6 +25,14 @@ import (
 // Note that this set is currently not complete, but should cover the most common use cases.
 // For the others, the implementation will be from the page.NoopPage.
 type OutputFormatContentProvider interface {
+	OutputFormatPageContentProvider
+
+	// for internal use.
+	ContentRenderer
+}
+
+// OutputFormatPageContentProvider holds the exported methods from Page that are "outputFormat aware".
+type OutputFormatPageContentProvider interface {
 	ContentProvider
 	TableOfContentsProvider
 	PageRenderProvider
@@ -46,7 +55,7 @@ type LazyContentProvider struct {
 func NewLazyContentProvider(f func() (OutputFormatContentProvider, error)) *LazyContentProvider {
 	lcp := LazyContentProvider{
 		init: lazy.New(),
-		cp:   NopPage,
+		cp:   NopCPageContentRenderer,
 	}
 	lcp.init.Add(func() (any, error) {
 		cp, err := f()
@@ -122,3 +131,8 @@ func (lcp *LazyContentProvider) TableOfContents() template.HTML {
 	lcp.init.Do()
 	return lcp.cp.TableOfContents()
 }
+
+func (lcp *LazyContentProvider) RenderContent(content []byte, renderTOC bool) (converter.Result, error) {
+	lcp.init.Do()
+	return lcp.cp.RenderContent(content, renderTOC)
+}
diff --git a/resources/page/page_nop.go b/resources/page/page_nop.go
index cdc5fd8b160..df3227f0a4d 100644
--- a/resources/page/page_nop.go
+++ b/resources/page/page_nop.go
@@ -16,10 +16,12 @@
 package page
 
 import (
+	"bytes"
 	"html/template"
 	"time"
 
 	"github.com/gohugoio/hugo/identity"
+	"github.com/gohugoio/hugo/markup/converter"
 
 	"github.com/gohugoio/hugo/hugofs/files"
 	"github.com/gohugoio/hugo/tpl"
@@ -41,7 +43,15 @@ import (
 )
 
 var (
-	NopPage Page = new(nopPage)
+	NopPage                 Page            = new(nopPage)
+	NopContentRenderer      ContentRenderer = new(nopContentRenderer)
+	NopCPageContentRenderer                 = struct {
+		OutputFormatPageContentProvider
+		ContentRenderer
+	}{
+		NopPage,
+		NopContentRenderer,
+	}
 	NilPage *nopPage
 )
 
@@ -513,3 +523,10 @@ func (p *nopPage) WordCount() int {
 func (p *nopPage) GetIdentity() identity.Identity {
 	return identity.NewPathIdentity("content", "foo/bar.md")
 }
+
+type nopContentRenderer int
+
+func (r *nopContentRenderer) RenderContent(content []byte, renderTOC bool) (converter.Result, error) {
+	b := &bytes.Buffer{}
+	return b, nil
+}


Test patch:
diff --git a/hugolib/shortcode_test.go b/hugolib/shortcode_test.go
index ec521729b8b..b5f27d6218c 100644
--- a/hugolib/shortcode_test.go
+++ b/hugolib/shortcode_test.go
@@ -1087,3 +1087,96 @@ Title: {{ .Get "title" | safeHTML }}
 	b.AssertFileContent("public/p1/index.html", `Title: Steve "Francia".`)
 
 }
+
+// Issue 10391.
+func TestNestedShortcodeCustomOutputFormat(t *testing.T) {
+	t.Parallel()
+
+	files := `
+-- config.toml --
+
+[outputFormats.Foobar]
+baseName = "foobar"
+isPlainText = true
+mediaType = "application/json"
+notAlternative = true
+
+[languages.en]
+languageName = "English"
+
+[languages.en.outputs]
+home = [ "HTML", "RSS", "Foobar" ]
+
+[languages.fr]
+languageName = "Français"
+
+[[module.mounts]]
+source = "content/en"
+target = "content"
+lang = "en"
+
+[[module.mounts]]
+source = "content/fr"
+target = "content"
+lang = "fr"
+
+-- layouts/_default/list.foobar.json --
+{{- $.Scratch.Add "data" slice -}}
+{{- range (where .Site.AllPages "Kind" "!=" "home") -}}
+	{{- $.Scratch.Add "data" (dict "content" (.Plain | truncate 10000) "type" .Type "full_url" .Permalink) -}}
+{{- end -}}
+{{- $.Scratch.Get "data" | jsonify -}}
+-- content/en/p1.md --
+---
+title: "p1"
+---
+
+### More information
+
+{{< tabs >}}
+{{% tab "Test" %}}
+
+It's a test
+
+{{% /tab %}}
+{{< /tabs >}}
+
+-- content/fr/p2.md --
+---
+title: Test
+---
+
+### Plus d'informations
+
+{{< tabs >}}
+{{% tab "Test" %}}
+
+C'est un test
+
+{{% /tab %}}
+{{< /tabs >}}
+
+-- layouts/shortcodes/tabs.html --
+<div>
+  <div class="tab-content">{{ .Inner }}</div>
+</div>
+
+-- layouts/shortcodes/tab.html --
+<div>{{ .Inner }}</div>
+
+-- layouts/_default/single.html --
+{{ .Content }}
+`
+
+	b := NewIntegrationTestBuilder(
+		IntegrationTestConfig{
+			T:           t,
+			TxtarString: files,
+			Running:     true,
+			Verbose:     true,
+		},
+	).Build()
+
+	b.AssertFileContent("public/fr/p2/index.html", `plus-dinformations`)
+
+}