Repository: gohugoio/hugo (language: go)

Issue:
Potential deadlock in GroupByParam/ByParam
I recently took a huge leap from v0.85.0 to v0.112.3 (and also tried v0.111.3) and have found my build it hanging more often than not. When it succeeds it builds fast, when it hangs it seems to do so forever. I have this issue locally on my mac, and also on Netlify. I have now wrapped my build script with `until timeout -s SIGABRT 5m hugo ; do : ; done` so that it is forced to retry after 5 minutes until it succeeds, and provides the useful go dump when it fails. I have a recent [Netlify build log](https://app.netlify.com/sites/compassionate-aryabhata-ce7373/deploys/6473963f2285590008ba3f75 ) here, where it took 3 attempts to build - search for "Start building sites" for the top of each attempt. The site code is public at [sparksp/lt-osp](https://github.com/sparksp/lt-osp/tree/rules-v4).

Hopefully someone is able to see something useful in the go dump that might lead to either an issue with my site, or an issue with hugo.

### Version

```
hugo v0.112.3-ba6f74e40420d76f15fc8c2358be90f7aca98e0e+extended linux/amd64 BuildDate=2023-05-24T14:42:50Z VendorInfo=gohugoio
```

### Does this issue reproduce with the latest release?
Yes.


Discussion / hints:
I can reproduce with [v0.97.0](https://github.com/gohugoio/hugo/releases/tag/v0.97.0) and later. The problem is here:

<https://github.com/sparksp/lt-osp/blob/rules-v4/themes/lt-osp/layouts/partials/lists/term/ladder.html>

If I comment out the contents, the build completes quickly.

To simplify the problem, replace the contents with:

```text
{{ range .Pages.ByWeight.GroupByParam "tier" }}
  {{ range .Pages.ByParam "ladder_group" }}
    {{ warnf "%[1]v (%[1]T)" .Params.ladder_group }}
  {{ end }}
{{ end }}
```

Then run `while true; do hugo; done` and wait for it to hang.

Then change it to this (remove the `.ByParam` method):

```text
{{ range .Pages.ByWeight.GroupByParam "tier" }}
  {{ range .Pages }}
    {{ warnf "%[1]v (%[1]T)" .Params.ladder_group }}
  {{ end }}
{{ end }}
```

No problems.

We made some changes to `.ByParam`  in v0.97.0 (see https://github.com/gohugoio/hugo/pull/9772). If you look at values displayed by the `warnf` function above, you will see that is a mix of strings ("1", "2", "3", "right") and nil.

---

You can use v0.112.x if you reduce the number of workers to 1.

```text
while true; do HUGO_NUMWORKERMULTIPLIER=1 hugo; done
```

Thank-you for looking at this so quickly, and having a work around! Do you know if this is something likely to be resolved in Hugo at some point? I don't want to put much effort into trying to change my logic (if I can), or adding `ladder_group`s to all my skill pages. If it might be resolved in the near future then I can live with the worker reduction. It does also seem like something that may trip other people up in the future. The [documentation](https://gohugo.io/templates/lists/#by-parameter) does imply that it's okay to use with mixed content
> If the parameter is not found at all in some entries, those entries will appear together at the end of the ordering.

Fix patch:
diff --git a/langs/language.go b/langs/language.go
index 4e089606c7b..f8d4f64b5a6 100644
--- a/langs/language.go
+++ b/langs/language.go
@@ -41,8 +41,11 @@ type Language struct {
 	translator    locales.Translator
 	timeFormatter htime.TimeFormatter
 	tag           language.Tag
-	collator      *Collator
-	location      *time.Location
+	// collator1 and collator2 are the same, we have 2 to prevent deadlocks.
+	collator1 *Collator
+	collator2 *Collator
+
+	location *time.Location
 
 	// This is just an alias of Site.Params.
 	params maps.Params
@@ -58,14 +61,20 @@ func NewLanguage(lang, defaultContentLanguage, timeZone string, languageConfig L
 		}
 	}
 
-	var coll *Collator
+	var coll1, coll2 *Collator
 	tag, err := language.Parse(lang)
 	if err == nil {
-		coll = &Collator{
+		coll1 = &Collator{
+			c: collate.New(tag),
+		}
+		coll2 = &Collator{
 			c: collate.New(tag),
 		}
 	} else {
-		coll = &Collator{
+		coll1 = &Collator{
+			c: collate.New(language.English),
+		}
+		coll2 = &Collator{
 			c: collate.New(language.English),
 		}
 	}
@@ -76,7 +85,8 @@ func NewLanguage(lang, defaultContentLanguage, timeZone string, languageConfig L
 		translator:     translator,
 		timeFormatter:  htime.NewTimeFormatter(translator),
 		tag:            tag,
-		collator:       coll,
+		collator1:      coll1,
+		collator2:      coll2,
 	}
 
 	return l, l.loadLocation(timeZone)
@@ -165,8 +175,12 @@ func GetLocation(l *Language) *time.Location {
 	return l.location
 }
 
-func GetCollator(l *Language) *Collator {
-	return l.collator
+func GetCollator1(l *Language) *Collator {
+	return l.collator1
+}
+
+func GetCollator2(l *Language) *Collator {
+	return l.collator2
 }
 
 type Collator struct {
diff --git a/resources/page/pages_sort.go b/resources/page/pages_sort.go
index b9b905cc227..32b1b3895c2 100644
--- a/resources/page/pages_sort.go
+++ b/resources/page/pages_sort.go
@@ -161,7 +161,7 @@ var collatorStringSort = func(getString func(Page) string) func(p Pages) {
 		// Pages may be a mix of multiple languages, so we need to use the language
 		// for the currently rendered Site.
 		currentSite := p[0].Site().Current()
-		coll := langs.GetCollator(currentSite.Language())
+		coll := langs.GetCollator1(currentSite.Language())
 		coll.Lock()
 		defer coll.Unlock()
 
@@ -173,7 +173,7 @@ var collatorStringSort = func(getString func(Page) string) func(p Pages) {
 
 var collatorStringCompare = func(getString func(Page) string, p1, p2 Page) int {
 	currentSite := p1.Site().Current()
-	coll := langs.GetCollator(currentSite.Language())
+	coll := langs.GetCollator1(currentSite.Language())
 	coll.Lock()
 	c := coll.CompareStrings(getString(p1), getString(p2))
 	coll.Unlock()
@@ -182,7 +182,9 @@ var collatorStringCompare = func(getString func(Page) string, p1, p2 Page) int {
 
 var collatorStringLess = func(p Page) (less func(s1, s2 string) bool, close func()) {
 	currentSite := p.Site().Current()
-	coll := langs.GetCollator(currentSite.Language())
+	// Make sure to use the second collator to prevent deadlocks.
+	// See issue 11039.
+	coll := langs.GetCollator2(currentSite.Language())
 	coll.Lock()
 	return func(s1, s2 string) bool {
 			return coll.CompareStrings(s1, s2) < 1
diff --git a/resources/page/taxonomy.go b/resources/page/taxonomy.go
index f50152f901a..3aa0c7a7b76 100644
--- a/resources/page/taxonomy.go
+++ b/resources/page/taxonomy.go
@@ -83,7 +83,7 @@ func (i Taxonomy) Alphabetical() OrderedTaxonomy {
 		return ia
 	}
 	currentSite := p.Site().Current()
-	coll := langs.GetCollator(currentSite.Language())
+	coll := langs.GetCollator1(currentSite.Language())
 	coll.Lock()
 	defer coll.Unlock()
 	name := func(i1, i2 *OrderedTaxonomyEntry) bool {
diff --git a/tpl/collections/sort.go b/tpl/collections/sort.go
index 83029b31026..4a2106039ad 100644
--- a/tpl/collections/sort.go
+++ b/tpl/collections/sort.go
@@ -46,7 +46,7 @@ func (ns *Namespace) Sort(l any, args ...any) (any, error) {
 		return nil, errors.New("can't sort " + reflect.ValueOf(l).Type().String())
 	}
 
-	collator := langs.GetCollator(ns.deps.Conf.Language())
+	collator := langs.GetCollator1(ns.deps.Conf.Language())
 
 	// Create a list of pairs that will be used to do the sort
 	p := pairList{Collator: collator, sortComp: ns.sortComp, SortAsc: true, SliceType: sliceType}


Test patch: