Repository: gohugoio/hugo (language: go)

Issue:
Add "cache identifier" argument to .GetRemote
Currently users grabbing data from an API have to rely on file caching to ensure the API is not solicited on every build. (To avoid limitations but also slower builds)
To refresh the response every hour, set up a hourly build and set your cache settings to:

```
caches:
  getresource:
    maxAge: 1h
```

Then file caching setting of Hugo identifies a cached "response"  with a hashed concatenation of the request's URL and the passed options (headers, body etc...)

## This presents several problem.

1. Some API requires the date and time to be included in the authorization header for extra security. This means the .`GetRemote` options will always be different thus producing a unique cache identifier every time.
2. Cache options are shared by ever request. Every `.GetRemote` call will be subject to the same `maxAge` for example. No way to set one API call to refresh every day, and another one every hour.

## Proposal

Add a third and optional parameter to `.GetRemote` to pass on a cache identifier.

Not only will it solves **1.** by giving control to users on the cache identifier (omitting some options), but it will also give more control to users, thus bypassing the **2.** limitation.

Cache for approximately one hour by including the current time (to the hour) in the cache id:

```
$cache_id := print $url (time.Format "2006-01-02T15")
$request := resource.GetRemote $url $options $cache_id
```

Cache for approximately one day by including the current date (to the day) in the cache id:

```
$cache_id := print $url (time.Format "2006-01-02")
$request := resource.GetRemote $url $options $cache_id
```


Discussion / hints:
>It would also move all the critical directories to the top of the list which is a nice bonus.

What?

I suggest you append a cache-buster to the URL, e.g. `https://example.org/api?hugo_cachebuster=12345`.

My last comment was in the wrong issue.

> I suggest you append a cache-buster to the URL, e.g. https://example.org/api?hugo_cachebuster=12345.

Yes that's a fair for point for **2.** but will not solve **1.**

>Yes that's a fair for point for 2. but will not solve 1.

OK, I misunderstood -- you want it to be cached ... I guess we could consider adding a `cacheKey` to the `options` map or something. But you could probably also solve this by hiding the header creating behind `partialCached`.



> But you could probably also solve this by hiding the header creating behind partialCached.

That would only cache the output on that one build scope though right?
(Sidenote: I was using cron to update the GA script hosted locally on WP once a day. Will this feature do a similar thing if a build is made with Hugo, e.g. if set for a day, any time after one day?)

Fix patch:
diff --git a/common/maps/maps.go b/common/maps/maps.go
index 5552da55d19..2d8a122ca61 100644
--- a/common/maps/maps.go
+++ b/common/maps/maps.go
@@ -109,6 +109,20 @@ func ToSliceStringMap(in any) ([]map[string]any, error) {
 	}
 }
 
+// LookupEqualFold finds key in m with case insensitive equality checks.
+func LookupEqualFold[T any | string](m map[string]T, key string) (T, bool) {
+	if v, found := m[key]; found {
+		return v, true
+	}
+	for k, v := range m {
+		if strings.EqualFold(k, key) {
+			return v, true
+		}
+	}
+	var s T
+	return s, false
+}
+
 type keyRename struct {
 	pattern glob.Glob
 	newKey  string
diff --git a/docs/content/en/hugo-pipes/introduction.md b/docs/content/en/hugo-pipes/introduction.md
index 83d64d1d3eb..bbafe55b209 100755
--- a/docs/content/en/hugo-pipes/introduction.md
+++ b/docs/content/en/hugo-pipes/introduction.md
@@ -53,6 +53,19 @@ With `resources.GetRemote`, the first argument is a remote URL:
 
 `resources.Get` and `resources.GetRemote` return `nil` if the resource is not found.
 
+### Caching
+
+By default, Hugo calculates a cache key based on the `URL` and the `options` (e.g. headers) given.
+
+
+{{< new-in "0.97.0" >}} You can override this by setting a `key` in the options map. This can be used to get more fine grained control over how often a remote resource is fetched, e.g.:
+
+
+```go-html-template
+{{ $cacheKey := print $url (now.Format "2006-01-02") }}
+{{ $resource := resource.GetRemote $url (dict "key" $cacheKey) }}
+```
+
 ### Error Handling
 
 {{< new-in "0.91.0" >}}
diff --git a/resources/resource_factories/create/remote.go b/resources/resource_factories/create/remote.go
index 56bd99cb16f..32dfafe5cab 100644
--- a/resources/resource_factories/create/remote.go
+++ b/resources/resource_factories/create/remote.go
@@ -27,6 +27,7 @@ import (
 	"strings"
 
 	"github.com/gohugoio/hugo/common/hugio"
+	"github.com/gohugoio/hugo/common/maps"
 	"github.com/gohugoio/hugo/common/types"
 	"github.com/gohugoio/hugo/helpers"
 	"github.com/gohugoio/hugo/media"
@@ -79,7 +80,7 @@ func (c *Client) FromRemote(uri string, optionsm map[string]any) (resource.Resou
 		return nil, errors.Wrapf(err, "failed to parse URL for resource %s", uri)
 	}
 
-	resourceID := helpers.HashString(uri, optionsm)
+	resourceID := calculateResourceID(uri, optionsm)
 
 	_, httpResponse, err := c.cacheGetResource.GetOrCreate(resourceID, func() (io.ReadCloser, error) {
 		options, err := decodeRemoteOptions(optionsm)
@@ -199,6 +200,13 @@ func (c *Client) validateFromRemoteArgs(uri string, options fromRemoteOptions) e
 	return nil
 }
 
+func calculateResourceID(uri string, optionsm map[string]any) string {
+	if key, found := maps.LookupEqualFold(optionsm, "key"); found {
+		return helpers.HashString(key)
+	}
+	return helpers.HashString(uri, optionsm)
+}
+
 func addDefaultHeaders(req *http.Request, accepts ...string) {
 	for _, accept := range accepts {
 		if !hasHeaderValue(req.Header, "Accept", accept) {


Test patch:
diff --git a/common/maps/maps_test.go b/common/maps/maps_test.go
index 53120dce75b..0b84d2dd7b3 100644
--- a/common/maps/maps_test.go
+++ b/common/maps/maps_test.go
@@ -171,3 +171,26 @@ func TestRenameKeys(t *testing.T) {
 		t.Errorf("Expected\n%#v, got\n%#v\n", expected, m)
 	}
 }
+
+func TestLookupEqualFold(t *testing.T) {
+	c := qt.New(t)
+
+	m1 := map[string]any{
+		"a": "av",
+		"B": "bv",
+	}
+
+	v, found := LookupEqualFold(m1, "b")
+	c.Assert(found, qt.IsTrue)
+	c.Assert(v, qt.Equals, "bv")
+
+	m2 := map[string]string{
+		"a": "av",
+		"B": "bv",
+	}
+
+	v, found = LookupEqualFold(m2, "b")
+	c.Assert(found, qt.IsTrue)
+	c.Assert(v, qt.Equals, "bv")
+
+}
diff --git a/resources/resource_factories/create/remote_test.go b/resources/resource_factories/create/remote_test.go
index cfd18269f8d..c2a3b7b3275 100644
--- a/resources/resource_factories/create/remote_test.go
+++ b/resources/resource_factories/create/remote_test.go
@@ -83,3 +83,14 @@ func TestDecodeRemoteOptions(t *testing.T) {
 	}
 
 }
+
+func TestCalculateResourceID(t *testing.T) {
+	c := qt.New(t)
+
+	c.Assert(calculateResourceID("foo", nil), qt.Equals, "5917621528921068675")
+	c.Assert(calculateResourceID("foo", map[string]any{"bar": "baz"}), qt.Equals, "7294498335241413323")
+
+	c.Assert(calculateResourceID("foo", map[string]any{"key": "1234", "bar": "baz"}), qt.Equals, "14904296279238663669")
+	c.Assert(calculateResourceID("asdf", map[string]any{"key": "1234", "bar": "asdf"}), qt.Equals, "14904296279238663669")
+	c.Assert(calculateResourceID("asdf", map[string]any{"key": "12345", "bar": "asdf"}), qt.Equals, "12191037851845371770")
+}