Repository: gruntwork-io/terragrunt

Issue:
Add a way to supply credentials for a third-party registry when using Terragrunt Provider Cache
## Describe the bug
Would like to use Terragrunt provider cache feature with `run-all`. We execute Terragrunt workloads exclusively in Github Actions CI/CD pipelines. Since it is non-interactive we cannot use commands like `terraform login` to auth with third-party provider registries.

We have the following `.terraformrc` configuration file:
```hcl
provider_installation {
    direct {
        exclude = ["registry.terraform.io/*/*"]
    }
    filesystem_mirror {
        path = "hcl-file-working-directory-on-the-ci-cd-runner/terraform.d/plugins/"
        include = ["registry.terraform.io/manually-downloaded/provider"] 
    }
    network_mirror {
        url = "https://myorg.jfrog.io/artifactory/api/terraform/terraform-registry-prod/providers/" #<-- REQUIRES CREDENTIALS
        exclude = ["registry.terraform.io/manually-downloaded/provider"]
    }
}
```
The way we supply credential for `myorg.jfrog.io` is by setting env var: `TF_TOKEN_myorg_jfrog_io: ${{secrets.JFROG_API_KEY}}` which is automatically recognized by Terraform. However, the problem arises when attempting to enable Terragrunt Provider Cache, by setting following vars:
`TERRAGRUNT_PROVIDER_CACHE=1`
`TERRAGRUNT_PROVIDER_CACHE_REGISTRY_NAMES=myorg.jfrog.io`
We get the following error:
![image](https://github.com/user-attachments/assets/8b988d59-998f-460d-a9f9-8b39e1c5d6c9)

Terragrunt is unable to pass through credentials specified in `TF_TOKEN_myorg_jfrog_io`

## Expected behavior
Terragrunt is able to read credentials from `TF_TOKEN_` variables.
OR
Introduce new env var and cli flag to supply credentials when using Terragrunt Provider Cache

## Additional Context
We run around 15 Terragrunt workloads in the pipeline at the same time. Each of those calls around 20 different terraform modules using `run-all` feature of Terragrunt. This results in having to download mostly same providers 300 times which is extremely inefficient as well as overwhelming for provider registry server resulting in provider querying errors. It would be absolutely amazing to have Terragrunt Provider Cache working 😊
## Versions
- Terragrunt version: 0.63.6
- Terraform version: 1.5.7
- Environment details: alpine linux container in Github Actions runner.

## Related Issues
https://github.com/gruntwork-io/terragrunt/issues/3287
https://github.com/gruntwork-io/terragrunt/issues/3278

Discussion:
@lorengordon: TIL about the `TF_TOKEN_...` env... It's not mentioned on the terraform environment variables page, https://developer.hashicorp.com/terraform/cli/config/environment-variables. I kept looking and eventually found the doc over here on the cli configuration page, https://developer.hashicorp.com/terraform/cli/config/config-file#environment-variable-credentials@nuryupin-kr: UPDATE:
I've also tried to use `credentials` block in the `.terraformrc` file instead of setting `TF_TOKEN...` env var but ended up with an exact same result.@lev-ok: > UPDATE: I've also tried to use `credentials` block in the `.terraformrc` file instead of setting `TF_TOKEN...` env var but ended up with an exact same result.

Yes, this will not work because Terragrunt Provider Cache is responsible for downloading assets and does not accept any credentials when making requests. I have already started working on this, the results will appear soon.@lev-ok: Resolved in [v0.64.2](https://github.com/gruntwork-io/terragrunt/releases/tag/v0.64.2) release.@nuryupin-kr: @levkohimins just tried v0.64.2 and seeing the exactly identical error. Tried both specifying credentials block and using an env var. Even tried to have them both and it still doesn't work. Had 'trace' level enabled for logs, but not seeing any useful info about why it's not working 😞
![image](https://github.com/user-attachments/assets/cff64dce-0d9e-450a-bfcc-4116c28468b0)

UPDATE:
It seems to be discovering the registry though:
![image](https://github.com/user-attachments/assets/efa3fc81-c475-4b26-bd55-4fcff5651281)
@lev-ok: Hey @nuryupin-kr,
I can't help you until you provide me with the actual configs: `.terraformrc`, and `terraform { required_providers { ...` from `*.tf`. Perhaps you could upload them to a private repository and give me access.

> UPDATE:
> It seems to be discovering the registry though:
> ![image](https://private-user-images.githubusercontent.com/94398695/353588146-efa3fc81-c475-4b26-bd55-4fcff5651281.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjI1MTAwMDMsIm5iZiI6MTcyMjUwOTcwMywicGF0aCI6Ii85NDM5ODY5NS8zNTM1ODgxNDYtZWZhM2ZjODEtYzQ3NS00YjI2LWJkNTUtNGZjZmY1NjUxMjgxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MDElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODAxVDEwNTUwM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRlODRiMzljZjVkNDQ4MDY2YTRlY2NmNzU5MDFkYTFmMGE3ZTBkNDkwMDIxZjdkMmIzOGZjNWE5MmExOWRmNDkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.kZq3E8-14VeNDGrZ8Y-Dg_bRbsbEo__-7_fbiugTzbw)

It is suspicious that the url "providers.v1" is empty.@nuryupin-kr: @levkohimins I spoke to my higher ups and unfortunately, I'm not allowed to share anything that has company specific data like domains etc. :( However, regarding `.terraformrc` - it's exactly as I showed in the original post, there is no other configuration, just the `provider_installation` block (except that I substituted real domain of course). Here I highlighted exact values that I substituted with dummy ones in `.terraformrc` to make it clear:
![image](https://github.com/user-attachments/assets/a0c3df82-456c-49bd-9681-6d4c067b5731)

Here is required providers:
![image](https://github.com/user-attachments/assets/57a6b56f-53fd-441b-83c8-7d28fc0ae6ec)

Here is the one that gets generated for the provider we download manually in the Github Workflow (not through provider registry)
![image](https://github.com/user-attachments/assets/d650334d-0150-465a-9eb4-1c2be75b69f9)

So far though, it's only been complaining about not being able to accept credentials while trying to pull official registry.terraform.io providers using our jfrog mirror. Really you can just ignore manually downloaded provider and `filesystem_mirror` configuration for now, since it seems like the issue is with credentials for jfrog in the `network_mirror`

Please let me know what other info I can provide to make things more clear

> It is suspicious that the url "providers.v1" is empty.

It does seem strange, and just for context all of our terragrunt HCLs are pulling modules directly from github repos. We don't host modules in the jfrog registry.@nuryupin-kr: @levkohimins Should I create a new issue?@lev-ok: Hi @nuryupin-kr, sorry for taking so long to reply, I'm busy with another critical task right now. No, you don't need to create a new task, I am reopening this one. @nuryupin-kr: Hi @levkohimins. Is there any updates on this issue? Would love to hear back.@yhakbar: Hey folks, we can't prioritize this work, as (as far as I know) a pretty small minority of users experience this issue.

If a member of the community wants to take up the mantle of addressing the request, we are more than happy to support you develop a solution.

We have documentation on [contributing to Terragrunt](https://terragrunt.gruntwork.io/docs/community/contributing/#developing-terragrunt), and encourage you to join [our Discord](https://discord.gg/SPu4Degs5f) to chat with us directly if you need quick guidance on best practices.

Solution patch:
diff --git a/cli/provider_cache.go b/cli/provider_cache.go
index 08698591..a8be2088 100644
--- a/cli/provider_cache.go
+++ b/cli/provider_cache.go
@@ -25,7 +25,6 @@ import (
 	"github.com/gruntwork-io/terragrunt/terraform/cliconfig"
 	"github.com/gruntwork-io/terragrunt/terraform/getproviders"
 	"github.com/gruntwork-io/terragrunt/util"
-	"golang.org/x/exp/maps"
 )
 
 const (
@@ -88,16 +87,16 @@ func InitProviderCacheServer(opts *options.TerragruntOptions) (*ProviderCache, e
 		opts.ProviderCacheToken = fmt.Sprintf("%s:%s", apiKeyAuth, opts.ProviderCacheToken)
 	}
 
-	userProviderDir, err := cliconfig.UserProviderDir()
+	cliCfg, err := cliconfig.LoadUserConfig()
 	if err != nil {
 		return nil, err
 	}
-	providerService := services.NewProviderService(opts.ProviderCacheDir, userProviderDir)
 
-	cliCfg, err := cliconfig.LoadUserConfig()
+	userProviderDir, err := cliconfig.UserProviderDir()
 	if err != nil {
 		return nil, err
 	}
+	providerService := services.NewProviderService(opts.ProviderCacheDir, userProviderDir, cliCfg.CredentialsSource())
 
 	var (
 		providerHandlers []handlers.ProviderHandler
@@ -114,9 +113,9 @@ func InitProviderCacheServer(opts *options.TerragruntOptions) (*ProviderCache, e
 		case *cliconfig.ProviderInstallationFilesystemMirror:
 			providerHandlers = append(providerHandlers, handlers.NewProviderFilesystemMirrorHandler(providerService, cacheProviderHTTPStatusCode, method))
 		case *cliconfig.ProviderInstallationNetworkMirror:
-			providerHandlers = append(providerHandlers, handlers.NewProviderNetworkMirrorHandler(providerService, cacheProviderHTTPStatusCode, method))
+			providerHandlers = append(providerHandlers, handlers.NewProviderNetworkMirrorHandler(providerService, cacheProviderHTTPStatusCode, method, cliCfg.CredentialsSource()))
 		case *cliconfig.ProviderInstallationDirect:
-			providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, method))
+			providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, method, cliCfg.CredentialsSource()))
 			directIsdefined = true
 		}
 		method.AppendExclude(excludeAddrs)
@@ -124,7 +123,7 @@ func InitProviderCacheServer(opts *options.TerragruntOptions) (*ProviderCache, e
 
 	if !directIsdefined {
 		// In a case if none of direct provider installation methods `cliCfg.ProviderInstallation.Methods` are specified.
-		providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, new(cliconfig.ProviderInstallationDirect)))
+		providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, new(cliconfig.ProviderInstallationDirect), cliCfg.CredentialsSource()))
 	}
 
 	cache := cache.NewServer(
@@ -149,7 +148,7 @@ func (cache *ProviderCache) TerraformCommandHook(ctx context.Context, opts *opti
 	var (
 		cliConfigFilename    = filepath.Join(opts.WorkingDir, localCLIFilename)
 		cacheRequestID       = uuid.New().String()
-		env                  = providerCacheEnvironment(opts, cliConfigFilename)
+		envs                 = providerCacheEnvironment(opts, cliConfigFilename)
 		commandsArgs         = convertToMultipleCommandsByPlatforms(args)
 		skipRunTargetCommand bool
 	)
@@ -178,7 +177,7 @@ func (cache *ProviderCache) TerraformCommandHook(ctx context.Context, opts *opti
 	// It's low cost operation, because it does not cache the same provider twice, but only new previously non-existent providers.
 
 	for _, args := range commandsArgs {
-		if output, err := runTerraformCommand(ctx, opts, args, env); err != nil {
+		if output, err := runTerraformCommand(ctx, opts, args, envs); err != nil {
 			return output, err
 		}
 	}
@@ -198,7 +197,7 @@ func (cache *ProviderCache) TerraformCommandHook(ctx context.Context, opts *opti
 
 	cloneOpts := opts.Clone(opts.TerragruntConfigPath)
 	cloneOpts.WorkingDir = opts.WorkingDir
-	maps.Copy(cloneOpts.Env, env)
+	cloneOpts.Env = envs
 
 	if skipRunTargetCommand {
 		return &util.CmdOutput{}, nil
@@ -297,10 +296,7 @@ func runTerraformCommand(ctx context.Context, opts *options.TerragruntOptions, a
 	cloneOpts.ErrWriter = errWriter
 	cloneOpts.WorkingDir = opts.WorkingDir
 	cloneOpts.TerraformCliArgs = args
-
-	if envs != nil {
-		maps.Copy(cloneOpts.Env, envs)
-	}
+	cloneOpts.Env = envs
 
 	// If the Terraform error matches `httpStatusCacheProviderReg` we ignore it and hide the log from users, otherwise we process the error as is.
 	if output, err := shell.RunTerraformCommandWithOutput(ctx, cloneOpts, cloneOpts.TerraformCliArgs...); err != nil && len(errWriter.Msgs()) == 0 {
@@ -312,10 +308,18 @@ func runTerraformCommand(ctx context.Context, opts *options.TerragruntOptions, a
 
 // providerCacheEnvironment returns TF_* name/value ENVs, which we use to force terraform processes to make requests through our cache server (proxy) instead of making direct requests to the origin servers.
 func providerCacheEnvironment(opts *options.TerragruntOptions, cliConfigFile string) map[string]string {
-	envs := make(map[string]string)
+	envs := opts.Env
 
 	for _, registryName := range opts.ProviderCacheRegistryNames {
 		envName := fmt.Sprintf(terraform.EnvNameTFTokenFmt, strings.ReplaceAll(registryName, ".", "_"))
+
+		// delete existing key case insensitive
+		for key := range envs {
+			if strings.EqualFold(key, envName) {
+				delete(envs, key)
+			}
+		}
+
 		// We use `TF_TOKEN_*` for authentication with our private registry (cache server).
 		// https://developer.hashicorp.com/terraform/cli/config/config-file#environment-variable-credentials
 		envs[envName] = opts.ProviderCacheToken
diff --git a/terraform/cache/handlers/provider_direct.go b/terraform/cache/handlers/provider_direct.go
index defa182f..67404e64 100644
--- a/terraform/cache/handlers/provider_direct.go
+++ b/terraform/cache/handlers/provider_direct.go
@@ -44,10 +44,10 @@ type ProviderDirectHandler struct {
 	cacheProviderHTTPStatusCode int
 }
 
-func NewProviderDirectHandler(providerService *services.ProviderService, cacheProviderHTTPStatusCode int, method *cliconfig.ProviderInstallationDirect) *ProviderDirectHandler {
+func NewProviderDirectHandler(providerService *services.ProviderService, cacheProviderHTTPStatusCode int, method *cliconfig.ProviderInstallationDirect, credsSource *cliconfig.CredentialsSource) *ProviderDirectHandler {
 	return &ProviderDirectHandler{
 		CommonProviderHandler:       NewCommonProviderHandler(method.Include, method.Exclude),
-		ReverseProxy:                &ReverseProxy{},
+		ReverseProxy:                &ReverseProxy{CredsSource: credsSource},
 		providerService:             providerService,
 		cacheProviderHTTPStatusCode: cacheProviderHTTPStatusCode,
 	}
diff --git a/terraform/cache/handlers/provider_network_mirror.go b/terraform/cache/handlers/provider_network_mirror.go
index d6c6c7fc..628563a7 100644
--- a/terraform/cache/handlers/provider_network_mirror.go
+++ b/terraform/cache/handlers/provider_network_mirror.go
@@ -13,6 +13,7 @@ import (
 	"github.com/gruntwork-io/terragrunt/terraform/cache/router"
 	"github.com/gruntwork-io/terragrunt/terraform/cache/services"
 	"github.com/gruntwork-io/terragrunt/terraform/cliconfig"
+	svchost "github.com/hashicorp/terraform-svchost"
 	"github.com/labstack/echo/v4"
 )
 
@@ -23,15 +24,17 @@ type ProviderNetworkMirrorHandler struct {
 	providerService             *services.ProviderService
 	cacheProviderHTTPStatusCode int
 	networkMirrorURL            string
+	credsSource                 *cliconfig.CredentialsSource
 }
 
-func NewProviderNetworkMirrorHandler(providerService *services.ProviderService, cacheProviderHTTPStatusCode int, networkMirror *cliconfig.ProviderInstallationNetworkMirror) ProviderHandler {
+func NewProviderNetworkMirrorHandler(providerService *services.ProviderService, cacheProviderHTTPStatusCode int, networkMirror *cliconfig.ProviderInstallationNetworkMirror, credsSource *cliconfig.CredentialsSource) ProviderHandler {
 	return &ProviderNetworkMirrorHandler{
 		CommonProviderHandler:       NewCommonProviderHandler(networkMirror.Include, networkMirror.Exclude),
 		Client:                      &http.Client{},
 		providerService:             providerService,
 		cacheProviderHTTPStatusCode: cacheProviderHTTPStatusCode,
 		networkMirrorURL:            networkMirror.URL,
+		credsSource:                 credsSource,
 	}
 }
 
@@ -46,7 +49,7 @@ func (handler *ProviderNetworkMirrorHandler) GetVersions(ctx echo.Context, provi
 	}
 
 	reqPath := path.Join(provider.RegistryName, provider.Namespace, provider.Name, "index.json")
-	if err := handler.request(ctx, http.MethodGet, reqPath, &mirrorData); err != nil {
+	if err := handler.do(ctx, http.MethodGet, reqPath, &mirrorData); err != nil {
 		return err
 	}
 
@@ -82,7 +85,7 @@ func (handler *ProviderNetworkMirrorHandler) GetPlatform(ctx echo.Context, provi
 	}
 
 	reqPath := path.Join(provider.RegistryName, provider.Namespace, provider.Name, provider.Version+".json")
-	if err := handler.request(ctx, http.MethodGet, reqPath, &mirrorData); err != nil {
+	if err := handler.do(ctx, http.MethodGet, reqPath, &mirrorData); err != nil {
 		return err
 	}
 
@@ -109,7 +112,7 @@ func (handler *ProviderNetworkMirrorHandler) Download(ctx echo.Context, provider
 	return ctx.NoContent(http.StatusNotImplemented)
 }
 
-func (handler *ProviderNetworkMirrorHandler) request(ctx echo.Context, method, reqPath string, value any) error {
+func (handler *ProviderNetworkMirrorHandler) do(ctx echo.Context, method, reqPath string, value any) error {
 	reqURL := fmt.Sprintf("%s/%s", strings.TrimRight(handler.networkMirrorURL, "/"), reqPath)
 
 	req, err := http.NewRequestWithContext(ctx.Request().Context(), method, reqURL, nil)
@@ -117,6 +120,13 @@ func (handler *ProviderNetworkMirrorHandler) request(ctx echo.Context, method, r
 		return errors.WithStackTrace(err)
 	}
 
+	if handler.credsSource != nil {
+		hostname := svchost.Hostname(req.URL.Hostname())
+		if creds := handler.credsSource.ForHost(hostname); creds != nil {
+			creds.PrepareRequest(req)
+		}
+	}
+
 	resp, err := handler.Client.Do(req)
 	if err != nil {
 		return errors.WithStackTrace(err)
diff --git a/terraform/cache/handlers/reverse_proxy.go b/terraform/cache/handlers/reverse_proxy.go
index b1c6615d..bdd30aa4 100644
--- a/terraform/cache/handlers/reverse_proxy.go
+++ b/terraform/cache/handlers/reverse_proxy.go
@@ -6,11 +6,14 @@ import (
 	"net/url"
 
 	"github.com/gruntwork-io/terragrunt/pkg/log"
+	"github.com/gruntwork-io/terragrunt/terraform/cliconfig"
+	svchost "github.com/hashicorp/terraform-svchost"
 	"github.com/labstack/echo/v4"
 )
 
 type ReverseProxy struct {
-	ServerURL *url.URL
+	ServerURL   *url.URL
+	CredsSource *cliconfig.CredentialsSource
 
 	Rewrite        func(*httputil.ProxyRequest)
 	ModifyResponse func(resp *http.Response) error
@@ -33,6 +36,13 @@ func (reverseProxy *ReverseProxy) NewRequest(ctx echo.Context, targetURL *url.UR
 			req.Out.Host = targetURL.Host
 			req.Out.URL = targetURL
 
+			if reverseProxy.CredsSource != nil {
+				hostname := svchost.Hostname(req.Out.URL.Hostname())
+				if creds := reverseProxy.CredsSource.ForHost(hostname); creds != nil {
+					creds.PrepareRequest(req.Out)
+				}
+			}
+
 			if reverseProxy.Rewrite != nil {
 				reverseProxy.Rewrite(req)
 			}
diff --git a/terraform/cache/helpers/http.go b/terraform/cache/helpers/http.go
index a12feba4..6ec8a768 100644
--- a/terraform/cache/helpers/http.go
+++ b/terraform/cache/helpers/http.go
@@ -15,11 +15,7 @@ import (
 	"github.com/hashicorp/go-getter/v2"
 )
 
-func Fetch(ctx context.Context, url string, dst io.Writer) error {
-	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
-	if err != nil {
-		return errors.WithStackTrace(err)
-	}
+func Fetch(ctx context.Context, req *http.Request, dst io.Writer) error {
 	req.Header.Add("Accept-Encoding", "gzip")
 
 	resp, err := (&http.Client{}).Do(req)
@@ -29,7 +25,7 @@ func Fetch(ctx context.Context, url string, dst io.Writer) error {
 	defer resp.Body.Close() //nolint:errcheck
 
 	if resp.StatusCode != http.StatusOK {
-		return fmt.Errorf("%s returned from %s", resp.Status, url)
+		return fmt.Errorf("%s returned from %s", resp.Status, req.URL)
 	}
 
 	reader, err := ResponseReader(resp)
@@ -47,14 +43,14 @@ func Fetch(ctx context.Context, url string, dst io.Writer) error {
 }
 
 // FetchToFile downloads the file from the given `url` into the specified `dst` file.
-func FetchToFile(ctx context.Context, url, dst string) error {
+func FetchToFile(ctx context.Context, req *http.Request, dst string) error {
 	file, err := os.Create(dst)
 	if err != nil {
 		return errors.WithStackTrace(err)
 	}
 	defer file.Close() //nolint:errcheck
 
-	if err := Fetch(ctx, url, file); err != nil {
+	if err := Fetch(ctx, req, file); err != nil {
 		return err
 	}
 
diff --git a/terraform/cache/services/provider_cache.go b/terraform/cache/services/provider_cache.go
index dfeeb622..71218cd7 100644
--- a/terraform/cache/services/provider_cache.go
+++ b/terraform/cache/services/provider_cache.go
@@ -6,6 +6,7 @@ import (
 	"crypto/sha256"
 	"encoding/hex"
 	"fmt"
+	"net/http"
 	"os"
 	"path"
 	"path/filepath"
@@ -16,10 +17,12 @@ import (
 	"github.com/gruntwork-io/terragrunt/pkg/log"
 	"github.com/gruntwork-io/terragrunt/terraform/cache/helpers"
 	"github.com/gruntwork-io/terragrunt/terraform/cache/models"
+	"github.com/gruntwork-io/terragrunt/terraform/cliconfig"
 	"github.com/gruntwork-io/terragrunt/terraform/getproviders"
 	"github.com/gruntwork-io/terragrunt/util"
 	"github.com/hashicorp/go-getter/v2"
 	"github.com/hashicorp/go-multierror"
+	svchost "github.com/hashicorp/terraform-svchost"
 	"github.com/sirupsen/logrus"
 	"golang.org/x/sync/errgroup"
 )
@@ -85,6 +88,8 @@ type ProviderCache struct {
 	packageDir      string
 	lockfilePath    string
 	archivePath     string
+
+	credsSource *cliconfig.CredentialsSource
 }
 
 func (cache *ProviderCache) DocumentSHA256Sums(ctx context.Context) ([]byte, error) {
@@ -94,7 +99,11 @@ func (cache *ProviderCache) DocumentSHA256Sums(ctx context.Context) ([]byte, err
 
 	var documentSHA256Sums = new(bytes.Buffer)
 
-	if err := helpers.Fetch(ctx, cache.SHA256SumsURL, documentSHA256Sums); err != nil {
+	req, err := cache.newRequest(ctx, cache.SHA256SumsURL)
+	if err != nil {
+		return nil, err
+	}
+	if err := helpers.Fetch(ctx, req, documentSHA256Sums); err != nil {
 		return nil, fmt.Errorf("failed to retrieve authentication checksums for provider %q: %w", cache.Provider, err)
 	}
 
@@ -109,7 +118,11 @@ func (cache *ProviderCache) Signature(ctx context.Context) ([]byte, error) {
 
 	var signature = new(bytes.Buffer)
 
-	if err := helpers.Fetch(ctx, cache.SHA256SumsSignatureURL, signature); err != nil {
+	req, err := cache.newRequest(ctx, cache.SHA256SumsSignatureURL)
+	if err != nil {
+		return nil, err
+	}
+	if err := helpers.Fetch(ctx, req, signature); err != nil {
 		return nil, fmt.Errorf("failed to retrieve authentication signature for provider %q: %w", cache.Provider, err)
 	}
 
@@ -204,7 +217,11 @@ func (cache *ProviderCache) warmUp(ctx context.Context) error {
 		cache.archivePath = cache.DownloadURL
 	} else {
 		if err := util.DoWithRetry(ctx, fmt.Sprintf("Fetching provider %s", cache.Provider), maxRetriesFetchFile, retryDelayFetchFile, logrus.DebugLevel, func(ctx context.Context) error {
-			return helpers.FetchToFile(ctx, cache.DownloadURL, cache.archivePath)
+			req, err := cache.newRequest(ctx, cache.DownloadURL)
+			if err != nil {
+				return err
+			}
+			return helpers.FetchToFile(ctx, req, cache.archivePath)
 		}); err != nil {
 			return err
 		}
@@ -226,6 +243,24 @@ func (cache *ProviderCache) warmUp(ctx context.Context) error {
 	return nil
 }
 
+func (cache *ProviderCache) newRequest(ctx context.Context, url string) (*http.Request, error) {
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return nil, errors.WithStackTrace(err)
+	}
+
+	if cache.credsSource == nil {
+		return req, nil
+	}
+
+	hostname := svchost.Hostname(req.URL.Hostname())
+	if creds := cache.credsSource.ForHost(hostname); creds != nil {
+		creds.PrepareRequest(req)
+	}
+
+	return req, nil
+}
+
 func (cache *ProviderCache) removeArchive() error {
 	if cache.archiveCached && util.FileExists(cache.archivePath) {
 		log.Debugf("Remove provider cached archive %s", cache.archivePath)
@@ -267,13 +302,16 @@ type ProviderService struct {
 
 	cacheMu      sync.RWMutex
 	cacheReadyMu sync.RWMutex
+
+	credsSource *cliconfig.CredentialsSource
 }
 
-func NewProviderService(cacheDir, userCacheDir string) *ProviderService {
+func NewProviderService(cacheDir, userCacheDir string, credsSource *cliconfig.CredentialsSource) *ProviderService {
 	return &ProviderService{
 		cacheDir:              cacheDir,
 		userCacheDir:          userCacheDir,
 		providerCacheWarmUpCh: make(chan *ProviderCache),
+		credsSource:           credsSource,
 	}
 }
 
@@ -312,8 +350,9 @@ func (service *ProviderService) CacheProvider(ctx context.Context, requestID str
 	packageName := fmt.Sprintf("%s-%s-%s-%s-%s", provider.RegistryName, provider.Namespace, provider.Name, provider.Version, provider.Platform())
 
 	cache := &ProviderCache{
-		Provider: provider,
-		started:  make(chan struct{}, 1),
+		Provider:    provider,
+		credsSource: service.credsSource,
+		started:     make(chan struct{}, 1),
 
 		userProviderDir: filepath.Join(service.userCacheDir, provider.Address(), provider.Version, provider.Platform()),
 		packageDir:      filepath.Join(service.cacheDir, provider.Address(), provider.Version, provider.Platform()),
diff --git a/terraform/cliconfig/config.go b/terraform/cliconfig/config.go
index 7c7064bd..74b5d923 100644
--- a/terraform/cliconfig/config.go
+++ b/terraform/cliconfig/config.go
@@ -6,6 +6,7 @@ import (
 	"github.com/gruntwork-io/go-commons/errors"
 	"github.com/hashicorp/hcl/v2/gohcl"
 	"github.com/hashicorp/hcl/v2/hclwrite"
+	svchost "github.com/hashicorp/terraform-svchost"
 )
 
 // ConfigHost is the structure of the "host" nested block within the CLI configuration, which can be used to override the default service host discovery behavior for a particular hostname.
@@ -110,3 +111,20 @@ func (cfg *Config) Save(configPath string) error {
 
 	return nil
 }
+
+// CredentialsSource creates and returns a service credentials source whose behavior depends on which "credentials" if are present in the receiving config.
+func (cfg *Config) CredentialsSource() *CredentialsSource {
+	configured := make(map[svchost.Hostname]string)
+	for _, creds := range cfg.Credentials {
+		host, err := svchost.ForComparison(creds.Name)
+		if err != nil {
+			// We expect the config was already validated by the time we get here, so we'll just ignore invalid hostnames.
+			continue
+		}
+		configured[host] = creds.Token
+	}
+
+	return &CredentialsSource{
+		configured: configured,
+	}
+}
diff --git a/terraform/cliconfig/credentials.go b/terraform/cliconfig/credentials.go
new file mode 100644
index 00000000..49fd3411
--- /dev/null
+++ b/terraform/cliconfig/credentials.go
@@ -0,0 +1,76 @@
+package cliconfig
+
+import (
+	"os"
+	"strings"
+
+	svchost "github.com/hashicorp/terraform-svchost"
+	svcauth "github.com/hashicorp/terraform-svchost/auth"
+)
+
+type CredentialsSource struct {
+	// configured describes the credentials explicitly configured in the CLI config via "credentials" blocks.
+	configured map[svchost.Hostname]string
+}
+
+func (s *CredentialsSource) ForHost(host svchost.Hostname) svcauth.HostCredentials {
+	// The first order of precedence for credentials is a host-specific environment variable
+	if envCreds := hostCredentialsFromEnv(host); envCreds != nil {
+		return envCreds
+	}
+
+	// Then, any credentials block present in the CLI config
+	if token, ok := s.configured[host]; ok {
+		return svcauth.HostCredentialsToken(token)
+	}
+
+	return nil
+}
+
+// hostCredentialsFromEnv returns a token credential by searching for a hostname-specific environment variable. The host parameter is expected to be in the "comparison" form, for example, hostnames containing non-ASCII characters like "café.fr" should be expressed as "xn--caf-dma.fr". If the variable based on the hostname is not defined, nil is returned.
+//
+// Hyphen and period characters are allowed in environment variable names, but are not valid POSIX variable names. However, it's still possible to set variable names with these characters using utilities like env or docker. Variable names may have periods translated to underscores and hyphens translated to double underscores in the variable name. For the example "café.fr", you may use the variable names "TF_TOKEN_xn____caf__dma_fr", "TF_TOKEN_xn--caf-dma_fr", or "TF_TOKEN_xn--caf-dma.fr"
+func hostCredentialsFromEnv(host svchost.Hostname) svcauth.HostCredentials {
+	token, ok := collectCredentialsFromEnv()[host]
+	if !ok {
+		return nil
+	}
+	return svcauth.HostCredentialsToken(token)
+}
+
+func collectCredentialsFromEnv() map[svchost.Hostname]string {
+	const prefix = "TF_TOKEN_"
+
+	ret := make(map[svchost.Hostname]string)
+	for _, ev := range os.Environ() {
+		eqIdx := strings.Index(ev, "=")
+		if eqIdx < 0 {
+			continue
+		}
+		name := ev[:eqIdx]
+		value := ev[eqIdx+1:]
+		if !strings.HasPrefix(name, prefix) {
+			continue
+		}
+		rawHost := name[len(prefix):]
+
+		// We accept double underscores in place of hyphens because hyphens are not valid identifiers in most shells and are therefore hard to set.
+		rawHost = strings.ReplaceAll(rawHost, "__", "-")
+
+		// We accept underscores in place of dots because dots are not valid identifiers in most shells and are therefore hard to set.
+		// Underscores are not valid in hostnames, so this is unambiguous for valid hostnames.
+		rawHost = strings.ReplaceAll(rawHost, "_", ".")
+
+		// Because environment variables are often set indirectly by OS libraries that might interfere with how they are encoded, we'll be tolerant of them being given either directly as UTF-8 IDNs or in Punycode form, normalizing to Punycode form here because that is what the OpenTofu credentials helper protocol will use in its requests.
+		dispHost := svchost.ForDisplay(rawHost)
+		hostname, err := svchost.ForComparison(dispHost)
+		if err != nil {
+			// Ignore invalid hostnames
+			continue
+		}
+
+		ret[hostname] = value
+	}
+
+	return ret
+}



Test patch:
diff --git a/cli/provider_cache_test.go b/cli/provider_cache_test.go
index 6be82d8f..df1e8677 100644
--- a/cli/provider_cache_test.go
+++ b/cli/provider_cache_test.go
@@ -110,8 +110,8 @@ func TestProviderCache(t *testing.T) {
 
 			errGroup, ctx := errgroup.WithContext(ctx)
 
-			providerService := services.NewProviderService(providerCacheDir, pluginCacheDir)
-			providerHandler := handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, new(cliconfig.ProviderInstallationDirect))
+			providerService := services.NewProviderService(providerCacheDir, pluginCacheDir, nil)
+			providerHandler := handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, new(cliconfig.ProviderInstallationDirect), nil)
 
 			testCase.opts = append(testCase.opts, cache.WithServices(providerService), cache.WithProviderHandlers(providerHandler))
 
@@ -155,5 +155,4 @@ func TestProviderCache(t *testing.T) {
 			require.NoError(t, err)
 		})
 	}
-
 }
diff --git a/test/integration_common_test.go b/test/integration_common_test.go
index e14301a8..eead55e1 100644
--- a/test/integration_common_test.go
+++ b/test/integration_common_test.go
@@ -25,6 +25,7 @@ import (
 	"time"
 
 	"github.com/gruntwork-io/terragrunt/util"
+	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
 	"github.com/NYTimes/gziphandler"
@@ -43,7 +44,7 @@ func testRunAllPlan(t *testing.T, args string) (string, string, string, error) {
 	return tmpEnvPath, stdout, stderr, err
 }
 
-func runNetworkMirrorServer(t *testing.T, ctx context.Context, urlPrefix, providerDir string) *url.URL {
+func runNetworkMirrorServer(t *testing.T, ctx context.Context, urlPrefix, providerDir, token string) *url.URL {
 	serverTLSConf, clientTLSConf := certSetup(t)
 
 	http.DefaultTransport = &http.Transport{
@@ -56,7 +57,14 @@ func runNetworkMirrorServer(t *testing.T, ctx context.Context, urlPrefix, provid
 
 	withGz := gziphandler.GzipHandler(http.StripPrefix(urlPrefix, fs))
 
-	mux.Handle(urlPrefix, withGz)
+	mux.HandleFunc(urlPrefix, func(resp http.ResponseWriter, req *http.Request) {
+		if token != "" {
+			authHeaders := req.Header.Values("Authorization")
+			assert.Contains(t, authHeaders, fmt.Sprintf("Bearer %s", token))
+		}
+
+		withGz.ServeHTTP(resp, req)
+	})
 
 	ln, err := tls.Listen("tcp", "localhost:8888", serverTLSConf)
 	require.NoError(t, err)
diff --git a/test/integration_serial_test.go b/test/integration_serial_test.go
index ba5e0ac2..b2b09d3f 100644
--- a/test/integration_serial_test.go
+++ b/test/integration_serial_test.go
@@ -138,7 +138,9 @@ func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
 		http.DefaultTransport = defaultTransport
 	}()
 
-	networkMirrorURL := runNetworkMirrorServer(t, ctx, "/providers/", providersNetkworMirrorPath)
+	token := "123456790"
+
+	networkMirrorURL := runNetworkMirrorServer(t, ctx, "/providers/", providersNetkworMirrorPath, token)
 	t.Logf("Network mirror URL: %s", networkMirrorURL)
 	t.Logf("Provdiers network mirror path: %s", providersNetkworMirrorPath)
 	t.Logf("Provdiers filesysmte mirror path: %s", providersFilesystemMirrorPath)
@@ -149,6 +151,11 @@ func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
 	require.NoError(t, err)
 	defer cliConfigFilename.Close()
 
+	tokenEnvName := fmt.Sprintf("TF_TOKEN_%s", strings.ReplaceAll(networkMirrorURL.Hostname(), ".", "_"))
+	err = os.Setenv(tokenEnvName, token)
+	require.NoError(t, err)
+	defer os.Unsetenv(tokenEnvName)
+
 	err = os.Setenv(terraform.EnvNameTFCLIConfigFile, cliConfigFilename.Name())
 	require.NoError(t, err)
 	defer os.Unsetenv(terraform.EnvNameTFCLIConfigFile)