Repository: gruntwork-io/terragrunt

Issue:
Issues when attempting to use air-gap/offline provider cache and private module (i.e. `provider_installation` possibly being ignored).
## Describe the bug

I am attempting to `terragrunt run-all plan` in an air-gapped/offline-only/private-intranet use case. I basically need to use a private/proprietary module, along with the public `github` provider to communicate with a private GitHub Enterprise Server (GHES) stack. This work is somewhat similar to what is described in <https://github.com/gruntwork-io/terragrunt/issues/3117>.

I'm able to get the [provider cache](https://terragrunt.gruntwork.io/docs/features/provider-cache/) to work with various modules I can pull from `registry.terraform.io`, but not with private modules. For example, I have a private module, `cheeseburger/pickle/3.0.0`, hosted on a private (slow) server, `terraform.food.lan`. Since the private server is slow and unreliable, I'm attempting to manually install the provider into the cache. However, all attempts to get the private module to be sourced via my Terragrunt provider cache fail with the following error:

```
ERRO[0078] remote https://terraform.food.lan/v1/providers/cheeseburger/pickle/versions unreachable, could not forward: context canceled
...
...
...
Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider
terraform.food.lan/cheeseburger/pickle: could not query provider registry for
terraform.food.lan/cheeseburger/pickle: the request failed after 2 attempts,
please try again later: Get
"http://127.0.0.1:5758/v1/providers//terraform.food.lan/cheeseburger/pickle/versions":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)
```

## Steps To Reproduce

First, I setup the Terragrunt provider cache in my shell:

```sh
export TERRAGRUNT_PROVIDER_CACHE="1"
export TERRAGRUNT_PROVIDER_CACHE_REGISTRY_NAMES="registry.terraform.io,terraform.food.lan"
export TERRAGRUNT_PROVIDER_CACHE_HOST=127.0.0.1
export TERRAGRUNT_PROVIDER_CACHE_PORT=5758
```

I create a `~/.terraformrc` file like so:

```hcl
plugin_cache_dir = "$HOME/.cache/terragrunt/providers"
disable_checkpoint = true
provider_installation {
  filesystem_mirror {
    path    = "/home/owner/.cache/terragrunt/providers"
    include = [
      "terraform.food.lan/*/*",
    ]
  }
  direct {
    exclude = [
      "terraform.food.lan/*/*",
    ]
  }
}
```

For additional context, my respective provider cache folders are located at the following paths (and that, with the exception of this use case where I'm trying to manually install a private provider into the Terragrunt provider cache, both Terragrunt and vanilla Terraform provider caches appear to be working as expected):

- Terragrunt: `~/.cache/terragrunt/providers`
- Terraform: `~/.terraform.d/plugins`

## Expected behavior

I expected `terragrunt run-all plan` to conclude normally (i.e. to successful completion), but without requiring any access to the private server, `terraform.food.lan`.

## Nice to haves

- [x] Terminal output
- [ ] Screenshots

## Versions

- Terragrunt version: `terragrunt version v0.59.6`
- OpenTofu/Terraform version: `Terraform v1.9.1 on linux_arm64`
- Environment details (Ubuntu 20.04, Windows 10, etc.): 
```sh
$> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble

$> uname -a
Linux darkstar 6.8.0-36-generic #36-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 10 13:20:23 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
```

## Additional context

N/A. Happy to provide additional info as we go.


Hints from issue comments:
@IAXES: After additional testing, it seems that `aws` and `github` providers, for example, are indeed being sourced from the local provider cache (can't get my private provider working this way, though).

However, **all** providers seem to be ignoring the `direct.exclude` statements, and are still attempting to query `versions` from the remote registries, preventing the air-gap use case from working (i.e. if local provider cache is properly populated, but remote version check fails, then the overall `terragrunt run-all apply` operation also fails early-on).

---

So, I think the public-versus-private provider topic I raised is a non issue. Instead, this seems to be entirely a function of Terragrunt and/or Terraform reaching out to remote provider registries when we don't want the tools to do this (either due to a misconfig on my end, or some feature/code-change needed to support air-gap deployments).

I've also tried going as far as [using the `dev_overrides` feature in the `provider_installation` block](https://developer.hashicorp.com/terraform/cli/config/config-file#development-overrides-for-provider-developers), but no luck. It seems like the entire issue hinges on "honor `direct.exclude` and `disable_checkpoint = true` statements in `~/.terraformrc`" in order for an air-gapped use case to work.@lev-ok: Resolved in [v0.63.3](https://github.com/gruntwork-io/terragrunt/releases/tag/v0.63.3) release.@antoniogomezalvarado: I'm using `terragrunt version v0.68.6`

With this config located at `~/.terraformrc`

```
 provider_installation {
  filesystem_mirror {
    path    = "/terragrunt-provider-cache-dir"
    include = ["local/custom/kafka-connect"]
  }
  direct {
    exclude = ["local/custom/kafka-connect"]
  }
}
```

And getting the following errors for all the other providers which I would expect to act as default:

```
  ╷
  │ Error: Failed to query available provider packages
  │ 
  │ Could not retrieve the list of available versions for provider
  │ hashicorp/aws: provider registry.terraform.io/hashicorp/aws was not found
  │ in any of the search locations
  │ 
  │ 
```
This is just an example i'm getting multiple errors for `registry.terraform.io/` what am I missing? 🤔 

Solution patch:
diff --git a/cli/commands/terraform/creds/providers/externalcmd/provider.go b/cli/commands/terraform/creds/providers/externalcmd/provider.go
index 139613ff..72754cb0 100644
--- a/cli/commands/terraform/creds/providers/externalcmd/provider.go
+++ b/cli/commands/terraform/creds/providers/externalcmd/provider.go
@@ -43,7 +43,6 @@ func (provider *Provider) GetCredentials(ctx context.Context) (*providers.Creden
 		args = parts[1:]
 	}
 
-	provider.terragruntOptions.Logger.Infof("Executing %s to obtain Terragrunt credentials", provider.Name())
 	output, err := shell.RunShellCommandWithOutput(ctx, provider.terragruntOptions, "", true, false, command, args...)
 	if err != nil {
 		return nil, err
diff --git a/cli/provider_cache.go b/cli/provider_cache.go
index 4b03ecfc..7b2d06f9 100644
--- a/cli/provider_cache.go
+++ b/cli/provider_cache.go
@@ -105,17 +105,19 @@ func InitProviderCacheServer(opts *options.TerragruntOptions) (*ProviderCache, e
 	for _, registryName := range opts.ProviderCacheRegistryNames {
 		excludeAddrs = append(excludeAddrs, fmt.Sprintf("%s/*/*", registryName))
 	}
-
 	for _, method := range cliCfg.ProviderInstallation.Methods {
 		switch method := method.(type) {
 		case *cliconfig.ProviderInstallationFilesystemMirror:
 			providerHandlers = append(providerHandlers, handlers.NewProviderFilesystemMirrorHandler(providerService, cacheProviderHTTPStatusCode, method))
+			method.AppendExclude(excludeAddrs)
 		case *cliconfig.ProviderInstallationNetworkMirror:
 			providerHandlers = append(providerHandlers, handlers.NewProviderNetworkMirrorHandler(providerService, cacheProviderHTTPStatusCode, method))
+			method.AppendExclude(excludeAddrs)
 		case *cliconfig.ProviderInstallationDirect:
 			providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, method))
+			method.RemoveExclude(excludeAddrs)
+			continue
 		}
-		method.AppendExclude(excludeAddrs)
 	}
 	providerHandlers = append(providerHandlers, handlers.NewProviderDirectHandler(providerService, cacheProviderHTTPStatusCode, new(cliconfig.ProviderInstallationDirect)))
 
diff --git a/docs/_docs/04_reference/cli-options.md b/docs/_docs/04_reference/cli-options.md
index 6519464b..65df489a 100644
--- a/docs/_docs/04_reference/cli-options.md
+++ b/docs/_docs/04_reference/cli-options.md
@@ -1318,7 +1318,7 @@ When this flag is set, Terragrunt will wrap Terraform `stdout` and `stderr` in J
 
 - [run-all](#run-all)
 
-Enables Terragrunt's provider caching. This forces Terraform to make provider requests through the Terragrunt Provider Cache server. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+Enables Terragrunt's provider caching. This forces Terraform to make provider requests through the Terragrunt Provider Cache server. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-provider-cache-dir
 
@@ -1328,7 +1328,7 @@ Enables Terragrunt's provider caching. This forces Terraform to make provider re
 
 - [run-all](#run-all)
 
-The path to the Terragrunt provider cache directory. By default, `terragrunt/providers` folder in the user cache directory: `$HOME/.cache` on Unix systems, `$HOME/Library/Caches` on Darwin, `%LocalAppData%` on Windows. The file structure of the cache directory is identical to the Terraform [plugin_cache_dir](https://developer.hashicorp.com/terraform/cli/config/config-file#provider-plugin-cache) directory. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+The path to the Terragrunt provider cache directory. By default, `terragrunt/providers` folder in the user cache directory: `$HOME/.cache` on Unix systems, `$HOME/Library/Caches` on Darwin, `%LocalAppData%` on Windows. The file structure of the cache directory is identical to the Terraform [plugin_cache_dir](https://developer.hashicorp.com/terraform/cli/config/config-file#provider-plugin-cache) directory. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-provider-cache-hostname
 
@@ -1338,7 +1338,7 @@ The path to the Terragrunt provider cache directory. By default, `terragrunt/pro
 
 - [run-all](#run-all)
 
-The hostname of the Terragrunt Provider Cache server. By default, 'localhost'. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+The hostname of the Terragrunt Provider Cache server. By default, 'localhost'. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-provider-cache-port
 
@@ -1348,7 +1348,7 @@ The hostname of the Terragrunt Provider Cache server. By default, 'localhost'. M
 
 - [run-all](#run-all)
 
-The port of the Terragrunt Provider Cache server. By default, assigned automatically. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+The port of the Terragrunt Provider Cache server. By default, assigned automatically. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-provider-cache-token
 
@@ -1358,7 +1358,7 @@ The port of the Terragrunt Provider Cache server. By default, assigned automatic
 
 - [run-all](#run-all)
 
-The Token for authentication on the Terragrunt Provider Cache server. By default, assigned automatically. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+The Token for authentication on the Terragrunt Provider Cache server. By default, assigned automatically. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-provider-cache-registry-names
 
@@ -1368,7 +1368,7 @@ The Token for authentication on the Terragrunt Provider Cache server. By default
 
 - [run-all](#run-all)
 
-The list of remote registries to cached by Terragrunt Provider Cache server. By default, 'registry.terraform.io', 'registry.opentofu.org'. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-caching/) for context.
+The list of remote registries to cached by Terragrunt Provider Cache server. By default, 'registry.terraform.io', 'registry.opentofu.org'. Make sure to read [Provider Caching](https://terragrunt.gruntwork.io/docs/features/provider-cache/) for context.
 
 ### terragrunt-out-dir
 
diff --git a/dynamodb/dynamo_lock_table.go b/dynamodb/dynamo_lock_table.go
index 4ca393f2..7151558f 100644
--- a/dynamodb/dynamo_lock_table.go
+++ b/dynamodb/dynamo_lock_table.go
@@ -2,10 +2,13 @@ package dynamodb
 
 import (
 	"fmt"
+	"net/http"
 	"time"
 
 	"github.com/aws/aws-sdk-go/aws"
 	"github.com/aws/aws-sdk-go/aws/awserr"
+	"github.com/aws/aws-sdk-go/aws/client"
+	"github.com/aws/aws-sdk-go/aws/request"
 	"github.com/aws/aws-sdk-go/service/dynamodb"
 	"github.com/gruntwork-io/go-commons/errors"
 	"github.com/gruntwork-io/terragrunt/aws_helper"
@@ -156,12 +159,34 @@ func tagTableIfTagsGiven(tags map[string]string, tableArn *string, client *dynam
 }
 
 // Delete the given table in DynamoDB
-func DeleteTable(tableName string, client *dynamodb.DynamoDB) error {
+func DeleteTable(tableName string, dbClient *dynamodb.DynamoDB) error {
+	const (
+		maxRetries    = 5
+		minRetryDelay = time.Second
+	)
+
 	tableCreateDeleteSemaphore.Acquire()
 	defer tableCreateDeleteSemaphore.Release()
 
-	_, err := client.DeleteTable(&dynamodb.DeleteTableInput{TableName: aws.String(tableName)})
-	return err
+	req, _ := dbClient.DeleteTableRequest(&dynamodb.DeleteTableInput{TableName: aws.String(tableName)})
+	// It is not always able to delete a table the first attempt, error: `StatusCode: 400, Attempt to change a resource which is still in use: Table tags are being updated: terragrunt_test_*`
+	req.Retryer = &DeleteTableRetryer{DefaultRetryer: client.DefaultRetryer{
+		NumMaxRetries: maxRetries,
+		MinRetryDelay: minRetryDelay,
+	}}
+	return req.Send()
+
+}
+
+type DeleteTableRetryer struct {
+	client.DefaultRetryer
+}
+
+func (retryer DeleteTableRetryer) ShouldRetry(req *request.Request) bool {
+	if req.HTTPResponse.StatusCode == http.StatusBadRequest {
+		return true
+	}
+	return retryer.DefaultRetryer.ShouldRetry(req)
 }
 
 // Return true if the given error is the error message returned by AWS when the resource already exists and is being
diff --git a/terraform/cliconfig/config.go b/terraform/cliconfig/config.go
index b4520235..2855c991 100644
--- a/terraform/cliconfig/config.go
+++ b/terraform/cliconfig/config.go
@@ -91,11 +91,11 @@ func (cfg *Config) AddHost(name string, services map[string]string) {
 //			exclude = ["example.com/*/*"]
 //		}
 //	}
-func (cfg *Config) AddProviderInstallationMethods(methods ...ProviderInstallationMethod) {
+func (cfg *Config) AddProviderInstallationMethods(newMethods ...ProviderInstallationMethod) {
 	if cfg.ProviderInstallation == nil {
 		cfg.ProviderInstallation = &ProviderInstallation{}
 	}
-	cfg.ProviderInstallation.Methods = append(cfg.ProviderInstallation.Methods, methods...)
+	cfg.ProviderInstallation.Methods = cfg.ProviderInstallation.Methods.Merge(newMethods...)
 }
 
 // Save marshalls and saves CLI config with the given config path.
diff --git a/terraform/cliconfig/provider_installation.go b/terraform/cliconfig/provider_installation.go
index 85ca597d..2703cb37 100644
--- a/terraform/cliconfig/provider_installation.go
+++ b/terraform/cliconfig/provider_installation.go
@@ -1,8 +1,47 @@
 package cliconfig
 
+import (
+	"encoding/json"
+	"fmt"
+	"sort"
+
+	"github.com/gruntwork-io/terragrunt/util"
+)
+
 // ProviderInstallation is the structure of the "provider_installation" nested block within the CLI configuration.
 type ProviderInstallation struct {
-	Methods []ProviderInstallationMethod `hcl:",block"`
+	Methods ProviderInstallationMethods `hcl:",block"`
+}
+
+type ProviderInstallationMethods []ProviderInstallationMethod
+
+func (methods ProviderInstallationMethods) Merge(withMethods ...ProviderInstallationMethod) ProviderInstallationMethods {
+	mergedMethods := methods
+
+	for _, withMethod := range withMethods {
+		var isMerged bool
+
+		for _, method := range methods {
+			if method.Merge(withMethod) {
+				isMerged = true
+				break
+			}
+		}
+
+		if !isMerged {
+			mergedMethods = append(mergedMethods, withMethod)
+		}
+	}
+
+	// place the `direct` method at the very end.
+	sort.Slice(mergedMethods, func(i, j int) bool {
+		if _, ok := mergedMethods[j].(*ProviderInstallationDirect); ok {
+			return true
+		}
+		return false
+	})
+
+	return mergedMethods
 }
 
 // ProviderInstallationMethod is an interface type representing the different installation path types and represents an installation method block inside a provider_installation block. The concrete implementations of this interface are:
@@ -10,7 +49,11 @@ type ProviderInstallation struct {
 //	ProviderInstallationDirect:           install from the provider's origin registry
 //	ProviderInstallationFilesystemMirror: install from a local filesystem mirror
 type ProviderInstallationMethod interface {
+	fmt.Stringer
+	AppendInclude(addrs []string)
 	AppendExclude(addrs []string)
+	RemoveExclude(addrs []string)
+	Merge(with ProviderInstallationMethod) bool
 }
 
 type ProviderInstallationDirect struct {
@@ -35,11 +78,54 @@ func NewProviderInstallationDirect(include, exclude []string) *ProviderInstallat
 	return res
 }
 
+func (method *ProviderInstallationDirect) Merge(with ProviderInstallationMethod) bool {
+	if with, ok := with.(*ProviderInstallationDirect); ok {
+		if with.Exclude != nil {
+			method.AppendExclude(*with.Exclude)
+		}
+		if with.Include != nil {
+			method.AppendInclude(*with.Include)
+		}
+		return true
+	}
+
+	return false
+}
+
+func (method *ProviderInstallationDirect) AppendInclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
+	if method.Include == nil {
+		method.Include = &[]string{}
+	}
+	*method.Include = util.RemoveDuplicatesFromList(append(*method.Include, addrs...))
+}
+
 func (method *ProviderInstallationDirect) AppendExclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
 	if method.Exclude == nil {
 		method.Exclude = &[]string{}
 	}
-	*method.Exclude = append(*method.Exclude, addrs...)
+	*method.Exclude = util.RemoveDuplicatesFromList(append(*method.Exclude, addrs...))
+}
+
+func (method *ProviderInstallationDirect) RemoveExclude(addrs []string) {
+	if len(addrs) == 0 || method.Exclude == nil {
+		return
+	}
+	*method.Exclude = util.RemoveSublistFromList(*method.Exclude, addrs)
+
+	if len(*method.Exclude) == 0 {
+		method.Exclude = nil
+	}
+}
+
+func (method *ProviderInstallationDirect) String() string {
+	b, _ := json.Marshal(method) //nolint:errcheck
+	return string(b)
 }
 
 type ProviderInstallationFilesystemMirror struct {
@@ -66,11 +152,54 @@ func NewProviderInstallationFilesystemMirror(path string, include, exclude []str
 	return res
 }
 
+func (method *ProviderInstallationFilesystemMirror) Merge(with ProviderInstallationMethod) bool {
+	if with, ok := with.(*ProviderInstallationFilesystemMirror); ok && method.Path == with.Path {
+		if with.Exclude != nil {
+			method.AppendExclude(*with.Exclude)
+		}
+		if with.Include != nil {
+			method.AppendInclude(*with.Include)
+		}
+		return true
+	}
+
+	return false
+}
+
+func (method *ProviderInstallationFilesystemMirror) AppendInclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
+	if method.Include == nil {
+		method.Include = &[]string{}
+	}
+	*method.Include = util.RemoveDuplicatesFromList(append(*method.Include, addrs...))
+}
+
 func (method *ProviderInstallationFilesystemMirror) AppendExclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
 	if method.Exclude == nil {
 		method.Exclude = &[]string{}
 	}
-	*method.Exclude = append(*method.Exclude, addrs...)
+	*method.Exclude = util.RemoveDuplicatesFromList(append(*method.Exclude, addrs...))
+}
+
+func (method *ProviderInstallationFilesystemMirror) RemoveExclude(addrs []string) {
+	if len(addrs) == 0 || method.Exclude == nil {
+		return
+	}
+	*method.Exclude = util.RemoveSublistFromList(*method.Exclude, addrs)
+
+	if len(*method.Exclude) == 0 {
+		method.Exclude = nil
+	}
+}
+
+func (method *ProviderInstallationFilesystemMirror) String() string {
+	b, _ := json.Marshal(method) //nolint:errcheck
+	return string(b)
 }
 
 type ProviderInstallationNetworkMirror struct {
@@ -97,9 +226,52 @@ func NewProviderInstallationNetworkMirror(url string, include, exclude []string)
 	return res
 }
 
+func (method *ProviderInstallationNetworkMirror) Merge(with ProviderInstallationMethod) bool {
+	if with, ok := with.(*ProviderInstallationNetworkMirror); ok && method.URL == with.URL {
+		if with.Exclude != nil {
+			method.AppendExclude(*with.Exclude)
+		}
+		if with.Include != nil {
+			method.AppendInclude(*with.Include)
+		}
+		return true
+	}
+
+	return false
+}
+
+func (method *ProviderInstallationNetworkMirror) AppendInclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
+	if method.Include == nil {
+		method.Include = &[]string{}
+	}
+	*method.Include = util.RemoveDuplicatesFromList(append(*method.Include, addrs...))
+}
+
 func (method *ProviderInstallationNetworkMirror) AppendExclude(addrs []string) {
+	if len(addrs) == 0 {
+		return
+	}
 	if method.Exclude == nil {
 		method.Exclude = &[]string{}
 	}
-	*method.Exclude = append(*method.Exclude, addrs...)
+	*method.Exclude = util.RemoveDuplicatesFromList(append(*method.Exclude, addrs...))
+}
+
+func (method *ProviderInstallationNetworkMirror) RemoveExclude(addrs []string) {
+	if len(addrs) == 0 || method.Exclude == nil {
+		return
+	}
+	*method.Exclude = util.RemoveSublistFromList(*method.Exclude, addrs)
+
+	if len(*method.Exclude) == 0 {
+		method.Exclude = nil
+	}
+}
+
+func (method *ProviderInstallationNetworkMirror) String() string {
+	b, _ := json.Marshal(method) //nolint:errcheck
+	return string(b)
 }



Test patch:
diff --git a/test/cliconfig.go b/test/cliconfig.go
index 90c52bd7..48129ad2 100644
--- a/test/cliconfig.go
+++ b/test/cliconfig.go
@@ -18,7 +18,7 @@ provider_installation {
 {{ if gt (len $method.Include) 0 }}
     include = [{{ range $index, $include := $method.Include }}{{ if $index }},{{ end }}"{{ $include }}"{{ end }}]
 {{ end }}{{ if gt (len $method.Exclude) 0 }}
-    include = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
+    exclude = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
 {{ end }}
   }
 {{ end }}{{ end }}
@@ -28,7 +28,7 @@ provider_installation {
 {{ if gt (len $method.Include) 0 }}
     include = [{{ range $index, $include := $method.Include }}{{ if $index }},{{ end }}"{{ $include }}"{{ end }}]
 {{ end }}{{ if gt (len $method.Exclude) 0 }}
-    include = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
+    exclude = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
 {{ end }}
   }
 {{ end }}{{ end }}
@@ -37,7 +37,7 @@ provider_installation {
 {{ if gt (len $method.Include) 0 }}
     include = [{{ range $index, $include := $method.Include }}{{ if $index }},{{ end }}"{{ $include }}"{{ end }}]
 {{ end }}{{ if gt (len $method.Exclude) 0 }}
-    include = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
+    exclude = [{{ range $index, $exclude := $method.Exclude }}{{ if $index }},{{ end }}"{{ $exclude }}"{{ end }}]
 {{ end }}
   }
 {{ end }}{{ end }}
diff --git a/test/fixture-provider-cache/mirror/app/main.tf b/test/fixture-provider-cache/mirror/app/main.tf
index ac294110..3db6ee5f 100644
--- a/test/fixture-provider-cache/mirror/app/main.tf
+++ b/test/fixture-provider-cache/mirror/app/main.tf
@@ -1,8 +1,15 @@
 terraform {
   required_providers {
     null = {
-      source  = "example.com/hashicorp/null"
-      version = "3.2.2"
+      source = "registry.opentofu.org/hashicorp/null"
+    }
+    aws = {
+      source  = "example.com/hashicorp/aws"
+      version = "5.59.0"
+    }
+    azurerm = {
+      source  = "example.com/hashicorp/azurerm"
+      version = "3.113.0"
     }
   }
 }
diff --git a/test/integration_serial_test.go b/test/integration_serial_test.go
index f7d3c6ed..22b6db12 100644
--- a/test/integration_serial_test.go
+++ b/test/integration_serial_test.go
@@ -44,8 +44,18 @@ func TestTerragruntProviderCacheWithFilesystemMirror(t *testing.T) {
 	fakeProvider := FakeProvider{
 		RegistryName: "example.com",
 		Namespace:    "hashicorp",
-		Name:         "null",
-		Version:      "3.2.2",
+		Name:         "aws",
+		Version:      "5.59.0",
+		PlatformOS:   runtime.GOOS,
+		PlatformArch: runtime.GOARCH,
+	}
+	fakeProvider.CreateMirror(t, providersMirrorPath)
+
+	fakeProvider = FakeProvider{
+		RegistryName: "example.com",
+		Namespace:    "hashicorp",
+		Name:         "azurerm",
+		Version:      "3.113.0",
 		PlatformOS:   runtime.GOOS,
 		PlatformArch: runtime.GOARCH,
 	}
@@ -69,13 +79,23 @@ func TestTerragruntProviderCacheWithFilesystemMirror(t *testing.T) {
 	cliConfigSettings := &CLIConfigSettings{
 		FilesystemMirrorMethods: []CLIConfigProviderInstallationFilesystemMirror{
 			{
-				Path: providersMirrorPath,
+				Path:    providersMirrorPath,
+				Include: []string{"example.com/*/*"},
 			},
 		},
 	}
 	createCLIConfig(t, cliConfigFilename, cliConfigSettings)
 
-	runTerragrunt(t, fmt.Sprintf("terragrunt run-all init --terragrunt-provider-cache --terragrunt-provider-cache-registry-names example.com --terragrunt-provider-cache-dir %s --terragrunt-log-level trace --terragrunt-non-interactive --terragrunt-working-dir %s", providerCacheDir, appPath))
+	runTerragrunt(t, fmt.Sprintf("terragrunt run-all init --terragrunt-provider-cache --terragrunt-provider-cache-registry-names example.com --terragrunt-provider-cache-registry-names registry.terraform.io --terragrunt-provider-cache-dir %s --terragrunt-log-level trace --terragrunt-non-interactive --terragrunt-working-dir %s", providerCacheDir, appPath))
+
+	expectedProviderInstallation := `provider_installation { "filesystem_mirror" { path = "%s" include = ["example.com/*/*"] exclude = ["example.com/*/*", "registry.terraform.io/*/*"] } "filesystem_mirror" { path = "%s" include = ["example.com/*/*", "registry.terraform.io/*/*"] } "direct" { } }`
+	expectedProviderInstallation = fmt.Sprintf(strings.Join(strings.Fields(expectedProviderInstallation), " "), providersMirrorPath, providerCacheDir)
+
+	terraformrcBytes, err := os.ReadFile(filepath.Join(appPath, ".terraformrc"))
+	require.NoError(t, err)
+	terraformrc := strings.Join(strings.Fields(string(terraformrcBytes)), " ")
+
+	assert.Contains(t, terraformrc, expectedProviderInstallation, "%s\n\n%s", terraformrc, expectedProviderInstallation)
 }
 
 func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
@@ -86,20 +106,31 @@ func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
 	rootPath := util.JoinPath(tmpEnvPath, TEST_FIXTURE_PROVIDER_CACHE_MIRROR)
 
 	appPath := filepath.Join(rootPath, "app")
-	providersMirrorPath := filepath.Join(rootPath, "providers-mirror")
+	providersNetkworMirrorPath := filepath.Join(rootPath, "providers-network-mirror")
+	providersFilesystemMirrorPath := filepath.Join(rootPath, "providers-filesystem-mirror")
 
 	ctx, cancel := context.WithCancel(context.Background())
 	defer cancel()
 
-	fakeProvider := FakeProvider{
+	netowrkProvider := FakeProvider{
 		RegistryName: "example.com",
 		Namespace:    "hashicorp",
-		Name:         "null",
-		Version:      "3.2.2",
+		Name:         "aws",
+		Version:      "5.59.0",
 		PlatformOS:   runtime.GOOS,
 		PlatformArch: runtime.GOARCH,
 	}
-	fakeProvider.CreateMirror(t, providersMirrorPath)
+	netowrkProvider.CreateMirror(t, providersNetkworMirrorPath)
+
+	filesystemProvider := FakeProvider{
+		RegistryName: "example.com",
+		Namespace:    "hashicorp",
+		Name:         "azurerm",
+		Version:      "3.113.0",
+		PlatformOS:   runtime.GOOS,
+		PlatformArch: runtime.GOARCH,
+	}
+	filesystemProvider.CreateMirror(t, providersFilesystemMirrorPath)
 
 	// when we run NetworkMirrorServer, we override the default transport to configure the self-signed certificate, we need to restor, after finishing we need to restore this value
 	defaultTransport := http.DefaultTransport
@@ -107,9 +138,10 @@ func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
 		http.DefaultTransport = defaultTransport
 	}()
 
-	networkMirrorURL := runNetworkMirrorServer(t, ctx, "/providers/", providersMirrorPath)
-	t.Logf("Provdiers mirror path: %s", providersMirrorPath)
+	networkMirrorURL := runNetworkMirrorServer(t, ctx, "/providers/", providersNetkworMirrorPath)
 	t.Logf("Network mirror URL: %s", networkMirrorURL)
+	t.Logf("Provdiers network mirror path: %s", providersNetkworMirrorPath)
+	t.Logf("Provdiers filesysmte mirror path: %s", providersFilesystemMirrorPath)
 
 	providerCacheDir := filepath.Join(rootPath, "providers-cache")
 
@@ -124,15 +156,37 @@ func TestTerragruntProviderCacheWithNetworkMirror(t *testing.T) {
 	t.Logf("%s=%s", terraform.EnvNameTFCLIConfigFile, cliConfigFilename.Name())
 
 	cliConfigSettings := &CLIConfigSettings{
+		DirectMethods: []CLIConfigProviderInstallationDirect{
+			{
+				Exclude: []string{"example.com/*/*"},
+			},
+		},
+		FilesystemMirrorMethods: []CLIConfigProviderInstallationFilesystemMirror{
+			{
+				Path:    providersFilesystemMirrorPath,
+				Include: []string{"example.com/hashicorp/azurerm"},
+			},
+		},
 		NetworkMirrorMethods: []CLIConfigProviderInstallationNetworkMirror{
 			{
-				URL: networkMirrorURL.String(),
+				URL:     networkMirrorURL.String(),
+				Include: []string{"example.com/hashicorp/aws"},
 			},
 		},
 	}
 	createCLIConfig(t, cliConfigFilename, cliConfigSettings)
 
-	runTerragrunt(t, fmt.Sprintf("terragrunt run-all init --terragrunt-provider-cache --terragrunt-provider-cache-registry-names example.com --terragrunt-provider-cache-dir %s --terragrunt-log-level trace --terragrunt-non-interactive --terragrunt-working-dir %s", providerCacheDir, appPath))
+	runTerragrunt(t, fmt.Sprintf("terragrunt run-all init --terragrunt-provider-cache --terragrunt-provider-cache-registry-names example.com --terragrunt-provider-cache-registry-names registry.terraform.io --terragrunt-provider-cache-dir %s --terragrunt-log-level trace --terragrunt-non-interactive --terragrunt-working-dir %s", providerCacheDir, appPath))
+
+	expectedProviderInstallation := `provider_installation { "filesystem_mirror" { path = "%s" include = ["example.com/hashicorp/azurerm"] exclude = ["example.com/*/*", "registry.terraform.io/*/*"] } "network_mirror" { url = "%s" include = ["example.com/hashicorp/aws"] exclude = ["example.com/*/*", "registry.terraform.io/*/*"] } "filesystem_mirror" { path = "%s" include = ["example.com/*/*", "registry.terraform.io/*/*"] } "direct" { } }`
+
+	expectedProviderInstallation = fmt.Sprintf(strings.Join(strings.Fields(expectedProviderInstallation), " "), providersFilesystemMirrorPath, networkMirrorURL.String(), providerCacheDir)
+
+	terraformrcBytes, err := os.ReadFile(filepath.Join(appPath, ".terraformrc"))
+	require.NoError(t, err)
+	terraformrc := strings.Join(strings.Fields(string(terraformrcBytes)), " ")
+
+	assert.Contains(t, terraformrc, expectedProviderInstallation, "%s\n\n%s", terraformrc, expectedProviderInstallation)
 }
 
 func TestTerragruntInputsFromDependency(t *testing.T) {