Repository: gruntwork-io/terragrunt

Issue:
Add option to only check top level dependency blocks
## Current behavior

If I have a module named MainModule with two dependency blocks, and those two dependencies each depend on a few other modules, and I run a `terragrunt plan` on MainModule, terragrunt will run a `terraform output -json` to check the state of all modules described in this scenario, not just on the two top level dependencies.

## What I don't love about this

### It is slower

In my real world terragrunt config, we have hundreds of modules, some of which have over a dozen dependencies, each of which have their own. Even with provider caching, I still see cases where a `terragrunt plan` takes over ten minutes because it checks all of the state in series without parallelization.

### It goes against my intuition of the permissions model

This one is highly opinionated, so I only mean it as a suggestion. But say I have the following modules:

- DevOrg module, which creates an AWS organization for dev resources
- ProdOrg module, which creates an AWS organization for prod resources
- CommonEcr module, which creates a Docker repo on AWS Ecr with read access from DevOrg and ProdOrg
- EcsDevModule, which depends on that ECR module to get the url of the docker image to use

Running a plan on EcsDevModule would have a deep level dependency on being able to access the Prod env state, which I do no like. Instead, I would prefer if I only had to give access to CommonsOrg state.

Hints from issue comments:
@yorinasub17: That's a fair point. This happens because getting the output calls `RunTerragrunt` [here](https://github.com/gruntwork-io/terragrunt/blob/6c304f91690badae3c2d64bf5b2c98d38368621e/config/dependency.go#L354), and that function is currently parsing the target config in full.

The implementation to fix this is a bit involved though because it means we need to update the `RunTerragrunt` routine to do "just-in-time" config parsing. That is, it needs to index all the terraform commands and construct a mapping so that it only parses the relevant pieces of the config for each command. E.g., `output` may only need the `terraform` block, while `apply` needs all the `inputs`, unless it has a plan file (you can see how this can get complicated).

Note that we can't just parse out the `terraform` block for calling `output` and be done with it. The crux of the issue is parsing out the `dependency` blocks from the target config, and avoiding that requires an update to the configuration parsing mechanism. Right now, we parse `dependency` blocks before `terraform` blocks (see https://terragrunt.gruntwork.io/docs/getting-started/configuration/#configuration-parsing-order) because we want to support the usage of `dependency` in setting `hooks` and `extra_arguments` and the like. Updating that means we need some kind graph based parsing mechanism so that we only parse `dependency` blocks if it is used: a very involved refactor!

I'd be willing to overhaul the configuration parsing mechanism, but right now we are buried with other initiatives to take on this. If anyone else in the community wants to take a crack at this, please submit an RFC detailing the approach given the changes involved.@dmattia: Awesome, thanks for the write up @yorinasub17! I won't be able to work on an RFC in the near future, but will keep this in my back log just in case I ever have time.@yorinasub17: I came up with a potential intermediate solution that I hope will cover 80% of use cases. Check out the PR description for more info: https://github.com/gruntwork-io/terragrunt/pull/1311@dmattia: I think that is an excellent solution!@yorinasub17: The intermediate optimization is now released as https://github.com/gruntwork-io/terragrunt/releases/tag/v0.23.35 (binaries will show up shortly).@yorinasub17: Keeping this issue open as this update does not allow this optimization for all use cases. In particular, we've been directing folks to start using `generate` blocks to manage state backends that are not `s3` or `gcs`, which means those folks won't benefit from the optimization.

Will keep this ticket open until we can support `generate` block based backend configuration.@dmattia: Just tested it out and it's working wonderfully!

<img width="391" alt="Screen Shot 2020-08-26 at 12 23 05 AM" src="https://user-images.githubusercontent.com/8922077/91258592-55acf200-e732-11ea-889f-243c3ae54869.png">

Pictured is a module with ~100 nested dependencies, but only the 11 declared dependencies that you see in the output before the state lock is acquired 😄 

When I use v0.23.34 without this optimization, you can see that there are many more dependencies:

<img width="393" alt="Screen Shot 2020-08-26 at 12 29 21 AM" src="https://user-images.githubusercontent.com/8922077/91259017-3febfc80-e733-11ea-89cf-b9f8532455ef.png">

The total time to run this module is down from 2m13s to 40s, but there's probs 15ish seconds just to aquire the state lock and refresh the state, so this is extremely efficient now in my mind.

Thank you so much for all your work on this!@yorinasub17: That's great to hear @dmattia ! Thanks for sharing!@github-actions: This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for raising this issue.

Solution patch:
diff --git a/Makefile b/Makefile
index 7a2cf748..af052150 100644
--- a/Makefile
+++ b/Makefile
@@ -25,4 +25,17 @@ install-pre-commit-hook:
 	@ln -s scripts/pre-commit .git/hooks/pre-commit
 	@echo "pre-commit hook installed."
 
-.PHONY: help fmtcheck fmt install-fmt-hook
+
+# This build target just for convenience for those building directly from
+# source. See also: .circleci/config.yml
+build: terragrunt
+terragrunt: $(shell find . \( -type d -name 'vendor' -prune \) \
+                        -o \( -type f -name '*.go'   -print \) )
+	set -xe ;\
+	vtag_maybe_extra=$$(git describe --tags --abbrev=12 --dirty --broken) ;\
+	go build -o $@ -ldflags "-X main.VERSION=$${vtag_maybe_extra}" .
+
+clean:
+	rm -f terragrunt
+
+.PHONY: help fmtcheck fmt install-fmt-hook clean
diff --git a/_docs/migration_guides/upgrading_to_terragrunt_0.12.x.md b/_docs/migration_guides/upgrading_to_terragrunt_0.12.x.md
deleted file mode 100644
index ff7e3f12..00000000
--- a/_docs/migration_guides/upgrading_to_terragrunt_0.12.x.md
+++ /dev/null
@@ -1,176 +0,0 @@
-# Upgrading to Terragrunt 0.12.x
-
-## Background
-
-Terragrunt was originally created to support two features that were not available in Terraform: defining remote state
-configuration in a file (rather than via CLI commands) and locking. As of version 0.9.0, Terraform now supports both of
-these features natively, so we have made some changes to Terragrunt:
-
-1. Terragrunt still supports remote state configuration, so you can take advantage of Terragrunt's interpolation
-   functions.
-1. Terragrunt no longer supports locking.
-
-
-## Migration guide
-
-The following sections outlines the steps you may need to take in order to migrate from Terragrunt <= v0.11.x
-to Terragrunt 0.12.x. If you are using Terraform <= 0.8.x, see the upgrade guide on the Terraform website
-[Upgrading to Terraform v0.9](https://www.terraform.io/upgrade-guides/0-9.html).
-
-Migration steps include:
-
-* [Define a backend in your Terraform files](#define-a-backend-in-your-terraform-files)
-* [Switch from Terragrunt locking to Terraform locking](#switch-from-terragrunt-locking-to-terraform-locking)
-* [Migrate from .terragrunt to terraform.tfvars](#migrate-from-terragrunt-to-terraformtfvars)
-
-
-### Define a backend in your Terraform files
-
-As of Terraform 0.9.0, you must define a `backend` in your Terraform code (the `.tf` files). The Terraform guides 
-[Upgrading to Terraform v0.9](https://www.terraform.io/upgrade-guides/0-9.html) and 
-[Backends: Migrating From 0.8.x and Earlier](https://www.terraform.io/docs/backends/legacy-0-8.html)
-may be useful to understand backends.
-
-For example, to use S3 as a remote state
-backend, you will need to add the following to your Terraform code:
-
-```hcl
-# main.tf
-terraform {
-  # The configuration for this backend will be filled in by Terragrunt
-  backend "s3" {}
-}
-```
-
-Note that you can leave the configuration of the `backend` empty and allow Terragrunt to provide that configuration
-instead. This allows you to keep your remote state configuration more DRY by taking advantage of Terragrunt's
-interpolation functions:
-
-```hcl
-# terraform.tfvars
-terragrunt = {
-  remote_state {
-    backend = "s3"
-    config {
-      bucket  = "my-terraform-state"
-      key     = "${path_relative_to_include()}/terraform.tfstate"
-      region  = "us-east-1"
-      encrypt = true
-    }
-  }
-}
-```
-
-Note: when switching from Terragrunt v0.11.x to v0.12.x, you also need to change the `encrypt` setting from a string (e.g. `"true"`) to a boolean (e.g. `true` without the quotes).
-
-### Switch from Terragrunt locking to Terraform locking
-
-Remove any `lock { ... }` blocks from your Terragrunt configurations, as these are no longer supported.
-
-If you were storing remote state in S3 and relying on DynamoDB as a locking mechanism, Terraform now supports that
-natively. To enable it, simply add the `lock_table` parameter to your S3 backend configuration. If you configure
-your S3 backend using Terragrunt, then Terragrunt will automatically create the `lock_table` for you if that table
-doesn't already exist:
-
-```hcl
-# terraform.tfvars
-terragrunt = {
-  remote_state {
-    backend = "s3"
-    config {
-      bucket  = "my-terraform-state"
-      key     = "${path_relative_to_include()}/terraform.tfstate"
-      region  = "us-east-1"
-      encrypt = true
-
-      # Tell Terraform to do locking using DynamoDB. Terragrunt will automatically
-      # create this table for you if it doesn't already exist.
-      lock_table = "my-lock-table"
-    }
-  }
-}
-```
-
-**NOTE**: We recommend using a completely new lock table name and NOT reusing the lock table from older versions of
-Terragrunt, as that older table had a different structure than what Terraform expects, and Terragrunt will not
-automatically recreate it.
-
-If you would like Terraform to automatically retry locks like Terragrunt did (this is particularly useful when
-running Terraform as part of an automated script, such as a CI build), you use an `extra_arguments` block:
-
-```hcl
-# terraform.tfvars
-terragrunt = {
-  remote_state {
-    backend = "s3"
-    config {
-      bucket  = "my-terraform-state"
-      key     = "${path_relative_to_include()}/terraform.tfstate"
-      region  = "us-east-1"
-      encrypt = true
-
-      # Tell Terraform to do locking using DynamoDB. Terragrunt will automatically
-      # create this table for you if it doesn't already exist.
-      lock_table = "my-lock-table"
-    }
-  }
-
-  terraform {
-    # Force Terraform to keep trying to acquire a lock for up to 20 minutes
-    # if someone else already has the lock.
-    extra_arguments "retry_lock" {
-      commands = [
-        "init",
-        "apply",
-        "refresh",
-        "import",
-        "plan",
-        "taint",
-        "untaint"
-      ]
-
-      arguments = [
-        "-lock-timeout=20m"
-      ]
-    }
-  }
-}
-```
-
-
-### Migrate from .terragrunt to terraform.tfvars
-
-The configuration in a `.terragrunt` file is identical to that of the `terraform.tfvars` file, except the
-`terraform.tfvars` file requires you to wrap that configuration in a `terragrunt = { ... }` block.
-
-For example, if this is your `.terragrunt` file:
-
-```hcl
-include {
-  path = "${find_in_parent_folders()}"
-}
-
-dependencies {
-  paths = ["../vpc", "../mysql", "../redis"]
-}
-```
-
-The equivalent `terraform.tfvars` file is:
-
-```hcl
-terragrunt = {
-  include {
-    path = "${find_in_parent_folders()}"
-  }
-
-  dependencies {
-    paths = ["../vpc", "../mysql", "../redis"]
-  }
-}
-```
-
-To migrate, all you need to do is:
-
-1. Copy all the contents of the `.terragrunt` file.
-1. Paste those contents into a `terragrunt = { ... }` block in a `terraform.tfvars` file.
-1. Delete the `.terragrunt` file.
diff --git a/config/config.go b/config/config.go
index 5e441035..2b8324be 100644
--- a/config/config.go
+++ b/config/config.go
@@ -90,6 +90,35 @@ func (remoteState *remoteStateConfigFile) String() string {
 	return fmt.Sprintf("remoteStateConfigFile{Backend = %v, Config = %v}", remoteState.Backend, remoteState.Config)
 }
 
+// Convert the parsed config file remote state struct to the internal representation struct of remote state
+// configurations.
+func (remoteState *remoteStateConfigFile) toConfig() (*remote.RemoteState, error) {
+	remoteStateConfig, err := parseCtyValueToMap(remoteState.Config)
+	if err != nil {
+		return nil, err
+	}
+
+	config := &remote.RemoteState{}
+	config.Backend = remoteState.Backend
+	if remoteState.Generate != nil {
+		config.Generate = &remote.RemoteStateGenerate{
+			Path:     remoteState.Generate.Path,
+			IfExists: remoteState.Generate.IfExists,
+		}
+	}
+	config.Config = remoteStateConfig
+
+	if remoteState.DisableInit != nil {
+		config.DisableInit = *remoteState.DisableInit
+	}
+
+	config.FillDefaults()
+	if err := config.Validate(); err != nil {
+		return nil, err
+	}
+	return config, err
+}
+
 type remoteStateConfigGenerate struct {
 	// We use cty instead of hcl, since we are using this type to convert an attr and not a block.
 	Path     string `cty:"path"`
@@ -679,30 +708,10 @@ func convertToTerragruntConfig(
 	}
 
 	if terragruntConfigFromFile.RemoteState != nil {
-		remoteStateConfig, err := parseCtyValueToMap(terragruntConfigFromFile.RemoteState.Config)
+		remoteState, err := terragruntConfigFromFile.RemoteState.toConfig()
 		if err != nil {
 			return nil, err
 		}
-
-		remoteState := &remote.RemoteState{}
-		remoteState.Backend = terragruntConfigFromFile.RemoteState.Backend
-		if terragruntConfigFromFile.RemoteState.Generate != nil {
-			remoteState.Generate = &remote.RemoteStateGenerate{
-				Path:     terragruntConfigFromFile.RemoteState.Generate.Path,
-				IfExists: terragruntConfigFromFile.RemoteState.Generate.IfExists,
-			}
-		}
-		remoteState.Config = remoteStateConfig
-
-		if terragruntConfigFromFile.RemoteState.DisableInit != nil {
-			remoteState.DisableInit = *terragruntConfigFromFile.RemoteState.DisableInit
-		}
-
-		remoteState.FillDefaults()
-		if err := remoteState.Validate(); err != nil {
-			return nil, err
-		}
-
 		terragruntConfig.RemoteState = remoteState
 	}
 
diff --git a/config/config_partial.go b/config/config_partial.go
index 7806aca1..3446d4f1 100644
--- a/config/config_partial.go
+++ b/config/config_partial.go
@@ -24,6 +24,7 @@ const (
 	TerraformSource
 	TerragruntFlags
 	TerragruntVersionConstraints
+	RemoteStateBlock
 )
 
 // terragruntInclude is a struct that can be used to only decode the include block.
@@ -79,6 +80,12 @@ type terragruntDependency struct {
 	Remain       hcl.Body     `hcl:",remain"`
 }
 
+// terragruntRemoteState is a struct that can be used to only decode the remote_state blocks in the terragrunt config
+type terragruntRemoteState struct {
+	RemoteState *remoteStateConfigFile `hcl:"remote_state,block"`
+	Remain      hcl.Body               `hcl:",remain"`
+}
+
 // DecodeBaseBlocks takes in a parsed HCL2 file and decodes the base blocks. Base blocks are blocks that should always
 // be decoded even in partial decoding, because they provide bindings that are necessary for parsing any block in the
 // file. Currently base blocks are:
@@ -148,6 +155,7 @@ func PartialParseConfigFile(
 // - TerragruntFlags: Parses the boolean flags `prevent_destroy` and `skip` in the config
 // - TerragruntVersionConstraints: Parses the attributes related to constraining terragrunt and terraform versions in
 //                                 the config.
+// - RemoteStateBlock: Parses the `remote_state` block in the config
 // Note that the following blocks are always decoded:
 // - locals
 // - include
@@ -263,6 +271,20 @@ func PartialParseConfigString(
 				output.TerraformBinary = *decoded.TerraformBinary
 			}
 
+		case RemoteStateBlock:
+			decoded := terragruntRemoteState{}
+			err := decodeHcl(file, filename, &decoded, terragruntOptions, contextExtensions)
+			if err != nil {
+				return nil, err
+			}
+			if decoded.RemoteState != nil {
+				remoteState, err := decoded.RemoteState.toConfig()
+				if err != nil {
+					return nil, err
+				}
+				output.RemoteState = remoteState
+			}
+
 		default:
 			return nil, InvalidPartialBlockName{decode}
 		}
diff --git a/config/dependency.go b/config/dependency.go
index d05dea2f..b340b208 100644
--- a/config/dependency.go
+++ b/config/dependency.go
@@ -6,6 +6,8 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
+	"io/ioutil"
+	"os"
 	"path/filepath"
 	"strings"
 	"sync"
@@ -19,6 +21,8 @@ import (
 
 	"github.com/gruntwork-io/terragrunt/errors"
 	"github.com/gruntwork-io/terragrunt/options"
+	"github.com/gruntwork-io/terragrunt/remote"
+	"github.com/gruntwork-io/terragrunt/shell"
 	"github.com/gruntwork-io/terragrunt/util"
 )
 
@@ -326,7 +330,7 @@ func getOutputJsonWithCaching(targetConfig string, terragruntOptions *options.Te
 	}
 
 	// Cache miss, so look up the output and store in cache
-	newJsonBytes, err := runTerragruntOutputJson(terragruntOptions, targetConfig)
+	newJsonBytes, err := getTerragruntOutputJson(terragruntOptions, targetConfig)
 	if err != nil {
 		return nil, err
 	}
@@ -382,20 +386,92 @@ func cloneTerragruntOptionsForDependencyOutput(terragruntOptions *options.Terrag
 	return targetOptions, nil
 }
 
-// runTerragruntOutputJson uses terragrunt running functions to extract the json output from the target config. Make a
-// copy of the terragruntOptions so that we can reuse the same execution environment.
-func runTerragruntOutputJson(terragruntOptions *options.TerragruntOptions, targetConfig string) ([]byte, error) {
-	targetOptions, err := cloneTerragruntOptionsForDependencyOutput(terragruntOptions, targetConfig)
+// Retrieve the outputs from the terraform state in the target configuration. This attempts to optimize the output
+// retrieval if the following conditions are true:
+// - State backends are managed with a `remote_state` block.
+// - The `remote_state` block is using the generator pattern (the `generate` attribute is set).
+// - The `remote_state` block does not depend on any `dependency` outputs.
+// If these conditions are met, terragrunt can optimize the retrieval to avoid recursively retrieving dependency outputs
+// by directly pulling down the state file. Otherwise, terragrunt will fallback to running `terragrunt output` on the
+// target module.
+func getTerragruntOutputJson(terragruntOptions *options.TerragruntOptions, targetConfig string) ([]byte, error) {
+	// Make a copy of the terragruntOptions so that we can reuse the same execution environment, but in the context of
+	// the target config.
+	targetTGOptions, err := cloneTerragruntOptionsForDependencyOutput(terragruntOptions, targetConfig)
 	if err != nil {
 		return nil, err
 	}
 
+	// First, attempt to parse the `remote_state` blocks without parsing/getting dependency outputs. If this is
+	// possible, proceed to routine that fetches remote state directly. Otherwise, fallback to calling
+	// `terragrunt output` directly.
+	remoteStateTGConfig, err := PartialParseConfigFile(targetConfig, targetTGOptions, nil, []PartialDecodeSectionType{RemoteStateBlock})
+	if err != nil || !canGetRemoteState(remoteStateTGConfig.RemoteState) {
+		terragruntOptions.Logger.Printf("WARNING: Could not parse remote_state block from target config %s", targetConfig)
+		terragruntOptions.Logger.Printf("WARNING: Falling back to terragrunt output.")
+		return runTerragruntOutputJson(targetTGOptions, targetConfig)
+	}
+	return getTerragruntOutputJsonFromRemoteState(targetTGOptions, targetConfig, remoteStateTGConfig.RemoteState)
+}
+
+// canGetRemoteState returns true if the remote state block is not nil and has the generate block configured.
+func canGetRemoteState(remoteState *remote.RemoteState) bool {
+	return remoteState != nil && remoteState.Generate != nil
+}
+
+// getTerragruntOutputJsonFromRemoteState will retrieve the outputs directly by using just the remote state block. This
+// uses terraform's feature where `output` and `init` can work without the real source, as long as you have the
+// `backend` configured.
+// To do this, this function will:
+// - Create a temporary folder
+// - Generate the backend.tf file with the backend configuration from the remote_state block
+// - Run terraform init and terraform output
+// - Clean up folder once json file is generated
+// NOTE: targetTGOptions should be in the context of the targetConfig.
+func getTerragruntOutputJsonFromRemoteState(targetTGOptions *options.TerragruntOptions, targetConfig string, remoteState *remote.RemoteState) ([]byte, error) {
+	util.Debugf(targetTGOptions.Logger, "Detected remote state block with generate config. Resolving dependency by pulling remote state.")
+
+	// Create working directory where we will run terraform in. Make sure it is cleaned up before the function returns.
+	tempWorkDir, err := ioutil.TempDir("", "")
+	if err != nil {
+		return nil, err
+	}
+	defer os.RemoveAll(tempWorkDir)
+
+	// Set the terraform working dir to the tempdir, and set stdout writer to ioutil.Discard so that output content is
+	// not logged.
+	targetTGOptions.WorkingDir = tempWorkDir
+	targetTGOptions.Writer = ioutil.Discard
+
+	// Generate the backend configuration in the working dir
+	if err := remoteState.GenerateTerraformCode(targetTGOptions); err != nil {
+		return nil, err
+	}
+
+	// The working directory is now set up to interact with the state, so pull it down to get the json output.
+	// First run init, then output.
+	if err := shell.RunTerraformCommand(targetTGOptions, "init"); err != nil {
+		return nil, err
+	}
+	out, err := shell.RunTerraformCommandWithOutput(targetTGOptions, "output", "-json")
+	if err != nil {
+		return nil, err
+	}
+	jsonString := out.Stdout
+	jsonBytes := []byte(strings.TrimSpace(jsonString))
+	util.Debugf(targetTGOptions.Logger, "Retrieved output from %s as json: %s", targetConfig, jsonString)
+	return jsonBytes, nil
+}
+
+// runTerragruntOutputJson uses terragrunt running functions to extract the json output from the target config.
+// NOTE: targetTGOptions should be in the context of the targetConfig.
+func runTerragruntOutputJson(targetTGOptions *options.TerragruntOptions, targetConfig string) ([]byte, error) {
 	// Update the stdout buffer so we can capture the output
 	var stdoutBuffer bytes.Buffer
 	stdoutBufferWriter := bufio.NewWriter(&stdoutBuffer)
-	targetOptions.Writer = stdoutBufferWriter
+	targetTGOptions.Writer = stdoutBufferWriter
 
-	err = targetOptions.RunTerragrunt(targetOptions)
+	err := targetTGOptions.RunTerragrunt(targetTGOptions)
 	if err != nil {
 		return nil, errors.WithStackTrace(err)
 	}
@@ -403,7 +479,7 @@ func runTerragruntOutputJson(terragruntOptions *options.TerragruntOptions, targe
 	stdoutBufferWriter.Flush()
 	jsonString := stdoutBuffer.String()
 	jsonBytes := []byte(strings.TrimSpace(jsonString))
-	util.Debugf(terragruntOptions.Logger, "Retrieved output from %s as json: %s", targetConfig, jsonString)
+	util.Debugf(targetTGOptions.Logger, "Retrieved output from %s as json: %s", targetConfig, jsonString)
 	return jsonBytes, nil
 }
 
diff --git a/docs/_docs/03_community/contributing.md b/docs/_docs/03_community/contributing.md
index ce823380..0023cd47 100644
--- a/docs/_docs/03_community/contributing.md
+++ b/docs/_docs/03_community/contributing.md
@@ -37,13 +37,13 @@ code. If there is anything you're unsure about, just ask!
 Sometimes, the scope of the feature proposal is large enough that it requires major updates to the code base to
 implement. In these situations, a maintainer may suggest writing up an RFC that describes the feature in more details
 than what can be reasonably captured in a Github Issue. RFCs are written in markdown and live in the directory
-`_docs/rfc`.
+`docs/_docs/05_rfc`.
 
 To write an RFC:
 
 - Clone the repository
 - Create a new branch
-- Copy the template (`_docs/rfc/TEMPLATE.md`) to a new file in the same directory.
+- Copy the template (`docs/_docs/05_rfc/template.md`) to a new file in the same directory.
 - Fill out the template
 - Open a PR for comments, prefixing the title with the term `[RFC]` to indicate that it is an RFC PR.
 
diff --git a/docs/_docs/04_reference/config-blocks-and-attributes.md b/docs/_docs/04_reference/config-blocks-and-attributes.md
index 6e06a59f..83deb64c 100644
--- a/docs/_docs/04_reference/config-blocks-and-attributes.md
+++ b/docs/_docs/04_reference/config-blocks-and-attributes.md
@@ -428,6 +428,35 @@ inputs = {
 }
 ```
 
+**Can I speed up dependency fetching?**
+
+`dependency` blocks are fetched in parallel at each source level, but will serially parse each recursive dependency. For
+example, consider the following chain of dependencies:
+
+```
+account --> vpc --> securitygroup --> ecs
+                                      ^
+                                     /
+                              ecr --
+```
+
+In this chain, the `ecr` and `securitygroup` module outputs will be fetched concurrently when applying the `ecs` module,
+but the outputs for `account` and `vpc` will be fetched serially as terragrunt needs to recursively walk through the
+tree to retrieve the outputs at each level.
+
+This recursive parsing happens due to the necessity to parse the entire `terragrunt.hcl` configuration (including
+`dependency` blocks) in full before being able to call `terraform output`.
+
+However, terragrunt includes an optimization to only fetch the lowest level outputs (`securitygroup` and `ecr` in this
+example) provided that the following conditions are met in the immediate dependencies:
+
+- The remote state is managed using `remote_state` blocks.
+- The `remote_state` block is in generate mode (the `generate` attribute is set).
+- The `remote_state` block itself does not depend on any `dependency` outputs (`locals` and `include` are ok).
+
+If these conditions are met, terragrunt will only parse out the `remote_state` blocks and use that to pull down the
+state for the target module without parsing the `dependency` blocks, avoiding the recursive dependency retrieval.
+
 
 ### dependencies
 
diff --git a/_docs/rfc/dynamo_table_config.md b/docs/_docs/05_rfc/dynamo_table_config.md
similarity index 86%
rename from _docs/rfc/dynamo_table_config.md
rename to docs/_docs/05_rfc/dynamo_table_config.md
index 80c17b7c..5d8ace07 100644
--- a/_docs/rfc/dynamo_table_config.md
+++ b/docs/_docs/05_rfc/dynamo_table_config.md
@@ -1,3 +1,15 @@
+---
+layout: collection-browser-doc
+title: Custom state configuration
+category: RFC
+categories_url: rfc
+excerpt: Allow further customization of Terraform Lock table for S3 Remote State.
+tags: ["rfc", "contributing", "community"]
+order: 503
+nav_title: Documentation
+nav_title_link: /docs/
+---
+
 # Allow further customization of Terraform Lock table for S3 Remote State
 
 **STATUS**: In proposal
diff --git a/_docs/rfc/for_each_iteration.md b/docs/_docs/05_rfc/for_each_iteration.md
similarity index 97%
rename from _docs/rfc/for_each_iteration.md
rename to docs/_docs/05_rfc/for_each_iteration.md
index de836ef2..a8946374 100644
--- a/_docs/rfc/for_each_iteration.md
+++ b/docs/_docs/05_rfc/for_each_iteration.md
@@ -1,3 +1,15 @@
+---
+layout: collection-browser-doc
+title: for_each to call terraform module multiple times
+category: RFC
+categories_url: rfc
+excerpt: for_each - looping variables to call module multiple times.
+tags: ["rfc", "contributing", "community"]
+order: 502
+nav_title: Documentation
+nav_title_link: /docs/
+---
+
 # RFC: for_each - looping variables to call module multiple times
 
 **STATUS: Won't Implement**
diff --git a/_docs/rfc/imports.md b/docs/_docs/05_rfc/imports.md
similarity index 99%
rename from _docs/rfc/imports.md
rename to docs/_docs/05_rfc/imports.md
index 6f508a2d..35e157b4 100644
--- a/_docs/rfc/imports.md
+++ b/docs/_docs/05_rfc/imports.md
@@ -1,3 +1,15 @@
+---
+layout: collection-browser-doc
+title: Imports
+category: RFC
+categories_url: rfc
+excerpt: Define new mechanisms for importing terragrunt config.
+tags: ["rfc", "contributing", "community"]
+order: 504
+nav_title: Documentation
+nav_title_link: /docs/
+---
+
 # Imports
 
 **STATUS**: In development
diff --git a/_docs/rfc/TEMPLATE.md b/docs/_docs/05_rfc/template.md
similarity index 87%
rename from _docs/rfc/TEMPLATE.md
rename to docs/_docs/05_rfc/template.md
index 2a114e1f..0f4cca64 100644
--- a/_docs/rfc/TEMPLATE.md
+++ b/docs/_docs/05_rfc/template.md
@@ -1,3 +1,15 @@
+---
+layout: collection-browser-doc
+title: RFC Template for Contributors
+category: RFC
+categories_url: rfc
+excerpt: This is a template you can use for proposing new major features to Terragrunt.
+tags: ["rfc", "contributing", "community"]
+order: 501
+nav_title: Documentation
+nav_title_link: /docs/
+---
+
 # RFC Template
 
 This is a template you can use for proposing new major features to Terragrunt. When creating a new RFC, copy this
diff --git a/_docs/migration_guides/upgrading_to_terragrunt_0.19.x.md b/docs/_docs/06_migration_guides/upgrading_to_terragrunt_0.19.x.md
similarity index 89%
rename from _docs/migration_guides/upgrading_to_terragrunt_0.19.x.md
rename to docs/_docs/06_migration_guides/upgrading_to_terragrunt_0.19.x.md
index fb8a040c..c21d217a 100644
--- a/_docs/migration_guides/upgrading_to_terragrunt_0.19.x.md
+++ b/docs/_docs/06_migration_guides/upgrading_to_terragrunt_0.19.x.md
@@ -1,16 +1,28 @@
+---
+layout: collection-browser-doc
+title: Terragrunt 0.19 migration guide
+category: upgrade
+categories_url: upgrade
+excerpt: Migration guide to upgrade to terragrunt 0.19.x
+tags: ["migration", "community"]
+order: 601
+nav_title: Documentation
+nav_title_link: /docs/
+---
+
 # Upgrading to Terragrunt 0.19.x
 
 ## Background
 
 Terraform 0.12 was released in May, 2019, and it included a few major changes:
 
-1. More strict rules around what can go in a `.tfvars` file. In particular, any variable defined in a `.tfvars` file 
+1. More strict rules around what can go in a `.tfvars` file. In particular, any variable defined in a `.tfvars` file
    that does not match a corresponding `variable` definition in your `.tf` files produces an error.
 1. A shift from HCL to HCL2 as the main syntax. This included support for first-class expressions (i.e., using variables
    and functions without having to wrap everything in `${...}`).
-   
-Before version 0.19.0, Terragrunt had you define its configuration in a `terragrunt = { ... }` variable in 
-a `terraform.tfvars` file, but due to item (1) this no longer works with Terraform 0.12 and newer. That means we had to 
+
+Before version 0.19.0, Terragrunt had you define its configuration in a `terragrunt = { ... }` variable in
+a `terraform.tfvars` file, but due to item (1) this no longer works with Terraform 0.12 and newer. That means we had to
 move to a new file format. This requires a migration, which is unfortunate, but as a nice benefit, item (2)
 gives us a nicer syntax and new functionality!
 
@@ -29,16 +41,16 @@ to Terragrunt 0.19.x and newer:
 1. [Rename a few built-in functions ](#rename-a-few-built-in-functions)
 1. [Use terraform \<0.12](#use-older-terraform)
 
-Check out [this PR in the terragrunt-infrastructure-live-example 
-repo](https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/pull/17) for an example of what the code 
-changes look like. 
+Check out [this PR in the terragrunt-infrastructure-live-example
+repo](https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/pull/17) for an example of what the code
+changes look like.
+
 
-   
 ### Move from terraform.tfvars to terragrunt.hcl
 
 Since Terraform 0.12 has more strict rules about what can go into `terraform.tfvars` files, you now need to move your
 Terragrunt configuration from `terraform.tfvars` to a `terragrunt.hcl` file, removing the `terragrunt = { ... }`
-wrapping along the way. 
+wrapping along the way.
 
 For example, if you had the following in `terraform.tfvars`:
 
@@ -54,7 +66,7 @@ terragrunt = {
       arguments = ["-var", "foo=42"]
     }
   }
-  
+
   remote_state {
     backend = "s3"
     config = {
@@ -63,8 +75,8 @@ terragrunt = {
       region         = "us-east-1"
       encrypt        = true
       dynamodb_table = "my-lock-table"
-    }   
-  }  
+    }
+  }
 }
 ```
 
@@ -90,15 +102,15 @@ remote_state {
     region         = "us-east-1"
     encrypt        = true
     dynamodb_table = "my-lock-table"
-  }   
-}  
+  }
+}
 ```
 
 
 ### Move input variables into inputs
 
-When we were using `terraform.tfvars` files for Terragrunt configuration, we were piggybacking on the fact that 
-Terraform [automatically loads variables from tfvars 
+When we were using `terraform.tfvars` files for Terragrunt configuration, we were piggybacking on the fact that
+Terraform [automatically loads variables from tfvars
 files](https://www.terraform.io/docs/configuration/variables.html#variable-definitions-tfvars-files) to set variables
 for our modules:
 
@@ -110,7 +122,7 @@ terragrunt = {
   terraform {
     # ...
   }
-  
+
   remote_state {
     # ...
   }
@@ -119,9 +131,9 @@ terragrunt = {
 # Input variables to set for your Terraform module
 instance_type  = "t2.micro"
 instance_count = 10
-``` 
+```
 
-With the move to `terragrunt.hcl`, we no longer get this behavior for free. However, Terragrunt can simulate this 
+With the move to `terragrunt.hcl`, we no longer get this behavior for free. However, Terragrunt can simulate this
 behavior for you if you define your input variables by specifying `inputs = { ... }`:
 
 ```hcl
@@ -132,15 +144,15 @@ terraform {
 }
 
 remote_state {
-  # ...  
+  # ...
 }
 
 # Input variables to set for your Terraform module
 inputs = {
   instance_type  = "t2.micro"
   instance_count = 10
-}  
-``` 
+}
+```
 
 Whenever you run a Terragrunt command, such as `terragrunt apply`, Terragrunt will make these variables available to
 your Terraform module as environment variables.
@@ -148,7 +160,7 @@ your Terraform module as environment variables.
 ### Use first-class expressions
 
 Terraform 0.11 only allowed special behavior, such as function calls, using "interpolation syntax," where you wrapped
-the code with `${...}`. Terragrunt included a handful of functions you could call using interpolation syntax, but 
+the code with `${...}`. Terragrunt included a handful of functions you could call using interpolation syntax, but
 _only_ within the `terragrunt = { ... }` block:
 
 ```hcl
@@ -168,7 +180,7 @@ terragrunt = {
 foo = "${get_env("FOO", "default")}"
 ```
 
-Terraform 0.12 has moved to HCL2, which has first-class support for expressions. That means you can call functions 
+Terraform 0.12 has moved to HCL2, which has first-class support for expressions. That means you can call functions
 without having to wrap them in `${...}`. Terragrunt embraces HCL2, and thanks to HCL2's nice parser, that means we not
 only get first-class expressions, but we can also use those expressions _everywhere_ in `terragrunt.hcl`!
 
@@ -209,7 +221,7 @@ foo {
 }
 ```
 
-Since Terragrunt uses HCL2, we now have to be more strict with which parts of the Terragrunt configuration are 
+Since Terragrunt uses HCL2, we now have to be more strict with which parts of the Terragrunt configuration are
 attributes and which are blocks:
 
 ```hcl
@@ -218,7 +230,7 @@ attributes and which are blocks:
 # terraform is a block, so make sure NOT to include an equals sign
 terraform {
   source = "git::git@github.com:foo/modules.git//frontend-app?ref=v0.0.3"
-  
+
   # extra_arguments is a block, so make sure NOT to include an equals sign
   extra_arguments "custom_vars" {
     commands  = ["apply", "plan"]
@@ -232,7 +244,7 @@ remote_state {
   # config is an attribute, so an equals sign is REQUIRED
   config = {
     bucket = "foo"
-    
+
     # s3_bucket_tags is an attribute, so an equals sign is REQUIRED
     s3_bucket_tags = {
       owner = "terragrunt integration test"
@@ -243,7 +255,7 @@ remote_state {
     dynamodb_table_tags = {
       owner = "terragrunt integration test"
       name = "Terraform lock table"
-    }    
+    }
   }
 }
 
@@ -261,16 +273,16 @@ dependencies {
 inputs = {
   instance_type  = "t2.micro"
   instance_count = 10
-}  
+}
 ```
 
 
-### Rename a few built-in functions 
+### Rename a few built-in functions
 
 Two built-in functions were renamed:
 
 1. `get_tfvars_dir()` is now called `get_terragrunt_dir()`.
-1. `get_parent_tfvars_dir()` is now called `get_parent_terragrunt_dir()`.   
+1. `get_parent_tfvars_dir()` is now called `get_parent_terragrunt_dir()`.
 
 Make sure to make the corresponding updates in your `terragrunt.hcl` file!
 



Test patch:
diff --git a/test/fixture-get-output/nested-optimization/.gitignore b/test/fixture-get-output/nested-optimization/.gitignore
new file mode 100644
index 00000000..79655ca1
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/.gitignore
@@ -0,0 +1 @@
+backend.tf
diff --git a/test/fixture-get-output/nested-optimization/deepdep/main.tf b/test/fixture-get-output/nested-optimization/deepdep/main.tf
new file mode 100644
index 00000000..cf92c131
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/deepdep/main.tf
@@ -0,0 +1,3 @@
+output "output" {
+  value = "The answer is 42"
+}
diff --git a/test/fixture-get-output/nested-optimization/deepdep/terragrunt.hcl b/test/fixture-get-output/nested-optimization/deepdep/terragrunt.hcl
new file mode 100644
index 00000000..97f3ba80
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/deepdep/terragrunt.hcl
@@ -0,0 +1 @@
+# Intentionally empty.
diff --git a/test/fixture-get-output/nested-optimization/dep/main.tf b/test/fixture-get-output/nested-optimization/dep/main.tf
new file mode 100644
index 00000000..fd008c9a
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/dep/main.tf
@@ -0,0 +1,4 @@
+variable "input" {}
+output "output" {
+  value = "No, ${var.input}"
+}
diff --git a/test/fixture-get-output/nested-optimization/dep/terragrunt.hcl b/test/fixture-get-output/nested-optimization/dep/terragrunt.hcl
new file mode 100644
index 00000000..ea795078
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/dep/terragrunt.hcl
@@ -0,0 +1,12 @@
+include {
+  path = find_in_parent_folders()
+}
+
+# Retrieve a dependency. In the test, we will destroy this state and verify we can still get the output.
+dependency "deepdep" {
+  config_path = "../deepdep"
+}
+
+inputs = {
+  input = dependency.deepdep.outputs.output
+}
diff --git a/test/fixture-get-output/nested-optimization/live/main.tf b/test/fixture-get-output/nested-optimization/live/main.tf
new file mode 100644
index 00000000..8932fbbd
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/live/main.tf
@@ -0,0 +1,4 @@
+variable "input" {}
+output "output" {
+  value = "They said, \"${var.input}\""
+}
diff --git a/test/fixture-get-output/nested-optimization/live/terragrunt.hcl b/test/fixture-get-output/nested-optimization/live/terragrunt.hcl
new file mode 100644
index 00000000..3de76183
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/live/terragrunt.hcl
@@ -0,0 +1,8 @@
+# Retrieve a dependency. In the test, we will destroy this state and verify we can still get the output.
+dependency "dep" {
+  config_path = "../dep"
+}
+
+inputs = {
+  input = dependency.dep.outputs.output
+}
diff --git a/test/fixture-get-output/nested-optimization/terragrunt.hcl b/test/fixture-get-output/nested-optimization/terragrunt.hcl
new file mode 100644
index 00000000..2a0976b1
--- /dev/null
+++ b/test/fixture-get-output/nested-optimization/terragrunt.hcl
@@ -0,0 +1,16 @@
+# Configure Terragrunt to automatically store tfstate files in an S3 bucket
+remote_state {
+  backend = "s3"
+  config = {
+    encrypt = true
+    bucket = "__FILL_IN_BUCKET_NAME__"
+    key = "${path_relative_to_include()}/terraform.tfstate"
+    region = "us-west-2"
+    dynamodb_table = "__FILL_IN_LOCK_TABLE_NAME__"
+    enable_lock_table_ssencryption = true
+  }
+  generate = {
+    path      = "backend.tf"
+    if_exists = "overwrite"
+  }
+}
diff --git a/test/integration_test.go b/test/integration_test.go
index f5a49af2..323c62a4 100644
--- a/test/integration_test.go
+++ b/test/integration_test.go
@@ -2030,6 +2030,50 @@ func TestYamlDecodeRegressions(t *testing.T) {
 	assert.Equal(t, outputs["test2"].Value, "1.00")
 }
 
+// We test the path with remote_state blocks by:
+// - Applying all modules initially
+// - Deleting the local state of the nested deep dependency
+// - Running apply on the root module
+// If output optimization is working, we should still get the same correct output even though the state of the upmost
+// module has been destroyed.
+func TestDependencyOutputOptimization(t *testing.T) {
+	t.Parallel()
+
+	expectedOutput := `They said, "No, The answer is 42"`
+	generatedUniqueId := uniqueId()
+
+	cleanupTerraformFolder(t, TEST_FIXTURE_GET_OUTPUT)
+	tmpEnvPath := copyEnvironment(t, TEST_FIXTURE_GET_OUTPUT)
+	rootPath := filepath.Join(tmpEnvPath, TEST_FIXTURE_GET_OUTPUT, "nested-optimization")
+	rootTerragruntConfigPath := filepath.Join(rootPath, config.DefaultTerragruntConfigPath)
+	livePath := filepath.Join(rootPath, "live")
+	deepDepPath := filepath.Join(rootPath, "deepdep")
+
+	s3BucketName := fmt.Sprintf("terragrunt-test-bucket-%s", strings.ToLower(generatedUniqueId))
+	lockTableName := fmt.Sprintf("terragrunt-test-locks-%s", strings.ToLower(generatedUniqueId))
+	defer deleteS3Bucket(t, TERRAFORM_REMOTE_STATE_S3_REGION, s3BucketName)
+	defer cleanupTableForTest(t, lockTableName, TERRAFORM_REMOTE_STATE_S3_REGION)
+	copyTerragruntConfigAndFillPlaceholders(t, rootTerragruntConfigPath, rootTerragruntConfigPath, s3BucketName, lockTableName, TERRAFORM_REMOTE_STATE_S3_REGION)
+
+	runTerragrunt(t, fmt.Sprintf("terragrunt apply-all --terragrunt-non-interactive --terragrunt-working-dir %s", rootPath))
+
+	// verify expected output
+	stdout, _, err := runTerragruntCommandWithOutput(t, fmt.Sprintf("terragrunt output -no-color -json --terragrunt-non-interactive --terragrunt-working-dir %s", livePath))
+	require.NoError(t, err)
+
+	outputs := map[string]TerraformOutput{}
+	require.NoError(t, json.Unmarshal([]byte(stdout), &outputs))
+	assert.Equal(t, expectedOutput, outputs["output"].Value)
+
+	// Now delete the deepdep state and verify still works
+	require.NoError(t, os.Remove(filepath.Join(deepDepPath, "terraform.tfstate")))
+	reout, _, err := runTerragruntCommandWithOutput(t, fmt.Sprintf("terragrunt output -no-color -json --terragrunt-non-interactive --terragrunt-working-dir %s", livePath))
+	require.NoError(t, err)
+
+	require.NoError(t, json.Unmarshal([]byte(reout), &outputs))
+	assert.Equal(t, expectedOutput, outputs["output"].Value)
+}
+
 func TestDependencyOutput(t *testing.T) {
 	t.Parallel()
 
@@ -3108,6 +3152,16 @@ func runTerragrunt(t *testing.T, command string) {
 	runTerragruntRedirectOutput(t, command, os.Stdout, os.Stderr)
 }
 
+func runTerragruntCommandWithOutput(t *testing.T, command string) (string, string, error) {
+	stdout := bytes.Buffer{}
+	stderr := bytes.Buffer{}
+	err := runTerragruntCommand(t, command, &stdout, &stderr)
+	if err != nil {
+		return "", "", err
+	}
+	return stdout.String(), stderr.String(), nil
+}
+
 func runTerragruntRedirectOutput(t *testing.T, command string, writer io.Writer, errwriter io.Writer) {
 	if err := runTerragruntCommand(t, command, writer, errwriter); err != nil {
 		stdout := "(see log output above)"