Repository: rancher/terraform-provider-rancher2

Issue:
[release/v13] feat: support custom data directory configuration
Backport #1665 to release/v13 for #1666 
Copied description:
## Issue: 
- #1450
<!-- link the issue or issues this PR resolves here -->
<!-- If your PR depends on changes from another pr link them here and describe why they are needed in your solution section. -->
 
## Problem

Rancher supports having a custom data directory that hasn't been implemented in the provider.
<!-- Describe the root cause of the issue you are resolving. This may include what behavior is observed and why it is not desirable. If this is a new feature describe why we need this feature and how it will be used. -->
 
## Solution

Implement the data_directories argument.
<!-- Describe what you changed to fix the issue. Relate your changes back to the original issue / feature and explain how this addresses the issue. -->
 
## Testing

<!-- Note: Confirm if the repro steps in the GitHub issue are valid, if not, please update the issue with accurate repro steps. -->

## Engineering Testing
### Manual Testing
<!-- Describe what manual testing you did (if no testing was done, explain why). -->

### Automated Testing
<!--If you added/updated unit/integration/validation tests, describe what cases they cover and do not cover. -->

## QA Testing Considerations
<!-- Highlight areas or (additional) cases that QA should test w.r.t a fresh install as well as the upgrade scenarios -->
 
### Regressions Considerations
<!-- Dedicated section to specifically call out any areas that with higher chance of regressions caused by this change, include estimation of probability of regressions -->

Hints from issue comments:
@matttrach: Release created for QA:
https://github.com/rancher/terraform-provider-rancher2/releases/tag/v13.0.0-rc4
@matttrach: Please assign someone to validate?@Josh-Diamond: Ticket #1667 - Test Results - ✅ 
---

Verified on Rancher `v2.13-e9cf13a74b1ebbfb73efad234e8dc748e7215c83-head` with Rancher2 `v13.0.0-rc.8`:

| Scenario | Test Case | Result |
|---|---|---|
| 1. | Using Rancher2 provider, provision a downstream RKE2 cluster with a custom data directory configured | ✅ |
| 1. | Using Rancher2 provider, provision a downstream K3s cluster with a custom data directory configured | ✅ |

---

Above scenarios were tested and validated with the following automation:
https://github.com/rancher/tfp-automation/blob/main/tests/rancher2/provisioning/custom_data_directory_test.go

Solution patch:
diff --git a/docs/resources/cluster_v2.md b/docs/resources/cluster_v2.md
index dd312d71..42f42297 100644
--- a/docs/resources/cluster_v2.md
+++ b/docs/resources/cluster_v2.md
@@ -991,6 +991,7 @@ see more information on [Resource Management for Pods and Containers](https://ku
 #### Arguments
 
 * `additional_manifest` - (Optional, string, must be in YAML format) The value of the additional manifest is delivered to the path `/var/lib/rancher/rke2/server/manifests/rancher/addons.yaml` or `/var/lib/rancher/k3s/server/manifests/rancher/addons.yaml` on the control plane nodes.
+* `data_directories` - (Optional, list, max length: 1) Data directory configuration for the kubernetes distro, system-agent, and provisioning resources.
 * `local_auth_endpoint` - (Deprecated) Use rancher2_cluster_v2.local_auth_endpoint instead.
 * `upgrade_strategy` - (Optional, list, max length: 1) Cluster upgrade strategy.
 * `chart_values` - (Optional, string, must be in YAML format) The value for the system charts installed by the distribution. For more information about how RKE2 or K3s manage packaged components, please refer to [RKE2 documentation](https://docs.rke2.io/helm) or [K3s documentation](https://docs.k3s.io/installation/packaged-components).
@@ -1004,6 +1005,14 @@ see more information on [Resource Management for Pods and Containers](https://ku
 * `etcd_snapshot_create` (Optional, list, max length: 1) Cluster V2 etcd snapshot create.
 * `etcd_snapshot_restore` (Optional, list, max length: 1) Cluster V2 etcd snapshot restore.
 
+#### `data_directories`
+
+##### Arguments
+
+* `system_agent` - (Optional, string) System agent data directory path, used for all system agent data on provisioning v2 downstream  cluster nodes. If unspecified, the `rancher-system-agent` will use the default: `/var/lib/rancher/agent`.
+* `provisioning` - (Optional, string) Provisioning data directory path, used for all provisioning data on provisioning v2 downstream cluster nodes. If unspecified, Rancher will use the default: `/var/lib/rancher/provisioning`.
+* `k8s_distro` - (Optional, string) Kubernetes distro data directory path, rendered as the `data-dir` argument for provisioning v2 RKE2/K3s clusters. If unspecified, the distro will use the default: `/var/lib/rancher/rke2` for RKE2, and `/var/lib/rancher/k3s` for K3s.
+
 #### `local_auth_endpoint`
 
 ##### Arguments
diff --git a/rancher2/schema_cluster_v2_rke_config.go b/rancher2/schema_cluster_v2_rke_config.go
index 8a2748a7..911c2072 100644
--- a/rancher2/schema_cluster_v2_rke_config.go
+++ b/rancher2/schema_cluster_v2_rke_config.go
@@ -179,6 +179,14 @@ func clusterV2RKEConfigFields() map[string]*schema.Schema {
 			Optional:    true,
 			Description: "Cluster V2 additional manifest",
 		},
+		"data_directories": {
+			Type:        schema.TypeList,
+			Optional:    true,
+			Description: "Cluster V2 data directories",
+			Elem: &schema.Resource{
+				Schema: clusterV2RKEConfigDataDirectoriesFields(),
+			},
+		},
 		"local_auth_endpoint": {
 			Type:        schema.TypeList,
 			MaxItems:    1,
diff --git a/rancher2/schema_cluster_v2_rke_config_data_directories.go b/rancher2/schema_cluster_v2_rke_config_data_directories.go
new file mode 100644
index 00000000..5d017d58
--- /dev/null
+++ b/rancher2/schema_cluster_v2_rke_config_data_directories.go
@@ -0,0 +1,29 @@
+package rancher2
+
+import (
+	"github.com/hashicorp/terraform-plugin-sdk/helper/schema"
+)
+
+//Types
+
+func clusterV2RKEConfigDataDirectoriesFields() map[string]*schema.Schema {
+	s := map[string]*schema.Schema{
+		"system_agent": {
+			Type:        schema.TypeString,
+			Optional:    true,
+			Description: "Desired System Agent data directory.",
+		},
+		"provisioning": {
+			Type:        schema.TypeString,
+			Optional:    true,
+			Description: "Desired provisioning data directory.",
+		},
+		"k8s_distro": {
+			Type:        schema.TypeString,
+			Optional:    true,
+			Description: "Desired k8s distro data directory.",
+		},
+	}
+
+	return s
+}
diff --git a/rancher2/structure_cluster_v2_rke_config.go b/rancher2/structure_cluster_v2_rke_config.go
index 7b61cb67..0d075aba 100644
--- a/rancher2/structure_cluster_v2_rke_config.go
+++ b/rancher2/structure_cluster_v2_rke_config.go
@@ -17,6 +17,7 @@ func flattenClusterV2RKEConfig(in *provisionv1.RKEConfig) []interface{} {
 		obj["additional_manifest"] = in.AdditionalManifest
 	}
 
+	obj["data_directories"] = flattenClusterV2RKEConfigDataDirectories(in.DataDirectories)
 	obj["upgrade_strategy"] = flattenClusterV2RKEConfigUpgradeStrategy(in.UpgradeStrategy)
 
 	if in.ChartValues.Data != nil && len(in.ChartValues.Data) > 0 {
@@ -73,6 +74,10 @@ func expandClusterV2RKEConfig(p []interface{}) *provisionv1.RKEConfig {
 		obj.AdditionalManifest = v
 	}
 
+	if v, ok := in["data_directories"].([]interface{}); ok && len(v) > 0 {
+		obj.DataDirectories = expandClusterV2RKEConfigDataDirectories(v)
+	}
+
 	if v, ok := in["upgrade_strategy"].([]interface{}); ok && len(v) > 0 {
 		obj.UpgradeStrategy = expandClusterV2RKEConfigUpgradeStrategy(v)
 	}
diff --git a/rancher2/structure_cluster_v2_rke_config_data_directories.go b/rancher2/structure_cluster_v2_rke_config_data_directories.go
new file mode 100644
index 00000000..47237273
--- /dev/null
+++ b/rancher2/structure_cluster_v2_rke_config_data_directories.go
@@ -0,0 +1,45 @@
+package rancher2
+
+import rkev1 "github.com/rancher/rancher/pkg/apis/rke.cattle.io/v1"
+
+func flattenClusterV2RKEConfigDataDirectories(in rkev1.DataDirectories) []any {
+	obj := make(map[string]any)
+
+	if in.SystemAgent != "" {
+		obj["system_agent"] = in.SystemAgent
+	}
+
+	if in.Provisioning != "" {
+		obj["provisioning"] = in.Provisioning
+	}
+
+	if in.K8sDistro != "" {
+		obj["k8s_distro"] = in.K8sDistro
+	}
+
+	return []any{obj}
+}
+
+func expandClusterV2RKEConfigDataDirectories(p []any) rkev1.DataDirectories {
+	obj := rkev1.DataDirectories{}
+
+	if p == nil || len(p) == 0 || p[0] == nil {
+		return obj
+	}
+
+	in := p[0].(map[string]interface{})
+
+	if v, ok := in["system_agent"].(string); ok && v != "" {
+		obj.SystemAgent = v
+	}
+
+	if v, ok := in["provisioning"].(string); ok && v != "" {
+		obj.Provisioning = v
+	}
+
+	if v, ok := in["k8s_distro"].(string); ok && v != "" {
+		obj.K8sDistro = v
+	}
+
+	return obj
+}



Test patch:
diff --git a/rancher2/structure_cluster_v2_rke_config_data_directories_test.go b/rancher2/structure_cluster_v2_rke_config_data_directories_test.go
new file mode 100644
index 00000000..dd0f0e38
--- /dev/null
+++ b/rancher2/structure_cluster_v2_rke_config_data_directories_test.go
@@ -0,0 +1,63 @@
+package rancher2
+
+import (
+	"testing"
+
+	rkev1 "github.com/rancher/rancher/pkg/apis/rke.cattle.io/v1"
+	"github.com/stretchr/testify/assert"
+)
+
+var (
+	testClusterV2RKEConfigDataDirectoriesConf      rkev1.DataDirectories
+	testClusterV2RKEConfigDataDirectoriesInterface []interface{}
+)
+
+func init() {
+	testClusterV2RKEConfigDataDirectoriesConf = rkev1.DataDirectories{
+		SystemAgent:  "/tmp/test/agent",
+		Provisioning: "/tmp/test/provisioning",
+		K8sDistro:    "/tmp/test/distro",
+	}
+
+	testClusterV2RKEConfigDataDirectoriesInterface = []any{
+		map[string]any{
+			"system_agent": "/tmp/test/agent",
+			"provisioning": "/tmp/test/provisioning",
+			"k8s_distro":   "/tmp/test/distro",
+		},
+	}
+}
+
+func Test_flattenClusterV2RKEConfigDataDirectories(t *testing.T) {
+	cases := []struct {
+		Input          rkev1.DataDirectories
+		ExpectedOutput []interface{}
+	}{
+		{
+			testClusterV2RKEConfigDataDirectoriesConf,
+			testClusterV2RKEConfigDataDirectoriesInterface,
+		},
+	}
+
+	for _, tc := range cases {
+		output := flattenClusterV2RKEConfigDataDirectories(tc.Input)
+		assert.Equal(t, tc.ExpectedOutput, output, "Unexpected output from flattener.")
+	}
+}
+
+func TestExpandClusterV2RKEConfigDataDirectories(t *testing.T) {
+	cases := []struct {
+		Input          []interface{}
+		ExpectedOutput rkev1.DataDirectories
+	}{
+		{
+			testClusterV2RKEConfigDataDirectoriesInterface,
+			testClusterV2RKEConfigDataDirectoriesConf,
+		},
+	}
+
+	for _, tc := range cases {
+		output := expandClusterV2RKEConfigDataDirectories(tc.Input)
+		assert.Equal(t, tc.ExpectedOutput, output, "Unexpected output from expander.")
+	}
+}
diff --git a/rancher2/structure_cluster_v2_rke_config_z_test.go b/rancher2/structure_cluster_v2_rke_config_z_test.go
index 94b24e3d..5018adfa 100644
--- a/rancher2/structure_cluster_v2_rke_config_z_test.go
+++ b/rancher2/structure_cluster_v2_rke_config_z_test.go
@@ -18,6 +18,7 @@ func init() {
 		MachinePools: testClusterV2RKEConfigMachinePoolsConf,
 	}
 	testClusterV2RKEConfigConf.AdditionalManifest = "additional_manifest"
+	testClusterV2RKEConfigConf.DataDirectories = testClusterV2RKEConfigDataDirectoriesConf
 	testClusterV2RKEConfigConf.UpgradeStrategy = testClusterV2RKEConfigUpgradeStrategyConf
 	testClusterV2RKEConfigConf.ChartValues = rkev1.GenericMap{
 		Data: map[string]interface{}{
@@ -43,6 +44,7 @@ func init() {
 	testClusterV2RKEConfigInterface = []interface{}{
 		map[string]interface{}{
 			"additional_manifest":     "additional_manifest",
+			"data_directories":        testClusterV2RKEConfigDataDirectoriesInterface,
 			"upgrade_strategy":        testClusterV2RKEConfigUpgradeStrategyInterface,
 			"chart_values":            "chart_one: one\nchart_two: two\n",
 			"machine_global_config":   "config_one: one\nconfig_two: two\n",