Repository: hashicorp/terraform-cdk

Issue #2024: Support accessing resources created by an iterator
Created: 2022-08-12T21:37:39Z
Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment


Description
Iterators allow you to create a dynamic amount of resources based on data only know at Terraform execution time; however, if you then need to reference those resources for some other purpose, there isn't a supported way (besides escape hatches). This is still dynamic information so not everything will be possible, but some operations should still work. We could support creating another iterator from the resource which could then be feed to another resource. We may also be able to expose list/map operations for direct access (via Terraform lookups).
References

https://discuss.hashicorp.com/t/how-to-iterate-over-objects-made-with-an-iterator/43115
#1094

Duplicate issue #3145: splat expressions and Fn.values not working as expected
Created: 2023-09-26T00:30:38Z
Expected Behavior
I have a use case where I need to lookup a list of objects at runtime and then extract a value out of the list. More specifically, I am using DataOpenstackNetworkingSubnetIdsV2 to run a search at runtime to find a list of ids which match a given criteria. Then I am using an iterator and DataOpenstackNetworkingSubnetV2 to lookup these ids to get the backing resources using a for_each statement. Both of these steps work fine and as expected. However, the issue comes in when I am trying to extract the data I need from the DataOpenstackNetworkingSubnetV2 namely the dynamic list of cidrs.
A simplified version of the above would look like:
list_of_ids = TerraformLocal(self.scope, f"list", ['f43657ef-a331-4108-883e-557fd4ff03d0', 'b13e710e-9276-493d-b561-6dc4cff2f5da', 'ad21a8c7-4ad4-4c28-8f18-4b87efaeec77'])
iterator = TerraformIterator.from_list(list=list_of_ids.as_list)
elements = DataOpenstackNetworkingSubnetV2(
     self.scope,
     f"elements",
     provider=self.provider,
     for_each=iterator,
     subnet_id=Token.as_string(iterator.value))
TerraformOutput(self.scope, f"outputs", value=elements)
TerraformOutput(self.scope, f"outputs2", value=elements.cidr)

The above TerraformOutput blocks generate the following terraform code:
"outputs": {
  "value": "${data.openstack_networking_subnet_v2.elements}"
},
"outputs2": {
  "value": "${data.openstack_networking_subnet_v2.elements.*.cidr}"
},

Actual Behavior
The first output works great and outputs all the objects correctly. The second output however isn't valid Terraform and results in the following error
Error: Unsupported attribute
│
│   on cdk.tf.json line 568, in output.outputs2:
│  568:       "value": "${data.openstack_networking_subnet_v2.elements.*.cidr}"
│
│ This object does not have an attribute named "cidr".

The problem is that data.openstack_networking_subnet_v2.elements is a map of maps. which looks like:
elements = { 
    "id1" = { "cidr" : ... },
    "id2" = { "cidr" : ... }
}

taking the splat expression of that object creates will just create a list of one element with then entire contents of elements in it. What I actually need is to generate the following command:
"${values(data.openstack_networking_subnet_v2.elements).*.cidr}"
Steps to Reproduce
To reproduce this issue run the following CDKTF Python or modify it to use any resource instead of DataOpenstackNetworkingSubnetV2
list_of_ids = TerraformLocal(self.scope, f"{self.seed_name}_list", ['f43657ef-a331-4108-883e-557fd4ff03d0', 'b13e710e-9276-493d-b561-6dc4cff2f5da', 'ad21a8c7-4ad4-4c28-8f18-4b87efaeec77'])
iterator = TerraformIterator.from_list(list=list_of_ids.as_list)
elements = DataOpenstackNetworkingSubnetV2(
     self.scope,
     f"elements",
     provider=self.provider,
     for_each=iterator,
     subnet_id=Token.as_string(iterator.value))
TerraformOutput(self.scope, f"outputs", value=elements)
TerraformOutput(self.scope, f"outputs2", value=elements.cidr)

Versions
cdktf debug
language: python
cdktf-cli: 0.18.0
node: v18.17.1
cdktf: 0.17.3
constructs: 10.2.69
jsii: 1.85.0
terraform: 1.5.7
arch: x64
os: linux 5.15.90.1-microsoft-standard-WSL2
python: Python 3.10.12
pip: pip 23.2.1 from /home/nick/.local/lib/python3.10/site-packages/pip (python 3.10)
pipenv: pipenv, version 2023.9.8
Providers
┌────────────────────────────────────────┬──────────────────┬─────────┬────────────┬───────────────────────────────┬─────────────────┐
│ Provider Name                          │ Provider Version │ CDKTF   │ Constraint │ Package Name                  │ Package Version │
├────────────────────────────────────────┼──────────────────┼─────────┼────────────┼───────────────────────────────┼─────────────────┤
│ terraform-provider-openstack/openstack │ 1.52.1           │         │            │                               │                 │
├────────────────────────────────────────┼──────────────────┼─────────┼────────────┼───────────────────────────────┼─────────────────┤
│ google                                 │ 4.75.0           │ ^0.17.0 │            │ cdktf-cdktf-provider-google   │ 8.0.6           │
├────────────────────────────────────────┼──────────────────┼─────────┼────────────┼───────────────────────────────┼─────────────────┤
│ local                                  │ 2.4.0            │ ^0.17.0 │            │ cdktf-cdktf-provider-local    │ 7.0.0           │
├────────────────────────────────────────┼──────────────────┼─────────┼────────────┼───────────────────────────────┼─────────────────┤
│ template                               │ 2.2.0            │ ^0.17.0 │            │ cdktf-cdktf-provider-template │ 7.0.0           │
├────────────────────────────────────────┼──────────────────┼─────────┼────────────┼───────────────────────────────┼─────────────────┤
│ vault                                  │ 3.18.0           │ ^0.17.0 │            │ cdktf-cdktf-provider-vault    │ 9.0.1           │
└────────────────────────────────────────┴──────────────────┴─────────┴────────────┴───────────────────────────────┴─────────────────┘
Gist
No response
Possible Solutions
I am not sure about the wider reaching impact of for_each in cdktf but it would seem that if you are making a splat expression from an object generated from a for_each you will always want to put it through Fn.values() first. As the generated resource will be of the type map of maps. So, one possibility is to insert that values call or add a function on the resource which will produce that. Someone with more experience with CDKTF would probably want to compare this case and solution with the count case. which i believe generates a list of maps.  This solution does have the benifit of being the most intuitive to use. namely it matches the non-splat use case where if no for_each is used the user would need to write elements.cidr to get the single cidr value.
Another solution would be to change the way Fn.values() works. Currently, it returns string object which seems wrong. Instead, it could return a list object capable of generating splat statements. This solution seems the safest. it would require the user to understand why a splat statement on the for_each resources don't work but it would at least match the solution you would need to do in terraform.
Workarounds
The workaround here what i did was first to put the for_each object through Fn.values() and store it into a local variable. Then i used the local variable to generate the correct terraform code in another local variable. This local variable then contained the data i needed. This data was then used in an iterator object to be useful.
Also not that Fn.values() requires you to use the .fqn and failes if you try and use the local variable which also seems like a bug.
local_var = TerraformLocal(self.scope, f"public_cidrs", Fn.values(public_subnets.fqn))
# This is generating the splat statement "${local_var.public_cidrs.*.cidr}" which isn't suported except though this trickery
self.local_public_subnets_cidrs = TerraformLocal(self.scope, f"public_cidrs2", f"${{local.{local_var.friendly_unique_id}.*.cidr}}")

Anything Else?
Fn.values seems like it has a number of issues. First it returns a str object making it impossible to use with other Fn. functions like fn.element. It also doesn't allow the direct use of variable/resources and requires the user to use the .fqn parameter to workaround it.
One thing i have found that i often need when trying to use cdktf, especially when trying to work around issues like this, is access to the full terraform name unmodified. For example, when i wrote f"${{local.{local.friendly_unique_id}.*.cidr}}" i needed to hardcode in the local. and use the friendly_unique_id instead the resource just having an attribute which returns the terraform name. the fqn comes close but it i always end up with ${} around the name in bad places when i try and use it. Consider a raw_fqn parameter which never adds the ${} to the fqn.
References
No response
Help Wanted

 I'm interested in contributing a fix myself

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Closing comment (2023-10-12T15:25:11Z): Closing as a duplicate of #2024 and #1641.