Package: tfquery
🌩️ tfquery 🌩️Run SQL queries on your Terraform infrastructure. Ask questions that are hard to answer🚀 What is tfquery?tfquery is a framework that allows running SQL queries on Terraform code. It's made to analyze your Terraform infrastructure, locate resources, run security compliance checks, spot misconfigured resources, develop CI benchmarks, and much more.tfquery is made to help in answering questions that are hard to answer about your infrastructure-as-code. It allows querying resources and analyzing its configuration using a SQL-powered framework.Why?infrastructure-as-code is the de-facto today for documenting and deploying infrastructure on cloud providers. As the organization grows, it becomes really hard to understand and analyze the deployed infrastructure. Grepping and searching for resources in Terraform state files is not enough. Terraform Modules are automating processes dynamically for infrastructure deployment, so searching for static resources is also not feasible for good visibility.With tfquery, you can run SQL queries on Terraform state files, and gain the best possible visibility.💡 Use tfquery toHave full coverage of your infrastructure, without being locked on a specific provider, including Amazon AWS, Microsoft Azure, Google Cloud Platform, Alibaba Cloud, IBM Cloud, Oracle Cloud, and many others.Analyze deployed resources configuration.Develop CI and monitoring checks for cloud infrastructure.Write custom queries to scan Terraform resources.tfquery vs. Cloud-specific SQL engines?There are cloud-specific SQL engines that allow you to run SQL queries to understand resources on their infrastructure, both are covered as provided service by the cloud provider, or given as an open-source tool or a product. The main difference tfquery brings:Maintainability: Cloud-specific SQL engines require maintenance in case of new services or breaking changes to existing ones. tfquery make use of Terraform schemas as a standard. tfquery will work on all given services, without the need to continuously update it with new API specs.Coverage: tfquery covers all the cloud providers that Terraform supports out of the box (thanks to Terraform Providers).📖 UsageRun SQL query on Terraform states>>>importtfquery>>>>>>result=tfquery.tfstate.run_query("terraform.tfstate","select count(*) from resources")>>print(result)[{'count(*)':86}]Parse all resources from a Terraform state file>>>importtfquery>>>>>>resources=tfquery.tfstate.parse_resources("terraform.tfstate")>>>print(f"[i] Resources Count:{len(resources)}")[i]ResourcesCount:1475Advanced UsageMigrate Version 3 to Version 4 Terraform statesThis is a parsing library to migrate the older Version 3 Terraform states to a Version 4 state. This is made to add backward compatibility for Terraform states that is made for releases older thanTerraform v0.11.>>>importtfquery>>>>>>tfstate_v3=tfquery.tfstate.load_file("terraform.tfstate")>>>tfstate_v4=tfquery.tfstate_v3_migration.upgrade_v3_tfstate(tfstate)🖲️ Command-Line (tfquery)TFquery is also available as a CLI tool. It can be used to run SQL queries directly on Terraform states, and for importing resources into persistent storage.mazin@hackbox$>tfquery-h

usage:tfquery[-h][--tfstateTFSTATE][--tfstate-dirTFSTATE_DIR][--queryQUERY][--dbDB_PATH][--interactive][--import]tfquery:RunSQLqueriesonyourTerraforminfrastructure.

optionalarguments:-h,--helpshowthishelpmessageandexit--tfstateTFSTATETerraform.tfstatefile.--tfstate-dirTFSTATE_DIRDirectoryofTerraform.tfstatefiles,forrunningqueriesonenvironments.--queryQUERY,-qQUERYSQLquerytoexecute.--dbDB_PATHDBpath(optional.default:temporarily-generateddatabase).--interactive,-iInteractivemode.--importImporttfstateintodatabase.ExamplesRun SQL query for a directory of multiple Terraform states (for multiple workspaces).:$tfquery-q'select count(*) as count from resources;'--tfstate-dir/path/to/terraform-states[i]DBPath:tfstate.db[+]Imported4203resourcesfrom./prod.tfstate.[i]DBPath:tfstate.db[+]Imported3675resourcesfrom./nonprod.tfstate.[i]DBPath:tfstate.db[+]Imported463resourcesfrom./qa.tfstate.Import Terraform states into Database.:$python3tfquery--tfstate/path/to/terraform.state--dbtfstate.db--import[i]DBPath:tfstate.db[+]Imported386resourcesfromterraform.tfstate.Run queries on imported resources in a database$tfquery--dbtfstate.db-q'select count(*) as count from resources;'[{"count":386}]💭 Awesome Queries & ScriptsFind all AWS S3 buckets without versioning being enabledimporttfquery,sysresults=tfquery.tfstate.run_query(sys.argv[1],"select * from resources where type = 'aws_s3_bucket'")forresultinresults:attributes=result["attributes"]if'versioning'notinattributesorlen(attributes["versioning"])==0:# print(result)continueforversioninginattributes["versioning"]:ifversioning["enabled"]isFalse:# print(result)passFind all AWS IAM users, and print their ARNsimporttfquery,sysresults=tfquery.tfstate.run_query(sys.argv[1],"select json_extract(attributes, '$.arn') as arn from resources where type = 'aws_iam_user';")forresultinresults:print(result["arn"])orimporttfquery,sysresults=tfquery.tfstate.run_query(sys.argv[1],"select attributes from resources where type = 'aws_iam_user';")forresultinresults:print(result["attributes"]["arn"])Find all resources in the environment, and show how many instances were deployedimporttfqueryresults=tfquery.tfstate.run_query("terraform.tfstate","select type, count(*) as count from resources group by type order BY count desc;")print(results)✨ Interested in tfquery?Post a Tweet about the project and tag@mazen160🙏🌟 Star it on Github 🌟Create a PR for a new awesome feature 💛Would like to sponsor the project? Contact me on email!💻 ContributionContribution is always welcome! Please feel free to report issues on Github and create PRs for new features.📌 Ideas to Start onWould like to contribute to tfquery? Here are some ideas that you may start with:Better documentation: would be great to enhance the documentation with additional examples and queries.CI: Implement CI along with test terraform states for Terragoat.Support dependencies for resources lookup: Create a new table called "dependencies", parse V4 Terraform states, and implement a many-to-one relation for dependencies of resources.More V3 --> V4 migration support: currently V3 resources migrations are supported. Dependencies are not migrated to the new V4 state. It will be great to continue on V3--> V4 support for Terraform states.General validation of Terraform states parser implementation: Validate current implementation of the parser, and enhance it where possible.Connect resources with terraform state base name: For environments with many workspaces, each workspace can have a different name, it would be nice to add a column for terraform state file base name, to help in querying across different workspaces.tfplan parsing: Allow parsing of tfplan files. This can be an opening addition for implementing a new CI security scanner for Terraform deployments.Logo design: a logo design would be great.Web interface representation withcoleifer/sqlite-web- Thanks@securityfufor the idea!Add a managed repository of pre-defined queries to enable teams to be able to query Terraform for different compliance and security controls.As you can see, there are many ways to support. Please help us make the project bigger for everyone!Installationmazin@hackbox$>gitclonehttps://github.com/mazen160/tfquery.git
mazin@hackbox$>cdtfquery
mazin@hackbox$>python3setup.pyinstallormazin@hackbox$>pipinstallgit+https://github.com/mazen160/tfquery📄 LicenseThe project is licensed under MIT License.💚 AuthorMazin AhmedWebsite:https://mazinahmed.netEmail:mazin [at] mazinahmed [dot] netTwitter:https://twitter.com/mazen160Linkedin:http://linkedin.com/in/infosecmazinahmed