terraform upgrade を試す

## はじめに
Terraform の Upgrade 方法について Terraform 本体と Module にわけて調査・試験したのでその内容をまとめる

切り戻し方法はどこかを参照したわけではないので、間違っている可能性があるので参考まで

## Terraform Upgrade Path
ここでは、v0.14.x から最新(2024.02.24時点)のリリースバージョン v1.7.4 へアップグレードするためのアップグレードパスを記載する

### 結論 (v0.14.x から最新 v1.7.4 へあげる場合)
v0.14.x -> v1.0.0 -> v1.7.4 の 2段階でアップグレードする

### v0.14.x -> v1.0.0
[Upgrading to Terraform v1.0 ドキュメンントには v0.14 からは v1.0.0 へ直接アップグレード可能](https://developer.hashicorp.com/terraform/language/v1.1.x/upgrade-guides/1-0#upgrading-to-terraform-v1-0)とあり、
[hashicorp ブログのアップグレードベストプラクティス](https://support.hashicorp.com/hc/en-us/articles/6302733655315-Terraform-Upgrade-Best-Practices)の推奨アップグレードパスには v1.0.0 を経由することが書かれているので、一度 v1.0.0 へアップグレードする

v0.14 以前を使用している場合は、マイナーバージョンを一つずつ v0.14.x まで上げてから v1.0.0 へアップグレードする



### v1.0.0 -> v1.7.4

[Terraform v1.x リリースと互換性が保証されている](https://developer.hashicorp.com/terraform/language/v1-compatibility-promises)ので、基本的には v1.x 系では**直接**最新バージョンへアップグレードができる (はず)

ただし、下記のようにバージョンによる注意事項があるため、使用しているバージョンによっては特定のバージョンを経由する必要がある

#### バージョンによる注意事項
##### v1.3.x ~ v.1.5.x を使用している場合
[v1.7.xへアップグレードする前にstateファイルの互換性問題](https://developer.hashicorp.com/terraform/language/v1.7.x/upgrade-guides#validations-and-checks)が v1.3.x ~ v.1.5.x で発生しているが、
v1.3.0 より前および v1.6.0 以降の Terraform バージョンは、この問題の影響を受けない
(Terraform versions prior to v1.3.0 and equal to or after v1.6.0 are not affected by this issue.)

##### v1.1.0, v.1.1.1 を使用している場合
[v1.1.0/v1.1.1 には State File に Bug がある](https://support.hashicorp.com/hc/en-us/articles/4412991179539-v1-1-0-v1-1-1-Bug-Terraform-applied-no-changes-and-unexpectedly-emptied-the-State-File) ため、v.1.1.2 を経由する必要がある

ここでは、2024.02.24 で最新パッチバージョンの v1.7.4 へアップグレードする

## Provider Upgrade Path (AWS Provider)
Provider はメジャーバージョンを一つずつ上げていく

ここでは AWS Provider を例に見ていく

各 Provider ページに Upgrade Guide があるので、まずはそこを読んで実施する

[Terraform AWS Provider Version 5 Upgrade Guide
](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-5-upgrade) の [Provider Version Configuration](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-5-upgrade#provider-version-configuration) を参照すると `Before upgrading to version 5.0.0, upgrade to the most recent 4.X version` とあるので、まずは事前に 4.x の最新バージョンまで上げてから 5.0.0 へアップグレードし、5.x の 2024.2.25 時点の最新バージョン 5.38.0 へアップグレードする

### 結論 Upgrade Path (4.30.0 から最新 5.38.0 へアップグレードする場合)
4.30.0 -> 4.67.0 -> 5.0.0 -> 5.38.0



## アップグレード切り戻し
切り戻し方法が見つからなかったので、stateファイルをアップグレード前を保存することで戻せることを確認する

## 検証

Terraform 本体と Provider の2つに分けてアップグレードの検証試験を実施する

### 実行環境

- Terraform Version : 1.1.7
- Provider : aws
- Provider Version : 4.30.0
- Test Code : 次に記載

#### Test Code (Upgrade 前)

`.terraform-version`(後述)で version を指定しており、
`main.tf` 内で provider とバージョンを指定している環境コード

terraform state ファイルの保存は `backend "s3"` で S3 を使用するようにしている (`bucket`名は環境差異のためマスク)

`profile`指定や`module`を使っていることは特に今回のテストには関係ない (vnetとかは命名ミス)

```.terraform-version
1.1.7
```

```main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "=4.30.0"
    }
  }
  # https://www.terraform.io/language/settings/backends/s3
  backend "s3" {
    bucket  = "[tfstate s3 bucket name]"
    key     = "infra/vpc/terraform.state"
    region  = "ap-northeast-1"
  }
}

# Configure the AWS Provider
provider "aws" {
  region  = "ap-northeast-1"
  profile = "infra"
  # shared_credentials_file = "~/.aws/credentials" # default path
}
```

```vpc.tf
module "vnet" {
  source     = "../../../../modules/vpc/"
  name       = "homelab-apne1-vpc"
  cidr_block = "172.26.0.0/16"
}
```

```module_vpc_main.tf
resource "aws_vpc" "main" {
  cidr_block       = var.cidr_block
  instance_tenancy = "default"

  tags = {
    Name = var.name
  }
}
```

#### 前提

試験前には `terraform plan` で `No changes.` であることを確認している
(vpc の id は出力例からマスクしている)

```sh:出力例
$ terraform plan
module.vnet.aws_vpc.main: Refreshing state... [id=vpc-0aXXXXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

$ terraform version
Terraform v1.1.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.30.0

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html
```

### ツール
Terraform Version 管理には [tfenv](https://github.com/tfutils/tfenv) 利用する

また、フォルダ内で `.terraform-version` ([tfenv ドキュメント](https://github.com/tfutils/tfenv?tab=readme-ov-file#terraform-version-file))を記載することで実行 Terraform Version を指定する 
(`.terraform-version` (例： 1.1.7 などバージョン数字だけ記載するファイル) を実行フォルダに置いておくと、tfenv で自動的にそのバージョンを指定して実行してくれる(なければ勝手にダウンロードしてくれる))

### Terraform Version Upgrade

#### 手順

1. S3 State File Buckup
1. Upgrade to v1.7.4
1. Downgrade to v1.1.7

#### 1. S3 State File Buckup

S3 から tfstate ファイルをダウンロードしてバックアップファイルとしてローカルに保存しておく

```sh
export AWS_BUCKET_NAME="自分が使用しているバケット名"
aws s3 cp --profile=infra s3://${AWS_BUCKET_NAME}/infra/vpc/terraform.state .
mv terraform.state terraform.state.backup
```

#### 2. Upgrade to v1.7.4

Terraform 本体を v1.7.4 へアップグレードする

`.terraform-version` ファイルを書き換える

```sh
echo 1.7.4 > .terraform-version
terraform version
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ terraform version
Terraform v1.7.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.30.0
```

</div></details>

version を変更したので、一度 `-reconfigure` を付けて `terraform init` をする

```sh
terraform init -reconfigure
terraform refresh
```

<details><summary>　出力例　</summary><div>

```sh:出力結果
$ terraform init -reconfigure


Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v4.30.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform refresh

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXXXXXXX]
$ terraform plan
module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
```
</div></details>


##### State File Diff 

Upgrade 後の State file を落としてみて、backup と比較すると下記のような感じだった

```diff
 diff terraform.state.backup terraform.state
3,4c3,4
<   "terraform_version": "1.1.7",
<   "serial": 0,
---
>   "terraform_version": "1.7.4",
>   "serial": 1,
52c52,53
<   ]
---
>   ],
>   "check_results": null
```

以上でアップグレード試験は完了

#### 3. Downgrade to v1.1.7

作業前のバージョンに切り戻しを試す

terraform の version を戻す

```sh
echo 1.1.7 > .terraform-version
terraform version
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ terraform version
Terraform v1.1.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.30.0

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html
```

</div></details>

次に、terraform.state ファイルをバックアップしていたものをアップロードして上書きする

```sh
aws s3 cp --profile=infra ./terraform.state.backup s3://${AWS_BUCKET_NAME}/infra/vpc/terraform.state
```

最後に `-reconfigure` して、`terraform plan` で `No changes.` であることを確認する

```sh
terraform init -reconfigure
terraform plan
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ terraform init -reconfigure

Initializing modules...

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v4.30.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform plan
module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
```
</div></details>

以上、terraform upgrade の切り戻し試験も終了

次は、upgrade 後 (切り戻し/downgrade せず) に実施する

### Provider Version (AWS) Upgrade

`terraform version` は `v1.7.4` へアップグレード後に実施している

#### 手順

1. S3 State File Buckup
1. Upgrade to 4.67.0
1. Upgrade to 5.0.0
1. Upgrade to 5.38.0
1. Downgrade to 4.54.0

#### 1. S3 State File Buckup

S3 から tfstate ファイルをダウンロードしてバックアップファイルとしてローカルに保存しておく

```sh
aws s3 cp --profile=infra s3://${AWS_BUCKET_NAME}/infra/vpc/terraform.state .
mv terraform.state terraform.state.module.backup
```

#### 2. Upgrade to 4.67.0

aws provider を 4.67.0 (4.x の最新バージョン) へアップグレードする

terraform 設定の `required_providers` の中の `version` を書き換える

```diff_terraform:main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
-     version = "=4.30.0"
+     version = "=4.67.0"
    }
  }
```

`terarform init -upgrade` でアップグレードする

```sh
terraform init -upgrade
terraform refresh
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ terraform init -upgrade

Initializing the backend...
Upgrading modules...
- vnet in ../../../../modules/vpc

Initializing provider plugins...
- Finding hashicorp/aws versions matching "4.67.0"...
- Installing hashicorp/aws v4.67.0...
- Installed hashicorp/aws v4.67.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform refresh

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXXXX]
```
</div></details>

##### State File Diff 

Upgrade 後の State file を落としてみて、backup と比較すると下記のような感じだった

```diff
$ diff terraform.state.module.backup terraform.state
4c4
<   "serial": 1,
---
>   "serial": 2,
28a29
>             "enable_network_address_usage_metrics": false,
```

#### 3. Upgrade to 5.0.0

同様に version を書き換えて実行する

```diff_terraform:main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
-     version = "=4.67.0"
+     version = "=5.0.0"
    }
  }
```

```sh
terraform init -upgrade
terraform refresh
terraform plan
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ terraform init -upgrade


Initializing the backend...
Upgrading modules...
- vnet in ../../../../modules/vpc

Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.0.0"...
- Installing hashicorp/aws v5.0.0...
- Installed hashicorp/aws v5.0.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform refresh

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXX]

$ terraform plan
module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
```

</div></details>

#### 4. Upgrade to 5.38.0

同様に version を書き換えて実行する

```diff_terraform:main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
-     version = "=5.0.0"
+     version = "=5.38.0"
    }
  }
```

```sh
terraform init -upgrade
terraform refresh
terraform plan
```

<details><summary>　出力例　</summary><div>

```sh:出力例
% terraform init -upgrade


Initializing the backend...
Upgrading modules...
- vnet in ../../../../modules/vpc

Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.38.0"...
- Installing hashicorp/aws v5.38.0...
- Installed hashicorp/aws v5.38.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform refresh

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXX]
$ terraform plan

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

$ terraform plan

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
```
</div></details>

##### State File Diff 

Upgrade 後の State file を落としてみて、backup と比較すると下記のような感じだった

```diff
$ diff terraform.state.module.backup terraform.state
4c4
<   "serial": 1,
---
>   "serial": 3,
25,26d24
<             "enable_classiclink": false,
<             "enable_classiclink_dns_support": false,
28a27
>             "enable_network_address_usage_metrics": false,
```

以上で Provider のアップグレードは完了

#### 5. Downgrade to 4.54.0

provider の切り戻しの試験をする

version を書き換える

```diff_terraform:main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
-     version = "=5.38.0"
+     version = "=4.30.0"
    }
  }
```

state file をバックアップファイルをアップロードして書き換える

```sh
aws s3 cp --profile=infra ./terraform.state.module.backup s3://${AWS_BUCKET_NAME}/infra/vpc/terraform.state
```

`.terraform.lock.hcl` に `version     = "5.38.0"`等の記載があるので、名前を変えておく (実施しないと update しろとかでる)

あとは `terraform init -reconfigure` してから、`terraform plan` で `No changes.` を確認する

```sh
mv .terraform.lock.hcl .terraform.lock.hcl.bak
terraform init -reconfigure
terraform plan
```

<details><summary>　出力例　</summary><div>

```sh:出力例
$ mv .terraform.lock.hcl .terraform.lock.hcl.bak
$ terraform init -reconfigure                   


Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "4.30.0"...
- Installing hashicorp/aws v4.30.0...
- Installed hashicorp/aws v4.30.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform plan

module.vnet.aws_vpc.main: Refreshing state... [id=vpc-XXXXXXXXXXXXXXXX]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
```

</div></details>

念の為、state ファイルをダウンロードしてバックアップファイルと比較したが diff はなかった

以上で切り戻し試験も完了

## おわりに

Terraform 本体と Provider のアップグレードを試して実施できることを確認した

今回の試験は、aws の vpc のみしかなく、パラメータもあまり指定してないので大きな影響はなかったが、
実際には Upgrade Guide などにある変更内容を確認して差分がある場合は対応なども必要になりそう

## 参考

https://support.hashicorp.com/hc/en-us/articles/6302733655315-Terraform-Upgrade-Best-Practices


https://developer.hashicorp.com/terraform/language/v1.7.x/upgrade-guides


https://developer.hashicorp.com/terraform/language/v1.1.x/upgrade-guides/1-0#upgrading-to-terraform-v1-0

https://developer.hashicorp.com/terraform/language/v1-compatibility-promises