Repository: skypilot-org/skypilot
Pull request: [Core/AWS] Skip association with no subnetId and fix autodown for spot instances
<!-- Describe the changes in this PR -->

Fixes #2920.

<!-- Describe the tests ran -->
<!-- Unit tests (tests/test_*.py) are part of GitHub CI; below are tests that launch on the cloud. -->

Tested (run the relevant ones):

- [ ] Code formatting: `bash format.sh`
- [ ] Any manual or new tests for this PR (please specify below)
  - [x] `sky launch --cloud aws --region us-west-2 -c test-west --cpus 2 echo hi`
- [ ] All smoke tests: `pytest tests/test_smoke.py` 
- [ ] Relevant individual smoke tests: `pytest tests/test_smoke.py::test_fill_in_the_name` 
- [ ] Backward compatibility tests: `bash tests/backward_comaptibility_tests.sh`


Problem statement:
[AWS] KeyError 'SubnetId' when multiple Main route tables appear in a same region
`sky launch --cloud aws --region us-west-2 --cpus 2 echo hi`
```
E 12-29 15:55:13 provisioner.py:88] Failed to configure the cloud for 'test-west' with the following error:
E 12-29 15:55:13 provisioner.py:88] KeyError: 'SubnetId'
W 12-29 15:55:14 cloud_vm_ray_backend.py:2246] sky.exceptions.ResourcesUnavailableError: Failed to acquire resources in all zones in us-west-2. Try changing resource requirements or use another region.
```

It is mainly because the route table associations can have both main:true and main:false in the same route table.
`aws ec2 describe-route-tables --region us-west-2`
```
{
    "RouteTables": [
        {
            "Associations": [
                {
                    "Main": false,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "SubnetId": "subnet-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                },
                {
                    "Main": false,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "SubnetId": "subnet-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                },
                {
                    "Main": false,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "SubnetId": "subnet-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                },
                {
                    "Main": false,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "SubnetId": "subnet-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                },
                {
                    "Main": true,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                }
            ],
            "PropagatingVgws": [],
            "RouteTableId": "rtb-xxxxxx",
            "Routes": [
                {
                    "DestinationCidrBlock": "10.0.0.0/16",
                    "GatewayId": "local",
                    "Origin": "CreateRouteTable",
                    "State": "active"
                },
                {
                    "DestinationCidrBlock": "0.0.0.0/0",
                    "GatewayId": "igw-xxxxxx",
                    "Origin": "CreateRoute",
                    "State": "active"
                }
            ],
            "Tags": [],
            "VpcId": "vpc-xxxxxx",
            "OwnerId": "xxxxxxxx"
        },
        {
            "Associations": [
                {
                    "Main": true,
                    "RouteTableAssociationId": "rtbassoc-xxxxxx",
                    "RouteTableId": "rtb-xxxxxx",
                    "AssociationState": {
                        "State": "associated"
                    }
                }
            ],
            "PropagatingVgws": [],
            "RouteTableId": "rtb-xxxxxx",
            "Routes": [
                {
                    "DestinationCidrBlock": "172.31.0.0/16",
                    "GatewayId": "local",
                    "Origin": "CreateRouteTable",
                    "State": "active"
                },
                {
                    "DestinationCidrBlock": "0.0.0.0/0",
                    "GatewayId": "igw-xxxxxx",
                    "Origin": "CreateRoute",
                    "State": "active"
                }
            ],
            "Tags": [],
            "VpcId": "vpc-xxxxxx",
            "OwnerId": "xxxxxx"
        }
    ]
}
```

Hints from issue discussion:

Commit message: Skip association if no subnetid

Patch to review:
diff --git a/sky/provision/aws/config.py b/sky/provision/aws/config.py
index b03835068f0..b113b1a5714 100644
--- a/sky/provision/aws/config.py
+++ b/sky/provision/aws/config.py
@@ -218,7 +218,10 @@ def _is_subnet_public(ec2, subnet_id, vpc_id: Optional[str]) -> bool:
         rt for rt in all_route_tables
         # An RT can be associated with multiple subnets, i.e.,
         # rt['Associations'] is a list of associations.
-        if any(assoc['SubnetId'] == subnet_id for assoc in rt['Associations'])
+        # There may be no subnet associated in a association, we skip it.s
+        if any(
+            assoc.get('SubnetId', '') == subnet_id
+            for assoc in rt['Associations'])
     ]
 
     # Check each route table for an internet gateway route


Code review comments:
diff_hunk: @@ -218,7 +218,10 @@ def _is_subnet_public(ec2, subnet_id, vpc_id: Optional[str]) -> bool:
         rt for rt in all_route_tables
         # An RT can be associated with multiple subnets, i.e.,
         # rt['Associations'] is a list of associations.
-        if any(assoc['SubnetId'] == subnet_id for assoc in rt['Associations'])
+        # There may be no subnet associated in a association, we skip it.s
path: sky/provision/aws/config.py
text: @user1:
```suggestion
        # There may be no subnet associated in an association.
```