Repository: getmoto/moto

Issue:
Waiting for VPC Peering Connection to become available
I know that you're working on various VPC issues, so you may be aware of this problem.  I wanted to cover my bases and report it in case you're not.  

I'm using release 2.2.4 of moto.

I have two tests (written in Terraform) that I run against a Terraform module that sets up a peered connection.  It doesn't matter which order I run the tests, the second one fails with the error below.  I'm struggling to come up with self-contained tests to demonstrate the problem, but if the error message can't help point you at the problem, I'll work on the tests some more.

```
module.vpc_pcx_requester.aws_route_table_association.public[0]: Creation complete after 0s [id=rtbassoc-60df5245]
 
Error: Error waiting for VPC Peering Connection to become available: 
Error waiting for VPC Peering Connection (pcx-e88f13a3) to become available
unexpected state 'deleted', wanted target 'pending-acceptance, active'. last error: %!s(<nil>)

  with module.generated_route_tables.aws_vpc_peering_connection.this,
  on ../../main.tf line 15, in resource "aws_vpc_peering_connection" "this":
  15: resource "aws_vpc_peering_connection" "this" { !
make: *** [pytest//home/kbalk/plus3it/tardigrade-ci/tests/terraform_pytest] Error 1
```


Discussion:
Hey @kbalk, not sure what would cause that honestly.
As far as I can tell, the `deleted` status only occurs when explicitly calling `delete_vpc_peering_connection`.

Are there any other logs related to the VPC Peering Connection (successful creation, warnings, etc)?
When the first (Terraform) test completes, it will destroy the resources and clean up.  Then the second (Terraform) test starts up.  So there is a delete operation in between.  I'll see what I can come up with to help test this.
I have a self-contained version of the tests, which I've attached, that works with AWS using the Terraform plan, apply and destroy commands.  With moto, the Terraform plan and apply works, but using the terraform envvar `TF_LOG=debug`, I can see it struggling with unmarshalling of error messages when performing the destroy.  I didn't try to re-run the test for the original problem when I saw the problems with the destroy.

To run the test, it's the usual scenario:

```
terraform init
terraform plan
terraform apply   # respond with "yes"
terraform destroy
```
[generate_routes.tar.gz](https://github.com/spulec/moto/files/7049795/generate_routes.tar.gz)

When I initially set the peer region to a region different than that of the default provider, both AWS and moto had problems.  But moto had another unmarshalling error message, so I thought I would mention that as well:

Note:  I'm using Terraform 1.0.0.

```
2021-08-25T16:14:24.484-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/25 16:14:24 [DEBUG] [aws-sdk-go] DEBUG: Retrying Request ec2/DescribeVpcs, attempt 12: timestamp=2021-08-25T16:14:24.483-0400
2021-08-25T16:14:24.484-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/25 16:14:24 [DEBUG] [aws-sdk-go] DEBUG: Request ec2/DescribeVpcs Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: localhost:5000
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.0.0 (+https://www.terraform.io) terraform-provider-aws/3.55.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.40.24 (go1.16; linux; amd64)
Content-Length: 59
Authorization: AWS4-HMAC-SHA256 Credential=test/20210825/us-east-2/ec2/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=68f7cb93d66927aaeb217f05f8f3f4f6ecdd5260c310cf2a8d0670694dc5dd81
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20210825T201424Z
Accept-Encoding: gzip

Action=DescribeVpcs&Version=2016-11-15&VpcId.1=vpc-57b62b40
-----------------------------------------------------: timestamp=2021-08-25T16:14:24.483-0400
2021-08-25T16:14:24.493-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/25 16:14:24 [DEBUG] [aws-sdk-go] DEBUG: Response ec2/DescribeVpcs Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.0 400 BAD REQUEST
Connection: close
Content-Length: 308
Access-Control-Allow-Origin: *
Content-Type: application/xml
Date: Wed, 25 Aug 2021 20:14:24 GMT
Server: amazon.com
Status: 400
X-Amzn-Errortype: InvalidVpcID.NotFound


-----------------------------------------------------: timestamp=2021-08-25T16:14:24.493-0400
2021-08-25T16:14:24.493-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/25 16:14:24 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
  <ErrorResponse>
    <Errors>
      <Error>
        <Code>InvalidVpcID.NotFound</Code>
        <Message>VpcID {'vpc-57b62b40'} does not exist.</Message>
        
      </Error>
    </Errors>
  <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
</ErrorResponse>: timestamp=2021-08-25T16:14:24.493-0400
2021-08-25T16:14:24.493-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/25 16:14:24 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ec2/DescribeVpcs failed, attempt 12/25, error SerializationError: failed to unmarshal error message
	status code: 400, request id: 
caused by: UnmarshalError: failed to unmarshal error message
	00000000  3c 3f 78 6d 6c 20 76 65  72 73 69 6f 6e 3d 22 31  |<?xml version="1|
00000010  2e 30 22 20 65 6e 63 6f  64 69 6e 67 3d 22 55 54  |.0" encoding="UT|
00000020  46 2d 38 22 3f 3e 0a 20  20 3c 45 72 72 6f 72 52  |F-8"?>.  <ErrorR|
00000030  65 73 70 6f 6e 73 65 3e  0a 20 20 20 20 3c 45 72  |esponse>.    <Er|
00000040  72 6f 72 73 3e 0a 20 20  20 20 20 20 3c 45 72 72  |rors>.      <Err|
00000050  6f 72 3e 0a 20 20 20 20  20 20 20 20 3c 43 6f 64  |or>.        <Cod|
00000060  65 3e 49 6e 76 61 6c 69  64 56 70 63 49 44 2e 4e  |e>InvalidVpcID.N|
00000070  6f 74 46 6f 75 6e 64 3c  2f 43 6f 64 65 3e 0a 20  |otFound</Code>. |
00000080  20 20 20 20 20 20 20 3c  4d 65 73 73 61 67 65 3e  |       <Message>|
00000090  56 70 63 49 44 20 7b 27  76 70 63 2d 35 37 62 36  |VpcID {'vpc-57b6|
000000a0  32 62 34 30 27 7d 20 64  6f 65 73 20 6e 6f 74 20  |2b40'} does not |
000000b0  65 78 69 73 74 2e 3c 2f  4d 65 73 73 61 67 65 3e  |exist.</Message>|
000000c0  0a 20 20 20 20 20 20 20  20 0a 20 20 20 20 20 20  |.        .      |
000000d0  3c 2f 45 72 72 6f 72 3e  0a 20 20 20 20 3c 2f 45  |</Error>.    </E|
000000e0  72 72 6f 72 73 3e 0a 20  20 3c 52 65 71 75 65 73  |rrors>.  <Reques|
000000f0  74 49 44 3e 37 61 36 32  63 34 39 66 2d 33 34 37  |tID>7a62c49f-347|
00000100  65 2d 34 66 63 34 2d 39  33 33 31 2d 36 65 38 65  |e-4fc4-9331-6e8e|
00000110  45 58 41 4d 50 4c 45 3c  2f 52 65 71 75 65 73 74  |EXAMPLE</Request|
00000120  49 44 3e 0a 3c 2f 45 72  72 6f 72 52 65 73 70 6f  |ID>.</ErrorRespo|
00000130  6e 73 65 3e                                       |nse>|

caused by: expected element type <Response> but have <ErrorResponse>: timestamp=2021-08-25T16:14:24.493-0400
data.aws_vpc.peer: Still reading... [3m20s elapsed]
```

Thanks for the logs @kbalk. Looks like this line says it all:
`expected element type <Response> but have <ErrorResponse>`

Our response-format is wrong, when trying to describe a VPC.

Interestingly, I could not produce this exactly with your example - I got the same error, but when trying to delete the RouteTable - deleting a VPC worked fine for me. 

I've created a branch that fixes both responses, for DescribeVPC and DescribeRouteTable. Would you mind trying to run your original tests against my branch, to verify that this fixes the problem?
https://github.com/bblommers/moto/tree/ec2-describeroutetable-errorformat
There's good news and bad news ...  Using the test case I sent you:

1.  No more invalid message problems ... woo hoo!
2. After I fetched your branch, the first test run completes ok, but the second time the Terraform apply fails, details at the very end.
3. I'm seeing a new error, the logs of which I've captured below.

When I run a test, I have moto_server running in a second window and I keep an eye on the HTTP response codes.  I'm still seeing the 400 response codes for this test, so I do the following to get more details: 

```
# NOTE:  I start moto_server at the beginning and I do not stop it in between tests.

$ export TF_LOG=debug
$ rm -rf .terraform* terraform*
$ terraform init
$ terraform plan
$ terraform apply
$ terraform destroy   # Shows 400 response codes

# To get the next error, continue with the commands ...
$ terraform plan
$ terraform apply
```

Here are the details on the 400 messages, i.e.:  
```
2021-08-26T11:40:02.843-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 11:40:02 
[DEBUG] [aws-sdk-go] <DeleteSubnetResponse xmlns="http://ec2.amazonaws.com/doc/2013-10-15/">
<requestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</requestId><return>true</return>
</DeleteSubnetResponse>: timestamp=2021-08-26T11:40:02.836-0400
module.vpc_pcx_requester.aws_route_table.private[2]: Destruction complete after 1s
2021-08-26T11:40:02.875-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 11:40:02 [DEBUG] [aws-sdk-go] DEBUG: Response ec2/DescribeRouteTables Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.0 400 BAD REQUEST
Connection: close
Content-Length: 291
Access-Control-Allow-Origin: *
Content-Type: application/xml
Date: Thu, 26 Aug 2021 15:40:02 GMT
Server: amazon.com
Status: 400
X-Amzn-Errortype: InvalidRouteTableID.NotFound


-----------------------------------------------------: timestamp=2021-08-26T11:40:02.874-0400
2021-08-26T11:40:02.875-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 11:40:02 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Errors>
    <Error>
      <Code>InvalidRouteTableID.NotFound</Code>
      <Message>The routeTable ID 'rtb-eaf0d38d' does not exist</Message>
    </Error>
  </Errors>
  <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
</Response>: timestamp=2021-08-26T11:40:02.874-0400
2021-08-26T11:40:02.875-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 11:40:02 [DEBUG] [aws-sdk-go] 
DEBUG: Validate Response ec2/DescribeRouteTables failed, attempt 0/25, error InvalidRouteTableID.NotFound: 
The routeTable ID 'rtb-eaf0d38d' does not exist
	status code: 400, request id: 7a62c49f-347e-4fc4-9331-6e8eEXAMPLE: timestamp=2021-08-26T11:40:02.874-0400
module.vpc_pcx_requester.aws_route_table.private[0]: Destruction complete after 1s
```

Info on the second apply error:

```
module.vpc_pcx_requester.aws_route_table_association.public[2]: 
Creation complete after 0s [id=rtbassoc-09762de3]
2021-08-26T12:02:18.179-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 12:02:18 
[DEBUG] [aws-sdk-go] DEBUG: Response ec2/DescribeRouteTables Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.0 200 OK
Connection: close
Content-Length: 1552
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=utf-8
Date: Thu, 26 Aug 2021 16:02:18 GMT
Server: amazon.com


-----------------------------------------------------: timestamp=2021-08-26T12:02:18.179-0400
2021-08-26T12:02:18.179-0400 [INFO]  provider.terraform-provider-aws_v3.55.0_x5: 2021/08/26 12:02:18 
[DEBUG] [aws-sdk-go] <DescribeRouteTablesResponse xmlns="http://ec2.amazonaws.com/doc/2013-10-15/">
<requestId>6f570b0b-9c18-4b07-bdec-73740dcf861a</requestId>
<routeTableSet><item><routeTableId>rtb-2842d66e</routeTableId><vpcId>vpc-23b0eb4e</vpcId>
<ownerId>123456789012</ownerId><routeSet><item><destinationCidrBlock>10.0.0.0/16</destinationCidrBlock>
<gatewayId>local</gatewayId><origin>CreateRouteTable</origin><state>active</state></item><item>
<destinationCidrBlock>0.0.0.0/0</destinationCidrBlock><gatewayId>igw-ec4abd5d</gatewayId>
<origin>CreateRoute</origin><state>active</state></item></routeSet><associationSet><item>
<routeTableAssociationId>rtbassoc-51401f98</routeTableAssociationId><routeTableId>rtb-2842d66e</routeTableId>
<main>true</main><subnetId>subnet-bd61bc66</subnetId><associationState><state>associated</state>
</associationState></item><item><routeTableAssociationId>rtbassoc-0cbf4dd2</routeTableAssociationId>
<routeTableId>rtb-2842d66e</routeTableId><main>true</main><subnetId>subnet-44ad4360</subnetId>
<associationState><state>associated</state></associationState></item><item>
<routeTableAssociationId>rtbassoc-09762de3</routeTableAssociationId><routeTableId>rtb-2842d66e</routeTableId>
<main>true</main><subnetId>subnet-3dce302f</subnetId><associationState><state>associated</state>
</associationState></item></associationSet><tagSet><item><resourceId>rtb-2842d66e</resourceId>
<resourceType>route-table</resourceType><key>Name</key><value>tardigrade-pcx-requester-kxdpqp-public</value>
</item></tagSet></item></routeTableSet></DescribeRouteTablesResponse>: 
timestamp=2021-08-26T12:02:18.179-0400
2021-08-26T12:02:18.180-0400 [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an
unexpected new value for module.vpc_pcx_requester.aws_route_table_association.public[2], but we are tolerating it 
because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .gateway_id: was null, but now cty.StringVal("")

│ Error: Error waiting for VPC Peering Connection to become available: Error waiting for VPC Peering Connection 
(pcx-ec2c358a) to become available: unexpected state 'deleted', wanted target 'pending-acceptance, active'. 
last error: %!s(<nil>)
│ 
│   with aws_vpc_peering_connection.this,
│   on main.tf line 15, in resource "aws_vpc_peering_connection" "this":
│   15: resource "aws_vpc_peering_connection" "this" {
```
Well, one step at the time I suppose. I'll merge the 'invalid response' fix into master for now, and I'll start looking into the VPC Peering Connection problem either tomorrow or this weekend.

Resolving patch:
diff --git a/moto/ec2/models.py b/moto/ec2/models.py
--- a/moto/ec2/models.py
+++ b/moto/ec2/models.py
@@ -3563,7 +3563,9 @@ def create_vpc_peering_connection(self, vpc, peer_vpc, tags=None):
                     vpc_pcx_cx.vpc_pcxs[vpc_pcx_id] = vpc_pcx
         return vpc_pcx
 
-    def get_all_vpc_peering_connections(self):
+    def get_all_vpc_peering_connections(self, vpc_peering_ids=None):
+        if vpc_peering_ids:
+            return [pcx for pcx in self.vpc_pcxs.values() if pcx.id in vpc_peering_ids]
         return self.vpc_pcxs.values()
 
     def get_vpc_peering_connection(self, vpc_pcx_id):
diff --git a/moto/ec2/responses/vpc_peering_connections.py b/moto/ec2/responses/vpc_peering_connections.py
--- a/moto/ec2/responses/vpc_peering_connections.py
+++ b/moto/ec2/responses/vpc_peering_connections.py
@@ -29,7 +29,8 @@ def delete_vpc_peering_connection(self):
         return template.render(vpc_pcx=vpc_pcx)
 
     def describe_vpc_peering_connections(self):
-        vpc_pcxs = self.ec2_backend.get_all_vpc_peering_connections()
+        ids = self._get_multi_param("VpcPeeringConnectionId")
+        vpc_pcxs = self.ec2_backend.get_all_vpc_peering_connections(vpc_peering_ids=ids)
         template = self.response_template(DESCRIBE_VPC_PEERING_CONNECTIONS_RESPONSE)
         return template.render(vpc_pcxs=vpc_pcxs)
 


Test patch:
diff --git a/tests/test_ec2/test_vpc_peering.py b/tests/test_ec2/test_vpc_peering.py
--- a/tests/test_ec2/test_vpc_peering.py
+++ b/tests/test_ec2/test_vpc_peering.py
@@ -227,6 +227,36 @@ def test_vpc_peering_connections_cross_region_fail():
     cm.value.response["Error"]["Code"].should.equal("InvalidVpcID.NotFound")
 
 
+@mock_ec2
+def test_describe_vpc_peering_connections_only_returns_requested_id():
+    # create vpc in us-west-1 and ap-northeast-1
+    ec2_usw1 = boto3.resource("ec2", region_name="us-west-1")
+    vpc_usw1 = ec2_usw1.create_vpc(CidrBlock="10.90.0.0/16")
+    ec2_apn1 = boto3.resource("ec2", region_name="ap-northeast-1")
+    vpc_apn1 = ec2_apn1.create_vpc(CidrBlock="10.20.0.0/16")
+    # create peering
+    vpc_pcx_usw1 = ec2_usw1.create_vpc_peering_connection(
+        VpcId=vpc_usw1.id, PeerVpcId=vpc_apn1.id, PeerRegion="ap-northeast-1"
+    )
+    vpc_pcx_usw2 = ec2_usw1.create_vpc_peering_connection(
+        VpcId=vpc_usw1.id, PeerVpcId=vpc_apn1.id, PeerRegion="ap-northeast-1"
+    )
+    # describe peering
+    ec2_usw1 = boto3.client("ec2", region_name="us-west-1")
+    all_pcx = ec2_usw1.describe_vpc_peering_connections()["VpcPeeringConnections"]
+    all_pcx.should.have.length_of(2)
+
+    both_pcx = ec2_usw1.describe_vpc_peering_connections(
+        VpcPeeringConnectionIds=[vpc_pcx_usw1.id, vpc_pcx_usw2.id]
+    )["VpcPeeringConnections"]
+    both_pcx.should.have.length_of(2)
+
+    one_pcx = ec2_usw1.describe_vpc_peering_connections(
+        VpcPeeringConnectionIds=[vpc_pcx_usw1.id]
+    )["VpcPeeringConnections"]
+    one_pcx.should.have.length_of(1)
+
+
 @mock_ec2
 def test_vpc_peering_connections_cross_region_accept():
     # create vpc in us-west-1 and ap-northeast-1