Repository: getmoto/moto

Issue:
Unpredictable results when filtering EC2 instances based on tags.
The following gist exposes the bug I have currently hit:

https://gist.github.com/esc/88780a70f9acb6a53f603ebbe5fcc8ec

which is:

```
import boto3
import moto


@moto.mock_ec2
def expose():
    ec2 = boto3.resource('ec2')
    blue, green = ec2.create_instances(
        ImageId='ANY_ID', MinCount=2, MaxCount=2)
    ec2.create_tags(Resources=[blue.instance_id],
                    Tags=[{"Key": "environment", "Value": "blue"},
                          {"Key": "application", "Value": "api"},
                          ])
    ec2.create_tags(Resources=[green.instance_id],
                    Tags=[{"Key": "environment", "Value": "green"},
                          {"Key": "application", "Value": "api"},
                          ])
    filter_ = [{'Name': 'tag-key', 'Values': ['application']},
               {'Name': 'tag-value', 'Values': ['api']},
               {'Name': 'tag-key', 'Values': ['environment']},
               {'Name': 'tag-value', 'Values': ['green']},
               ]
    instances = list(ec2.instances.filter(Filters=filter_))
    print(instances)

expose()
```

Assuming I have understood the way in which EC2 instances are filtered correctly, the above should _only_ return a single instances, namely the `green` one. The problem is that filtering on EC2 instances produces unpredictable results, sometimes it works and I get only the green instance, but sometimes it fails and I get both instances. The behavior is loosely correlated with the phase of the moon (and essentially a result of the implementation of the filter in moto and the fact that python dictionaries are not ordered).

Observe:

```
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py                                                     
[ec2.Instance(id='i-ddfcda29')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-da6b71e8'), ec2.Instance(id='i-7cc6583c')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-fbd09878'), ec2.Instance(id='i-9942c0a7')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-58a64557')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-fe6b4589'), ec2.Instance(id='i-0b58c502')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-e0d763bd'), ec2.Instance(id='i-a7287241')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-2a5c6f0c'), ec2.Instance(id='i-c9a5bf46')]
esc@omnitech ~/git/moto-bug (git)-[master] % python expose_moto_bug.py
[ec2.Instance(id='i-a47f2988')]
```

Does anyone know what the correct behavior should be and can someone else confirm this is reproducible. Note: I am using Python3.



Discussion:
I don't believe you can use more than one of the same filters at once. In this instance, you can only use `tag-key` once. By passing two, there is undefined behavior about which one it will use.

http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html

`tag-key - The key of a tag assigned to the resource. This filter is independent of the tag-value filter. For example, if you use both the filter "tag-key=Purpose" and the filter "tag-value=X", you get any resources assigned both the tag key Purpose (regardless of what the tag's value is), and the tag value X (regardless of what the tag's key is). If you want to list only resources where Purpose is X, see the tag:key=value filter.`

You may want to use the `tag:key=value` filter instead. Let me know if this doesn't match what you are seeing against the real Amazon API.

I'll try with `tag:key=value` and report back.

In any case, my example above works fine on AWS but not using moto.

In the above, you should probably return the instance which has key values `application` and `environment` as well as the value values `api` and `green`. In this case it doesn't matter, if `application` maps to `api` or `green`, its just important that _a_ key with a specific value and a value with a specific value is present (ignoring if one maps onto the other). I believe that is what the docs say too, and in fact the behavior is actually well defined:

```
you get any resources assigned both the tag key Purpose (regardless of what the tag's value is), and the tag value X (regardless of what the tag's key is).
```

Are you seeing it working consistently against AWS? My understanding is that you can't pass the same filter type twice.

Yes, I run this setup in production where I query with multiple `tag-key` filters.


Resolving patch:
diff --git a/moto/ec2/utils.py b/moto/ec2/utils.py
--- a/moto/ec2/utils.py
+++ b/moto/ec2/utils.py
@@ -252,7 +252,8 @@ def dhcp_configuration_from_querystring(querystring, option="DhcpConfiguration")
 
 def filters_from_querystring(querystring_dict):
     response_values = {}
-    for key, value in querystring_dict.items():
+    last_tag_key = None
+    for key, value in sorted(querystring_dict.items()):
         match = re.search(r"Filter.(\d).Name", key)
         if match:
             filter_index = match.groups()[0]
@@ -262,6 +263,10 @@ def filters_from_querystring(querystring_dict):
                 for filter_key, filter_value in querystring_dict.items()
                 if filter_key.startswith(value_prefix)
             ]
+            if value[0] == "tag-key":
+                last_tag_key = "tag:" + filter_values[0]
+            elif last_tag_key and value[0] == "tag-value":
+                response_values[last_tag_key] = filter_values
             response_values[value[0]] = filter_values
     return response_values
 
@@ -329,6 +334,8 @@ def tag_filter_matches(obj, filter_name, filter_values):
         tag_values = get_obj_tag_names(obj)
     elif filter_name == "tag-value":
         tag_values = get_obj_tag_values(obj)
+    elif filter_name.startswith("tag:"):
+        tag_values = get_obj_tag_values(obj)
     else:
         tag_values = [get_obj_tag(obj, filter_name) or ""]
 


Test patch:
diff --git a/tests/test_ec2/test_tags.py b/tests/test_ec2/test_tags.py
--- a/tests/test_ec2/test_tags.py
+++ b/tests/test_ec2/test_tags.py
@@ -468,3 +468,36 @@ def test_delete_tag_empty_resource():
     ex.exception.response["Error"]["Message"].should.equal(
         "The request must contain the parameter resourceIdSet"
     )
+
+
+@mock_ec2
+def test_retrieve_resource_with_multiple_tags():
+    ec2 = boto3.resource("ec2", region_name="us-west-1")
+    blue, green = ec2.create_instances(ImageId="ANY_ID", MinCount=2, MaxCount=2)
+    ec2.create_tags(
+        Resources=[blue.instance_id],
+        Tags=[
+            {"Key": "environment", "Value": "blue"},
+            {"Key": "application", "Value": "api"},
+        ],
+    )
+    ec2.create_tags(
+        Resources=[green.instance_id],
+        Tags=[
+            {"Key": "environment", "Value": "green"},
+            {"Key": "application", "Value": "api"},
+        ],
+    )
+    green_instances = list(ec2.instances.filter(Filters=(get_filter("green"))))
+    green_instances.should.equal([green])
+    blue_instances = list(ec2.instances.filter(Filters=(get_filter("blue"))))
+    blue_instances.should.equal([blue])
+
+
+def get_filter(color):
+    return [
+        {"Name": "tag-key", "Values": ["application"]},
+        {"Name": "tag-value", "Values": ["api"]},
+        {"Name": "tag-key", "Values": ["environment"]},
+        {"Name": "tag-value", "Values": [color]},
+    ]