Repository: getmoto/moto
Pull request #5981 (2023-02-24 23:15:26)

Techdebt: MyPy ElasticTranscoder

codecov[bot]: # [Codecov](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#5981](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (d33f0c8) into [master](https://codecov.io/gh/getmoto/moto/commit/dae4f4947e8d092c87246befc8c2694afc096b7e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (dae4f49) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

```diff
@@           Coverage Diff           @@
##           master    #5981   +/-   ##
=======================================
  Coverage   96.22%   96.22%           
=======================================
  Files         760      760           
  Lines       74805    74808    +3     
=======================================
+ Hits        71983    71986    +3     
  Misses       2822     2822           
```

| Flag | Coverage Δ | |
|---|---|---|
| servertests | `37.00% <92.59%> (+<0.01%)` | :arrow_up: |
| unittests | `96.16% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [moto/core/common\_types.py](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9jb3JlL2NvbW1vbl90eXBlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [moto/elastictranscoder/models.py](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9lbGFzdGljdHJhbnNjb2Rlci9tb2RlbHMucHk=) | `100.00% <100.00%> (ø)` | |
| [moto/elastictranscoder/responses.py](https://codecov.io/gh/getmoto/moto/pull/5981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9lbGFzdGljdHJhbnNjb2Rlci9yZXNwb25zZXMucHk=) | `98.76% <100.00%> (+0.03%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)


Patch:
diff --git a/IMPLEMENTATION_COVERAGE.md b/IMPLEMENTATION_COVERAGE.md
index c602ce7b1..7bd4458ab 100644
--- a/IMPLEMENTATION_COVERAGE.md
+++ b/IMPLEMENTATION_COVERAGE.md
@@ -2391,7 +2391,7 @@
 
 ## ecs
 <details>
-<summary>80% implemented</summary>
+<summary>78% implemented</summary>
 
 - [X] create_capacity_provider
 - [X] create_cluster
@@ -2402,6 +2402,7 @@
 - [X] delete_capacity_provider
 - [X] delete_cluster
 - [X] delete_service
+- [ ] delete_task_definitions
 - [X] delete_task_set
 - [X] deregister_container_instance
 - [X] deregister_task_definition
@@ -3197,7 +3198,7 @@
 - [ ] get_job_bookmark
 - [X] get_job_run
 - [ ] get_job_runs
-- [ ] get_jobs
+- [X] get_jobs
 - [ ] get_mapping
 - [ ] get_ml_task_run
 - [ ] get_ml_task_runs
@@ -6880,6 +6881,7 @@
 - support-app
 - synthetics
 - timestream-query
+- tnb
 - transfer
 - translate
 - voice-id
diff --git a/docs/docs/services/ecs.rst b/docs/docs/services/ecs.rst
index c3ddd7d99..735415407 100644
--- a/docs/docs/services/ecs.rst
+++ b/docs/docs/services/ecs.rst
@@ -36,6 +36,7 @@ ecs
 - [X] delete_capacity_provider
 - [X] delete_cluster
 - [X] delete_service
+- [ ] delete_task_definitions
 - [X] delete_task_set
   
         The Force-parameter is not yet implemented
diff --git a/docs/docs/services/glue.rst b/docs/docs/services/glue.rst
index 219f6b80c..b3a559923 100644
--- a/docs/docs/services/glue.rst
+++ b/docs/docs/services/glue.rst
@@ -124,7 +124,7 @@ glue
 - [ ] get_job_bookmark
 - [X] get_job_run
 - [ ] get_job_runs
-- [ ] get_jobs
+- [X] get_jobs
 - [ ] get_mapping
 - [ ] get_ml_task_run
 - [ ] get_ml_task_runs
diff --git a/moto/__init__.py b/moto/__init__.py
index 18d8a72d9..b25359aba 100644
--- a/moto/__init__.py
+++ b/moto/__init__.py
@@ -88,7 +88,7 @@ mock_firehose = lazy_load(".firehose", "mock_firehose")
 mock_forecast = lazy_load(".forecast", "mock_forecast")
 mock_greengrass = lazy_load(".greengrass", "mock_greengrass")
 mock_glacier = lazy_load(".glacier", "mock_glacier")
-mock_glue = lazy_load(".glue", "mock_glue")
+mock_glue = lazy_load(".glue", "mock_glue", boto3_name="glue")
 mock_guardduty = lazy_load(".guardduty", "mock_guardduty")
 mock_iam = lazy_load(".iam", "mock_iam")
 mock_iot = lazy_load(".iot", "mock_iot")
diff --git a/moto/backend_index.py b/moto/backend_index.py
index 7e2831c35..7dc48e205 100644
--- a/moto/backend_index.py
+++ b/moto/backend_index.py
@@ -1,4 +1,4 @@
-# autogenerated by /Users/danaronson/GitHub/forks/moto/scripts/update_backend_index.py
+# autogenerated by /Users/dan/Sites/moto/scripts/update_backend_index.py
 import re
 
 backend_url_patterns = [
diff --git a/moto/core/common_types.py b/moto/core/common_types.py
index 3f04dfbf6..0e1741721 100644
--- a/moto/core/common_types.py
+++ b/moto/core/common_types.py
@@ -1,5 +1,5 @@
-from typing import Dict, Tuple, TypeVar, Union
+from typing import Any, Dict, Tuple, TypeVar, Union
 
 
-TYPE_RESPONSE = Tuple[int, Dict[str, str], Union[str, bytes]]
+TYPE_RESPONSE = Tuple[int, Dict[str, Any], Union[str, bytes]]
 TYPE_IF_NONE = TypeVar("TYPE_IF_NONE")
diff --git a/moto/ec2/utils.py b/moto/ec2/utils.py
index b633bf9c9..3e0e609bc 100644
--- a/moto/ec2/utils.py
+++ b/moto/ec2/utils.py
@@ -395,6 +395,8 @@ def tag_filter_matches(obj: Any, filter_name: str, filter_values: List[str]) ->
     for tag_value in tag_values:
         if any(regex.match(tag_value) for regex in regex_filters):
             return True
+        if tag_value in filter_values:
+            return True
 
     return False
 
diff --git a/moto/elastictranscoder/models.py b/moto/elastictranscoder/models.py
index 53bb42526..d836c4c98 100644
--- a/moto/elastictranscoder/models.py
+++ b/moto/elastictranscoder/models.py
@@ -1,3 +1,4 @@
+from typing import Any, Dict, List, Tuple
 from moto.core import BaseBackend, BackendDict, BaseModel
 from moto.moto_api._internal import mock_random as random
 import string
@@ -6,14 +7,14 @@ import string
 class Pipeline(BaseModel):
     def __init__(
         self,
-        account_id,
-        region,
-        name,
-        input_bucket,
-        output_bucket,
-        role,
-        content_config,
-        thumbnail_config,
+        account_id: str,
+        region: str,
+        name: str,
+        input_bucket: str,
+        output_bucket: str,
+        role: str,
+        content_config: Dict[str, Any],
+        thumbnail_config: Dict[str, Any],
     ):
         a = "".join(random.choice(string.digits) for _ in range(13))
         b = "".join(random.choice(string.ascii_lowercase) for _ in range(6))
@@ -31,7 +32,7 @@ class Pipeline(BaseModel):
         if "Permissions" not in self.thumbnail_config:
             self.thumbnail_config["Permissions"] = []
 
-    def update(self, name, input_bucket, role):
+    def update(self, name: str, input_bucket: str, role: str) -> None:
         if name:
             self.name = name
         if input_bucket:
@@ -39,7 +40,7 @@ class Pipeline(BaseModel):
         if role:
             self.role = role
 
-    def to_dict(self):
+    def to_dict(self) -> Dict[str, Any]:
         return {
             "Id": self.id,
             "Name": self.name,
@@ -60,19 +61,19 @@ class Pipeline(BaseModel):
 
 
 class ElasticTranscoderBackend(BaseBackend):
-    def __init__(self, region_name, account_id):
+    def __init__(self, region_name: str, account_id: str):
         super().__init__(region_name, account_id)
-        self.pipelines = {}
+        self.pipelines: Dict[str, Pipeline] = {}
 
     def create_pipeline(
         self,
-        name,
-        input_bucket,
-        output_bucket,
-        role,
-        content_config,
-        thumbnail_config,
-    ):
+        name: str,
+        input_bucket: str,
+        output_bucket: str,
+        role: str,
+        content_config: Dict[str, Any],
+        thumbnail_config: Dict[str, Any],
+    ) -> Tuple[Pipeline, List[str]]:
         """
         The following parameters are not yet implemented:
         AWSKMSKeyArn, Notifications
@@ -88,26 +89,28 @@ class ElasticTranscoderBackend(BaseBackend):
             thumbnail_config,
         )
         self.pipelines[pipeline.id] = pipeline
-        warnings = []
+        warnings: List[str] = []
         return pipeline, warnings
 
-    def list_pipelines(self):
+    def list_pipelines(self) -> List[Dict[str, Any]]:
         return [p.to_dict() for _, p in self.pipelines.items()]
 
-    def read_pipeline(self, pipeline_id):
+    def read_pipeline(self, pipeline_id: str) -> Pipeline:
         return self.pipelines[pipeline_id]
 
-    def update_pipeline(self, pipeline_id, name, input_bucket, role):
+    def update_pipeline(
+        self, pipeline_id: str, name: str, input_bucket: str, role: str
+    ) -> Tuple[Pipeline, List[str]]:
         """
         The following parameters are not yet implemented:
         AWSKMSKeyArn, Notifications, ContentConfig, ThumbnailConfig
         """
         pipeline = self.read_pipeline(pipeline_id)
         pipeline.update(name, input_bucket, role)
-        warnings = []
+        warnings: List[str] = []
         return pipeline, warnings
 
-    def delete_pipeline(self, pipeline_id):
+    def delete_pipeline(self, pipeline_id: str) -> None:
         self.pipelines.pop(pipeline_id)
 
 
diff --git a/moto/elastictranscoder/responses.py b/moto/elastictranscoder/responses.py
index 476f1eef8..f2aa81091 100644
--- a/moto/elastictranscoder/responses.py
+++ b/moto/elastictranscoder/responses.py
@@ -1,18 +1,20 @@
+from typing import Any, Optional
+from moto.core.common_types import TYPE_RESPONSE
 from moto.core.responses import BaseResponse
-from .models import elastictranscoder_backends
+from .models import elastictranscoder_backends, ElasticTranscoderBackend
 import json
 import re
 
 
 class ElasticTranscoderResponse(BaseResponse):
-    def __init__(self):
+    def __init__(self) -> None:
         super().__init__(service_name="elastictranscoder")
 
     @property
-    def elastictranscoder_backend(self):
+    def elastictranscoder_backend(self) -> ElasticTranscoderBackend:
         return elastictranscoder_backends[self.current_account][self.region]
 
-    def pipelines(self, request, full_url, headers):
+    def pipelines(self, request: Any, full_url: str, headers: Any) -> TYPE_RESPONSE:
         self.setup_class(request, full_url, headers)
         if request.method == "POST":
             return self.create_pipeline()
@@ -21,7 +23,9 @@ class ElasticTranscoderResponse(BaseResponse):
         else:
             return self.update_pipeline()
 
-    def individual_pipeline(self, request, full_url, headers):
+    def individual_pipeline(
+        self, request: Any, full_url: str, headers: Any
+    ) -> TYPE_RESPONSE:
         self.setup_class(request, full_url, headers)
         if request.method == "GET":
             return self.read_pipeline()
@@ -30,7 +34,7 @@ class ElasticTranscoderResponse(BaseResponse):
         else:
             return self.update_pipeline()
 
-    def create_pipeline(self):
+    def create_pipeline(self) -> TYPE_RESPONSE:
         name = self._get_param("Name")
         input_bucket = self._get_param("InputBucket")
         output_bucket = self._get_param("OutputBucket")
@@ -65,14 +69,14 @@ class ElasticTranscoderResponse(BaseResponse):
             json.dumps({"Pipeline": pipeline.to_dict(), "Warnings": warnings}),
         )
 
-    def list_pipelines(self):
+    def list_pipelines(self) -> TYPE_RESPONSE:
         return (
             200,
             {},
             json.dumps({"Pipelines": self.elastictranscoder_backend.list_pipelines()}),
         )
 
-    def validate_pipeline_id(self, pipeline_id):
+    def validate_pipeline_id(self, pipeline_id: str) -> Optional[TYPE_RESPONSE]:
         r = "^\\d{13}-\\w{6}$"
         if not re.match(r, pipeline_id):
             msg = f"1 validation error detected: Value '{pipeline_id}' at 'id' failed to satisfy constraint: Member must satisfy regular expression pattern: {r}"
@@ -88,7 +92,7 @@ class ElasticTranscoderResponse(BaseResponse):
                 json.dumps({"message": msg}),
             )
 
-    def read_pipeline(self):
+    def read_pipeline(self) -> TYPE_RESPONSE:
         _id = self.path.rsplit("/", 1)[-1]
         err = self.validate_pipeline_id(_id)
         if err:
@@ -96,7 +100,7 @@ class ElasticTranscoderResponse(BaseResponse):
         pipeline = self.elastictranscoder_backend.read_pipeline(_id)
         return 200, {}, json.dumps({"Pipeline": pipeline.to_dict()})
 
-    def update_pipeline(self):
+    def update_pipeline(self) -> TYPE_RESPONSE:
         _id = self.path.rsplit("/", 1)[-1]
         name = self._get_param("Name")
         input_bucket = self._get_param("InputBucket")
@@ -114,12 +118,12 @@ class ElasticTranscoderResponse(BaseResponse):
             json.dumps({"Pipeline": pipeline.to_dict(), "Warnings": warnings}),
         )
 
-    def delete_pipeline(self):
+    def delete_pipeline(self) -> TYPE_RESPONSE:
         _id = self.path.rsplit("/", 1)[-1]
         self.elastictranscoder_backend.delete_pipeline(pipeline_id=_id)
         return 200, {}, "{}"
 
-    def err(self, msg):
+    def err(self, msg: str) -> TYPE_RESPONSE:
         return (
             400,
             {"status": 400, "x-amzn-ErrorType": "ValidationException"},
diff --git a/moto/glue/models.py b/moto/glue/models.py
index 70e794f49..3194f4cb5 100644
--- a/moto/glue/models.py
+++ b/moto/glue/models.py
@@ -69,6 +69,12 @@ class GlueBackend(BaseBackend):
             "limit_default": 100,
             "unique_attribute": "name",
         },
+        "get_jobs": {
+            "input_token": "next_token",
+            "limit_key": "max_results",
+            "limit_default": 100,
+            "unique_attribute": "name",
+        },
     }
 
     def __init__(self, region_name, account_id):
@@ -329,6 +335,10 @@ class GlueBackend(BaseBackend):
         except KeyError:
             raise JobNotFoundException(name)
 
+    @paginate(pagination_model=PAGINATION_MODEL)
+    def get_jobs(self):
+        return [job for _, job in self.jobs.items()]
+
     def start_job_run(self, name):
         job = self.get_job(name)
         return job.start_job_run()
diff --git a/moto/glue/responses.py b/moto/glue/responses.py
index fd797d6c2..64ddd107c 100644
--- a/moto/glue/responses.py
+++ b/moto/glue/responses.py
@@ -343,6 +343,19 @@ class GlueResponse(BaseResponse):
         job = self.glue_backend.get_job(name)
         return json.dumps({"Job": job.as_dict()})
 
+    def get_jobs(self):
+        next_token = self._get_param("NextToken")
+        max_results = self._get_int_param("MaxResults")
+        jobs, next_token = self.glue_backend.get_jobs(
+            next_token=next_token, max_results=max_results
+        )
+        return json.dumps(
+            dict(
+                Jobs=[job.as_dict() for job in jobs],
+                NextToken=next_token,
+            )
+        )
+
     def start_job_run(self):
         name = self.parameters.get("JobName")
         job_run_id = self.glue_backend.start_job_run(name)
diff --git a/setup.cfg b/setup.cfg
index 9140bce3f..9cec5d4fa 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -229,7 +229,7 @@ disable = W,C,R,E
 enable = anomalous-backslash-in-string, arguments-renamed, dangerous-default-value, deprecated-module, function-redefined, import-self, redefined-builtin, redefined-outer-name, reimported, pointless-statement, super-with-arguments, unused-argument, unused-import, unused-variable, useless-else-on-loop, wildcard-import
 
 [mypy]
-files= moto/a*,moto/b*,moto/c*,moto/d*,moto/ebs/,moto/ec2,moto/ec2instanceconnect,moto/ecr,moto/ecs,moto/efs,moto/eks,moto/elasticache,moto/elasticbeanstalk,moto/es,moto/moto_api
+files= moto/a*,moto/b*,moto/c*,moto/d*,moto/ebs/,moto/ec2,moto/ec2instanceconnect,moto/ecr,moto/ecs,moto/efs,moto/eks,moto/elasticache,moto/elasticbeanstalk,moto/elastictranscoder,moto/es,moto/moto_api
 show_column_numbers=True
 show_error_codes = True
 disable_error_code=abstract


Test patch:
diff --git a/tests/test_ec2/test_instances.py b/tests/test_ec2/test_instances.py
index 0c670c80b..491baa9e7 100644
--- a/tests/test_ec2/test_instances.py
+++ b/tests/test_ec2/test_instances.py
@@ -1,5 +1,6 @@
 import base64
 import ipaddress
+import json
 import warnings
 from unittest import SkipTest, mock
 from uuid import uuid4
@@ -769,9 +770,15 @@ def test_get_instances_filtering_by_tag():
     tag1_val = str(uuid4())
     tag2_name = str(uuid4())[0:6]
     tag2_val = str(uuid4())
+    tag3_name = str(uuid4())[0:6]
 
-    instance1.create_tags(Tags=[{"Key": tag1_name, "Value": tag1_val}])
-    instance1.create_tags(Tags=[{"Key": tag2_name, "Value": tag2_val}])
+    instance1.create_tags(
+        Tags=[
+            {"Key": tag1_name, "Value": tag1_val},
+            {"Key": tag2_name, "Value": tag2_val},
+            {"Key": tag3_name, "Value": json.dumps(["entry1", "entry2"])},
+        ]
+    )
     instance2.create_tags(Tags=[{"Key": tag1_name, "Value": tag1_val}])
     instance2.create_tags(Tags=[{"Key": tag2_name, "Value": "wrong value"}])
     instance3.create_tags(Tags=[{"Key": tag2_name, "Value": tag2_val}])
@@ -812,6 +819,16 @@ def test_get_instances_filtering_by_tag():
     res["Reservations"][0]["Instances"][0]["InstanceId"].should.equal(instance1.id)
     res["Reservations"][0]["Instances"][1]["InstanceId"].should.equal(instance3.id)
 
+    # We should be able to use tags containing special characters
+    res = client.describe_instances(
+        Filters=[
+            {"Name": f"tag:{tag3_name}", "Values": [json.dumps(["entry1", "entry2"])]}
+        ]
+    )
+    res["Reservations"].should.have.length_of(1)
+    res["Reservations"][0]["Instances"].should.have.length_of(1)
+    res["Reservations"][0]["Instances"][0]["InstanceId"].should.equal(instance1.id)
+
 
 @mock_ec2
 def test_get_instances_filtering_by_tag_value():
diff --git a/tests/test_glue/test_glue.py b/tests/test_glue/test_glue.py
index 507b7affc..820bbc14e 100644
--- a/tests/test_glue/test_glue.py
+++ b/tests/test_glue/test_glue.py
@@ -109,6 +109,53 @@ def test_get_job_exists():
     job.should.have.key("GlueVersion")
 
 
+@mock_glue
+def test_get_jobs_job_name_exists():
+    client = create_glue_client()
+    test_job_name = create_test_job(client)
+    response = client.get_jobs()
+    response["Jobs"].should.have.length_of(1)
+    response["Jobs"][0].should.have.key("Name").equals(test_job_name)
+
+
+@mock_glue
+def test_get_jobs_with_max_results():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    response = client.get_jobs(MaxResults=2)
+    response["Jobs"].should.have.length_of(2)
+    response.should.have.key("NextToken")
+
+
+@mock_glue
+def test_get_jobs_from_next_token():
+    client = create_glue_client()
+    create_test_jobs(client, 10)
+    first_response = client.get_jobs(MaxResults=3)
+    response = client.get_jobs(NextToken=first_response["NextToken"])
+    response["Jobs"].should.have.length_of(7)
+
+
+@mock_glue
+def test_get_jobs_with_max_results_greater_than_actual_results():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    response = client.get_jobs(MaxResults=10)
+    response["Jobs"].should.have.length_of(4)
+
+
+@mock_glue
+def test_get_jobs_next_token_logic_does_not_create_infinite_loop():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    first_response = client.get_jobs(MaxResults=1)
+    next_token = first_response["NextToken"]
+    while next_token:
+        response = client.get_jobs(NextToken=next_token)
+        next_token = response.get("NextToken")
+    assert not next_token
+
+
 @mock_glue
 def test_list_jobs_with_max_results():
     client = create_glue_client()