Repository: getmoto/moto
Pull request #5974 (2023-02-24 12:19:52)

Introduce glue.get_jobs call
Implements `GetJobs` to the Moto Glue Client in accordance with the [get_jobs()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_jobs) boto specification.

In a nutshell, `get_jobs()` isn't far dissimilar to `list_jobs()`, however it does not accept **any** filter parameters (only pagination parameters). In addition, whilst `list_jobs()` returns a list of the jobs' `Name` attributes, `get_jobs()` returns the Job object similarly (or identically) to the data returned from `create_job()`.

Testing has been introduced, drawing from @dana-orca's contribution for `list_jobs()` to maintain consistency.

As a final point, the `scaffold.py` script updated the `mock_glue` `lazy_load` call with the `boto3_name` parameter locally, so I've updated the original call with this change also.

codecov[bot]: # [Codecov](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#5974](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (4e8b810) into [master](https://codecov.io/gh/getmoto/moto/commit/c875e7aa0e4499af8417c3fa3d068e7b6ea8b8cb?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (c875e7a) will **decrease** coverage by `0.01%`.
> The diff coverage is `88.88%`.

```diff
@@            Coverage Diff             @@
##           master    #5974      +/-   ##
==========================================
- Coverage   96.22%   96.22%   -0.01%     
==========================================
  Files         760      760              
  Lines       74759    74769      +10     
==========================================
+ Hits        71937    71946       +9     
- Misses       2822     2823       +1     
```

| Flag | Coverage Δ | |
|---|---|---|
| servertests | `37.00% <33.33%> (+<0.01%)` | :arrow_up: |
| unittests | `96.16% <88.88%> (-0.01%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [moto/backend\_index.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9iYWNrZW5kX2luZGV4LnB5) | `100.00% <ø> (ø)` | |
| [moto/glue/models.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9nbHVlL21vZGVscy5weQ==) | `97.00% <66.66%> (-0.17%)` | :arrow_down: |
| [moto/\_\_init\_\_.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9fX2luaXRfXy5weQ==) | `98.62% <100.00%> (ø)` | |
| [moto/glue/responses.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9nbHVlL3Jlc3BvbnNlcy5weQ==) | `99.69% <100.00%> (+<0.01%)` | :arrow_up: |
| [moto/elasticache/models.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9lbGFzdGljYWNoZS9tb2RlbHMucHk=) | `100.00% <0.00%> (ø)` | |
| [moto/elasticache/exceptions.py](https://codecov.io/gh/getmoto/moto/pull/5974?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bW90by9lbGFzdGljYWNoZS9leGNlcHRpb25zLnB5) | `100.00% <0.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

danielward: The coverage report has nicely highlighted that there's some redundant code I need to look into. Will take a look now.
danielward: Resolved the issue and hoping to see an improvement in the coverage as a result also.

Also, on a slightly related topic, `list_jobs` supports pagination because the API suggests that's possible, however there actually isn't a paginator and I've raised a botocore issue about it here: https://github.com/boto/botocore/issues/2876

Patch:
diff --git a/IMPLEMENTATION_COVERAGE.md b/IMPLEMENTATION_COVERAGE.md
index c602ce7b1..7bd4458ab 100644
--- a/IMPLEMENTATION_COVERAGE.md
+++ b/IMPLEMENTATION_COVERAGE.md
@@ -2391,7 +2391,7 @@
 
 ## ecs
 <details>
-<summary>80% implemented</summary>
+<summary>78% implemented</summary>
 
 - [X] create_capacity_provider
 - [X] create_cluster
@@ -2402,6 +2402,7 @@
 - [X] delete_capacity_provider
 - [X] delete_cluster
 - [X] delete_service
+- [ ] delete_task_definitions
 - [X] delete_task_set
 - [X] deregister_container_instance
 - [X] deregister_task_definition
@@ -3197,7 +3198,7 @@
 - [ ] get_job_bookmark
 - [X] get_job_run
 - [ ] get_job_runs
-- [ ] get_jobs
+- [X] get_jobs
 - [ ] get_mapping
 - [ ] get_ml_task_run
 - [ ] get_ml_task_runs
@@ -6880,6 +6881,7 @@
 - support-app
 - synthetics
 - timestream-query
+- tnb
 - transfer
 - translate
 - voice-id
diff --git a/docs/docs/services/ecs.rst b/docs/docs/services/ecs.rst
index c3ddd7d99..735415407 100644
--- a/docs/docs/services/ecs.rst
+++ b/docs/docs/services/ecs.rst
@@ -36,6 +36,7 @@ ecs
 - [X] delete_capacity_provider
 - [X] delete_cluster
 - [X] delete_service
+- [ ] delete_task_definitions
 - [X] delete_task_set
   
         The Force-parameter is not yet implemented
diff --git a/docs/docs/services/glue.rst b/docs/docs/services/glue.rst
index 219f6b80c..b3a559923 100644
--- a/docs/docs/services/glue.rst
+++ b/docs/docs/services/glue.rst
@@ -124,7 +124,7 @@ glue
 - [ ] get_job_bookmark
 - [X] get_job_run
 - [ ] get_job_runs
-- [ ] get_jobs
+- [X] get_jobs
 - [ ] get_mapping
 - [ ] get_ml_task_run
 - [ ] get_ml_task_runs
diff --git a/moto/__init__.py b/moto/__init__.py
index 18d8a72d9..b25359aba 100644
--- a/moto/__init__.py
+++ b/moto/__init__.py
@@ -88,7 +88,7 @@ mock_firehose = lazy_load(".firehose", "mock_firehose")
 mock_forecast = lazy_load(".forecast", "mock_forecast")
 mock_greengrass = lazy_load(".greengrass", "mock_greengrass")
 mock_glacier = lazy_load(".glacier", "mock_glacier")
-mock_glue = lazy_load(".glue", "mock_glue")
+mock_glue = lazy_load(".glue", "mock_glue", boto3_name="glue")
 mock_guardduty = lazy_load(".guardduty", "mock_guardduty")
 mock_iam = lazy_load(".iam", "mock_iam")
 mock_iot = lazy_load(".iot", "mock_iot")
diff --git a/moto/backend_index.py b/moto/backend_index.py
index 7e2831c35..7dc48e205 100644
--- a/moto/backend_index.py
+++ b/moto/backend_index.py
@@ -1,4 +1,4 @@
-# autogenerated by /Users/danaronson/GitHub/forks/moto/scripts/update_backend_index.py
+# autogenerated by /Users/dan/Sites/moto/scripts/update_backend_index.py
 import re
 
 backend_url_patterns = [
diff --git a/moto/cognitoidp/models.py b/moto/cognitoidp/models.py
index 3c6e97c62..0c1cda80e 100644
--- a/moto/cognitoidp/models.py
+++ b/moto/cognitoidp/models.py
@@ -567,6 +567,9 @@ class CognitoIdpUserPool(BaseModel):
 
     def create_id_token(self, client_id: str, username: str) -> Tuple[str, int]:
         extra_data = self.get_user_extra_data_by_client_id(client_id, username)
+        user = self._get_user(username)
+        if len(user.groups) > 0:
+            extra_data["cognito:groups"] = [group.group_name for group in user.groups]
         id_token, expires_in = self.create_jwt(
             client_id, username, "id", extra_data=extra_data
         )
diff --git a/moto/glue/models.py b/moto/glue/models.py
index 70e794f49..3194f4cb5 100644
--- a/moto/glue/models.py
+++ b/moto/glue/models.py
@@ -69,6 +69,12 @@ class GlueBackend(BaseBackend):
             "limit_default": 100,
             "unique_attribute": "name",
         },
+        "get_jobs": {
+            "input_token": "next_token",
+            "limit_key": "max_results",
+            "limit_default": 100,
+            "unique_attribute": "name",
+        },
     }
 
     def __init__(self, region_name, account_id):
@@ -329,6 +335,10 @@ class GlueBackend(BaseBackend):
         except KeyError:
             raise JobNotFoundException(name)
 
+    @paginate(pagination_model=PAGINATION_MODEL)
+    def get_jobs(self):
+        return [job for _, job in self.jobs.items()]
+
     def start_job_run(self, name):
         job = self.get_job(name)
         return job.start_job_run()
diff --git a/moto/glue/responses.py b/moto/glue/responses.py
index fd797d6c2..64ddd107c 100644
--- a/moto/glue/responses.py
+++ b/moto/glue/responses.py
@@ -343,6 +343,19 @@ class GlueResponse(BaseResponse):
         job = self.glue_backend.get_job(name)
         return json.dumps({"Job": job.as_dict()})
 
+    def get_jobs(self):
+        next_token = self._get_param("NextToken")
+        max_results = self._get_int_param("MaxResults")
+        jobs, next_token = self.glue_backend.get_jobs(
+            next_token=next_token, max_results=max_results
+        )
+        return json.dumps(
+            dict(
+                Jobs=[job.as_dict() for job in jobs],
+                NextToken=next_token,
+            )
+        )
+
     def start_job_run(self):
         name = self.parameters.get("JobName")
         job_run_id = self.glue_backend.start_job_run(name)


Test patch:
diff --git a/tests/test_cognitoidp/test_cognitoidp.py b/tests/test_cognitoidp/test_cognitoidp.py
index c8425128c..fb91777bc 100644
--- a/tests/test_cognitoidp/test_cognitoidp.py
+++ b/tests/test_cognitoidp/test_cognitoidp.py
@@ -1472,6 +1472,59 @@ def test_group_in_access_token():
     claims["cognito:groups"].should.equal([group_name])
 
 
+@mock_cognitoidp
+def test_group_in_id_token():
+    conn = boto3.client("cognito-idp", "us-west-2")
+
+    username = str(uuid.uuid4())
+    temporary_password = "P2$Sword"
+    user_pool_id = conn.create_user_pool(PoolName=str(uuid.uuid4()))["UserPool"]["Id"]
+    user_attribute_name = str(uuid.uuid4())
+    user_attribute_value = str(uuid.uuid4())
+    group_name = str(uuid.uuid4())
+    client_id = conn.create_user_pool_client(
+        UserPoolId=user_pool_id,
+        ClientName=str(uuid.uuid4()),
+        ReadAttributes=[user_attribute_name],
+    )["UserPoolClient"]["ClientId"]
+
+    conn.create_group(GroupName=group_name, UserPoolId=user_pool_id)
+
+    conn.admin_create_user(
+        UserPoolId=user_pool_id,
+        Username=username,
+        TemporaryPassword=temporary_password,
+        UserAttributes=[{"Name": user_attribute_name, "Value": user_attribute_value}],
+    )
+
+    conn.admin_add_user_to_group(
+        UserPoolId=user_pool_id, Username=username, GroupName=group_name
+    )
+
+    result = conn.admin_initiate_auth(
+        UserPoolId=user_pool_id,
+        ClientId=client_id,
+        AuthFlow="ADMIN_NO_SRP_AUTH",
+        AuthParameters={"USERNAME": username, "PASSWORD": temporary_password},
+    )
+
+    # A newly created user is forced to set a new password
+    result["ChallengeName"].should.equal("NEW_PASSWORD_REQUIRED")
+    result["Session"].should_not.equal(None)
+
+    # This sets a new password and logs the user in (creates tokens)
+    new_password = "P2$Sword"
+    result = conn.respond_to_auth_challenge(
+        Session=result["Session"],
+        ClientId=client_id,
+        ChallengeName="NEW_PASSWORD_REQUIRED",
+        ChallengeResponses={"USERNAME": username, "NEW_PASSWORD": new_password},
+    )
+
+    claims = jwt.get_unverified_claims(result["AuthenticationResult"]["IdToken"])
+    claims["cognito:groups"].should.equal([group_name])
+
+
 @mock_cognitoidp
 def test_create_group_with_duplicate_name_raises_error():
     conn = boto3.client("cognito-idp", "us-west-2")
diff --git a/tests/test_glue/test_glue.py b/tests/test_glue/test_glue.py
index 507b7affc..820bbc14e 100644
--- a/tests/test_glue/test_glue.py
+++ b/tests/test_glue/test_glue.py
@@ -109,6 +109,53 @@ def test_get_job_exists():
     job.should.have.key("GlueVersion")
 
 
+@mock_glue
+def test_get_jobs_job_name_exists():
+    client = create_glue_client()
+    test_job_name = create_test_job(client)
+    response = client.get_jobs()
+    response["Jobs"].should.have.length_of(1)
+    response["Jobs"][0].should.have.key("Name").equals(test_job_name)
+
+
+@mock_glue
+def test_get_jobs_with_max_results():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    response = client.get_jobs(MaxResults=2)
+    response["Jobs"].should.have.length_of(2)
+    response.should.have.key("NextToken")
+
+
+@mock_glue
+def test_get_jobs_from_next_token():
+    client = create_glue_client()
+    create_test_jobs(client, 10)
+    first_response = client.get_jobs(MaxResults=3)
+    response = client.get_jobs(NextToken=first_response["NextToken"])
+    response["Jobs"].should.have.length_of(7)
+
+
+@mock_glue
+def test_get_jobs_with_max_results_greater_than_actual_results():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    response = client.get_jobs(MaxResults=10)
+    response["Jobs"].should.have.length_of(4)
+
+
+@mock_glue
+def test_get_jobs_next_token_logic_does_not_create_infinite_loop():
+    client = create_glue_client()
+    create_test_jobs(client, 4)
+    first_response = client.get_jobs(MaxResults=1)
+    next_token = first_response["NextToken"]
+    while next_token:
+        response = client.get_jobs(NextToken=next_token)
+        next_token = response.get("NextToken")
+    assert not next_token
+
+
 @mock_glue
 def test_list_jobs_with_max_results():
     client = create_glue_client()