internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: localstack/localstack

Issue:
Lambda containers not cleaned up with LAMBDA_EXECUTOR=docker and LAMBDA_REMOTE_DOCKER='true'
# Steps to reproduce

* Run `localstack` with `LAMBDA_EXECUTOR=docker` and `LAMBDA_REMOTE_DOCKER=true`.
* Create and execute multiple lambda functions.
* Run `docker ps -a` and observe multiple `lambci` containers

This does not happen with `LAMBDA_EXECUTOR=docker-reuse` or with `LAMBDA_REMOTE_DOCKER=true`


Discussion:

Resolving patch:
diff --git a/localstack/services/awslambda/lambda_executors.py b/localstack/services/awslambda/lambda_executors.py
--- a/localstack/services/awslambda/lambda_executors.py
+++ b/localstack/services/awslambda/lambda_executors.py
@@ -578,6 +578,7 @@ def prepare_execution(self, func_arn, env_vars, runtime, command, handler, lambd
                 ' %s'
                 ' %s'
                 ' %s'  # network
+                ' --rm'
                 ' "lambci/lambda:%s" %s'
                 ')";'
                 '%s cp "%s/." "$CONTAINER_ID:/var/task"; '


Test patch:
Repository: localstack/localstack

Issue:
Presigned S3 url doesnt notify sqs 
<!-- Love localstack? Please consider supporting our collective:
👉  https://opencollective.com/localstack/donate -->
I have configured s3 bucket with event configuration to sqs for every object creation. When I try out aws cli command I get the notification correctly. 

When I try using presigned url with curl/postman command, I dont get the sqs notification. **Is this a known issue and are there any work arounds?**


Discussion:
@whummer
Sounds similar to this:  https://github.com/localstack/localstack/issues/945

Resolving patch:
diff --git a/localstack/services/s3/s3_listener.py b/localstack/services/s3/s3_listener.py
--- a/localstack/services/s3/s3_listener.py
+++ b/localstack/services/s3/s3_listener.py
@@ -600,10 +600,10 @@ def return_response(self, method, path, data, headers, response):
 
         should_send_notifications = all([
             method in ('PUT', 'POST', 'DELETE'),
-            '/' in path[1:] or bucket_name_in_host,
+            '/' in path[1:] or bucket_name_in_host or key,
             # check if this is an actual put object request, because it could also be
             # a put bucket request with a path like this: /bucket_name/
-            bucket_name_in_host or (len(path[1:].split('/')) > 1 and len(path[1:].split('/')[1]) > 0),
+            bucket_name_in_host or key or (len(path[1:].split('/')) > 1 and len(path[1:].split('/')[1]) > 0),
             self.is_query_allowable(method, parsed.query)
         ])
 


Test patch:
diff --git a/tests/integration/test_s3.py b/tests/integration/test_s3.py
--- a/tests/integration/test_s3.py
+++ b/tests/integration/test_s3.py
@@ -292,6 +292,27 @@ def test_s3_put_object_on_presigned_url(self):
         # clean up
         self._delete_bucket(bucket_name, [object_key])
 
+    def test_s3_post_object_on_presigned_post(self):
+        bucket_name = 'test-bucket-%s' % short_uid()
+        self.s3_client.create_bucket(Bucket=bucket_name)
+        body = 'something body'
+        # get presigned URL
+        object_key = 'test-presigned-post-key'
+        presigned_request = self.s3_client.generate_presigned_post(
+            Bucket=bucket_name,
+            Key=object_key
+        )
+        # put object
+        files = {'file': body}
+        response = requests.post(presigned_request['url'], data=presigned_request['fields'], files=files, verify=False)
+        self.assertEqual(response.status_code, 200)
+        # get object and compare results
+        downloaded_object = self.s3_client.get_object(Bucket=bucket_name, Key=object_key)
+        download_object = downloaded_object['Body'].read()
+        self.assertEqual(to_str(body), to_str(download_object))
+        # clean up
+        self._delete_bucket(bucket_name, [object_key])
+
     def test_s3_delete_response_content_length_zero(self):
         bucket_name = 'test-bucket-%s' % short_uid()
         self.s3_client.create_bucket(Bucket=bucket_name)
Repository: localstack/localstack

Issue:
lambda executor should include stdout in exceptoin message

https://github.com/localstack/localstack/blob/master/localstack/services/awslambda/lambda_executors.py#L67

```python
   def run_lambda_executor(self, cmd, env_vars={}, async=False):
        process = run(cmd, async=True, stderr=subprocess.PIPE, outfile=subprocess.PIPE, env_vars=env_vars)
        if async:
            result = '{"async": "%s"}' % async
            log_output = 'Lambda executed asynchronously'
        else:
            return_code = process.wait()
            result = to_str(process.stdout.read())
            log_output = to_str(process.stderr.read())

            if return_code != 0:
                raise Exception('Lambda process returned error status code: %s. Output:\n%s' %
                    (return_code, log_output))
        return result, log_output
```

When the return code is not zero, the executor function will raise an Exception with return code and stderr information, but the stdout is missing.

When there are errors in lambda, not only the stderr, but also the stdout is necessary for analyzing, so the stdout should also be included in the Exception message.


Discussion:

Resolving patch:
diff --git a/localstack/services/awslambda/lambda_executors.py b/localstack/services/awslambda/lambda_executors.py
--- a/localstack/services/awslambda/lambda_executors.py
+++ b/localstack/services/awslambda/lambda_executors.py
@@ -153,8 +153,8 @@ def run_lambda_executor(self, cmd, event=None, env_vars={}):
             log_output += '\n%s' % additional_logs
 
         if return_code != 0:
-            raise Exception('Lambda process returned error status code: %s. Output:\n%s' %
-                (return_code, log_output))
+            raise Exception('Lambda process returned error status code: %s. Result: %s. Output:\n%s' %
+                (return_code, result, log_output))
 
         return result, log_output
 
diff --git a/localstack/services/cloudformation/cloudformation_listener.py b/localstack/services/cloudformation/cloudformation_listener.py
--- a/localstack/services/cloudformation/cloudformation_listener.py
+++ b/localstack/services/cloudformation/cloudformation_listener.py
@@ -100,7 +100,7 @@ def get_template_body(req_data):
         return body
     url = req_data.get('TemplateURL')
     if url:
-        response = safe_requests.get(url)
+        response = safe_requests.get(url, verify=False)
         if response.status_code >= 400:
             # check if this is an S3 URL, then get the file directly from there
             if '://localhost' in url or re.match(r'.*s3(\-website)?\.([^\.]+\.)?amazonaws.com.*', url):


Test patch:
Repository: localstack/localstack

Issue:
Regression: S3 502 error from headBucket request 
When executing:

```
s3AsyncClient.headBucket(HeadBucketRequest.builder()
                    .bucket("local-stack-test-bucket")
                    .build()).join();
```

We get this exception thrown (Java): java.util.concurrent.CompletionException: software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 502, Request ID: null)

I narrowed this down to db4cddea4e4844c1241cca5ac22ecbde08e52064 and found the problem. PR incoming shortly.


Discussion:

Resolving patch:
diff --git a/localstack/services/generic_proxy.py b/localstack/services/generic_proxy.py
--- a/localstack/services/generic_proxy.py
+++ b/localstack/services/generic_proxy.py
@@ -210,7 +210,7 @@ def forward(self, method):
             self.close_connection = 1
 
         def is_full_url(url):
-            return re.match(r'[a-zA-Z+]://.+', url)
+            return re.match(r'[a-zA-Z]+://.+', url)
 
         path = self.path
         if is_full_url(path):


Test patch:
Repository: localstack/localstack

Issue:
Standardize usage with terraform
As a user of localstack I would like a documented standardized way to run terraform scripts against a running localstack to create AWS infrastructure so that I don't have to duplicate work that involves programatically creating the resources when I use terraform to create resources in my cloud environment.


Discussion:
Thanks for reporting @danielptm . Very much agree that the documentation around Terraform usage could be improved. We should try to keep the details in the main README short and crisp, though.

Basically, it boils down to configuring the endpoint URLs for each service in the `provider` section. Here is a simple demo project that highlights how this can be done: https://github.com/localstack/localstack-pro-samples/blob/master/terraform-resources/test.tf

Do you think you could create a small pull request and add a few instructions/pointers to the README? That would be really appreciated, thanks!
Sure. Once I succeed at doing it I will make the PR. Thanks!
FYI, we also have a page in the TF docs about this you may be able to link to: https://www.terraform.io/docs/providers/aws/guides/custom-service-endpoints.html#localstack

Resolving patch:
diff --git a/localstack/services/awslambda/lambda_api.py b/localstack/services/awslambda/lambda_api.py
--- a/localstack/services/awslambda/lambda_api.py
+++ b/localstack/services/awslambda/lambda_api.py
@@ -28,6 +28,7 @@
     LAMBDA_RUNTIME_NODEJS610,
     LAMBDA_RUNTIME_NODEJS810,
     LAMBDA_RUNTIME_JAVA8,
+    LAMBDA_RUNTIME_JAVA11,
     LAMBDA_RUNTIME_DOTNETCORE2,
     LAMBDA_RUNTIME_DOTNETCORE21,
     LAMBDA_RUNTIME_GOLANG,
@@ -542,7 +543,7 @@ def set_archive_code(code, lambda_name, zip_file_content=None):
 
 def is_java_lambda(lambda_details):
     runtime = getattr(lambda_details, 'runtime', lambda_details)
-    return runtime == LAMBDA_RUNTIME_JAVA8
+    return runtime in [LAMBDA_RUNTIME_JAVA8, LAMBDA_RUNTIME_JAVA11]
 
 
 def set_function_code(code, lambda_name, lambda_cwd=None):
diff --git a/localstack/services/awslambda/lambda_executors.py b/localstack/services/awslambda/lambda_executors.py
--- a/localstack/services/awslambda/lambda_executors.py
+++ b/localstack/services/awslambda/lambda_executors.py
@@ -27,11 +27,16 @@
 
 LAMBDA_RUNTIME_PYTHON27 = 'python2.7'
 LAMBDA_RUNTIME_PYTHON36 = 'python3.6'
+LAMBDA_RUNTIME_PYTHON37 = 'python3.7'
+LAMBDA_RUNTIME_PYTHON38 = 'python3.8'
 LAMBDA_RUNTIME_NODEJS = 'nodejs'
+LAMBDA_RUNTIME_NODEJS43 = 'nodejs4.3'
 LAMBDA_RUNTIME_NODEJS610 = 'nodejs6.10'
 LAMBDA_RUNTIME_NODEJS810 = 'nodejs8.10'
 LAMBDA_RUNTIME_NODEJS10X = 'nodejs10.x'
+LAMBDA_RUNTIME_NODEJS12X = 'nodejs12.x'
 LAMBDA_RUNTIME_JAVA8 = 'java8'
+LAMBDA_RUNTIME_JAVA11 = 'java11'
 LAMBDA_RUNTIME_DOTNETCORE2 = 'dotnetcore2.0'
 LAMBDA_RUNTIME_DOTNETCORE21 = 'dotnetcore2.1'
 LAMBDA_RUNTIME_GOLANG = 'go1.x'


Test patch:
Repository: localstack/localstack

Issue:
Support newer Lambda regions like eu-west-2
We're using an old version of `boto` (through `moto`) to detect the available regions for Lambda (and other APIs). We should switch from `boto` to the newer `boto3` library, which includes the up-to-date endpoints.

```
code/localstack/localstack/services/cloudformation/cloudformation_starter.py", line 502, in Lambda_create_from_cloudformation_json
    return Lambda_create_from_cloudformation_json_orig(resource_name, cloudformation_json, region_name)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/moto/awslambda/models.py", line 519, in create_from_cloudformation_json
    backend = lambda_backends[region_name]
KeyError: 'eu-west-2'
```


Discussion:

Resolving patch:
diff --git a/localstack/services/cloudformation/cloudformation_starter.py b/localstack/services/cloudformation/cloudformation_starter.py
--- a/localstack/services/cloudformation/cloudformation_starter.py
+++ b/localstack/services/cloudformation/cloudformation_starter.py
@@ -3,6 +3,7 @@
 import logging
 import traceback
 import six
+import boto3.session
 from moto.s3 import models as s3_models
 from moto.iam import models as iam_models
 from moto.sqs import models as sqs_models
@@ -654,6 +655,12 @@ def cf_describe_stack_events(self):
 
     responses.CloudFormationResponse.describe_stack_events = cf_describe_stack_events
 
+    # fix Lambda regions in moto - see https://github.com/localstack/localstack/issues/1961
+
+    for region in boto3.session.Session().get_available_regions('lambda'):
+        if region not in lambda_models.lambda_backends:
+            lambda_models.lambda_backends[region] = lambda_models.LambdaBackend(region)
+
 
 def inject_stats_endpoint():
     """ Inject a simple /_stats endpoint into the moto server backend Web app. """


Test patch:
Repository: localstack/localstack

Issue:
S3 Uploads with underscores in metadata hang
It appears there was a regression introduced in #1965 -- when I try to upload an object with metadata, the server appears to hang.  The test for it does not actually set a `Body`; when a `Body` is added, the test hangs as well.


Discussion:

Resolving patch:
diff --git a/localstack/services/s3/s3_listener.py b/localstack/services/s3/s3_listener.py
--- a/localstack/services/s3/s3_listener.py
+++ b/localstack/services/s3/s3_listener.py
@@ -828,7 +828,7 @@ def forward_request(self, method, path, data, headers):
                 return set_object_lock(bucket, data)
 
         if modified_data is not None or headers_changed:
-            return Request(data=modified_data, headers=headers, method=method)
+            return Request(data=modified_data or data, headers=headers, method=method)
         return True
 
     def get_201_reponse(self, key, bucket_name):


Test patch:
diff --git a/tests/integration/test_s3.py b/tests/integration/test_s3.py
--- a/tests/integration/test_s3.py
+++ b/tests/integration/test_s3.py
@@ -252,7 +252,7 @@ def test_s3_put_metadata_underscores(self):
         # put object
         object_key = 'key-with-metadata'
         metadata = {'test_meta_1': 'foo'}
-        self.s3_client.put_object(Bucket=bucket_name, Key=object_key, Metadata=metadata)
+        self.s3_client.put_object(Bucket=bucket_name, Key=object_key, Metadata=metadata, Body='foo')
         metadata_saved = self.s3_client.head_object(Bucket=bucket_name, Key=object_key)['Metadata']
         self.assertEqual(metadata, metadata_saved)
Repository: localstack/localstack

Issue:
Is it possible to support dynamodbstreams ListStreams?
See also  https://github.com/localstack/localstack/blob/master/localstack/services/dynamodbstreams/dynamodbstreams_api.py#L70 


Discussion:

Resolving patch:
diff --git a/localstack/constants.py b/localstack/constants.py
--- a/localstack/constants.py
+++ b/localstack/constants.py
@@ -2,7 +2,7 @@
 import localstack_client.config
 
 # LocalStack version
-VERSION = '0.10.7'
+VERSION = '0.10.8'
 
 # constant to represent the "local" region, i.e., local machine
 REGION_LOCAL = 'local'
@@ -31,7 +31,7 @@
 LOCALHOST = 'localhost'
 
 # version of the Maven dependency with Java utility code
-LOCALSTACK_MAVEN_VERSION = '0.2.0'
+LOCALSTACK_MAVEN_VERSION = '0.2.1'
 
 # map of default service APIs and ports to be spun up (fetch map from localstack_client)
 DEFAULT_SERVICE_PORTS = localstack_client.config.get_service_ports()
diff --git a/localstack/services/dynamodbstreams/dynamodbstreams_api.py b/localstack/services/dynamodbstreams/dynamodbstreams_api.py
--- a/localstack/services/dynamodbstreams/dynamodbstreams_api.py
+++ b/localstack/services/dynamodbstreams/dynamodbstreams_api.py
@@ -66,8 +66,7 @@ def post_request():
     kinesis = aws_stack.connect_to_service('kinesis')
     if action == '%s.ListStreams' % ACTION_HEADER_PREFIX:
         result = {
-            'Streams': list(DDB_STREAMS.values()),
-            'LastEvaluatedStreamArn': 'TODO'
+            'Streams': list(DDB_STREAMS.values())
         }
     elif action == '%s.DescribeStream' % ACTION_HEADER_PREFIX:
         for stream in DDB_STREAMS.values():


Test patch:
Repository: localstack/localstack

Issue:
SNS logs location
Hi,

I want to see, what's coming from outside to my SNS service in localstack like payload.
Where can I see it? I have set `DEBUG=1` but I still not see anything.


Discussion:

Resolving patch:
diff --git a/localstack/services/sns/sns_listener.py b/localstack/services/sns/sns_listener.py
--- a/localstack/services/sns/sns_listener.py
+++ b/localstack/services/sns/sns_listener.py
@@ -218,6 +218,8 @@ def publish_message(topic_arn, req_data, subscription_arn=None):
     message = req_data['Message'][0]
     sqs_client = aws_stack.connect_to_service('sqs')
 
+    LOGGER.debug('Publishing message to TopicArn: %s | Message:  %s' % (topic_arn, message))
+
     for subscriber in SNS_SUBSCRIPTIONS.get(topic_arn, []):
         if subscription_arn not in [None, subscriber['SubscriptionArn']]:
             continue


Test patch:
Repository: localstack/localstack

Issue:
USE_SSL=1 breaks cloudwatch logging
[x] bug report
[ ] feature request

# Detailed description
cloudwatch logging doesn't work if USE_SSL is enabled. The problem seems to be in 
https://github.com/localstack/localstack/blob/f9d5c1a77088bf65e10d05a298ef7ab61fc64314/localstack/utils/cloudwatch/cloudwatch_util.py#L57
If protocol is https ,the service is enabled resolver returns False. Logging works normally when USE_SSL=0
Happens at least with 0.11.1, but seems to be the same problem with older versions also.

## Expected behavior
Cloudwatch logging works
...

## Actual behavior
Nothing is logged to cloudwatch logs

# Steps to reproduce
set USE_SSL=1

## Command used to start LocalStack
docker-compose up



Discussion:

Resolving patch:
diff --git a/localstack/utils/common.py b/localstack/utils/common.py
--- a/localstack/utils/common.py
+++ b/localstack/utils/common.py
@@ -396,7 +396,7 @@ def is_port_open(port_or_url, http_path=None, expect_success=True, protocols=['t
         return True
     url = '%s://%s:%s%s' % (protocol, host, port, http_path)
     try:
-        response = safe_requests.get(url)
+        response = safe_requests.get(url, verify=False)
         return not expect_success or response.status_code < 400
     except Exception:
         return False


Test patch:
Repository: localstack/localstack

Issue:
Usage of `docker inspect ..` is fragile, depends on how and what built the docker image
[x] bug report
[ ] feature request

# Detailed description
`lambda_executor.py` current retrieves the container entrypoint from the docker image via `docker inspect --format="{{ .ContainerConfig.Entrypoint }}" ..`. This is fragile and may be missing depending on how the image in question is built. There is a `config` block _and_ a `containerconfig` block that are mostly the same, but sometimes different depending what built and what version of that thing built the image, for example we are seeing the entrypoint missing on images built with Docker for Mac 2.5.0.1, but not on earlier versions, others using `podman` are noticing the fragility in other projects:

https://github.com/containers/podman/issues/2017

## Expected behavior
entrypoint value is picked up from a validly built container

## Actual behavior
entrypoint is sometimes an empty string, which then for a `provided` lambda executor ends up with a script error trying to execute the handler name.

The simple fix is to change `--format="{{ .ContainerConfig.Entrypoint }}"` to `--format="{{ .Config.Entrypoint }}"` which seems like the more canonical way of getting that value.



┆Issue is synchronized with this [Jira Task](https://localstack.atlassian.net/browse/LOC-54) by [Unito](https://www.unito.io/learn-more)



Discussion:
Thanks for digging into this issue @nhoughto !

> The simple fix is to change --format="{{ .ContainerConfig.Entrypoint }}" to --format="{{ .Config.Entrypoint }}" which seems like the more canonical way of getting that value.

Sounds good to me - would you be able to create a small pull request with this change? Thanks for your help.

Resolving patch:
diff --git a/localstack/services/awslambda/lambda_executors.py b/localstack/services/awslambda/lambda_executors.py
--- a/localstack/services/awslambda/lambda_executors.py
+++ b/localstack/services/awslambda/lambda_executors.py
@@ -469,7 +469,7 @@ def prime_docker_container(self, runtime, func_arn, env_vars, lambda_cwd):
             LOG.debug('Getting the entrypoint for image: %s' % (docker_image))
             cmd = (
                 '%s image inspect'
-                ' --format="{{ .ContainerConfig.Entrypoint }}"'
+                ' --format="{{ .Config.Entrypoint }}"'
                 ' %s'
             ) % (docker_cmd, docker_image)
 


Test patch:
Repository: localstack/localstack

Issue:
EventBridge PutEvents not working when Detail is missing
# Type of request: This is a ...

[X ] bug report
[ ] feature request

# Detailed description
Calling PutEvents operation returns 500 if no `Detail` is specified. This case is similar to #3043

## Expected behavior
According to the doc https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_PutEventsRequestEntry.html `Details` is optional. 

```
 Detail
    A valid JSON string. There is no other schema imposed. The JSON string may contain fields and nested subobjects.
    Type: String
    Required: No
```

So this should works and set `Detail` as `{}`

```
eb_client = boto3.client('events', endpoint_url='http://localhost:4587')
eb_client.put_events(
    Entries=[
        {
            'DetailType': 'Test'
        }
    ]
)
```

## Actual behavior
500 is returned

# Steps to reproduce

1. Start LocalStack with SERVICES=events
2. Run the client code

## Command used to start LocalStack

```
docker run --rm --name localstack -p 4587:4587 -e SERVICES=events -e DEBUG=true localstack/localstack
```

## Client code (AWS SDK code snippet, or sequence of "awslocal" commands)

CLI:

```
aws events put-events --endpoint-url http://localhost:4587 --entries '[{"DetailType": "Test"}]'
```

Or via python:

```
eb_client = boto3.client('events', endpoint_url='http://localhost:4587')
eb_client.put_events(
    Entries=[
        {
            'DetailType': 'Test',
            'Detail': '{}'
        }
    ]
)
```



Discussion:

Resolving patch:
diff --git a/localstack/services/events/events_starter.py b/localstack/services/events/events_starter.py
--- a/localstack/services/events/events_starter.py
+++ b/localstack/services/events/events_starter.py
@@ -155,7 +155,7 @@ def events_handler_put_events(self):
                 'time': datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ'),
                 'region': self.region,
                 'resources': event.get('Resources', []),
-                'detail': json.loads(event.get('Detail')),
+                'detail': json.loads(event.get('Detail', '{}')),
             }
 
             targets = []


Test patch:
diff --git a/tests/integration/test_events.py b/tests/integration/test_events.py
--- a/tests/integration/test_events.py
+++ b/tests/integration/test_events.py
@@ -701,6 +701,18 @@ def test_put_event_without_source(self):
         )
         self.assertIn('Entries', response)
 
+    def test_put_event_without_detail(self):
+        self.events_client = aws_stack.connect_to_service('events', Region='eu-west-1')
+
+        response = self.events_client.put_events(
+            Entries=[
+                {
+                    'DetailType': 'Test',
+                }
+            ]
+        )
+        self.assertIn('Entries', response)
+
     def test_put_event_with_content_base_rule_in_pattern(self):
         queue_name = 'queue-{}'.format(short_uid())
         rule_name = 'rule-{}'.format(short_uid())