Repository: localstack/localstack

Issue:
Create Lambda function using SAM template
Is it possible to create a lambda function using SAM template? For example:

```
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
     TestLambda:
            Type: 'AWS::Serverless::Function'
             Properties:
                    CodeUri: ./testLambda.zip
                     Handler: testLambda.handler
                      .......
```

When I try to package to s3 in localstack using the following command:

aws --endpoint-url=http://localhost:4581 cloudformation package --template-file test.yaml --output-template-file output.yaml --s3-bucket test-bucket

I get an error as follows:
Unable to upload artifact ./testLambda.zip referenced by CodeUri parameter of TestLambda resource. An error occurred (AccessDenied) when calling the PutObject operation: Access Denied.

Any help will be appreciated.


Discussion:
+1
Having the same issue
+1
Following Example at:
https://docs.aws.amazon.com/lambda/latest/dg/serverless-deploy-wt.html#serverless-deploy

> An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.

+1
Having the same issue.
I encountered the same error as @emeraldBazaar .

> Unable to upload artifact ./testLambda.zip referenced by CodeUri parameter of TestLambda resource. An error occurred (AccessDenied) when calling the PutObject operation: Access Denied.

As I debugged the issue, it became apparent that this is a problem with `aws-cli`. You'll notice that the command you ran doesn't include an endpoint URL for S3. As a result, the CLI is attempting to upload the zip file to `test-bucket` in the real AWS environment. Since you likely don't have permissions to upload to `test-bucket`, you are getting that permissions error.

Unfortunately, `aws-cli` doesn't allow you to pass in a custom S3 endpoint URL. I'm working on a PR that will enable this option on the `aws-cli`, but it might be a while before that gets merged.
I think I got around this issue by packaging my code and uploading it myself, but then I started getting this error when running deploy.

https://github.com/localstack/localstack/issues/760

The deploy command eventually returns this:
`HTTPConnectionPool(host='localhost', port=4581): Read timed out.`
Also, even if an `--s3-endpoint-url` argument is added, it doesn't appear that SAM resource types are currently an option with localstack.

https://github.com/localstack/localstack/blob/master/localstack/utils/cloudformation/template_deployer.py#L20

Resolving patch:
diff --git a/localstack/services/cloudformation/cloudformation_listener.py b/localstack/services/cloudformation/cloudformation_listener.py
--- a/localstack/services/cloudformation/cloudformation_listener.py
+++ b/localstack/services/cloudformation/cloudformation_listener.py
@@ -1,8 +1,12 @@
 import re
+import os
+import json
 import uuid
+import boto3
 import logging
-from requests.models import Response
+from requests.models import Request, Response
 from six.moves.urllib import parse as urlparse
+from samtranslator.translator.transform import transform as transform_sam
 from localstack.utils.aws import aws_stack
 from localstack.utils.common import to_str, obj_to_xml, safe_requests
 from localstack.utils.analytics import event_publisher
@@ -62,6 +66,34 @@ def validate_template(req_data):
         return response
 
 
+def transform_template(req_data):
+    template_body = get_template_body(req_data)
+    parsed = template_deployer.parse_template(template_body)
+
+    policy_map = {
+        # SAM Transformer expects this map to be non-empty, but apparently the content doesn't matter (?)
+        'dummy': 'entry'
+        # 'AWSLambdaBasicExecutionRole': 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole',
+    }
+
+    class MockPolicyLoader(object):
+        def load(self):
+            return policy_map
+
+    if parsed.get('Transform') == 'AWS::Serverless-2016-10-31':
+        # Note: we need to fix boto3 region, otherwise AWS SAM transformer fails
+        region_before = os.environ.get('AWS_DEFAULT_REGION')
+        if boto3.session.Session().region_name is None:
+            os.environ['AWS_DEFAULT_REGION'] = aws_stack.get_region()
+        try:
+            transformed = transform_sam(parsed, {}, MockPolicyLoader())
+            return transformed
+        finally:
+            os.environ.pop('AWS_DEFAULT_REGION', None)
+            if region_before is not None:
+                os.environ['AWS_DEFAULT_REGION'] = region_before
+
+
 def get_template_body(req_data):
     body = req_data.get('TemplateBody')
     if body:
@@ -115,6 +147,13 @@ def forward_request(self, method, path, data, headers):
         if req_data:
             if action == 'ValidateTemplate':
                 return validate_template(req_data)
+            if action == 'CreateStack':
+                modified_request = transform_template(req_data)
+                if modified_request:
+                    req_data.pop('TemplateURL', None)
+                    req_data['TemplateBody'] = json.dumps(modified_request)
+                    data = urlparse.urlencode(req_data, doseq=True)
+                    return Request(data=data, headers=headers, method=method)
 
         return True
 
diff --git a/localstack/utils/cloudformation/template_deployer.py b/localstack/utils/cloudformation/template_deployer.py
--- a/localstack/utils/cloudformation/template_deployer.py
+++ b/localstack/utils/cloudformation/template_deployer.py
@@ -1,6 +1,8 @@
 import re
+import os
 import json
 import yaml
+import base64
 import logging
 import traceback
 import moto.cloudformation.utils
@@ -8,6 +10,8 @@
 from six import string_types
 from localstack.utils import common
 from localstack.utils.aws import aws_stack
+from localstack.utils.testutil import create_zip_file
+from localstack.services.awslambda.lambda_api import get_handler_file_from_name
 
 ACTION_CREATE = 'create'
 PLACEHOLDER_RESOURCE_NAME = '__resource_name__'
@@ -17,6 +21,13 @@
 # list of resource types that can be updated
 UPDATEABLE_RESOURCES = ['Lambda::Function', 'ApiGateway::Method']
 
+# create safe yaml loader that parses date strings as string, not date objects
+NoDatesSafeLoader = yaml.SafeLoader
+NoDatesSafeLoader.yaml_implicit_resolvers = {
+    k: [r for r in v if r[0] != 'tag:yaml.org,2002:timestamp'] for
+    k, v in NoDatesSafeLoader.yaml_implicit_resolvers.items()
+}
+
 
 def str_or_none(o):
     return o if o is None else json.dumps(o) if isinstance(o, (dict, list)) else str(o)
@@ -38,6 +49,21 @@ def lambda_get_params():
     return lambda params, **kwargs: params
 
 
+def get_lambda_code_param(params, **kwargs):
+    code = params.get('Code', {})
+    zip_file = code.get('ZipFile')
+    if zip_file and not common.is_base64(zip_file):
+        tmp_dir = common.new_tmp_dir()
+        handler_file = get_handler_file_from_name(params['Handler'], runtime=params['Runtime'])
+        tmp_file = os.path.join(tmp_dir, handler_file)
+        common.save_file(tmp_file, zip_file)
+        zip_file = create_zip_file(tmp_file, get_content=True)
+        code['ZipFile'] = common.to_str(base64.b64encode(zip_file))
+        code['ZipFile'] = zip_file
+        common.rm_rf(tmp_dir)
+    return code
+
+
 # maps resource types to functions and parameters for creation
 RESOURCE_TO_FUNCTION = {
     'S3::Bucket': {
@@ -83,7 +109,7 @@ def lambda_get_params():
                 'Runtime': 'Runtime',
                 'Role': 'Role',
                 'Handler': 'Handler',
-                'Code': 'Code',
+                'Code': get_lambda_code_param,
                 'Description': 'Description',
                 'Environment': 'Environment',
                 'Timeout': 'Timeout',
@@ -290,11 +316,11 @@ def parse_template(template):
     try:
         return json.loads(template)
     except Exception:
-        yaml.add_multi_constructor('', moto.cloudformation.utils.yaml_tag_constructor)
+        yaml.add_multi_constructor('', moto.cloudformation.utils.yaml_tag_constructor, Loader=NoDatesSafeLoader)
         try:
             return yaml.safe_load(template)
         except Exception:
-            return yaml.load(template, Loader=yaml.Loader)
+            return yaml.load(template, Loader=NoDatesSafeLoader)
 
 
 def template_to_json(template):
@@ -621,6 +647,17 @@ def update_resource(resource_id, resources, stack_name):
         return client.put_method(**kwargs)
 
 
+def fix_account_id_in_arns(params):
+    def fix_ids(o, **kwargs):
+        if isinstance(o, dict):
+            for k, v in o.items():
+                if common.is_string(v):
+                    o[k] = aws_stack.fix_account_id_in_arns(v)
+        return o
+    result = common.recurse_object(params, fix_ids)
+    return result
+
+
 def convert_data_types(func_details, params):
     """ Convert data types in the "params" object, with the type defs
         specified in the 'types' attribute of "func_details". """
@@ -730,7 +767,7 @@ def deploy_resource_via_sdk_function(resource_id, resources, resource_type, func
             params[param_key] = str(param_value).lower() == 'true'
 
     # convert any moto account IDs (123456789012) in ARNs to our format (000000000000)
-    params = json.loads(aws_stack.fix_account_id_in_arns(json.dumps(params)))
+    params = fix_account_id_in_arns(params)
     # convert data types (e.g., boolean strings to bool)
     params = convert_data_types(func_details, params)
     # remove None values, as they usually raise boto3 errors
diff --git a/localstack/utils/common.py b/localstack/utils/common.py
--- a/localstack/utils/common.py
+++ b/localstack/utils/common.py
@@ -300,6 +300,11 @@ def is_string_or_bytes(s):
     return is_string(s) or isinstance(s, six.string_types) or isinstance(s, bytes)
 
 
+def is_base64(s):
+    regex = r'^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$'
+    return is_string(s) and re.match(regex, s)
+
+
 def md5(string):
     m = hashlib.md5()
     m.update(to_bytes(string))


Test patch:
diff --git a/tests/integration/test_cloudformation.py b/tests/integration/test_cloudformation.py
--- a/tests/integration/test_cloudformation.py
+++ b/tests/integration/test_cloudformation.py
@@ -1,8 +1,9 @@
 import os
+import json
 import unittest
 from localstack.constants import TEST_AWS_ACCOUNT_ID
 from localstack.utils.aws import aws_stack
-from localstack.utils.common import load_file, retry, short_uid
+from localstack.utils.common import load_file, retry, short_uid, to_str
 from localstack.utils.cloudformation import template_deployer
 from botocore.exceptions import ClientError
 from botocore.parsers import ResponseParserError
@@ -12,13 +13,31 @@
 TEST_TEMPLATE_2 = os.path.join(THIS_FOLDER, 'templates', 'template2.yaml')
 TEST_TEMPLATE_3 = """
 AWSTemplateFormatVersion: 2010-09-09
-Transform: AWS::Serverless-2016-10-31
 Resources:
   S3Setup:
     Type: AWS::S3::Bucket
     Properties:
       BucketName: test-%s
 """ % short_uid()
+TEST_TEMPLATE_4 = """
+AWSTemplateFormatVersion: 2010-09-09
+Transform: AWS::Serverless-2016-10-31
+Parameters:
+  LambdaRuntime:
+    Type: String
+    Default: python3.6
+Resources:
+  MyFunc:
+    Type: AWS::Serverless::Function
+    Properties:
+      FunctionName: %s
+      Handler: index.handler
+      Runtime:
+        Ref: LambdaRuntime
+      InlineCode: |
+        def handler(event, context):
+            return {'hello': 'world'}
+"""
 
 
 def bucket_exists(name):
@@ -201,3 +220,18 @@ def test_create_change_set(self):
             TemplateBody=TEST_TEMPLATE_3, ChangeSetName='nochanges')
         self.assertIn(':%s:changeSet/nochanges/' % TEST_AWS_ACCOUNT_ID, response['Id'])
         self.assertIn(':%s:stack/' % TEST_AWS_ACCOUNT_ID, response['StackId'])
+
+    def test_sam_template(self):
+        cloudformation = aws_stack.connect_to_service('cloudformation')
+        awslambda = aws_stack.connect_to_service('lambda')
+
+        # deploy template
+        stack_name = 'stack-%s' % short_uid()
+        func_name = 'test-%s' % short_uid()
+        template = TEST_TEMPLATE_4 % func_name
+        cloudformation.create_stack(StackName=stack_name, TemplateBody=template)
+
+        # run Lambda test invocation
+        result = awslambda.invoke(FunctionName=func_name)
+        result = json.loads(to_str(result['Payload'].read()))
+        self.assertEqual(result, {'hello': 'world'})