Repository: localstack/localstack

Issue:
bug: StateMachine yaml Definition is not supported (DefinitionString works)
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

when describing using cloudformation YAML with Definition, LocalStack fails to create StateMachine. Switching to YAML with DefinitionString works as expected. 

Examples taken from here
https://docs.aws.amazon.com/step-functions/latest/dg/development-options.html#development-options-format

Attaching working example 
[localstack_stepfunctions_and_serverless.zip](https://github.com/localstack/localstack/files/7791134/localstack_stepfunctions_and_serverless.z




### Expected Behavior

YAML with Definition is much more preferable in a complex state machine description with bunch of substitutions, references and parameters

### How are you starting LocalStack?

With the `localstack` script

### Steps To Reproduce

#### How are you starting localstack (e.g., `bin/localstack` command, arguments, or `docker-compose.yml`)

    localstack --debug start -d

#### Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

   awslocal s3 mb s3://s4-echo-serverless-deployment-share-local
sls deploy --stage local


### Environment

```markdown
- OS: OSX Monterey
- LocalStack: latest
```


### Anything else?

Error when using YAML with Definition
[error.txt](https://github.com/localstack/localstack/files/7791149/error.txt)

Success when using YAML with DefinitionString
[success.txt](https://github.com/localstack/localstack/files/7791151/success.txt)




Discussion:
The same issue applies to `DefinitionUri` as well when used with YAML, although with a more specific error. Tested with the latest LocalStack, 2.2.0, and in this case my YAML starts with a comment `#`:

`template.yaml`: 

```yaml
MyStateMachine:
  Type: AWS::Serverless::StateMachine
  Properties:
    DefinitionUri: mystatemachine.asl.yaml
```

```python
Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/cloudformation/engine/template_deployer.py", line 1265, in _run
    self.do_apply_changes_in_loop(changes, stack)
  File "/opt/code/localstack/localstack/services/cloudformation/engine/template_deployer.py", line 1338, in do_apply_changes_in_loop
    self.apply_change(change, stack=stack)
  File "/opt/code/localstack/localstack/services/cloudformation/engine/template_deployer.py", line 1468, in apply_change
    executor.deploy_loop(resource_provider_payload)  # noqa
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 601, in deploy_loop
    event = self.execute_action(resource_provider, payload)
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 650, in execute_action
    return resource_provider.create(request)
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 394, in create
    return self.create_or_delete(request)
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 516, in create_or_delete
    result = invoke_function(
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 258, in invoke_function
    raise e
  File "/opt/code/localstack/localstack/services/cloudformation/resource_provider.py", line 238, in invoke_function
    result = function(**params)
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/botocore/client.py", line 534, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/botocore/client.py", line 976, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidDefinition: An error occurred (InvalidDefinition) when calling the CreateStateMachine operation: Invalid State Machine Definition: ''INVALID_JSON_DESCRIPTION: Unexpected character ('#' (code 35)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
```

Resolving patch:
diff --git a/localstack/services/stepfunctions/resource_providers/aws_stepfunctions_statemachine.py b/localstack/services/stepfunctions/resource_providers/aws_stepfunctions_statemachine.py
--- a/localstack/services/stepfunctions/resource_providers/aws_stepfunctions_statemachine.py
+++ b/localstack/services/stepfunctions/resource_providers/aws_stepfunctions_statemachine.py
@@ -1,6 +1,7 @@
 # LocalStack Resource Provider Scaffolding v2
 from __future__ import annotations
 
+import json
 import re
 from pathlib import Path
 from typing import Optional, TypedDict
@@ -129,14 +130,20 @@ def create(
         )
 
     def _get_definition(self, model, s3_client):
-        definition_str = model.get("DefinitionString")
-        s3_location = model.get("DefinitionS3Location")
-        if not definition_str and s3_location:
+        if "DefinitionString" in model:
+            definition_str = model.get("DefinitionString")
+        elif "DefinitionS3Location" in model:
             # TODO: currently not covered by tests - add a test to mimick the behavior of "sam deploy ..."
-
+            s3_location = model.get("DefinitionS3Location")
             LOG.debug("Fetching state machine definition from S3: %s", s3_location)
             result = s3_client.get_object(Bucket=s3_location["Bucket"], Key=s3_location["Key"])
             definition_str = to_str(result["Body"].read())
+        elif "Definition" in model:
+            definition = model.get("Definition")
+            definition_str = json.dumps(definition)
+        else:
+            definition_str = None
+
         substitutions = model.get("DefinitionSubstitutions")
         if substitutions is not None:
             definition_str = _apply_substitutions(definition_str, substitutions)


Test patch:
diff --git a/tests/aws/services/stepfunctions/v2/test_sfn_api.py b/tests/aws/services/stepfunctions/v2/test_sfn_api.py
--- a/tests/aws/services/stepfunctions/v2/test_sfn_api.py
+++ b/tests/aws/services/stepfunctions/v2/test_sfn_api.py
@@ -1,6 +1,7 @@
 import json
 
 import pytest
+import yaml
 
 from localstack.aws.api.lambda_ import Runtime
 from localstack.testing.pytest import markers
@@ -694,3 +695,120 @@ def test_describe_state_machine_for_execution(
             executionArn=execution_arn
         )
         sfn_snapshot.match("describe_resp", describe_resp)
+
+    @markers.aws.validated
+    @pytest.mark.parametrize("encoder_function", [json.dumps, yaml.dump])
+    def test_cloudformation_definition_create_describe(
+        self,
+        create_iam_role_for_sfn,
+        sfn_snapshot,
+        aws_client,
+        encoder_function,
+    ):
+        snf_role_arn = create_iam_role_for_sfn()
+        sfn_snapshot.add_transformer(RegexTransformer(snf_role_arn, "snf_role_arn"))
+
+        state_machine_name = f"statemachine{short_uid()}"
+        sfn_snapshot.add_transformer(RegexTransformer(state_machine_name, "state_machine_name"))
+        definition = BaseTemplate.load_sfn_template(BaseTemplate.BASE_PASS_RESULT)
+        stack_name = f"test-create-describe-yaml-{short_uid()}"
+        cloudformation_template = {
+            "Resources": {
+                "MyStateMachine": {
+                    "Type": "AWS::StepFunctions::StateMachine",
+                    "Properties": {
+                        "StateMachineName": state_machine_name,
+                        "Definition": definition,
+                        "RoleArn": snf_role_arn,
+                    },
+                }
+            }
+        }
+        cloudformation_template = encoder_function(cloudformation_template)
+
+        aws_client.cloudformation.create_stack(
+            StackName=stack_name,
+            TemplateBody=cloudformation_template,
+        )
+        aws_client.cloudformation.get_waiter("stack_create_complete").wait(StackName=stack_name)
+
+        list_state_machines_response = aws_client.stepfunctions.list_state_machines()
+        state_machine = next(
+            (
+                sm
+                for sm in list_state_machines_response["stateMachines"]
+                if sm["name"] == state_machine_name
+            ),
+            None,
+        )
+        state_machine_arn = state_machine["stateMachineArn"]
+        sfn_snapshot.add_transformer(RegexTransformer(state_machine_arn, "state_machine_arn"))
+
+        describe_state_machine_response = aws_client.stepfunctions.describe_state_machine(
+            stateMachineArn=state_machine_arn
+        )
+        sfn_snapshot.match("describe_state_machine_response", describe_state_machine_response)
+
+        aws_client.cloudformation.delete_stack(StackName=stack_name)
+        aws_client.cloudformation.get_waiter("stack_delete_complete").wait(StackName=stack_name)
+
+        aws_client.stepfunctions.delete_state_machine(stateMachineArn=state_machine_arn)
+
+    @markers.aws.validated
+    @pytest.mark.parametrize("encoder_function", [json.dumps, yaml.dump])
+    def test_cloudformation_definition_string_create_describe(
+        self,
+        create_iam_role_for_sfn,
+        sfn_snapshot,
+        aws_client,
+        encoder_function,
+    ):
+        snf_role_arn = create_iam_role_for_sfn()
+        sfn_snapshot.add_transformer(RegexTransformer(snf_role_arn, "snf_role_arn"))
+
+        state_machine_name = f"statemachine{short_uid()}"
+        sfn_snapshot.add_transformer(RegexTransformer(state_machine_name, "state_machine_name"))
+        definition = BaseTemplate.load_sfn_template(BaseTemplate.BASE_PASS_RESULT)
+        definition_string = json.dumps(definition)
+        stack_name = f"test-create-describe-yaml-{short_uid()}"
+        cloudformation_template = {
+            "Resources": {
+                "MyStateMachine": {
+                    "Type": "AWS::StepFunctions::StateMachine",
+                    "Properties": {
+                        "StateMachineName": state_machine_name,
+                        "DefinitionString": definition_string,
+                        "RoleArn": snf_role_arn,
+                    },
+                }
+            }
+        }
+        cloudformation_template = encoder_function(cloudformation_template)
+
+        aws_client.cloudformation.create_stack(
+            StackName=stack_name,
+            TemplateBody=cloudformation_template,
+        )
+        aws_client.cloudformation.get_waiter("stack_create_complete").wait(StackName=stack_name)
+
+        list_state_machines_response = aws_client.stepfunctions.list_state_machines()
+        state_machine = next(
+            (
+                sm
+                for sm in list_state_machines_response["stateMachines"]
+                if sm["name"] == state_machine_name
+            ),
+            None,
+        )
+        state_machine_arn = state_machine["stateMachineArn"]
+        sfn_snapshot.add_transformer(RegexTransformer(state_machine_arn, "state_machine_arn"))
+
+        describe_state_machine_response = aws_client.stepfunctions.describe_state_machine(
+            stateMachineArn=state_machine_arn
+        )
+        sfn_snapshot.match("describe_state_machine_response", describe_state_machine_response)
+
+        aws_client.cloudformation.delete_stack(StackName=stack_name)
+        aws_client.cloudformation.get_waiter("stack_delete_complete").wait(StackName=stack_name)
+
+        aws_client.stepfunctions.delete_state_machine(stateMachineArn=state_machine_arn)
diff --git a/tests/aws/services/stepfunctions/v2/test_sfn_api.snapshot.json b/tests/aws/services/stepfunctions/v2/test_sfn_api.snapshot.json
--- a/tests/aws/services/stepfunctions/v2/test_sfn_api.snapshot.json
+++ b/tests/aws/services/stepfunctions/v2/test_sfn_api.snapshot.json
@@ -1226,5 +1226,153 @@
     "recorded-content": {
       "describe_nonexistent_sm": "An error occurred (StateMachineDoesNotExist) when calling the DescribeStateMachine operation: State Machine Does Not Exist: 'sm_nonexistent_arn'"
     }
+  },
+  "tests/aws/services/stepfunctions/v2/test_sfn_api.py::TestSnfApi::test_cloudformation_definition_create_describe[dumps]": {
+    "recorded-date": "17-11-2023, 17:01:05",
+    "recorded-content": {
+      "describe_state_machine_response": {
+        "creationDate": "datetime",
+        "definition": {
+          "Comment": "BASE_PASS_RESULT",
+          "StartAt": "State_1",
+          "States": {
+            "State_1": {
+              "End": true,
+              "Result": {
+                "Arg1": "argument1"
+              },
+              "Type": "Pass"
+            }
+          }
+        },
+        "loggingConfiguration": {
+          "includeExecutionData": false,
+          "level": "OFF"
+        },
+        "name": "state_machine_name",
+        "roleArn": "snf_role_arn",
+        "stateMachineArn": "arn:aws:states:<region>:111111111111:stateMachine:state_machine_name",
+        "status": "ACTIVE",
+        "tracingConfiguration": {
+          "enabled": false
+        },
+        "type": "STANDARD",
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 200
+        }
+      }
+    }
+  },
+  "tests/aws/services/stepfunctions/v2/test_sfn_api.py::TestSnfApi::test_cloudformation_definition_create_describe[dump]": {
+    "recorded-date": "17-11-2023, 17:02:36",
+    "recorded-content": {
+      "describe_state_machine_response": {
+        "creationDate": "datetime",
+        "definition": {
+          "Comment": "BASE_PASS_RESULT",
+          "StartAt": "State_1",
+          "States": {
+            "State_1": {
+              "End": true,
+              "Result": {
+                "Arg1": "argument1"
+              },
+              "Type": "Pass"
+            }
+          }
+        },
+        "loggingConfiguration": {
+          "includeExecutionData": false,
+          "level": "OFF"
+        },
+        "name": "state_machine_name",
+        "roleArn": "snf_role_arn",
+        "stateMachineArn": "arn:aws:states:<region>:111111111111:stateMachine:state_machine_name",
+        "status": "ACTIVE",
+        "tracingConfiguration": {
+          "enabled": false
+        },
+        "type": "STANDARD",
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 200
+        }
+      }
+    }
+  },
+  "tests/aws/services/stepfunctions/v2/test_sfn_api.py::TestSnfApi::test_cloudformation_definition_string_create_describe[dumps]": {
+    "recorded-date": "17-11-2023, 17:06:22",
+    "recorded-content": {
+      "describe_state_machine_response": {
+        "creationDate": "datetime",
+        "definition": {
+          "Comment": "BASE_PASS_RESULT",
+          "StartAt": "State_1",
+          "States": {
+            "State_1": {
+              "Type": "Pass",
+              "Result": {
+                "Arg1": "argument1"
+              },
+              "End": true
+            }
+          }
+        },
+        "loggingConfiguration": {
+          "includeExecutionData": false,
+          "level": "OFF"
+        },
+        "name": "state_machine_name",
+        "roleArn": "snf_role_arn",
+        "stateMachineArn": "arn:aws:states:<region>:111111111111:stateMachine:state_machine_name",
+        "status": "ACTIVE",
+        "tracingConfiguration": {
+          "enabled": false
+        },
+        "type": "STANDARD",
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 200
+        }
+      }
+    }
+  },
+  "tests/aws/services/stepfunctions/v2/test_sfn_api.py::TestSnfApi::test_cloudformation_definition_string_create_describe[dump]": {
+    "recorded-date": "17-11-2023, 17:07:38",
+    "recorded-content": {
+      "describe_state_machine_response": {
+        "creationDate": "datetime",
+        "definition": {
+          "Comment": "BASE_PASS_RESULT",
+          "StartAt": "State_1",
+          "States": {
+            "State_1": {
+              "Type": "Pass",
+              "Result": {
+                "Arg1": "argument1"
+              },
+              "End": true
+            }
+          }
+        },
+        "loggingConfiguration": {
+          "includeExecutionData": false,
+          "level": "OFF"
+        },
+        "name": "state_machine_name",
+        "roleArn": "snf_role_arn",
+        "stateMachineArn": "arn:aws:states:<region>:111111111111:stateMachine:state_machine_name",
+        "status": "ACTIVE",
+        "tracingConfiguration": {
+          "enabled": false
+        },
+        "type": "STANDARD",
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 200
+        }
+      }
+    }
   }
 }