Repository: aws/aws-cli

Issue:
cloudformation deploy does not honour tabs in JSON templates
Most of our templates are using tabs for the indentation and everything was fine until we tried to feed the template to `aws cloudformation deploy` which did not like the template despite that every other function in AWS CLI works with the template just fine:

```
[galaxy@athena templates]$ aws cloudformation validate-template --template-body file://codepipeline.template
{
    "CapabilitiesReason": "The following resource(s) require capabilities: [AWS::IAM::Role]", 
    "Description": "Playing with CodeCommit, CodeBuild, and CodeDeploy", 
    "Parameters": [
```
and the only function that fails to parse the template is:
```
[galaxy@athena templates]$ aws cloudformation deploy --stack-name "galaxy-ccc" --template-file codepipeline.template --capabilities CAPABILITY_IAM

while scanning for the next token
found character '\t' that cannot start any token
  in "<string>", line 2, column 1:
    	"AWSTemplateFormatVersion": "20 ... 
    ^
```
A quick fix is to replace tabs with spaces:
```
[galaxy@athena templates]$ sed 's,\t, ,g' codepipeline.template > c.template
[galaxy@athena templates]$ aws cloudformation deploy --stack-name "galaxy-ccc" --template-file c.template --capabilities CAPABILITY_IAM
Waiting for changeset to be created..
Waiting for stack create/update to complete
```

... but it would mean that we would need to fix all our templates which are valid JSON just to workaround a bug in the tool! :(


Discussion:
The `deploy` command is a hand written customization.  It looks like it's relying on the fact that it can use the yaml parser to parse JSON whereas the rest of the CLI uses the `json` module.

```
>>> import yaml
>>> yaml.load(StringIO('{"foo":\t"bar"}'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
    return loader.get_single_data()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
    node = self.get_single_node()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/composer.py", line 35, in get_single_node
    if not self.check_event(StreamEndEvent):
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/parser.py", line 143, in parse_implicit_document_start
    StreamEndToken):
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/scanner.py", line 257, in fetch_more_tokens
    % ch.encode('utf-8'), self.get_mark())
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
  in "<file>", line 1, column 8
>>> import json
>>> json.loads('{"foo":\t"bar"}')
{u'foo': u'bar'}
```
We should be able to first try to parse this as JSON and then fall back to yaml if that fails.

Resolving patch:
diff --git a/awscli/customizations/cloudformation/yamlhelper.py b/awscli/customizations/cloudformation/yamlhelper.py
--- a/awscli/customizations/cloudformation/yamlhelper.py
+++ b/awscli/customizations/cloudformation/yamlhelper.py
@@ -10,11 +10,12 @@
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
-
+import json
 import yaml
-from awscli.compat import six
 from yaml.resolver import ScalarNode, SequenceNode
 
+from awscli.compat import six
+
 
 def intrinsics_multi_constructor(loader, tag_prefix, node):
     """
@@ -63,7 +64,13 @@ def yaml_dump(dict_to_dump):
 
 
 def yaml_parse(yamlstr):
-
-    yaml.SafeLoader.add_multi_constructor("!", intrinsics_multi_constructor)
-
-    return yaml.safe_load(yamlstr)
+    """Parse a yaml string"""
+    try:
+        # PyYAML doesn't support json as well as it should, so if the input
+        # is actually just json it is better to parse it with the standard
+        # json parser.
+        return json.loads(yamlstr)
+    except ValueError:
+        yaml.SafeLoader.add_multi_constructor(
+            "!", intrinsics_multi_constructor)
+        return yaml.safe_load(yamlstr)


Test patch:
diff --git a/tests/unit/customizations/cloudformation/test_yamlhelper.py b/tests/unit/customizations/cloudformation/test_yamlhelper.py
--- a/tests/unit/customizations/cloudformation/test_yamlhelper.py
+++ b/tests/unit/customizations/cloudformation/test_yamlhelper.py
@@ -84,3 +84,8 @@ def test_yaml_getatt(self):
         actual_output = yaml_parse(input)
         self.assertEquals(actual_output, output)
 
+    def test_parse_json_with_tabs(self):
+        template = '{\n\t"foo": "bar"\n}'
+        output = yaml_parse(template)
+        self.assertEqual(output, {'foo': 'bar'})
+