Repository: cdent/gabbi

Issue:
Data <@filename isn't relative to the YAML file
The [docs say](https://gabbi.readthedocs.io/en/latest/format.html#data):

> If the value is a string that begins with <@ then the rest of the string is treated as the name of a file to be loaded from the same directory as the YAML file. 

But I haven't found it works like this unless I cd into the directory containing the yaml file.

`_load_data_file` says:
```python
path = os.path.join(self.test_directory, os.path.basename(filename))
```
this does a few things:
- makes the path begin with `self.test_directory` (defaults to `.`, the current working directory)
- disguards any folders in the <@ `filename` path
- appends `filename`

This means, if I'm in `/`, and I have a test that says `data: <@cake.jpg`, I cannot run my tests as advised in the docs:
`gabbi-run -- /my/test.yaml /my/other.yaml`

`FileNotFoundError: [Errno 2] No such file or directory: './cake.jpg'`

So in our test running script, we have to start with `cd tests/`.

My preference, would be to make the path relative to the location of the yaml file. That way, I can run my tests from whatever directory.

Obviously this isn't possible when redirecting with `<` as the filename is never seen by gabbi. So I'm happy is discuss other ideas.

---

Furthermore I can't keep my test data in a child directory, unless I cd into that directory like:
```shell
cd yaml_tests/test_data
gabbi-run -- ../test_things.yaml
```

So for that reason, I'd like to allow directories like `<@test_data/filename.txt` to be included in the path.

---

Happy to write the patch, if we come up with an approach here.

Comments:

Patch:
diff --git a/docs/source/format.rst b/docs/source/format.rst
index cc1b129..51dcf24 100644
--- a/docs/source/format.rst
+++ b/docs/source/format.rst
@@ -269,8 +269,9 @@ flexibility when doing a ``POST`` or ``PUT``. If the value is not a
 string (that is, it is a sequence or structure) it is treated as a
 data structure which is turned into a JSON string. If the value is a
 string that begins with ``<@`` then the rest of the string is treated
-as the name of a file to be loaded from the same directory as the YAML
-file. If the value is an undecorated string, that's the value.
+as a filepath to be loaded. The path is relative to the test directory
+and may not traverse up into parent directories. If the value is an
+undecorated string, that's the value.
 
 When reading from a file care should be taken to ensure that a
 reasonable content-type is set for the data as this will control if any
diff --git a/gabbi/runner.py b/gabbi/runner.py
index ac98dea..9f45351 100644
--- a/gabbi/runner.py
+++ b/gabbi/runner.py
@@ -14,6 +14,7 @@
 
 import argparse
 from importlib import import_module
+import os
 import sys
 import unittest
 
@@ -84,8 +85,9 @@ def run():
     else:
         for input_file in input_files:
             with open(input_file, 'r') as fh:
+                data_dir = os.path.dirname(input_file)
                 success = run_suite(fh, handler_objects, host, port,
-                                    prefix, force_ssl, failfast)
+                                    prefix, force_ssl, failfast, data_dir)
             if not failure:  # once failed, this is considered immutable
                 failure = not success
             if failure and failfast:
@@ -95,7 +97,7 @@ def run():
 
 
 def run_suite(handle, handler_objects, host, port, prefix, force_ssl=False,
-              failfast=False):
+              failfast=False, data_dir='.'):
     """Run the tests from the YAML in handle."""
     data = utils.load_yaml(handle)
     if force_ssl:
@@ -106,7 +108,7 @@ def run_suite(handle, handler_objects, host, port, prefix, force_ssl=False,
 
     loader = unittest.defaultTestLoader
     test_suite = suitemaker.test_suite_from_dict(
-        loader, 'input', data, '.', host, port, None, None, prefix=prefix,
+        loader, 'input', data, data_dir, host, port, None, None, prefix=prefix,
         handlers=handler_objects)
 
     result = ConciseTestRunner(


Test patch:
diff --git a/gabbi/tests/gabbits_runner/subdir/sample.json b/gabbi/tests/gabbits_runner/subdir/sample.json
new file mode 100644
index 0000000..ddbce20
--- /dev/null
+++ b/gabbi/tests/gabbits_runner/subdir/sample.json
@@ -0,0 +1,1 @@
+{"items": {"house": "blue"}}
diff --git a/gabbi/tests/gabbits_runner/test_data.yaml b/gabbi/tests/gabbits_runner/test_data.yaml
new file mode 100644
index 0000000..35d056a
--- /dev/null
+++ b/gabbi/tests/gabbits_runner/test_data.yaml
@@ -0,0 +1,8 @@
+tests:
+
+- name: POST data from file
+  verbose: true
+  POST: /
+  request_headers:
+    content-type: application/json
+  data: <@subdir/sample.json
diff --git a/gabbi/tests/test_runner.py b/gabbi/tests/test_runner.py
index bf882ab..1b86235 100644
--- a/gabbi/tests/test_runner.py
+++ b/gabbi/tests/test_runner.py
@@ -22,6 +22,7 @@ from wsgi_intercept.interceptor import Urllib3Interceptor
 
 from gabbi import exception
 from gabbi.handlers import base
+from gabbi.handlers.jsonhandler import JSONHandler
 from gabbi import runner
 from gabbi.tests.simple_wsgi import SimpleWsgi
 
@@ -249,6 +250,28 @@ class RunnerTest(unittest.TestCase):
         self.assertIn('{\n', output)
         self.assertIn('}\n', output)
 
+    def test_data_dir_good(self):
+        """Confirm that data dir is the test file's dir."""
+        sys.argv = ['gabbi-run', 'http://%s:%s/foo' % (self.host, self.port)]
+
+        sys.argv.append('--')
+        sys.argv.append('gabbi/tests/gabbits_runner/test_data.yaml')
+        with self.server():
+            try:
+                runner.run()
+            except SystemExit as err:
+                self.assertSuccess(err)
+
+        # Compare the verbose output of tests with pretty printed
+        # data.
+        with open('gabbi/tests/gabbits_runner/subdir/sample.json') as data:
+            data = JSONHandler.loads(data.read())
+            expected_string = JSONHandler.dumps(data, pretty=True)
+
+        sys.stdout.seek(0)
+        output = sys.stdout.read()
+        self.assertIn(expected_string, output)
+
     def assertSuccess(self, exitError):
         errors = exitError.args[0]
         if errors: