Repository: frictionlessdata/datapackage-py (python)

Issue:
Provide a flag to open data packages with not safe paths?
# Overview

We could allow to open data package with not safe paths if user explicitly choose it:

```python
package = Package(descriptor, safe=False)
```

---

_From @vitorbaptista_ 

This is what I'm trying to do:

```python
import datapackage

descriptor = {
    'name': 'my-datapackage',
    'resources': [
      {
        'name': 'data',
        'path': '/tmp/my-data.csv'
      }
    ]
}
dp = datapackage.Package(descriptor)
# *** tableschema.exceptions.DataPackageException: Local path "/tmp/my-data.csv" is not safe
```

Reading the `helpers.is_safe_path()` method:

https://github.com/frictionlessdata/datapackage-py/blob/3d25cfd2013c0ddb6b5f9ceb7724dcd111fc9905/datapackage/helpers.py#L189-L196

Makes me think that what I'm trying to do isn't possible anymore. Any known workarounds?

(cc @roll)

Pull request:
Add `unsafe` option to Package and Resource
- fixes #171

Patch:
diff --git a/Makefile b/Makefile
index f715439..7330e90 100644
--- a/Makefile
+++ b/Makefile
@@ -26,7 +26,7 @@ readme:
 	pip install md-toc
 	pip install referencer
 	referencer $(PACKAGE) README.md --in-place
-	md_toc -p README.md github --header-levels 3
+	md_toc -p github --header-levels 3 README.md
 	sed -i '/(#$(PACKAGE)-py)/,+2d' README.md
 
 release:
diff --git a/README.md b/README.md
index 472400b..392355e 100644
--- a/README.md
+++ b/README.md
@@ -561,7 +561,15 @@ Commands:
 
 ### `Package`
 ```python
-Package(self, descriptor=None, base_path=None, strict=False, storage=None, schema=None, default_base_path=None, **options)
+Package(self,
+        descriptor=None,
+        base_path=None,
+        strict=False,
+        unsafe=False,
+        storage=None,
+        schema=None,
+        default_base_path=None,
+        **options)
 ```
 Package representation
 
@@ -571,6 +579,10 @@ __Arguments__
 - __strict (bool)__: strict flag to alter validation behavior.
         Setting it to `True` leads to throwing errors
         on any operation with invalid descriptor
+- __unsafe (bool)__:
+        if `True` unsafe paths will be allowed. For more inforamtion
+        https://specs.frictionlessdata.io/data-resource/#data-location.
+        Default to `False`
 - __storage (str/tableschema.Storage)__: storage name like `sql` or storage instance
 - __options (dict)__: storage options to use for storage creation
 
@@ -578,6 +590,7 @@ __Raises__
 - `DataPackageException`: raises error if something goes wrong
 
 
+
 #### `package.base_path`
 Package's base path
 
@@ -586,6 +599,7 @@ __Returns__
 `str/None`: returns the data package base path
 
 
+
 #### `package.descriptor`
 Package's descriptor
 
@@ -594,6 +608,7 @@ __Returns__
 `dict`: descriptor
 
 
+
 #### `package.errors`
 Validation errors
 
@@ -604,6 +619,7 @@ __Returns__
 `Exception[]`: validation errors
 
 
+
 #### `package.profile`
 Package's profile
 
@@ -612,6 +628,7 @@ __Returns__
 `Profile`: an instance of `Profile` class
 
 
+
 #### `package.resource_names`
 Package's resource names
 
@@ -620,6 +637,7 @@ __Returns__
 `str[]`: returns an array of resource names
 
 
+
 #### `package.resources`
 Package's resources
 
@@ -628,6 +646,7 @@ __Returns__
 `Resource[]`: returns an array of `Resource` instances
 
 
+
 #### `package.valid`
 Validation status
 
@@ -638,9 +657,10 @@ __Returns__
 `bool`: validation status
 
 
+
 #### `package.get_resource`
 ```python
-package.get_resource(self, name)
+package.get_resource(name)
 ```
 Get data package resource by name.
 
@@ -652,9 +672,10 @@ __Returns__
 `Resource/None`: returns `Resource` instances or null if not found
 
 
+
 #### `package.add_resource`
 ```python
-package.add_resource(self, descriptor)
+package.add_resource(descriptor)
 ```
 Add new resource to data package.
 
@@ -671,9 +692,10 @@ __Returns__
 `Resource/None`: returns added `Resource` instance or null if not added
 
 
+
 #### `package.remove_resource`
 ```python
-package.remove_resource(self, name)
+package.remove_resource(name)
 ```
 Remove data package resource by name.
 
@@ -690,9 +712,10 @@ __Returns__
 `Resource/None`: returns removed `Resource` instances or null if not found
 
 
+
 #### `package.get_group`
 ```python
-package.get_group(self, name)
+package.get_group(name)
 ```
 Returns a group of tabular resources by name.
 
@@ -709,9 +732,10 @@ __Returns__
 `Group/None`: returns a `Group` instance or null if not found
 
 
+
 #### `package.infer`
 ```python
-package.infer(self, pattern=False)
+package.infer(pattern=False)
 ```
 Infer a data package metadata.
 
@@ -730,9 +754,10 @@ __Returns__
 `dict`: returns data package descriptor
 
 
+
 #### `package.commit`
 ```python
-package.commit(self, strict=None)
+package.commit(strict=None)
 ```
 Update data package instance if there are in-place changes in the descriptor.
 
@@ -763,9 +788,14 @@ __Returns__
 `bool`: returns true on success and false if not modified
 
 
+
 #### `package.save`
 ```python
-package.save(self, target=None, storage=None, merge_groups=False, to_base_path=False, **options)
+package.save(target=None,
+             storage=None,
+             merge_groups=False,
+             to_base_path=False,
+             **options)
 ```
 Saves this data package
 
@@ -832,7 +862,14 @@ __Returns__
 
 ### `Resource`
 ```python
-Resource(self, descriptor={}, base_path=None, strict=False, storage=None, package=None, **options)
+Resource(self,
+         descriptor={},
+         base_path=None,
+         strict=False,
+         unsafe=False,
+         storage=None,
+         package=None,
+         **options)
 ```
 Resource represenation
 
@@ -842,6 +879,10 @@ __Arguments__
 - __strict (bool)__:
         strict flag to alter validation behavior.  Setting it to `true`
         leads to throwing errors on any operation with invalid descriptor
+- __unsafe (bool)__:
+        if `True` unsafe paths will be allowed. For more inforamtion
+        https://specs.frictionlessdata.io/data-resource/#data-location.
+        Default to `False`
 - __storage (str/tableschema.Storage)__: storage name like `sql` or storage instance
 - __options (dict)__: storage options to use for storage creation
 
@@ -849,9 +890,11 @@ __Raises__
 - `DataPackageException`: raises error if something goes wrong
 
 
+
 #### `resource.data`
 Return resource data
 
+
 #### `resource.descriptor`
 Package's descriptor
 
@@ -860,6 +903,7 @@ __Returns__
 `dict`: descriptor
 
 
+
 #### `resource.errors`
 Validation errors
 
@@ -870,6 +914,7 @@ __Returns__
 `Exception[]`: validation errors
 
 
+
 #### `resource.group`
 Group name
 
@@ -878,6 +923,7 @@ __Returns__
 `str`: group name
 
 
+
 #### `resource.headers`
 Resource's headers
 
@@ -888,6 +934,7 @@ __Returns__
 `str[]/None`: returns data source headers
 
 
+
 #### `resource.inline`
 Whether resource inline
 
@@ -896,6 +943,7 @@ __Returns__
 `bool`: returns true if resource is inline
 
 
+
 #### `resource.local`
 Whether resource local
 
@@ -904,6 +952,7 @@ __Returns__
 `bool`: returns true if resource is local
 
 
+
 #### `resource.multipart`
 Whether resource multipart
 
@@ -912,6 +961,7 @@ __Returns__
 `bool`: returns true if resource is multipart
 
 
+
 #### `resource.name`
 Resource name
 
@@ -920,6 +970,7 @@ __Returns__
 `str`: name
 
 
+
 #### `resource.package`
 Package instance if the resource belongs to some package
 
@@ -928,6 +979,7 @@ __Returns__
 `Package/None`: a package instance if available
 
 
+
 #### `resource.profile`
 Resource's profile
 
@@ -936,6 +988,7 @@ __Returns__
 `Profile`: an instance of `Profile` class
 
 
+
 #### `resource.remote`
 Whether resource remote
 
@@ -944,6 +997,7 @@ __Returns__
 `bool`: returns true if resource is remote
 
 
+
 #### `resource.schema`
 Resource's schema
 
@@ -957,6 +1011,7 @@ __Returns__
 `tableschema.Schema`: schema
 
 
+
 #### `resource.source`
 Resource's source
 
@@ -968,9 +1023,11 @@ __Returns__
 `list/str`: returns `data` or `path` property
 
 
+
 #### `resource.table`
 Return resource table
 
+
 #### `resource.tabular`
 Whether resource tabular
 
@@ -979,6 +1036,7 @@ __Returns__
 `bool`: returns true if resource is tabular
 
 
+
 #### `resource.valid`
 Validation status
 
@@ -989,9 +1047,10 @@ __Returns__
 `bool`: validation status
 
 
+
 #### `resource.iter`
 ```python
-resource.iter(self, integrity=False, relations=False, **options)
+resource.iter(integrity=False, relations=False, **options)
 ```
 Iterates through the resource data and emits rows cast based on table schema.
 
@@ -999,28 +1058,35 @@ Iterates through the resource data and emits rows cast based on table schema.
 
 __Arguments__
 
-- __keyed (bool)__:
+
+    keyed (bool):
         yield keyed rows in a form of `{header1: value1, header2: value2}`
         (default is false; the form of rows is `[value1, value2]`)
-- __extended (bool)__:
+
+    extended (bool):
         yield extended rows in a for of `[rowNumber, [header1, header2], [value1, value2]]`
         (default is false; the form of rows is `[value1, value2]`)
-- __cast (bool)__:
+
+    cast (bool):
         disable data casting if false
         (default is true)
-- __integrity (bool)__:
+
+    integrity (bool):
         if true actual size in BYTES and SHA256 hash of the file
         will be checked against `descriptor.bytes` and `descriptor.hash`
         (other hashing algorithms are not supported and will be skipped silently)
-- __relations (bool)__:
+
+    relations (bool):
         if true foreign key fields will be checked and resolved to its references
-- __foreign_keys_values (dict)__:
+
+    foreign_keys_values (dict):
         three-level dictionary of foreign key references optimized
         to speed up validation process in a form of
         `{resource1: {(fk_field1, fk_field2): {(value1, value2): {one_keyedrow}, ... }}}`.
         If not provided but relations is true, it will be created
         before the validation process by *index_foreign_keys_values* method
-- __exc_handler (func)__:
+
+    exc_handler (func):
         optional custom exception handler callable.
         Can be used to defer raising errors (i.e. "fail late"), e.g.
         for data validation purposes. Must support the signature below
@@ -1059,9 +1125,13 @@ __Returns__
 `Iterator[list]`: yields rows
 
 
+
 #### `resource.read`
 ```python
-resource.read(self, integrity=False, relations=False, foreign_keys_values=False, **options)
+resource.read(integrity=False,
+              relations=False,
+              foreign_keys_values=False,
+              **options)
 ```
 Read the whole resource and return as array of rows
 
@@ -1076,9 +1146,10 @@ __Returns__
 `list[]`: returns rows
 
 
+
 #### `resource.check_integrity`
 ```python
-resource.check_integrity(self)
+resource.check_integrity()
 ```
 Checks resource integrity
 
@@ -1096,9 +1167,10 @@ __Returns__
 `bool`: returns True if no issues
 
 
+
 #### `resource.check_relations`
 ```python
-resource.check_relations(self, foreign_keys_values=False)
+resource.check_relations(foreign_keys_values=False)
 ```
 Check relations
 
@@ -1114,9 +1186,10 @@ __Returns__
 `bool`: returns True if no issues
 
 
+
 #### `resource.drop_relations`
 ```python
-resource.drop_relations(self)
+resource.drop_relations()
 ```
 Drop relations
 
@@ -1129,9 +1202,10 @@ __Returns__
 `bool`: returns True
 
 
+
 #### `resource.raw_iter`
 ```python
-resource.raw_iter(self, stream=False)
+resource.raw_iter(stream=False)
 ```
 Iterate over data chunks as bytes.
 
@@ -1145,9 +1219,10 @@ __Returns__
 `bytes[]/filelike`: returns bytes[]/filelike
 
 
+
 #### `resource.raw_read`
 ```python
-resource.raw_read(self)
+resource.raw_read()
 ```
 Returns resource data as bytes.
 
@@ -1156,9 +1231,10 @@ __Returns__
 `bytes`: returns resource data in bytes
 
 
+
 #### `resource.infer`
 ```python
-resource.infer(self, **options)
+resource.infer(**options)
 ```
 Infer resource metadata
 
@@ -1175,9 +1251,10 @@ __Returns__
 `dict`: returns resource descriptor
 
 
+
 #### `resource.commit`
 ```python
-resource.commit(self, strict=None)
+resource.commit(strict=None)
 ```
 Update resource instance if there are in-place changes in the descriptor.
 
@@ -1192,9 +1269,10 @@ __Returns__
 `bool`: returns true on success and false if not modified
 
 
+
 #### `resource.save`
 ```python
-resource.save(self, target, storage=None, to_base_path=False, **options)
+resource.save(target, storage=None, to_base_path=False, **options)
 ```
 Saves this resource
 
@@ -1231,6 +1309,7 @@ __Arguments__
 - __Resource[]__: list of TABULAR resources
 
 
+
 #### `group.headers`
 Group's headers
 
@@ -1239,6 +1318,7 @@ __Returns__
 `str[]/None`: returns headers
 
 
+
 #### `group.name`
 Group name
 
@@ -1247,6 +1327,7 @@ __Returns__
 `str`: name
 
 
+
 #### `group.schema`
 Resource's schema
 
@@ -1255,27 +1336,30 @@ __Returns__
 `tableschema.Schema`: schema
 
 
+
 #### `group.iter`
 ```python
-group.iter(self, **options)
+group.iter(**options)
 ```
 Iterates through the group data and emits rows cast based on table schema.
 
 > It concatenates all the resources and has the same API as `resource.iter`
 
 
+
 #### `group.read`
 ```python
-group.read(self, limit=None, **options)
+group.read(limit=None, **options)
 ```
 Read the whole group and return as array of rows
 
 > It concatenates all the resources and has the same API as `resource.read`
 
 
+
 #### `group.check_relations`
 ```python
-group.check_relations(self)
+group.check_relations()
 ```
 Check group's relations
 
@@ -1298,6 +1382,7 @@ __Raises__
 - `DataPackageException`: raises error if something goes wrong
 
 
+
 #### `profile.jsonschema`
 JSONSchema content
 
@@ -1306,6 +1391,7 @@ __Returns__
 `dict`: returns profile's JSON Schema contents
 
 
+
 #### `profile.name`
 Profile name
 
@@ -1314,9 +1400,10 @@ __Returns__
 `str/None`: name if available
 
 
+
 #### `profile.validate`
 ```python
-profile.validate(self, descriptor)
+profile.validate(descriptor)
 ```
 Validate a data package `descriptor` against the profile.
 
@@ -1384,6 +1471,7 @@ except DataPackageException as exception:
 ```
 
 
+
 #### `datapackageexception.errors`
 List of nested errors
 
@@ -1392,6 +1480,7 @@ __Returns__
 `DataPackageException[]`: list of nested errors
 
 
+
 #### `datapackageexception.multiple`
 Whether it's a nested exception
 
@@ -1400,42 +1489,49 @@ __Returns__
 `bool`: whether it's a nested exception
 
 
+
 ### `TableSchemaException`
 ```python
 TableSchemaException(self, message, errors=[])
 ```
 Base class for all TableSchema exceptions.
 
+
 ### `LoadError`
 ```python
 LoadError(self, message, errors=[])
 ```
 All loading errors.
 
+
 ### `CastError`
 ```python
 CastError(self, message, errors=[])
 ```
 All value cast errors.
 
+
 ### `IntegrityError`
 ```python
 IntegrityError(self, message, errors=[])
 ```
 All integrity errors.
 
+
 ### `RelationError`
 ```python
 RelationError(self, message, errors=[])
 ```
 All relations errors.
 
+
 ### `StorageError`
 ```python
 StorageError(self, message, errors=[])
 ```
 All storage errors.
 
+
 ## Contributing
 
 > The project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).
diff --git a/data/table.csv b/data/table.csv
new file mode 100644
index 0000000..6d09540
--- /dev/null
+++ b/data/table.csv
@@ -0,0 +1,3 @@
+id,name
+1,english
+2,中国人
diff --git a/datapackage/package.py b/datapackage/package.py
index 9f0d558..c95d003 100644
--- a/datapackage/package.py
+++ b/datapackage/package.py
@@ -36,6 +36,10 @@ class Package(object):
         strict (bool): strict flag to alter validation behavior.
             Setting it to `True` leads to throwing errors
             on any operation with invalid descriptor
+        unsafe (bool):
+            if `True` unsafe paths will be allowed. For more inforamtion
+            https\\://specs.frictionlessdata.io/data-resource/#data-location.
+            Default to `False`
         storage (str/tableschema.Storage): storage name like `sql` or storage instance
         options (dict): storage options to use for storage creation
 
@@ -46,7 +50,7 @@ class Package(object):
 
     # Public
 
-    def __init__(self, descriptor=None, base_path=None, strict=False, storage=None,
+    def __init__(self, descriptor=None, base_path=None, strict=False, unsafe=False, storage=None,
                  # Deprecated
                  schema=None, default_base_path=None, **options):
 
@@ -114,6 +118,7 @@ class Package(object):
         self.__base_path = base_path
         self.__storage = storage
         self.__strict = strict
+        self.__unsafe = unsafe
         self.__resources = []
         self.__errors = []
 
@@ -462,7 +467,7 @@ class Package(object):
                 encoding = None
             json_target = target
             if not os.path.isabs(json_target) and to_base_path:
-                if not helpers.is_safe_path(target):
+                if not self.__unsafe and not helpers.is_safe_path(target):
                     raise exceptions.DataPackageException('Target path "%s" is not safe', target)
                 json_target = os.path.join(self.__base_path, target)
             else:
@@ -527,8 +532,9 @@ class Package(object):
             if (not resource or resource.descriptor != descriptor or
                     (resource.schema and resource.schema.foreign_keys)):
                 updated_resource = Resource(descriptor,
-                    strict=self.__strict,
                     base_path=self.__base_path,
+                    strict=self.__strict,
+                    unsafe=self.__unsafe,
                     storage=self.__storage,
                     package=self)
                 if not resource:
diff --git a/datapackage/resource.py b/datapackage/resource.py
index 0935405..899acb4 100644
--- a/datapackage/resource.py
+++ b/datapackage/resource.py
@@ -34,6 +34,10 @@ class Resource(object):
         strict (bool):
             strict flag to alter validation behavior.  Setting it to `true`
             leads to throwing errors on any operation with invalid descriptor
+        unsafe (bool):
+            if `True` unsafe paths will be allowed. For more inforamtion
+            https\\://specs.frictionlessdata.io/data-resource/#data-location.
+            Default to `False`
         storage (str/tableschema.Storage): storage name like `sql` or storage instance
         options (dict): storage options to use for storage creation
 
@@ -44,7 +48,7 @@ class Resource(object):
 
     # Public
 
-    def __init__(self, descriptor={}, base_path=None, strict=False, storage=None,
+    def __init__(self, descriptor={}, base_path=None, strict=False, unsafe=False, storage=None,
                  # Internal
                  package=None, **options):
 
@@ -77,6 +81,7 @@ class Resource(object):
         self.__storage = storage
         self.__relations = None
         self.__strict = strict
+        self.__unsafe = unsafe
         self.__table = None
         self.__errors = []
         self.__table_options = options
@@ -605,7 +610,7 @@ class Resource(object):
                 encoding = None
             json_target = target
             if not os.path.isabs(json_target) and to_base_path:
-                if not helpers.is_safe_path(target):
+                if not self.__unsafe and not helpers.is_safe_path(target):
                     raise exceptions.DataPackageException('Target path "%s" is not safe', target)
                 json_target = os.path.join(self.__base_path, target)
             else:
@@ -626,8 +631,9 @@ class Resource(object):
         self.__source_inspection = _inspect_source(
             self.__current_descriptor.get('data'),
             self.__current_descriptor.get('path'),
-            self.__base_path,
-            self.__storage)
+            base_path=self.__base_path,
+            unsafe=self.__unsafe,
+            storage=self.__storage)
 
         # Instantiate profile
         self.__profile = Profile(self.__current_descriptor.get('profile'))
@@ -762,7 +768,7 @@ _DIALECT_KEYS = [
 ]
 
 
-def _inspect_source(data, path, base_path, storage):
+def _inspect_source(data, path, base_path=None, unsafe=False, storage=None):
     inspection = {}
 
     # Normalize path
@@ -803,7 +809,7 @@ def _inspect_source(data, path, base_path, storage):
         else:
 
             # Path is not safe
-            if not helpers.is_safe_path(path[0]):
+            if not unsafe and not helpers.is_safe_path(path[0]):
                 raise exceptions.DataPackageException(
                     'Local path "%s" is not safe' % path[0])
 
@@ -823,7 +829,8 @@ def _inspect_source(data, path, base_path, storage):
 
     # Multipart Local/Remote
     elif len(path) > 1:
-        inspections = list(map(lambda item: _inspect_source(None, item, base_path, None), path))
+        inspect = lambda item: _inspect_source(None, item, base_path=base_path, unsafe=unsafe)
+        inspections = list(map(inspect, path))
         inspection.update(inspections[0])
         inspection['source'] = list(map(lambda item: item['source'], inspections))
         inspection['multipart'] = True


Test patch:
diff --git a/tests/test_package.py b/tests/test_package.py
index 334222b..af043c2 100644
--- a/tests/test_package.py
+++ b/tests/test_package.py
@@ -774,6 +774,17 @@ def test_local_with_resources_outside_base_path_isnt_safe(tmpfile):
         Package(tmpfile.name, {})
 
 
+def test_local_relative_parent_path_with_unsafe_option_issue_171():
+    descriptor = {'resources': [{'name': 'table', 'path': 'data/../data/table.csv'}]}
+    # unsafe=false (default)
+    with pytest.raises(exceptions.DataPackageException) as excinfo:
+        package = Package(descriptor)
+    assert 'is not safe' in str(excinfo.value)
+    # unsafe=true
+    package = Package(descriptor, unsafe=True)
+    assert package.get_resource('table').read() == [['1', 'english'], ['2', '中国人']]
+
+
 @pytest.mark.skip(reason='Wait for specs-v1.rc2 resource.data/path')
 def test_zip_with_relative_resources_paths_is_safe(datapackage_zip):
     package = Package(datapackage_zip.name, {})
diff --git a/tests/test_resource.py b/tests/test_resource.py
index 0acef4c..c06c5fb 100644
--- a/tests/test_resource.py
+++ b/tests/test_resource.py
@@ -385,6 +385,17 @@ def test_source_multipart_remote_path_remote_and_base_path_remote():
     assert resource.multipart == True
 
 
+def test_source_relative_parent_path_with_unsafe_option_issue_171():
+    descriptor = {'path': 'data/../data/table.csv'}
+    # unsafe=false (default)
+    with pytest.raises(exceptions.DataPackageException) as excinfo:
+        resource = Resource(descriptor)
+    assert 'is not safe' in str(excinfo.value)
+    # unsafe=true
+    resource = Resource(descriptor, unsafe=True)
+    assert resource.read() == [['1', 'english'], ['2', '中国人']]
+
+
 # Resource.table
 
 def test_descriptor_table():