Repository: jazzband/tablib
Pull request #484 (2020-12-05 11:55:25)

Export JSON/YAML as Unicode

codecov[bot]: # [Codecov](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=h1) Report
> Merging [#484](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=desc) (e107f08) into [master](https://codecov.io/gh/jazzband/tablib/commit/e8f54811c7128d6ee849d072f27459e9b9322034?el=desc) (e8f5481) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/jazzband/tablib/pull/484/graphs/tree.svg?width=650&height=150&src=pr&token=ns7MUP4H8t)](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master     #484   +/-   ##
=======================================
  Coverage   90.66%   90.66%           
=======================================
  Files          28       28           
  Lines        2613     2615    +2     
=======================================
+ Hits         2369     2371    +2     
  Misses        244      244           
```


| [Impacted Files](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/tablib/formats/\_json.py](https://codecov.io/gh/jazzband/tablib/pull/484/diff?src=pr&el=tree#diff-c3JjL3RhYmxpYi9mb3JtYXRzL19qc29uLnB5) | `97.36% <100.00%> (ø)` | |
| [src/tablib/formats/\_yaml.py](https://codecov.io/gh/jazzband/tablib/pull/484/diff?src=pr&el=tree#diff-c3JjL3RhYmxpYi9mb3JtYXRzL195YW1sLnB5) | `100.00% <100.00%> (ø)` | |
| [tests/test\_tablib.py](https://codecov.io/gh/jazzband/tablib/pull/484/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF90YWJsaWIucHk=) | `98.56% <100.00%> (+<0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=footer). Last update [e8f5481...e107f08](https://codecov.io/gh/jazzband/tablib/pull/484?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/HISTORY.md b/HISTORY.md
index e260784..d7ab064 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -5,6 +5,8 @@
 ### Breaking changes
 
 - Dropped Python 3.5 support
+- JSON-exported data is no longer forced to ASCII characters.
+- YAML-exported data is no longer forced to ASCII characters.
 
 ### Improvements
 
diff --git a/src/tablib/formats/_json.py b/src/tablib/formats/_json.py
index dd8c379..a0f9812 100644
--- a/src/tablib/formats/_json.py
+++ b/src/tablib/formats/_json.py
@@ -23,12 +23,16 @@ class JSONFormat:
     @classmethod
     def export_set(cls, dataset):
         """Returns JSON representation of Dataset."""
-        return json.dumps(dataset.dict, default=serialize_objects_handler)
+        return json.dumps(
+            dataset.dict, default=serialize_objects_handler, ensure_ascii=False
+        )
 
     @classmethod
     def export_book(cls, databook):
         """Returns JSON representation of Databook."""
-        return json.dumps(databook._package(), default=serialize_objects_handler)
+        return json.dumps(
+            databook._package(), default=serialize_objects_handler, ensure_ascii=False
+        )
 
     @classmethod
     def import_set(cls, dset, in_stream):
diff --git a/src/tablib/formats/_yaml.py b/src/tablib/formats/_yaml.py
index 1b653d7..323387f 100644
--- a/src/tablib/formats/_yaml.py
+++ b/src/tablib/formats/_yaml.py
@@ -13,13 +13,16 @@ class YAMLFormat:
     @classmethod
     def export_set(cls, dataset):
         """Returns YAML representation of Dataset."""
-
-        return yaml.safe_dump(dataset._package(ordered=False), default_flow_style=None)
+        return yaml.safe_dump(
+            dataset._package(ordered=False), default_flow_style=None, allow_unicode=True
+        )
 
     @classmethod
     def export_book(cls, databook):
         """Returns YAML representation of Databook."""
-        return yaml.safe_dump(databook._package(ordered=False), default_flow_style=None)
+        return yaml.safe_dump(
+            databook._package(ordered=False), default_flow_style=None, allow_unicode=True
+        )
 
     @classmethod
     def import_set(cls, dset, in_stream):


Test patch:
diff --git a/tests/test_tablib.py b/tests/test_tablib.py
index ccb28e1..96a7386 100755
--- a/tests/test_tablib.py
+++ b/tests/test_tablib.py
@@ -1096,11 +1096,13 @@ class JSONTests(BaseTestCase):
         headers = self.headers + ('address_id',)
         founders = tablib.Dataset(headers=headers, title='Founders')
         founders.append(('John', 'Adams', 90, address_id))
+        founders.append(('名字', '李', 60, ''))
         founders_json = founders.export('json')
 
         expected_json = (
             '[{"first_name": "John", "last_name": "Adams", "gpa": 90, '
-            '"address_id": "%s"}]' % str(address_id)
+            '"address_id": "%s"}, {"first_name": "名字", "last_name": "李", '
+            '"gpa": 60, "address_id": ""}]' % str(address_id)
         )
 
         self.assertEqual(founders_json, expected_json)
@@ -1153,10 +1155,12 @@ class YAMLTests(BaseTestCase):
     def test_yaml_export(self):
         """YAML export"""
 
+        self.founders.append(('名字', '李', 60))
         expected = """\
 - {first_name: John, gpa: 90, last_name: Adams}
 - {first_name: George, gpa: 67, last_name: Washington}
 - {first_name: Thomas, gpa: 50, last_name: Jefferson}
+- {first_name: 名字, gpa: 60, last_name: 李}
 """
         output = self.founders.yaml
         self.assertEqual(output, expected)