Repository: jazzband/tablib (python)

Issue:
Support importing ragged CSV text into a Dataset
Right now Dataset._validate prevents importing data where not all rows have the same width. This is not possible in some formats but is common in CSV files. Example:

``` python
from tablib import Dataset
from tablib.formats import csv
ds = Dataset()
csv.import_set(ds, "a,b,c\nx,y")
```

The result is:

``` python
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tablib/formats/_csv.py", line 48, in import_set
    dset.append(row)
  File "/usr/local/lib/python2.7/site-packages/tablib/core.py", line 668, in append
    self.rpush(row, tags)
  File "/usr/local/lib/python2.7/site-packages/tablib/core.py", line 652, in rpush
    self.insert(self.height, row=row, tags=tags)
  File "/usr/local/lib/python2.7/site-packages/tablib/core.py", line 643, in insert
    self._validate(row)
  File "/usr/local/lib/python2.7/site-packages/tablib/core.py", line 283, in _validate
    raise InvalidDimensions
tablib.core.InvalidDimensions
```

It may not be a good idea to actually represent ragged data in a Dataset, but it should be possible to import ragged CSV text into a uniform-width Dataset by filling in empty elements to make all rows match the width of the widest row.


Pull request description:
Fixes #226 - Allow importing ragged CSV files

Patch:
diff --git a/HISTORY.md b/HISTORY.md
index e5dd53c..4078e9a 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -9,6 +9,11 @@
   will remain in Tablib 1.x and will be fixed (reversed) in Tablib 2.0.0 (#453). If you
   count on the broken behavior, please update your code when you upgrade to Tablib 2.x.
 
+### Improvements
+
+- Tablib is now able to import CSV content where not all rows have the same
+  length. Missing columns on any line receive the empty string (#226).
+
 ## 1.0.0 (2020-01-13)
 
 ### Breaking changes
diff --git a/src/tablib/formats/_csv.py b/src/tablib/formats/_csv.py
index 14d7bb2..b0bf435 100644
--- a/src/tablib/formats/_csv.py
+++ b/src/tablib/formats/_csv.py
@@ -46,6 +46,8 @@ class CSVFormat:
             if (i == 0) and (headers):
                 dset.headers = row
             elif row:
+                if i > 0 and len(row) < dset.width:
+                    row += [''] * (dset.width - len(row))
                 dset.append(row)
 
     @classmethod


Test patch:
diff --git a/tests/test_tablib.py b/tests/test_tablib.py
index b20b669..3acdd44 100755
--- a/tests/test_tablib.py
+++ b/tests/test_tablib.py
@@ -804,6 +804,25 @@ class CSVTests(BaseTestCase):
         data.csv = csv_text
         self.assertEqual(data.width, 7)
 
+    def test_csv_import_set_ragged(self):
+        """Import CSV set when not all rows have the same length."""
+        csv_text = (
+            "H1,H2,H3\n"
+            "A,B\n"
+            "C,D,E\n"
+            "\n"
+            "F\n"
+        )
+        dataset = tablib.import_set(csv_text, format="csv")
+        self.assertEqual(
+            str(dataset),
+            'H1|H2|H3\n'
+            '--|--|--\n'
+            'A |B |  \n'
+            'C |D |E \n'
+            'F |  |  '
+        )
+
     def test_csv_export(self):
         """Verify exporting dataset object as CSV."""