Repository: dask/dask

Issue:
dask.bag.to_textfiles does not store each element in a line
Not sure if this is intended, but in the [documentation](http://dask.pydata.org/en/latest/bag.html) it is mentioned the behaviour of `bag.to_textfiles()` should be

```
Write bag to disk, one filename per partition, one line per element
```

However, as reproduced in the following example, it actually stores everything in a line

```
import dask.bag as db

db.from_sequence(range(10), 10) \
    .map(str) \
    .to_textfiles('foo.*.txt')
```

and the output file reads

```
0123456789
```

After some tinkering, replacing this loop in the `write()` function (in `dask/bag/core.py`) from

```
        for line in data:
            f.write(line.encode(encoding))
```

to

```
        for i, line in enumerate(data):
            f.write((line if i == 0 else '\n{}'.format(line)).encode(encoding))
```

seems to work as mentioned in the documentation



Discussion:
When I run this code:

``` python
In [1]: import dask.bag as db
In [2]: db.from_sequence(range(10), npartitions=10).map(str).to_textfiles('foo.*.txt')
```

I get files like these:

```
mrocklin@notebook:~/tmp/foo$ ls
foo.0.txt  foo.2.txt  foo.4.txt  foo.6.txt  foo.8.txt
foo.1.txt  foo.3.txt  foo.5.txt  foo.7.txt  foo.9.txt
```

Where each file has just one line because each partition has just one element.

```
$ cat foo.0.txt
0
```

This seems correct to me.

However, if I do this same example with fewer partitions

``` python
In [4]: db.from_sequence(range(10), npartitions=3).map(str).to_textfiles('foo.*.txt')
```

Then I find that my files lack endlines, as you suggest

```
$ cat foo.0.txt 
0123
```

So perhaps we should map a function that ensures text has endlines

``` python
def ensure_endline(line):
    return line if line.endswith(os.linesep) else line + os.linesep
```


Resolving patch:
diff --git a/dask/bag/core.py b/dask/bag/core.py
--- a/dask/bag/core.py
+++ b/dask/bag/core.py
@@ -165,10 +165,12 @@ def get_compression(path, compression=compression):
     dsk = dict(((name, i), (write, (b.name, i), path, get_compression(path),
                             encoding))
                for i, path in enumerate(paths))
-    if compute:
-        return Bag(merge(b.dask, dsk), name, b.npartitions).compute()
 
-    return Bag(merge(b.dask, dsk), name, b.npartitions)
+    result = Bag(merge(b.dask, dsk), name, b.npartitions)
+    if compute:
+        result.compute()
+    else:
+        return result
 
 
 def finalize(results):
@@ -1066,9 +1068,23 @@ def write(data, filename, compression, encoding):
             os.makedirs(dirname)
 
     f = open(filename, mode='wb', compression=compression)
+
+    # Check presence of endlines
+    data = iter(data)
+    firstline = next(data)
+    if not (firstline.endswith(os.linesep) or firstline.endswith('\n')):
+        sep = os.linesep if firstline.endswith(os.linesep) else '\n'
+        firstline = firstline + sep
+        data = (line + sep for line in data)
+    f.write(firstline.encode(encoding))
+
     try:
+        lastline = ''
         for line in data:
-            f.write(line.encode(encoding))
+            f.write(lastline.encode(encoding))
+            lastline = line
+        f.write(lastline.rstrip(os.linesep).encode(encoding))
+
     finally:
         f.close()
 


Test patch:
diff --git a/dask/bag/tests/test_bag.py b/dask/bag/tests/test_bag.py
--- a/dask/bag/tests/test_bag.py
+++ b/dask/bag/tests/test_bag.py
@@ -727,6 +727,15 @@ def test_to_textfiles_inputs():
     assert raises(ValueError, lambda: B.to_textfiles(5))
 
 
+def test_to_textfiles_endlines():
+    b = db.from_sequence(['a', 'b', 'c'], npartitions=1)
+    with tmpfile() as fn:
+        b.to_textfiles([fn])
+        with open(fn, 'r') as f:
+            result = f.readlines()
+        assert result == ['a\n', 'b\n', 'c']
+
+
 def test_bz2_stream():
     text = '\n'.join(map(str, range(10000)))
     compressed = bz2.compress(text.encode())