Repository: mars-project/mars

Issue:
[BUG] Creating dataframe from identical tensor yields incorrect result
**Describe the bug**

When creating dataframe from identical tensor (for example, `mt.ones`) the result may be not correct, due to different _dataframe chunks_ may have the same key.

Under distributed settings (with `LocalClusterSession` or `WebSession`) it may raise an exception when `merge_chunks`.

**To Reproduce**
To help us reproducing this bug, please provide information below:
1. Your Python version
2. The version of Mars you use: master
3. Versions of crucial packages, such as numpy, scipy and protobuf
4. Full stack of the error.
5. Minimized code to reproduce the error.

```python
In [1]: from mars import dataframe as md
In [2]: from mars import tensor as mt
In [3]: md.DataFrame(mt.ones((4,4), chunk_size=2)).execute()
Out[3]:
     2    3    2    3
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
```

**Expected behavior**

```python
     0    1    2    3
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
2  1.0  1.0  1.0  1.0
3  1.0  1.0  1.0  1.0
```

**Additional context**
Add any other context about the problem here.

[BUG] Creating dataframe from identical tensor yields incorrect result
**Describe the bug**

When creating dataframe from identical tensor (for example, `mt.ones`) the result may be not correct, due to different _dataframe chunks_ may have the same key.

Under distributed settings (with `LocalClusterSession` or `WebSession`) it may raise an exception when `merge_chunks`.

**To Reproduce**
To help us reproducing this bug, please provide information below:
1. Your Python version
2. The version of Mars you use: master
3. Versions of crucial packages, such as numpy, scipy and protobuf
4. Full stack of the error.
5. Minimized code to reproduce the error.

```python
In [1]: from mars import dataframe as md
In [2]: from mars import tensor as mt
In [3]: md.DataFrame(mt.ones((4,4), chunk_size=2)).execute()
Out[3]:
     2    3    2    3
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
```

**Expected behavior**

```python
     0    1    2    3
0  1.0  1.0  1.0  1.0
1  1.0  1.0  1.0  1.0
2  1.0  1.0  1.0  1.0
3  1.0  1.0  1.0  1.0
```

**Additional context**
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/mars/dataframe/datasource/from_tensor.py b/mars/dataframe/datasource/from_tensor.py
--- a/mars/dataframe/datasource/from_tensor.py
+++ b/mars/dataframe/datasource/from_tensor.py
@@ -88,6 +88,7 @@ def tile(cls, op):
             out_op = op.copy().reset_key()
             if in_chunk.ndim == 1:
                 i, = in_chunk.index
+                column_stop = 1
                 index = (in_chunk.index[0], 0)
                 columns_value = parse_index(out_df.columns.to_pandas()[0:1], store_data=True)
             else:
@@ -104,9 +105,10 @@ def tile(cls, op):
             else:
                 index_value = parse_index(pd.RangeIndex(start=index_stop - in_chunk.shape[0], stop=index_stop))
 
+            out_op.extra_params['index_stop'] = index_stop
+            out_op.extra_params['column_stop'] = column_stop
             out_chunk = out_op.new_chunk([in_chunk], shape=in_chunk.shape, index=index,
-                                         index_value=index_value,
-                                         columns_value=columns_value)
+                                         index_value=index_value, columns_value=columns_value)
             out_chunks.append(out_chunk)
 
         new_op = op.copy()


Test patch:
diff --git a/mars/dataframe/datasource/tests/test_datasource_execution.py b/mars/dataframe/datasource/tests/test_datasource_execution.py
--- a/mars/dataframe/datasource/tests/test_datasource_execution.py
+++ b/mars/dataframe/datasource/tests/test_datasource_execution.py
@@ -85,6 +85,13 @@ def testFromTensorExecution(self):
         pdf_expected = pd.DataFrame(np.array([1, 2, 3]))
         pd.testing.assert_frame_equal(pdf_expected, result3)
 
+        # test converted from identical chunks
+        tensor4 = mt.ones((10, 10), chunk_size=3)
+        df4 = from_tensor(tensor4)
+        result4 = self.executor.execute_dataframe(df4, concat=True)[0]
+        pdf_expected = pd.DataFrame(self.executor.execute_tensor(tensor4, concat=True)[0])
+        pd.testing.assert_frame_equal(pdf_expected, result4)
+
     def testFromRecordsExecution(self):
         dtype = np.dtype([('x', 'int'), ('y', 'double'), ('z', '<U16')])