Repository: mars-project/mars

Issue:
[BUG] `check_consistent_length` failed in Mars learn
<!--
Thank you for your contribution!

Please review https://github.com/mars-project/mars/blob/master/CONTRIBUTING.rst before opening an issue.
-->

**Describe the bug**

`check_consistent_length` failed in Mars learn.

**To Reproduce**
To help us reproducing this bug, please provide information below:
1. Your Python version
2. The version of Mars you use
3. Versions of crucial packages, such as numpy, scipy and protobuf
4. Full stack of the error.
5. Minimized code to reproduce the error.

Error message:

```
File "/Users/letao/PycharmProjects/mermaid/frameworks/feras/feras/backend/testmars.py", line 17, in test_1
    fpr, tpr, thresholds = metrics.roc_curve(y, pred, pos_label=2)
  File "/Users/letao/PycharmProjects/venv/lib/python3.7/site-packages/mars/learn/metrics/_ranking.py", line 277, in roc_curve
    session=session, run_kwargs=run_kwargs)
  File "/Users/letao/PycharmProjects/venv/lib/python3.7/site-packages/mars/learn/metrics/_ranking.py", line 128, in _binary_clf_curve
    check_consistent_length(y_true, y_score, sample_weight)
  File "/Users/letao/PycharmProjects/venv/lib/python3.7/site-packages/mars/learn/utils/validation.py", line 87, in check_consistent_length
    " samples: %r" % [int(l) for l in lengths])
  File "/Users/letao/PycharmProjects/venv/lib/python3.7/site-packages/mars/learn/utils/validation.py", line 87, in <listcomp>
    " samples: %r" % [int(l) for l in lengths])
ValueError: cannot convert float NaN to integer
```



Discussion:

Resolving patch:
diff --git a/mars/learn/utils/validation.py b/mars/learn/utils/validation.py
--- a/mars/learn/utils/validation.py
+++ b/mars/learn/utils/validation.py
@@ -25,6 +25,7 @@
     DataConversionWarning = UserWarning
 
 from ... import tensor as mt
+from ...core import ExecutableTuple
 from ...lib.sparse import issparse
 from ...tensor import Tensor
 from .checks import check_non_negative_then_return_value, \
@@ -80,7 +81,23 @@ def check_consistent_length(*arrays):
         Objects that will be checked for consistent length.
     """
 
-    lengths = [_num_samples(X) for X in arrays if X is not None]
+    new_arrays = []
+    lengths = []
+    to_execute = []
+    for X in arrays:
+        if X is not None:
+            n = _num_samples(X)
+            if np.isnan(n):
+                to_execute.append(X)
+            new_arrays.append(X)
+            lengths.append(n)
+    # unknown length exists
+    if len(to_execute) > 0:
+        # update shape
+        ExecutableTuple(to_execute).execute()
+        # get length again
+        lengths = [_num_samples(X) for X in new_arrays]
+
     uniques = np.unique(lengths)
     if len(uniques) > 1:
         raise ValueError("Found input variables with inconsistent numbers of"


Test patch:
diff --git a/mars/learn/utils/tests/test_validation.py b/mars/learn/utils/tests/test_validation.py
--- a/mars/learn/utils/tests/test_validation.py
+++ b/mars/learn/utils/tests/test_validation.py
@@ -31,7 +31,7 @@
     from sklearn.utils._testing import assert_raise_message, assert_raises_regex
     import pytest
 
-    from mars.learn.utils.validation import check_array
+    from mars.learn.utils.validation import check_array, check_consistent_length
 except ImportError:
     sklearn = None
 
@@ -245,3 +245,12 @@ def test_check_array_complex_data_error(_):
         X = sp.coo_matrix([[0, 1 + 2j], [0, 0]])
         assert_raises_regex(
             ValueError, "Complex data not supported", check_array, X)
+
+    def testCheckConsistentLength(self):
+        t = mt.random.RandomState(0).rand(10, 5)
+        t2 = t[t[:, 0] < 0.5]
+        t3 = t[t[:, 1] < 0.1]
+
+        check_consistent_length(t2, t2.copy())
+        with self.assertRaises(ValueError):
+            check_consistent_length(t2, t3)