Repository: biolab/orange3

Issue:
OWPredictions: support two sparse inputs.
** Is your feature request related to a problem? Please describe. **
I created a model to cluster items, but when imputing new data is presenting an error.
TypeError: ufunc 'isnan' not supported for input types, and inputs could not be safely coerced to any supported types according to casting rule '' safe ''


** Describe the solution you would like **
Let the predictive model work

** Describe the alternatives you considered **
None

** Additional Context **
I am a beginner and I am looking to learn a little more of this tool. I used google translation, so I apologize for any errors.

----

Traceback (most recent call last):
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\canvas\scheme\widgetsscheme.py", line 1082, in process_signals_for_widget
    widget.handleNewSignals()
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\widgets\evaluate\owpredictions.py", line 262, in handleNewSignals
    self._call_predictors()
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\widgets\evaluate\owpredictions.py", line 279, in _call_predictors
    pred, prob = self.predict(slot.predictor, self.data)
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\widgets\evaluate\owpredictions.py", line 618, in predict
    return cls.predict_discrete(predictor, data)
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\widgets\evaluate\owpredictions.py", line 625, in predict_discrete
    return predictor(data, Model.ValueProbs)
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\classification\base_classification.py", line 22, in __call__
    prediction = super().__call__(data, ret=ret)
  File "C:\Users\55119\AppData\Local\Orange\lib\site-packages\Orange\base.py", line 236, in __call__
    and not np.isnan(data.X).all():
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

OrderedDict([('data',
              [[24x1=1, 95=1, ex=1, excess=1, lev=1, ...] {- PNEU 24X1.95 EXCESS EX PR LEV},
 [1=1, 26x1=1, 2x2=1, manga=1, pneu=1, ...] {- PNEU 26X1.1/2X2 MANGA TURBO},
 [1=1, 26x1=1, 2x2=1, 91=1, k=1, ...] {- PNEU 26X1.1/2X2 K 91},
 [26x1=1, 95=1, levorim=1, pneu=1] {LEVORIM - PNEU 26X1.95},
 [26x1=1, 816=1, 95=1, k=1, pneu=1, ...] {- PNEU 26X1.95 K 816},
 ...
]),
             ('fix_dim',
              <function Model.__call__.<locals>.fix_dim at 0x00000146A8394F28>),
             ('one_d', False),
             ('ret', 2),
             ('self',
              LogisticRegressionClassifier(skl_model=LogisticRegression(C=1, class_weight=None, dual=False, fit_intercept=True,
                   intercept_scaling=1.0, l1_ratio=None, max_iter=100,
                   multi_class='ovr', n_jobs=1, penalty='l2', random_state=None,
                   solver='liblinear', tol=0.0001, verbose=0, warm_start=False))  # params={'penalty': 'l2', 'dual': False, 'tol': 0.0001, 'C': 1, 'fit_intercept': True, 'intercept_scaling': 1.0, 'class_weight': None, 'random_state': None, 'solver': 'liblinear', 'max_iter': 100, 'multi_class': 'ovr', 'verbose': 0, 'n_jobs': 1})])



Discussion:
Could you please attach a screenshot of the workflow and perhaps some canonical data set with which it is possible to reproduce the issue (iris, housing, something from single cell data sets?).
Also please report your version of Orange.
Version: 3.22.0

![screen](https://user-images.githubusercontent.com/52385242/63854661-0a6fd000-c974-11e9-98bf-d7b7b782f3e5.png)


[pneu.xlsx](https://github.com/biolab/orange3/files/3550778/pneu.xlsx) (CORPUS)
[Tinta_S.xlsx](https://github.com/biolab/orange3/files/3550781/Tinta_S.xlsx) (CORPUS 1)


Nice find! This can be reproduced with any example where BoW is constructed for Corpus 1 and Corpus 2 separately.

On another note, this is not the correct (optimal) way of constructing a predictive modeling workflow for text. It should be like this:
<img width="517" alt="Screen Shot 2019-08-28 at 14 26 49" src="https://user-images.githubusercontent.com/12524972/63855520-e32ef800-c99f-11e9-8f91-2b8ac2b85a8b.png">

Orange will automatically apply Preprocess Text and Bag of Words to the second data set. I tried this with your data and it works (but of course in the prediction data you only have one class, so there's an error for that).

On another note, Corpus should also enable setting the target variable in the widget itself. I will write an issue for this.
To reproduce for the others. This fails:
<img width="523" alt="Screen Shot 2019-08-28 at 14 30 36" src="https://user-images.githubusercontent.com/12524972/63855735-6b150200-c9a0-11e9-9466-8f0891d07a3f.png">
Use any dataset from Corpus.
Thank you for the tips.
As I understand it, is there a mistake or am I misused?
So the crash should not happen, that is a bug.
But your workflow should be like this:
<img width="517" alt="Screen Shot 2019-08-28 at 14 26 49" src="https://user-images.githubusercontent.com/12524972/63856885-f8f1ec80-c9a2-11e9-8d5c-2635634d095c.png">
It is a better way of predicting on new data.
OK thank you

Resolving patch:
diff --git a/Orange/base.py b/Orange/base.py
--- a/Orange/base.py
+++ b/Orange/base.py
@@ -2,7 +2,6 @@
 import itertools
 from collections import Iterable
 import re
-import threading
 
 import numpy as np
 import scipy
@@ -13,6 +12,7 @@
 from Orange.data.util import one_hot
 from Orange.misc.wrapper_meta import WrapperMeta
 from Orange.preprocess import Continuize, RemoveNaNColumns, SklImpute, Normalize
+from Orange.statistics.util import all_nan
 from Orange.util import Reprable
 
 __all__ = ["Learner", "Model", "SklLearner", "SklModel",
@@ -320,9 +320,9 @@ def data_to_model_domain():
 
             if self.original_domain.attributes != data.domain.attributes \
                     and data.X.size \
-                    and not np.isnan(data.X).all():
+                    and not all_nan(data.X):
                 new_data = data.transform(self.original_domain)
-                if np.isnan(new_data.X).all():
+                if all_nan(new_data.X):
                     raise DomainTransformationError(
                         "domain transformation produced no defined values")
                 return new_data.transform(self.domain)
diff --git a/Orange/statistics/util.py b/Orange/statistics/util.py
--- a/Orange/statistics/util.py
+++ b/Orange/statistics/util.py
@@ -659,6 +659,32 @@ def any_nan(x, axis=None):
     return result
 
 
+def all_nan(x, axis=None):
+    """
+    Check if all of the values in a matrix is nan. Works for sparse matrix too.
+    """
+    if not sp.issparse(x):
+        return np.isnan(x).all(axis=axis)
+
+    if axis is None:
+        # when x.nnz < actual shape there are zero values which are not nan
+        return np.prod(x.shape) == x.nnz and np.isnan(x.data).all()
+
+    if axis == 0:
+        x = x.tocsc()
+    elif axis == 1:
+        x = x.tocsr()
+
+    ax = x.ndim - axis - 1
+    axis_len = x.shape[axis]
+    result = np.zeros(x.shape[ax], dtype=bool)
+    for i in range(x.shape[ax]):
+        vals = x.data[x.indptr[i]:x.indptr[i + 1]]
+        result[i] = axis_len == len(vals) and np.isnan(vals).all()
+
+    return result
+
+
 def FDR(p_values: Iterable, dependent=False, m=None, ordered=False) -> Iterable:
     """ `False Discovery Rate <http://en.wikipedia.org/wiki/False_discovery_rate>`_
         correction on a list of p-values.


Test patch:
diff --git a/Orange/tests/test_statistics.py b/Orange/tests/test_statistics.py
--- a/Orange/tests/test_statistics.py
+++ b/Orange/tests/test_statistics.py
@@ -11,7 +11,8 @@
 from Orange.data.util import assure_array_dense
 from Orange.statistics.util import bincount, countnans, contingency, digitize, \
     mean, nanmax, nanmean, nanmedian, nanmin, nansum, nanunique, stats, std, \
-    unique, var, nanstd, nanvar, nanmode, nan_to_num, FDR, isnan, any_nan
+    unique, var, nanstd, nanvar, nanmode, nan_to_num, FDR, isnan, any_nan, \
+    all_nan
 from sklearn.utils import check_random_state
 
 
@@ -730,6 +731,51 @@ def test_axis_1_with_nans(self, array):
         np.testing.assert_equal(result, expected)
 
 
+class TestAllNans(unittest.TestCase):
+    def setUp(self) -> None:
+        # pylint: disable=bad-whitespace
+        self.x_with_nans = np.array([
+            [0.,     1.,     0.,     np.nan, 3.,     5.],
+            [np.nan, np.nan, np.nan, np.nan, np.nan, np.nan],
+            [0.,     0.,     0.,     np.nan, 7.,     6.],
+            [0.,     0.,     0.,     np.nan, 7.,     np.nan],
+        ])
+        self.x_no_nans = np.arange(12).reshape((3, 4))
+        self.x_only_nans = (np.ones(12) * np.nan).reshape((3, 4))
+
+    @dense_sparse
+    def test_axis_none_without_nans(self, array):
+        self.assertFalse(all_nan(array(self.x_no_nans)))
+
+    @dense_sparse
+    def test_axis_none_with_nans(self, array):
+        self.assertTrue(all_nan(array(self.x_only_nans)))
+
+    @dense_sparse
+    def test_axis_0_without_nans(self, array):
+        expected = np.array([0, 0, 0, 0], dtype=bool)
+        result = all_nan(array(self.x_no_nans), axis=0)
+        np.testing.assert_equal(result, expected)
+
+    @dense_sparse
+    def test_axis_0_with_nans(self, array):
+        expected = np.array([0, 0, 0, 1, 0, 0], dtype=bool)
+        result = all_nan(array(self.x_with_nans), axis=0)
+        np.testing.assert_equal(result, expected)
+
+    @dense_sparse
+    def test_axis_1_without_nans(self, array):
+        expected = np.array([0, 0, 0], dtype=bool)
+        result = all_nan(array(self.x_no_nans), axis=1)
+        np.testing.assert_equal(result, expected)
+
+    @dense_sparse
+    def test_axis_1_with_nans(self, array):
+        expected = np.array([0, 1, 0, 0], dtype=bool)
+        result = all_nan(array(self.x_with_nans), axis=1)
+        np.testing.assert_equal(result, expected)
+
+
 class TestNanModeFixedInScipy(unittest.TestCase):
 
     @unittest.expectedFailure