Repository: biolab/orange3

Issue:
Save Data widget - can not same Excel files when unknows are present
When I use the SAVE DATA widget, this does not respond, it doesn't save anything and doesn't export any spreadsheet or any format that it's supposed to be generated by using this widget. I have removed the program and re-installed it, also I have tried to use the widget in other terminals. Didn't work. 
ORANGE VERSION 3.35


Discussion:
Thanks for the report. Unfortunately it is insufficient; we cannot reproduce the problem. What does your workflow look like?

Could you try with just two widgets? Take the File widget, connect it to Save Data, load the Iris data set (in File) and try saving it. What happens?
![imagen](https://github.com/biolab/orange3/assets/141933056/c5e73afb-c2f8-4016-8a22-fed5c22f1996)
Thanks, Janezd, above you'll find my workflow. When I did it with two widgets it worked. I could save the data in a spreadsheet. Nevertheless, when I attempt to run the above workflow, it doesn't work. 
Huh, this is too huge. If the data in FINAL PREDICTIONS looks OK, I don't see how it would mess with Save Data.

Could you try connecting Save Data directly to Select Columns, without having Data Table (FINAL PREDICTIONS) in between?
I don't see it either. 
As you requested, directly applied the SAVE DATA widget to Select Columns. It didn't work, no file was generated.
![imagen](https://github.com/biolab/orange3/assets/141933056/a2d9b32b-c0f5-4438-b24f-e4e50b8f617e)
Additionally, I already tried to save that table at the very beginning of the workflow, didn't work. (Upper left side of the previous image).
> Additionally, I already tried to save that table at the very beginning of the workflow, didn't work. (Upper left side of the previous image).

You mean, you connected Save to File at the top left? If so, can you share this data? Or at least tell us whether there's anything peculiar in it?

Otherwise, can you save the data from Main Table?

Can you construct a minimal workflow with this problem? Just File -> Gradient Boosting -> Predictions -> Save and another File -> Predictions?

What happens if you connect File -> Predictions instead of Main Table -> Prediction?
-"You mean, you connected Save to File at the top left? If so, can you share this data? Or at least tell us whether there's anything peculiar in it?"
Yep, I meant I connected FILE-->DATA TABLE-->SAVE DATA. Didn't work either (Image below). I'd love to share this data. Sadly it is kind of confidential due to corporate secrecy. Anyway, this data is just mainly, samples and their respective chemical data and descriptive data. The file is composed of nearly 341 instances, 59 features and target with 9 values. Definitely, nothing peculiar in it.
![imagen](https://github.com/biolab/orange3/assets/141933056/fca2cc56-ed0e-49ef-a333-de3013741ad9)

-"Otherwise, can you save the data from Main Table?"
I think this question is answered above.

-"Can you construct a minimal workflow with this problem? Just File -> Gradient Boosting -> Predictions -> Save and another File -> Predictions?"
Necessarily, I have to use these two sets of data (training and prediction data) to have an output from the prediction model.
However, I followed your instructions to re-arrange the workflow. It didn't generate any output file
![imagen](https://github.com/biolab/orange3/assets/141933056/951b7626-69c1-4e4a-bead-152ec02b4c66)

-"What happens if you connect File -> Predictions instead of Main Table -> Prediction?"
I think this is what you meant. It didn't generate any output file.
![imagen](https://github.com/biolab/orange3/assets/141933056/fbc0a7df-d075-4880-9abf-1f9ceca66830)







Can you try this with a different data set, say one that comes with Orange? So that we see whether it's related to data set or your setup?

When you say it doesn't export anything in any format, I assume it opens the file dialog, you specify the directory and the name, but file is not written? Could there be anything related to your file? Any non-ASCII characters anywhere in the path name, like /Users/mó/data. (Orange should handle this, but one never knows.)
I tried using the IRIS data set in a simplified workflow, nevertheless, with the same aim and the same prediction model. The "Save data" widget worked and generated a CSV. 
I tried to mimic the original  workflow in a simplified way, but this time with my data (the former one. The same that I was usising in the above workflow) and It worked. Finally, I  could generate a CSV file. 
![imagen](https://github.com/biolab/orange3/assets/141933056/44f02d97-a614-47e2-9470-f56614e533f8)


**My question is: Why isn't the initial workflow working? Is it due it has a lot of widgets? because when I reduced the length, it worked. These are my first steps in Orange, therefore I'll be grateful for any feedback you can deliver Janezd.**

> ![imagen](https://user-images.githubusercontent.com/141933056/259816522-c5e73afb-c2f8-4016-8a22-fed5c22f1996.png)

The number of widgets should have no effect on saving. Based on what I see, it is impossible to tell why it didn't work. If the Table widget can show the data, the Save Table should be able to save it.
Hello,

I'm new here and have kind of the same issue.
The strange thing is if I save the data as .CSV a file is created. When I save as an .XLSX no file is created?
I added a primary Save Data (test) to one of the input files and that one does save .XLSX as .CSV files.

Anyone an idea what I'm doing wrong or is this a bug?
![afbeelding](https://github.com/biolab/orange3/assets/55947148/84465abd-40b3-4fef-ae09-7b4793392504)

I'm using the latest version 3.36.1 on a Windows 10 pc.

Thanx!

[Hubo_winkels_NL.xlsx](https://github.com/biolab/orange3/files/13071841/Hubo_winkels_NL.xlsx)
[postcode_geo_locatie.xlsx](https://github.com/biolab/orange3/files/13071843/postcode_geo_locatie.xlsx)

PS. how do I upload the Orange .OWS file?
> PS. how do I upload the Orange .OWS file?

Ah .ZIP is supported ;)
[13 widgets uit Hubo data.zip](https://github.com/biolab/orange3/files/13072215/13.widgets.uit.Hubo.data.zip)

I think I found out what is wrong.
Apparently it is impossible to write data to Excel if not all data is available (nan in the Data Table) where writing to text (CSV) doesn't have a problem with that.
![13 widgets uit Hubo data, postcode=nummer Data Table weergave_org](https://github.com/biolab/orange3/assets/55947148/48909edc-f493-489a-a41f-7253ccb6999b)

So the solution (in my case) is to filter all missing data out (using Select Columns) or make the missing data available in some other way.
![Hubo data 11) Data Table (centroids), postcode=nr](https://github.com/biolab/orange3/assets/55947148/db474eda-63e8-4bff-8571-95df1ac1ed0d)

Problem solved, the Save Data to Excel works fine now!

@C-Denninger, thank you, this is it! A proper fix is on the way.

Resolving patch:
diff --git a/Orange/data/io_base.py b/Orange/data/io_base.py
--- a/Orange/data/io_base.py
+++ b/Orange/data/io_base.py
@@ -15,6 +15,7 @@
 from glob import glob
 
 import numpy as np
+import pandas
 
 from Orange.data import Table, Domain, Variable, DiscreteVariable, \
     StringVariable, ContinuousVariable, TimeVariable
@@ -665,7 +666,7 @@ def formatter(cls, var):
         elif var.is_discrete:
             return lambda value: "" if isnan(value) else var.values[int(value)]
         elif var.is_string:
-            return lambda value: value
+            return lambda value: "" if pandas.isnull(value) else value
         else:
             return var.repr_val
 


Test patch:
diff --git a/Orange/data/tests/test_io.py b/Orange/data/tests/test_io.py
--- a/Orange/data/tests/test_io.py
+++ b/Orange/data/tests/test_io.py
@@ -122,7 +122,7 @@ def setUp(self):
             self.domain,
             np.array([[1, 0.5], [2, np.nan], [np.nan, 1.0625]]),
             np.array([3, 1, 7]),
-            np.array(["foo bar baz".split()]).T
+            np.array([["foo", "bar", np.nan]], dtype=object).T
         )
 
     def test_write_tab(self):
@@ -137,7 +137,7 @@ def test_write_tab(self):
 class\tmeta\t\t
 3\tfoo\ty\t0.500
 1\tbar\tz\t
-7\tbaz\t\t1.06250""".strip())
+7\t\t\t1.06250""".strip())
         finally:
             os.remove(fname)
 
@@ -149,7 +149,8 @@ def test_roundtrip_xlsx(self):
             data = ExcelReader(fname).read()
             np.testing.assert_equal(data.X, self.data.X)
             np.testing.assert_equal(data.Y, self.data.Y)
-            np.testing.assert_equal(data.metas, self.data.metas)
+            np.testing.assert_equal(data.metas[:2], self.data.metas[:2])
+            self.assertEqual(data.metas[2, 0], "")
             np.testing.assert_equal(data.domain, self.data.domain)
         finally:
             os.remove(fname)