Repository: biolab/orange3

Issue:
Widget Save data not working propperly
<!-- 
Thanks for taking the time to report a bug!
If you're raising an issue about an add-on (i.e., installed via Options > Add-ons), raise an issue in the relevant add-on's issue tracker instead. See: https://github.com/biolab?q=orange3
To fix the bug, we need to be able to reproduce it. Please answer the following questions to the best of your ability.
-->


**What's wrong?**

When push "save as" nothing happens or sometimes says "it is not responding"
<!-- Be specific, clear, and concise. Include screenshots if relevant. -->
<!-- If you're getting an error message, copy it, and enclose it with three backticks (```). -->





**How can we reproduce the problem?** 
<!-- Upload a zip with the .ows file and data. -->
<!-- Describe the steps (open this widget, click there, then add this...) -->





**What's your environment?**
<!-- To find your Orange version, see "Help → About → Version" or `Orange.version.full_version` in code -->
- Operating system: windows 10
- Orange version: 3.29.3
- How you installed Orange: on orange website --> windows version



Discussion:
Thanks for the report, but we'll need more. What are the circumstances in which it happenned, what kind of data did you use and, most importantly, how can we replicate this?
Hi:

 

Please see attached files. CSV file is a list of 536 abstracts from scopus.

 

I am still an orange beginner. Then maybe I am trying to do something that is not possible.

 

The error occurs when I try to get text data as the simple workflow will show you. But this error happened with other workflow as well, when using tree methods in heart csv file.

 

Thanks for your time,

 

Gilson 

 

 

De: Janez Demšar ***@***.***> 
Enviada em: sábado, 3 de julho de 2021 16:09
Para: biolab/orange3 ***@***.***>
Cc: ProfOliveira ***@***.***>; Author ***@***.***>
Assunto: Re: [biolab/orange3] Widget Save data not working propperly (#5512)

 

Thanks for the report, but we'll need more. What are the circumstances in which it happenned, what kind of data did you use and, most importantly, how can we replicate this?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <https://github.com/biolab/orange3/issues/5512#issuecomment-873456153> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUWUXWCRYWWA4ZKDTALAETLTV5N4BANCNFSM47XKFGOA> .  <https://github.com/notifications/beacon/AUWUXWCA4X3PHK6I5OEBSGDTV5N4BA5CNFSM47XKFGOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGQH6EGI.gif> 



-- 
Este email foi escaneado pelo Avast antivírus.
https://www.avast.com/antivirus

@ProfOliveira There's nothing attached. It doesn't work if you are replying by email, you'll have to submit the workflow directly via Github.

[workflow and csv file.zip](https://github.com/biolab/orange3/files/6765875/workflow.and.csv.file.zip)



When running this workflow, I get the following error:
```
Traceback (most recent call last):
  File "/Users/ajda/orange/orange3/Orange/widgets/utils/save/owsavebase.py", line 206, in save_file
    self.save_file_as()
  File "/Users/ajda/orange/orange3/Orange/widgets/utils/save/owsavebase.py", line 192, in save_file_as
    filename, selected_filter = self.get_save_filename()
  File "/Users/ajda/orange/orange3/Orange/widgets/utils/save/owsavebase.py", line 333, in get_save_filename
    filename = self.initial_start_dir()
  File "/Users/ajda/orange/orange3/Orange/widgets/data/owsave.py", line 125, in initial_start_dir
    data_name += self.writer.EXTENSIONS[0]
  File "/Users/ajda/orange/orange3/Orange/widgets/utils/save/owsavebase.py", line 167, in writer
    return self.get_filters()[self.filter]
KeyError: 'Microsoft Excel 97-2004 spreadsheet (*.xls)'
```

🤔 
Is this some old workflow? To my knowledge, Orange dropped support for xls in 2019. Please choose another format.

Crashing is still no OK. It may also occur if some add-on provides a new format and a workflow is used in installation without that add-on. In this case, the widget should indicate an error (with a red icon etc).
Hi guys! Sorry I didn´t updated . It is sorted. I had an old version installed on anaconda. Maybe it was running in background (not an experte sorry if it is bullshit). The point is after excluding anaconda it worked.
Thanks for this update.

I think this issue still needs resolving -- the widget should properly indicate that the file format is no longer supported.
Note for discussion on Friday: I tried to resolve this, but the widget (and in particular its derived classes -- all save widgets inherit from a common base) assume that the writer exists, and do so in many different places. Setting the error is not a problem, but clearing is. Since such writers can only come from stored settings, it would be simpler to silently switch to an existing format.
Sorry prof. Demšar:

 

I did not want to be inconvenient. In fact, I just closed the discussion because  I got ashamed of posting something that it was sorted with a simple uninstall of Anaconda.

 

About the tone I used in the closing message, please accept a second (and strong) apology! I believed it was ok to do so, and I did not intend to be unpolite.

 

I work with quantitative methods (especially statistics) over all my academic career, and orange opened new opportunities for me. As I usually joke, I am an old young doctor. I reorganized my two subjects in the master course, and now I am planning to use Orange in my papers as well. It seems orange use in academia can be better explored in my research field.

 

Therefore, I am very (very) thankful for all orange developers.

 

“Setting the error is not a problem, but clearing is.” : this is amazingly true! Please: if you really need more details I can try to recall all my user´s history:

 

i)                 I installed orange in early 2020 and used it since then;

ii)                Because of the pandemic, I started to study python or R. I am a very busy person (and relatively old --> 55). Some weeks ago, I gave up on coding;

iii)              Before “ii” I installed some other open sources solutions, all based on R (JASP, JAMOVI, and Bluesky). I am still using they are useful open sources solutions to me;

iv)              During my vacation in early 2021, I studied python by myself and I decide to install anaconda;

v)                As I saw there was an orange version on anaconda, I thought it would be practical to use it on anaconda;

vi)              Later I realized the version of orange inside anaconda was an old one, and then I uninstalled it;

vii)             After preparing a lecture (using orange), the issue occurred. Maybe it is a coincidence but it was after I updated my orange;

viii)           The remaining you know. I posted on Github. I am not a programmer, but if you need, please ask for any additional information, I will do my best to try to accomplish it.

 

The above are all the details I have to tell you.

 

Ps. Currently, I am using Orange for text mining in literature reviews. My focus is topic modelling, and I am struggling to solve the paradigm of using TM without determining the optimal number of topics. 

 

Again, Orange and similar approaches (as Knime) are amazing. I am your fan!

 

Congratulations on your remarkable and inclusive initiative,

 

Best regards,

 

Gilson Adamczuk Oliveira – Full Professor at UTFPR

 

From: Janez Demšar ***@***.***> 
Sent: domingo, 22 de agosto de 2021 12:10
To: biolab/orange3 ***@***.***>
Cc: ProfOliveira ***@***.***>; State change ***@***.***>
Subject: Re: [biolab/orange3] Widget Save data not working propperly (#5512)

 

Note for discussion on Friday: I tried to resolve this, but the widget (and in particular its derived classes -- all save widgets inherit from a common base) assume that the writer exists, and do so in many different places. Setting the error is not a problem, but clearing is. Since such writers can only come from stored settings, it would be simpler to silently switch to an existing format.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub <https://github.com/biolab/orange3/issues/5512#issuecomment-903283316> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUWUXWFNWGOT45OLNXC5Y4DT6EHN3ANCNFSM47XKFGOA> .
Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>  or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .  <https://github.com/notifications/beacon/AUWUXWAQ3R3QNHHAWF6M4TTT6EHN3A5CNFSM47XKFGOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGXLQE5A.gif> 



-- 
Este email foi escaneado pelo Avast antivírus.
https://www.avast.com/antivirus

There must had been some misunderstanding: I see no reason why you should apologize at all. All your posts were just fine. We are also happy if users close issues themselves, and there's no problem if we need to reopen it when we feel that there's some more work to be done.

By setting an error and clearing it I meant the coding perspective. Orange should indicated to the user (with red indicator and so forth) that the widget is trying to use an unsupported format. In the code, there is a perfect place to show this indicator, but no good place to remove the indicator when the problem is resolved. It was just a note for our discussion about how to actually program this.

Resolving patch:
diff --git a/Orange/widgets/data/owsave.py b/Orange/widgets/data/owsave.py
--- a/Orange/widgets/data/owsave.py
+++ b/Orange/widgets/data/owsave.py
@@ -67,6 +67,9 @@ def dataset(self, data):
         self.on_new_input()
 
     def do_save(self):
+        if self.writer is None:
+            super().do_save()  # This will do nothing but indicate an error
+            return
         if self.data.is_sparse() and not self.writer.SUPPORT_SPARSE_DATA:
             return
         self.writer.write(self.filename, self.data, self.add_type_annotations)
@@ -75,7 +78,8 @@ def update_messages(self):
         super().update_messages()
         self.Error.unsupported_sparse(
             shown=self.data is not None and self.data.is_sparse()
-            and self.filename and not self.writer.SUPPORT_SPARSE_DATA)
+            and self.filename
+            and self.writer is not None and not self.writer.SUPPORT_SPARSE_DATA)
 
     def send_report(self):
         self.report_data_brief(self.data)
@@ -83,9 +87,9 @@ def send_report(self):
         noyes = ["No", "Yes"]
         self.report_items((
             ("File name", self.filename or "not set"),
-            ("Format", writer.DESCRIPTION),
+            ("Format", writer and writer.DESCRIPTION),
             ("Type annotations",
-             writer.OPTIONAL_TYPE_ANNOTATIONS
+             writer and writer.OPTIONAL_TYPE_ANNOTATIONS
              and noyes[self.add_type_annotations])
         ))
 
@@ -122,6 +126,9 @@ def initial_start_dir(self):
         else:
             data_name = getattr(self.data, 'name', '')
             if data_name:
+                if self.writer is None:
+                    self.filter = self.default_filter()
+                assert self.writer is not None
                 data_name += self.writer.EXTENSIONS[0]
             return os.path.join(self.last_dir or _userhome, data_name)
 
diff --git a/Orange/widgets/utils/save/owsavebase.py b/Orange/widgets/utils/save/owsavebase.py
--- a/Orange/widgets/utils/save/owsavebase.py
+++ b/Orange/widgets/utils/save/owsavebase.py
@@ -43,6 +43,7 @@ class Information(widget.OWWidget.Information):
 
     class Error(widget.OWWidget.Error):
         no_file_name = widget.Msg("File name is not set.")
+        unsupported_format = widget.Msg("File format is unsupported.\n{}")
         general_error = widget.Msg("{}")
 
     want_main_area = False
@@ -80,7 +81,7 @@ def __init__(self, start_row=0):
 
         # This cannot be done outside because `filters` is defined by subclass
         if not self.filter:
-            self.filter = next(iter(self.get_filters()))
+            self.filter = self.default_filter()
 
         self.grid = grid = QGridLayout()
         gui.widgetBox(self.controlArea, orientation=grid, box=True)
@@ -98,6 +99,10 @@ def __init__(self, start_row=0):
         self.adjustSize()
         self.update_messages()
 
+    def default_filter(self):
+        """Returns the first filter in the list"""
+        return next(iter(self.get_filters()))
+
     @property
     def last_dir(self):
         # Not the best name, but kept for compatibility
@@ -158,13 +163,20 @@ def get_filters(cls):
     @property
     def writer(self):
         """
-        Return the active writer
+        Return the active writer or None if there is no writer for this filter
 
         The base class uses this property only in `do_save` to find the writer
         corresponding to the filter. Derived classes (e.g. OWSave) may also use
         it elsewhere.
+
+        Filter may not exist if it comes from settings saved in Orange with
+        some add-ons that are not (or no longer) present, or if support for
+        some extension was dropped, like the old Excel format.
         """
-        return self.get_filters()[self.filter]
+        filters = self.get_filters()
+        if self.filter not in filters:
+            return None
+        return filters[self.filter]
 
     def on_new_input(self):
         """
@@ -194,6 +206,7 @@ def save_file_as(self):
             return
         self.filename = filename
         self.filter = selected_filter
+        self.Error.unsupported_format.clear()
         self.bt_save.setText(f"Save as {self.stored_name}")
         self.update_messages()
         self._try_save()
@@ -233,6 +246,9 @@ def do_save(self):
         a single format.
         """
         # This method is separated out because it will usually be overriden
+        if self.writer is None:
+            self.Error.unsupported_format(self.filter)
+            return
         self.writer.write(self.filename, self.data)
 
     def update_messages(self):


Test patch:
diff --git a/Orange/widgets/data/tests/test_owsave.py b/Orange/widgets/data/tests/test_owsave.py
--- a/Orange/widgets/data/tests/test_owsave.py
+++ b/Orange/widgets/data/tests/test_owsave.py
@@ -363,7 +363,7 @@ def test_migration_to_version_2(self):
         OWSave.migrate_settings(settings)
         self.assertTrue(settings["filter"] in OWSave.get_filters())
 
-        # Unsupported file format (is this possible?)
+        # Unsupported file format
         settings = {**const_settings,
                     'compress': True, 'compression': 'lzma (.xz)',
                     'filetype': 'Bar file (.bar)'}
@@ -401,6 +401,33 @@ def test_save_uncompressed(self):
                     if hasattr(writer, "read"):
                         self.assertEqual(len(writer(filename).read()), 150)
 
+    def test_unsupported_file_format(self):
+        widget = self.create_widget(
+            OWSave,
+            stored_settings=dict(
+                filter="Unsupported filter (*.foo)", stored_name="test.foo",
+                __version__=2)
+        )
+        filters = widget.get_filters()
+        def_filter = filters[widget.default_filter()]
+
+        iris = Table("iris")
+        self.send_signal(widget.Inputs.data, iris)
+
+        # With unsupported format from settings, the widget should indicate
+        # an error
+        with patch.object(def_filter, "write"):
+            widget.save_file()
+            self.assertTrue(widget.Error.unsupported_format.is_shown())
+            def_filter.write.assert_not_called()
+
+        # Without file name, if `filter` is set to unsupported format,
+        # initial_start_dir should choose the default filter
+        widget.stored_name = ""
+        widget.initial_start_dir()
+        self.assertIs(filters[widget.filter], def_filter)
+        self.assertIs(widget.writer, def_filter)
+
 
 @unittest.skipUnless(sys.platform == "linux", "Tests for dialog on Linux")
 class TestOWSaveLinuxDialog(OWSaveTestBase):
diff --git a/Orange/widgets/utils/save/tests/test_owsavebase.py b/Orange/widgets/utils/save/tests/test_owsavebase.py
--- a/Orange/widgets/utils/save/tests/test_owsavebase.py
+++ b/Orange/widgets/utils/save/tests/test_owsavebase.py
@@ -276,6 +276,40 @@ def test_save_button_label(self):
             stored_settings=dict(stored_path="", stored_name="c.foo"))
         self.assertTrue(w.bt_save.text().endswith(" c.foo"))
 
+    def test_invalid_filter(self):
+        writer = Mock()
+
+        class OWSaveNoWriter(OWSaveBase):
+            name = "Mock save"
+            writers = {}
+            filters = {"csv (*.csv)": writer}
+
+        w = self.create_widget(
+            OWSaveNoWriter,
+            stored_settings=dict(
+                filter="Unsupported format (*.foo)", stored_path='test.foo')
+        )
+        w.data = Mock()
+        self.assertIsNone(w.writer)
+
+        w.do_save()
+        self.assertTrue(w.Error.unsupported_format.is_shown())
+
+        name = "/home/u/orange/a/b/c.csv"
+        w.get_save_filename = Mock(return_value=(name, "csv (*.csv)"))
+        w.save_file_as()
+        self.assertFalse(w.Error.unsupported_format.is_shown())
+        call_name, _ = writer.write.call_args[0]
+        self.assertPathEqual(call_name, name)
+
+    def test_default_filter(self):
+        class OWSave(OWSaveBase):
+            name = "Mock save"
+            filters = {"csv (*.csv)": Mock(), "txt (*.txt)": Mock()}
+
+        widget = self.create_widget(OWSave)
+        self.assertEqual(widget.default_filter(), "csv (*.csv)")
+
 
 class TestOWSaveBase(WidgetTest):
     # Tests for OWSaveBase methods with filters as list
@@ -317,6 +351,14 @@ def test_base_methods(self):
         self.assertIs(widget.valid_filters(), widget.get_filters())
         self.assertIs(widget.default_valid_filter(), widget.filter)
 
+    def test_default_filter(self):
+        class OWSave(OWSaveBase):
+            name = "Mock save"
+            filters = ["csv (*.csv)", "txt (*.txt)"]
+
+        widget = self.create_widget(OWSave)
+        self.assertEqual(widget.default_filter(), OWSave.filters[0])
+
 
 class TestOWSaveUtils(unittest.TestCase):
     def test_replace_extension(self):