Repository: biolab/orange3

Issue:
Multiple sheets in Excel Files
Orange does not support multiple sheets in Excel files. Add this functionality to the file widget. Don't ruin the GUI layout.



Discussion:
@janezd  i did a quick fix to select multiple sheets in Excel files in the FIle Widget 

```

    def browse_file(self, in_demos=False):
        if in_demos:
            try:
                start_file = get_sample_datasets_dir()
            except AttributeError:
                start_file = ""
            if not start_file or not os.path.exists(start_file):
                widgets_dir = os.path.dirname(gui.__file__)
                orange_dir = os.path.dirname(widgets_dir)
                start_file = os.path.join(orange_dir, "doc", "datasets")
            if not start_file or not os.path.exists(start_file):
                d = os.getcwd()
                if os.path.basename(d) == "canvas":
                    d = os.path.dirname(d)
                start_file = os.path.join(os.path.dirname(d), "doc", "datasets")
            if not os.path.exists(start_file):
                QtGui.QMessageBox.information(
                    None, "File",
                    "Cannot find the directory with example data sets")
                return
        else:
            if self.recent_paths:
                start_file = self.recent_paths[0].abspath
            else:
                start_file = os.path.expanduser("~/")

        filename = QtGui.QFileDialog.getOpenFileNames(
            self, 'Open Orange Data File', start_file, self.dlg_formats)
        if not filename:
            return

        self._add_path(filename)
        self.set_file_list()
        self.source = self.LOCAL_FILE
        self.load_data()

```

Make a PR so we can test it. Thanks!

Also be wary of quick fixes. In particular, make sure it works with Data Table (and its tabs functionality).

What Exactly should i do here . Should i make a gui for displaying the Multiple Excel files in Data Table ? 

GUI is already there. Data Table displays tabs when fed multiple data sets. Use this functionality to switch between Excel Sheets. Also implement an option to select which sheet to output in the File widget and Data Table widget. Think also about merging and concatenating data from different sheets.

@ajdapretnar 
Thanks ! 

@ajdapretnar  Please review this workflow . tell me if i am wrong !
1) select multiple excel files in file widget 
2) output of the file widget should be multiple excel files with switching tabs 

Basically We need to access multiple excel files  in Data Table with one instance of File widget and Data Table  right ?

The initial idea was to simply switch between the sheets. So if you have Sheet 1 and Sheet 2, can you switch between them in the Data Table?
Of course, it needs to be discussed where the selection would take place. Link editor would be one option, but what if the user has many sheets?
@janezd Would the sheet selection take place in the File widget or in later widgets? This seems a bit more complicated than we though... Many details to discuss here.

The idea is that you select the sheet in the File widget.

When the user opens an Excel file (a single file!) the File widget checks whether there are multiple (non-empty) sheets. If there is just one, it loads it just like now. If there are multiple, a new combo appears in the widget (well, the combo is always there, but hidden when it is not needed) that allows the user to select the sheet. The combo is populated with the sheet names and the first sheet is selected and loaded by default.

Furthermore, assume I select the third sheet. Then I close Orange and open it again: the third sheet must be selected. Also, if I load another file and select, say, the fifth sheet, but then load the first file again, the third sheet must again be selected. Therefore, the widget must remember the selected sheet for each file that was loaded in the past.

The latter is somewhat more difficult since it is accomplished by context settings. You can implement the rest first and then get some assistance with storing the settings.

I doing first half module first ! 
Thanks @janezd 

@janezd i was able to select multiple files with file widget with the auto save and remember file name . But i am facing  problem to create a new tab within the table widget > 
The problem is I am sending the output signals from filewidget(multiple files paths) to tablewidget that contains  but input of table widget is not handling that . I saw the testing of the widget it is succesfully creating multiple tabs with multiple Table() instances and save_dataset() Handling the est to create tabs. So How can i Use that in this case ? 

Please read my explanation above. We could, in principle, adapt the table widget to select a data set out of the list. But we would have to do the same for _all_ other widgets, too. We'd like to do exactly the opposite - move the selection of a sheet (from a single file) to the file widget.

@janezd I am not able to get the whole thing as I am new to this project but I want to learn through this issue  . It Would be nice if you please elaborate more  Thanks  :+1: 

> We'd like to do exactly the opposite - move the selection of a sheet (from a single file) to the file widget.

Sure.

As I wrote above, when the user opens an Excel file (a single file!) the File widget should check whether there are multiple (non-empty) sheets. If there is just one, it loads it just like now. If there are multiple, a new combo appears in the widget (well, the combo is always there, but hidden when it is not needed) that allows the user to select the sheet. The combo is populated with the sheet names and the first sheet is selected and loaded by default.

Furthermore, assume I select the third sheet. Then I close Orange and open it again: the third sheet must be selected. Also, if I load another file and select, say, the fifth sheet, but then load the first file again, the third sheet must again be selected. Therefore, the widget must remember the selected sheet for each file that was loaded in the past.

The latter is somewhat more difficult since it is accomplished by context settings. You can implement the rest first and then get some assistance with storing the settings.

@janezd For Mutiple sheets I need to embed xlrd module for fetching multiple sheets from a single file  . So that will change function from_file in Table instance in Orange.data.table.py which furthur depends on .io.py because Table take the path as args . [Event handling]  Later , further changes into combobox widgets will be reflected  according to sheet names in excel file (single file) 
Is it what we want ?

Don't fetch multiple sheets, fetch just their names, so you can show them in the combo. Loading multiple sheets may take too much time since sheets can be huge, and they can be in incorrect format or empty.

So, always load just the currently selected sheet. Method `Orange.data.io.ExcelFormat.read_file` already accept sheet as an argument; you can append the sheet name to the filename, using `:` as a separator (see https://github.com/biolab/orange3/blob/master/Orange/data/io.py#L658).

@janezd Totally Agreed ! 


Resolving patch:
diff --git a/Orange/widgets/data/owfile.py b/Orange/widgets/data/owfile.py
--- a/Orange/widgets/data/owfile.py
+++ b/Orange/widgets/data/owfile.py
@@ -1,21 +1,22 @@
 import os
 from warnings import catch_warnings
-
+from xlrd import open_workbook, XLRDError
 from PyQt4 import QtGui, QtCore
 from PyQt4.QtGui import QSizePolicy as Policy
 
 from Orange.widgets import widget, gui
-from Orange.widgets.settings import Setting
+from Orange.widgets.settings import Setting, ContextHandler, ContextSetting
 from Orange.widgets.utils.itemmodels import PyListModel
 from Orange.widgets.utils.filedialogs import RecentPathsWComboMixin
 from Orange.data.table import Table, get_sample_datasets_dir
-from Orange.data.io import FileFormat
+from Orange.data.io import FileFormat, ExcelFormat
 
 # Backward compatibility: class RecentPath used to be defined in this module,
 # and it is used in saved (pickled) settings. It must be imported into the
 # module's namespace so that old saved settings still work
 from Orange.widgets.utils.filedialogs import RecentPath
 
+
 def add_origin(examples, filename):
     """Adds attribute with file location to each variable"""
     vars = examples.domain.variables + examples.domain.metas
@@ -42,6 +43,22 @@ def add_name(self, url, name):
         self.modelReset.emit()
 
 
+class XlsContextHandler(ContextHandler):
+    def new_context(self, filename, sheet):
+        context = super().new_context()
+        context.filename = filename
+        return context
+
+    # noinspection PyMethodOverriding
+    def match(self, context, filename, sheets):
+        context_sheet = context.values.get("xls_sheet")
+        if context.filename == filename and context_sheet in sheets:
+            return ContextHandler.PERFECT_MATCH
+        if context_sheet in sheets:
+            return 1
+        return ContextHandler.NO_MATCH
+
+
 class OWFile(widget.OWWidget, RecentPathsWComboMixin):
     name = "File"
     id = "orange.widgets.data.file"
@@ -62,8 +79,10 @@ class OWFile(widget.OWWidget, RecentPathsWComboMixin):
 
     LOCAL_FILE, URL = range(2)
 
+    settingsHandler = XlsContextHandler()
     recent_urls = Setting([])
     source = Setting(LOCAL_FILE)
+    xls_sheet = ContextSetting("")
     sheet_names = Setting({})
     url = Setting("")
 
@@ -94,20 +113,42 @@ def __init__(self):
         self.file_combo.setSizePolicy(Policy.MinimumExpanding, Policy.Fixed)
         self.file_combo.activated[int].connect(self.select_file)
         box.layout().addWidget(self.file_combo)
-        button = gui.button(
-            box, self, '...', callback=self.browse_file, autoDefault=False)
-        button.setIcon(self.style().standardIcon(QtGui.QStyle.SP_DirOpenIcon))
-        button.setSizePolicy(Policy.Maximum, Policy.Fixed)
-        button = gui.button(
-            box, self, "Reload", callback=self.reload, autoDefault=False)
-        button.setIcon(self.style().standardIcon(QtGui.QStyle.SP_BrowserReload))
-        button.setSizePolicy(Policy.Fixed, Policy.Fixed)
-        layout.addWidget(box, 0, 1,  QtCore.Qt.AlignVCenter)
+        layout.addWidget(box, 0, 1)
+
+        file_button = gui.button(
+            None, self, '...', callback=self.browse_file, autoDefault=False)
+        file_button.setIcon(self.style().standardIcon(
+            QtGui.QStyle.SP_DirOpenIcon))
+        file_button.setSizePolicy(Policy.Maximum, Policy.Fixed)
+        layout.addWidget(file_button, 0, 2)
+
+        reload_button = gui.button(
+            None, self, "Reload", callback=self.reload, autoDefault=False)
+        reload_button.setIcon(self.style().standardIcon(
+            QtGui.QStyle.SP_BrowserReload))
+        reload_button.setSizePolicy(Policy.Fixed, Policy.Fixed)
+        layout.addWidget(reload_button, 0, 3)
+
+        self.sheet_box = gui.hBox(None, addToLayout=False, margin=0)
+        self.sheet_combo = gui.comboBox(None, self, "xls_sheet",
+                                        callback=self.load_data,
+                                        sendSelectedValue=True)
+        self.sheet_combo.setSizePolicy(
+            Policy.MinimumExpanding, Policy.Fixed)
+        self.sheet_label = QtGui.QLabel()
+        self.sheet_label.setText('Sheet')
+        self.sheet_label.setSizePolicy(
+            Policy.MinimumExpanding, Policy.Fixed)
+        self.sheet_box.layout().addWidget(
+            self.sheet_label, QtCore.Qt.AlignLeft)
+        self.sheet_box.layout().addWidget(
+            self.sheet_combo, QtCore.Qt.AlignVCenter)
+        layout.addWidget(self.sheet_box, 2, 1)
+        self.sheet_box.hide()
 
         rb_button = gui.appendRadioButton(vbox, "URL", addToLayout=False)
-        layout.addWidget(rb_button, 1, 0, QtCore.Qt.AlignVCenter)
+        layout.addWidget(rb_button, 3, 0, QtCore.Qt.AlignVCenter)
 
-        box = gui.hBox(vbox, addToLayout=False)
         self.url_combo = url_combo = QtGui.QComboBox()
         url_model = NamedURLModel(self.sheet_names)
         url_model.wrap(self.recent_urls)
@@ -119,9 +160,8 @@ def __init__(self):
         url_edit = url_combo.lineEdit()
         l, t, r, b = url_edit.getTextMargins()
         url_edit.setTextMargins(l + 5, t, r, b)
-        box.layout().addWidget(url_combo)
+        layout.addWidget(url_combo, 3, 1, 3, 3)
         url_combo.activated.connect(self._url_set)
-        layout.addWidget(box, 1, 1, QtCore.Qt.AlignVCenter)
 
         box = gui.vBox(self.controlArea, "Info")
         self.info = gui.widgetLabel(box, 'No data loaded.')
@@ -140,6 +180,8 @@ def __init__(self):
         self.warnings.setSizePolicy(Policy.Ignored, Policy.MinimumExpanding)
 
         self.set_file_list()
+        if self.last_path() is not None:
+            self.fill_sheet_combo(self.last_path())
         # Must not call open_file from within __init__. open_file
         # explicitly re-enters the event loop (by a progress bar)
         QtCore.QTimer.singleShot(0, self.load_data)
@@ -150,12 +192,15 @@ def reload(self):
             path = self.recent_paths[0]
             if basename in [path.relpath, path.basename]:
                 self.source = self.LOCAL_FILE
+                if self.is_multisheet_excel(path.abspath):
+                    self.fill_sheet_combo(path.abspath)
                 return self.load_data()
         self.select_file(len(self.recent_paths) + 1)
 
     def select_file(self, n):
         if n < len(self.recent_paths):
             super().select_file(n)
+            self.fill_sheet_combo(self.last_path())
         # TODO: This is weird. Has it remained here from "Browse data sets"
         # or from when this combo was editable? A `n` this large can come from
         # `reload`, but ... how?!
@@ -197,8 +242,28 @@ def browse_file(self, in_demos=False):
 
         self.add_path(filename)
         self.source = self.LOCAL_FILE
+        self.fill_sheet_combo(filename)
         self.load_data()
 
+    def fill_sheet_combo(self, path):
+        if os.path.exists(path) and self.is_multisheet_excel(path):
+            self.closeContext()
+            self.sheet_combo.clear()
+            self.sheet_box.show()
+            book = open_workbook(path)
+            sheet_names = [str(book.sheet_by_index(i).name)
+                           for i in range(book.nsheets)]
+            self.sheet_combo.addItems(sheet_names)
+            self.openContext(path, sheet_names)
+        else:
+            self.sheet_box.hide()
+
+    @staticmethod
+    def is_multisheet_excel(fn):
+        try:
+            return open_workbook(fn).nsheets > 1
+        except XLRDError:
+            return False
 
     # Open a file, create data from it and send it over the data channel
     def load_data(self):
@@ -219,6 +284,11 @@ def load_from_file():
                     fn = os.path.join(".", basename)
                     self.information("Loading '{}' from the current directory."
                                      .format(basename))
+            if self.is_multisheet_excel(fn):
+                data = ExcelFormat.read_file(fn + ':' + self.xls_sheet)
+                if data:
+                    return data, fn
+
             try:
                 return load(Table.from_file, fn)
             except Exception as exc:
@@ -240,7 +310,7 @@ def load_from_network():
                 data, url = load(Table.from_url, url)
             except:
                 self.warnings.setText(
-                        "URL '{}' does not contain valid data".format(url))
+                    "URL '{}' does not contain valid data".format(url))
                 # Don't remove from recent_urls:
                 # resource may reappear, or the user mistyped it
                 # and would like to retrieve it from history and fix it.
@@ -321,6 +391,8 @@ def get_ext_name(filename):
                        self.loaded_file[len(home):].lstrip("/").lstrip("\\")
             else:
                 name = self.loaded_file
+            if self.sheet_combo.isVisible():
+                name += " ({})".format(self.sheet_combo.currentText())
             self.report_items("File", [("File name", name),
                                        ("Format", get_ext_name(name))])
         else:
@@ -330,7 +402,6 @@ def get_ext_name(filename):
         self.report_data("Data", self.data)
 
 
-
 if __name__ == "__main__":
     import sys
     a = QtGui.QApplication(sys.argv)
diff --git a/Orange/widgets/settings.py b/Orange/widgets/settings.py
--- a/Orange/widgets/settings.py
+++ b/Orange/widgets/settings.py
@@ -587,6 +587,7 @@ def write_defaults_file(self, settings_file):
     def pack_data(self, widget):
         """Call the inherited method, then add local contexts to the pickle."""
         data = super().pack_data(widget)
+        self.settings_from_widget(widget)
         data["context_settings"] = widget.context_settings
         return data
 


Test patch: