Repository: biolab/orange3

Issue:
Widget save data shuffle the table columns
Hi
I'm just starting using Orange and I realised that the widget "Save Data" when writing the xcel file shuffle the columns..
![image](https://user-images.githubusercontent.com/116362134/197203535-adcd2215-94a8-4b5f-842c-25bc60ecaa95.png)

This is how the Save data sees the input table (just the first columns
![image](https://user-images.githubusercontent.com/116362134/197204275-0b9b8555-1eb2-4686-9d9b-330cae2cbbd5.png)



This is how it is going on the file 

![image](https://user-images.githubusercontent.com/116362134/197203925-4de217b9-8e69-4907-a354-81179f1f3d78.png)



<!-- 
Thanks for taking the time to report a bug!
If you're raising an issue about an add-on (i.e., installed via Options > Add-ons), raise an issue in the relevant add-on's issue tracker instead. See: https://github.com/biolab?q=orange3
To fix the bug, we need to be able to reproduce it. Please answer the following questions to the best of your ability.
-->


**What's wrong?**
<!-- Be specific, clear, and concise. Include screenshots if relevant. -->
<!-- If you're getting an error message, copy it, and enclose it with three backticks (```). -->





**How can we reproduce the problem?** 
<!-- Upload a zip with the .ows file and data. -->
<!-- Describe the steps (open this widget, click there, then add this...) -->





**What's your environment?**
<!-- To find your Orange version, see "Help → About → Version" or `Orange.version.full_version` in code -->
- Operating system:
- Orange version:
- How you installed Orange:



Discussion:
Ha!

Data tables have three kinds of columns: "attributes", "class(es)" and "meta attributes". The order is defined only within each group. If the data in the file contains a mixture of the three (e.g. meta attributes, class, another meta attribute, attribute, meta attribute ...), this order is lost -- except that columns within each group will maintain their relative order.

- Table widget always shows class variables, followed by meta attributes, followed by ordinary attributes.
- Save widget always saves attributes, followed by classes, followed by meta attributes.

This is indeed a discrepancy. The latter was the original order, which was also used by Table, but at some point we changed it. In other words, Save does not change the order, it just uses a different order. :) File widget, on the other hand, just forgets the order.

Thanks for your observation. We definitely need to discuss this.
Thank you for the quick response!
Got you..it would be nice in future to be able edit how the table is saved,
you have fantastic graphic features but sometime also the text needs to
have its form :)

Il giorno ven 21 ott 2022 alle ore 15:24 Janez Demšar <
***@***.***> ha scritto:

> Ha!
>
> Data tables have three kinds of columns: "attributes", "class(es)" and
> "meta attributes". The order is defined only within each group. If the data
> in the file contains a mixture of the three (e.g. meta attributes, class,
> another meta attribute, attribute, meta attribute ...), this order is lost
> -- except that columns within each group will maintain their relative order.
>
>    - Table widget always shows class variables, followed by meta
>    attributes, followed by ordinary attributes.
>    - Save widget always saves attributes, followed by classes, followed
>    by meta attributes.
>
> This is indeed a discrepancy. The latter was the original order, which was
> also used by Table, but at some point we changed it. In other words, Save
> does not change the order, it just uses a different order. :) File widget,
> on the other hand, just forgets the order.
>
> Thanks for your observation. We definitely need to discuss this.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/biolab/orange3/issues/6180#issuecomment-1286958169>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A3XYXFW2OEYDQDZMCU6XXDLWEKKSJANCNFSM6AAAAAARLFCSCM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>

Decision: Save data will use the same order as the Table widget.

Resolving patch:
diff --git a/Orange/data/io.py b/Orange/data/io.py
--- a/Orange/data/io.py
+++ b/Orange/data/io.py
@@ -322,14 +322,14 @@ def _get_active_sheet(self) -> openpyxl.worksheet.worksheet.Worksheet:
     @classmethod
     def write_file(cls, filename, data, with_annotations=False):
         vars = list(chain((ContinuousVariable('_w'),) if data.has_weights() else (),
-                          data.domain.attributes,
                           data.domain.class_vars,
-                          data.domain.metas))
+                          data.domain.metas,
+                          data.domain.attributes))
         formatters = [cls.formatter(v) for v in vars]
         zipped_list_data = zip(data.W if data.W.ndim > 1 else data.W[:, np.newaxis],
-                               data.X,
                                data.Y if data.Y.ndim > 1 else data.Y[:, np.newaxis],
-                               data.metas)
+                               data.metas,
+                               data.X)
         names = cls.header_names(data)
         headers = (names,)
         if with_annotations:
diff --git a/Orange/data/io_base.py b/Orange/data/io_base.py
--- a/Orange/data/io_base.py
+++ b/Orange/data/io_base.py
@@ -604,9 +604,9 @@ def write_file(fn):
     @staticmethod
     def header_names(data):
         return ['weights'] * data.has_weights() + \
-               [v.name for v in chain(data.domain.attributes,
-                                      data.domain.class_vars,
-                                      data.domain.metas)]
+               [v.name for v in chain(data.domain.class_vars,
+                                      data.domain.metas,
+                                      data.domain.attributes)]
 
     @staticmethod
     def header_types(data):
@@ -623,9 +623,9 @@ def _vartype(var):
             raise NotImplementedError
 
         return ['continuous'] * data.has_weights() + \
-               [_vartype(v) for v in chain(data.domain.attributes,
-                                           data.domain.class_vars,
-                                           data.domain.metas)]
+               [_vartype(v) for v in chain(data.domain.class_vars,
+                                           data.domain.metas,
+                                           data.domain.attributes)]
 
     @staticmethod
     def header_flags(data):
@@ -633,10 +633,10 @@ def header_flags(data):
             ['weight'] * data.has_weights(),
             (Flags.join([flag], *('{}={}'.format(*a) for a in
                                   sorted(var.attributes.items())))
-             for flag, var in chain(zip(repeat(''), data.domain.attributes),
-                                    zip(repeat('class'),
+             for flag, var in chain(zip(repeat('class'),
                                         data.domain.class_vars),
-                                    zip(repeat('meta'), data.domain.metas)))))
+                                    zip(repeat('meta'), data.domain.metas),
+                                    zip(repeat(''), data.domain.attributes)))))
 
     @classmethod
     def write_headers(cls, write, data, with_annotations=True):
@@ -667,15 +667,15 @@ def write_data(cls, write, data):
         """`write` is a callback that accepts an iterable"""
         vars_ = list(
             chain((ContinuousVariable('_w'),) if data.has_weights() else (),
-                  data.domain.attributes,
                   data.domain.class_vars,
-                  data.domain.metas))
+                  data.domain.metas,
+                  data.domain.attributes))
 
         formatters = [cls.formatter(v) for v in vars_]
         for row in zip(data.W if data.W.ndim > 1 else data.W[:, np.newaxis],
-                       data.X,
                        data.Y if data.Y.ndim > 1 else data.Y[:, np.newaxis],
-                       data.metas):
+                       data.metas,
+                       data.X):
             write([fmt(v) for fmt, v in zip(formatters, flatten(row))])
 
 


Test patch:
diff --git a/Orange/data/tests/test_io.py b/Orange/data/tests/test_io.py
--- a/Orange/data/tests/test_io.py
+++ b/Orange/data/tests/test_io.py
@@ -1,8 +1,12 @@
+import os
 import unittest
+from tempfile import NamedTemporaryFile
+
 import numpy as np
 
 from Orange.data import ContinuousVariable, DiscreteVariable, StringVariable, \
-    TimeVariable
+    TimeVariable, Domain, Table
+from Orange.data.io import TabReader, ExcelReader
 from Orange.data.io_util import guess_data_type
 from Orange.misc.collections import natural_sorted
 
@@ -108,6 +112,48 @@ def test_guess_data_type_values_order(self):
         self.assertEqual(DiscreteVariable, coltype)
         self.assertListEqual(res, valuemap)
 
+class TestWriters(unittest.TestCase):
+    def setUp(self):
+        self.domain = Domain([DiscreteVariable("a", values=tuple("xyz")),
+                         ContinuousVariable("b", number_of_decimals=3)],
+                        ContinuousVariable("c", number_of_decimals=0),
+                        [StringVariable("d")])
+        self.data = Table.from_numpy(
+            self.domain,
+            np.array([[1, 0.5], [2, np.nan], [np.nan, 1.0625]]),
+            np.array([3, 1, 7]),
+            np.array(["foo bar baz".split()]).T
+        )
+
+    def test_write_tab(self):
+        with NamedTemporaryFile(suffix=".tab", delete=False) as f:
+            fname = f.name
+        try:
+            TabReader.write(fname, self.data)
+            with open(fname, encoding="utf-8") as f:
+                self.assertEqual(f.read().strip(), """
+c\td\ta\tb
+continuous\tstring\tx y z\tcontinuous
+class\tmeta\t\t
+3.0\tfoo\ty\t0.5
+1.0\tbar\tz\t
+7.0\tbaz\t\t1.0625""".strip())
+        finally:
+            os.remove(fname)
+
+    def test_roundtrip_xlsx(self):
+        with NamedTemporaryFile(suffix=".xlsx", delete=False) as f:
+            fname = f.name
+        try:
+            ExcelReader.write(fname, self.data)
+            data = ExcelReader(fname).read()
+            np.testing.assert_equal(data.X, self.data.X)
+            np.testing.assert_equal(data.Y, self.data.Y)
+            np.testing.assert_equal(data.metas, self.data.metas)
+            np.testing.assert_equal(data.domain, self.data.domain)
+        finally:
+            os.remove(fname)
+
 
 if __name__ == "__main__":
     unittest.main()