Repository: biolab/orange3

Issue:
Find informative projections: Keep reproducible order of ranked variable pairs
<!-- 
Thanks for taking the time to report a bug!
If you're raising an issue about an add-on (i.e., installed via Options > Add-ons), raise an issue in the relevant add-on's issue tracker instead. See: https://github.com/biolab?q=orange3
To fix the bug, we need to be able to reproduce it. Please answer the following questions to the best of your ability.
-->


**What's wrong?**
<!-- Be specific, clear, and concise. Include screenshots if relevant. -->
<!-- If you're getting an error message, copy it, and enclose it with three backticks (```). -->
Running Find informative projections sometimes returns different order of variable pairs. This happens when two or more pairs have the same score and they are ranked randomly. This is a little confusing when the order on two machines is not the same.

Ideally, we would have reproducible ranking all the time. The easy way is to order variable pairs with same scores alphabetically.



**How can we reproduce the problem?** 
<!-- Upload a zip with the .ows file and data. -->
<!-- Describe the steps (open this widget, click there, then add this...) -->
Datasets (Auto MPG) - Scatter Plot. Find informative projections.
<img width="432" alt="Screenshot 2022-03-17 at 08 55 46" src="https://user-images.githubusercontent.com/12524972/158764213-25cc44bf-3ab4-4690-8183-a371ab5cb9e6.png">
<img width="432" alt="Screenshot 2022-03-17 at 08 55 50" src="https://user-images.githubusercontent.com/12524972/158764232-23c15bbb-42dd-4e13-9e8c-e8ac3949b6ce.png">



**What's your environment?**
<!-- To find your Orange version, see "Help → About → Version" or `Orange.version.full_version` in code -->
- Operating system: OSX
- Orange version: 3.32.dev
- How you installed Orange: pip



Discussion:
Does a problem also appear on the same computer and Orange installation?
Comment & Idea: `Orange.widgets.visualize.utils.run_vizrank`, which does the actual work, calls `bisect.bisect_left(scores, score)` to find the position for the score, where `scores` is a list of scores and `score` is the newly computed score. If we wanted to sort by attribute names (or whatever) as secondary keys, we could add this additional info (tuple of names) to `scores`.

While this is doable (the `score` attribute is rather internal-ish), but I'd very much prefer if we investigate another approach. There is (I suppose) nothing random in `bisect.bisect_left`. VizRank explores attributes in some specific order, defined by some heuristic. Maybe it would suffice to make this heuristic stable, e.g. by observing attribute names in case of ties.
As a matter of fact -- it already sorts by name. :) https://github.com/biolab/orange3/blob/master/Orange/widgets/visualize/owscatterplot.py#L98.

Randomness must be coming from ReliefF's sampling.

Resolving patch:
diff --git a/Orange/widgets/visualize/owscatterplot.py b/Orange/widgets/visualize/owscatterplot.py
--- a/Orange/widgets/visualize/owscatterplot.py
+++ b/Orange/widgets/visualize/owscatterplot.py
@@ -94,7 +94,8 @@ def score_heuristic(self):
         data = self.master.data.transform(domain)
         relief = ReliefF if isinstance(domain.class_var, DiscreteVariable) \
             else RReliefF
-        weights = relief(n_iterations=100, k_nearest=self.minK)(data)
+        weights = relief(
+            n_iterations=100, k_nearest=self.minK, random_state=0)(data)
         attrs = sorted(zip(weights, domain.attributes),
                        key=lambda x: (-x[0], x[1].name))
         return [a for _, a in attrs]


Test patch: