Repository: biolab/orange3

Issue:
Force regression through origin (zero)
**relevant info**
macOS 10.13.6, orange 3.26.0 from Anaconda distribution (conda 4.8.4).

**Is your feature request related to a problem? Please describe.**
No.

**Here, write a clear and concise description of what the problem is. Ex. I'm always frustrated when [...]**
When using the Canvas, Model > Linear Regression and Educational > Polynomial Regression do not allow forcing of the regression through 0, i.e. Coefficients always include intercept and independent variable terms.

**Describe the solution you'd like**
Add GUI elements (e.g., checkbox) to Linear Regression and Polynomial Regression widgets that allow the user to specify the forcing of the regression through the origin.

(Separately, perhaps add a Model > Polynomial Regression feature separate from the Educational add-on?)

**Add a clear and concise description of what you want to happen here**.
Return regression coefficients so fit `y = m_1x^n + m_2x^{n-1} + ... + m_nx + b` is calculated such that `b = 0`. (May only be feasible/valid for polynomials of first order.)

**Describe alternatives you've considered**
n/a

**Additional context**
n/a

Thanks! (And apologies if this feature exists. However, a search of the GUI and documentation was not fruitful.)



Discussion:
The feature would indeed be trivial to add and there's plenty of space in the GUI. However we do not want to add redundant features (especially not to educational add-on), because the user (in this case - a student) may accidentally enable them. It would be just another point where something can go wrong.

Educational add-on is essentially a collection of widgets that we ourselves use for teaching, so it is tailored specifically for our courses. We use this widget primarily to show overfitting and how to fight it with regularization. Although our lectures are unscripted and largely driven by students, we (at least I) never encountered a situation where I'd need this feature. I am however sincerely open to use cases that you may present.

You also suggested "Polynomial Regression feature separate from the Educational add-on". Do you think polynomial regression model with a single independent variable would be useful in real world?

(By the way -- we should be discussing this in [another repo](https://github.com/biolab/orange3-educational).)
I think @jkmgeo was not referring to Educational add-on as such, but generally having this feature for Linear Regression (and a new widget Polynomial Regression?). Or did I get it wrong? 
In an educational widget, especially, I'd like to have an option to remove the bias term from the model. To actually show students how important the bias really is.
My bad. Of course. Alright, let's do it. In both, I guess.
Thanks, all, for the fast replies. 

@ajdapretnar is right. I was requesting the addition of a new Polynomial Regression widget in the Model menu as a secondary request. My main request is to add a "force through zero" feature/toggle for the Linear Regression widget. 

The reason I'm requesting this is that I use a lot of chemistry data, and particularly the ratio of two elements. It would be nice to be able to select a subset of my data on a binary A vs. B plot (e.g., by selecting in a scatter plot) and get a sense of the elemental ratios (A/B) via linear regression forced through zero during data exploration work. Alternatively, adding "force through zero" and "display regression equation" toggles beneath the "Show regression line" option in the scatter plot GUI would suit me, but I don't want to impose too much on what the devs think is the best implementation. (I don't think the "Treat variables as independent" option located there right now is the same, but correct me if I'm wrong.)

Cheers!


Resolving patch:
diff --git a/Orange/regression/linear.py b/Orange/regression/linear.py
--- a/Orange/regression/linear.py
+++ b/Orange/regression/linear.py
@@ -28,8 +28,10 @@ def score(self, data):
 class LinearRegressionLearner(SklLearner, _FeatureScorerMixin):
     __wraps__ = skl_linear_model.LinearRegression
 
-    def __init__(self, preprocessors=None):
+    # Arguments are needed for signatures, pylint: disable=unused-argument
+    def __init__(self, preprocessors=None, fit_intercept=True):
         super().__init__(preprocessors=preprocessors)
+        self.params = vars()
 
     def fit(self, X, Y, W=None):
         model = super().fit(X, Y, W)
@@ -39,6 +41,7 @@ def fit(self, X, Y, W=None):
 class RidgeRegressionLearner(LinearRegressionLearner):
     __wraps__ = skl_linear_model.Ridge
 
+    # Arguments are needed for signatures, pylint: disable=unused-argument
     def __init__(self, alpha=1.0, fit_intercept=True,
                  normalize=False, copy_X=True, max_iter=None,
                  tol=0.001, solver='auto', preprocessors=None):
@@ -49,6 +52,7 @@ def __init__(self, alpha=1.0, fit_intercept=True,
 class LassoRegressionLearner(LinearRegressionLearner):
     __wraps__ = skl_linear_model.Lasso
 
+    # Arguments are needed for signatures, pylint: disable=unused-argument
     def __init__(self, alpha=1.0, fit_intercept=True, normalize=False,
                  precompute=False, copy_X=True, max_iter=1000,
                  tol=0.0001, warm_start=False, positive=False,
@@ -60,6 +64,7 @@ def __init__(self, alpha=1.0, fit_intercept=True, normalize=False,
 class ElasticNetLearner(LinearRegressionLearner):
     __wraps__ = skl_linear_model.ElasticNet
 
+    # Arguments are needed for signatures, pylint: disable=unused-argument
     def __init__(self, alpha=1.0, l1_ratio=0.5, fit_intercept=True,
                  normalize=False, precompute=False, max_iter=1000,
                  copy_X=True, tol=0.0001, warm_start=False, positive=False,
@@ -71,6 +76,7 @@ def __init__(self, alpha=1.0, l1_ratio=0.5, fit_intercept=True,
 class ElasticNetCVLearner(LinearRegressionLearner):
     __wraps__ = skl_linear_model.ElasticNetCV
 
+    # Arguments are needed for signatures, pylint: disable=unused-argument
     def __init__(self, l1_ratio=0.5, eps=0.001, n_alphas=100, alphas=None,
                  fit_intercept=True, normalize=False, precompute='auto',
                  max_iter=1000, tol=0.0001, cv=5, copy_X=True,
@@ -83,6 +89,7 @@ class SGDRegressionLearner(LinearRegressionLearner):
     __wraps__ = skl_linear_model.SGDRegressor
     preprocessors = SklLearner.preprocessors + [Normalize()]
 
+    # Arguments are needed for signatures, pylint: disable=unused-argument
     def __init__(self, loss='squared_loss', penalty='l2', alpha=0.0001,
                  l1_ratio=0.15, fit_intercept=True, max_iter=5, tol=1e-3,
                  shuffle=True, epsilon=0.1, n_jobs=1, random_state=None,
@@ -109,13 +116,15 @@ class PolynomialLearner(Learner):
     preprocessors = SklLearner.preprocessors
 
     def __init__(self, learner=LinearRegressionLearner(), degree=2,
-                 preprocessors=None):
+                 preprocessors=None, include_bias=True):
         super().__init__(preprocessors=preprocessors)
         self.degree = degree
         self.learner = learner
+        self.include_bias = include_bias
 
     def fit(self, X, Y, W=None):
-        polyfeatures = skl_preprocessing.PolynomialFeatures(self.degree)
+        polyfeatures = skl_preprocessing.PolynomialFeatures(
+            self.degree, include_bias=self.include_bias)
         X = polyfeatures.fit_transform(X)
         clf = self.learner
         if W is None or not self.supports_weights:
@@ -140,6 +149,7 @@ def __str__(self):
 
 class PolynomialModel(Model):
     def __init__(self, model, polyfeatures):
+        super().__init__()
         self.model = model
         self.polyfeatures = polyfeatures
 
diff --git a/Orange/widgets/model/owlinearregression.py b/Orange/widgets/model/owlinearregression.py
--- a/Orange/widgets/model/owlinearregression.py
+++ b/Orange/widgets/model/owlinearregression.py
@@ -39,6 +39,7 @@ class Outputs(OWBaseLearner.Outputs):
     reg_type = settings.Setting(OLS)
     alpha_index = settings.Setting(0)
     l2_ratio = settings.Setting(0.5)
+    fit_intercept = settings.Setting(True)
     autosend = settings.Setting(True)
 
     alphas = list(chain([x / 10000 for x in range(1, 10)],
@@ -51,6 +52,11 @@ class Outputs(OWBaseLearner.Outputs):
 
     def add_main_layout(self):
         # this is part of init, pylint: disable=attribute-defined-outside-init
+        box = gui.hBox(self.controlArea, "Parameters")
+        gui.checkBox(box, self, "fit_intercept",
+                     "Fit intercept (unchecking it fixes it to zero)",
+                     callback=self._intercept_changed)
+
         box = gui.hBox(self.controlArea, "Regularization")
         gui.radioButtons(box, self, "reg_type",
                          btnLabels=self.REGULARIZATION_TYPES,
@@ -93,6 +99,9 @@ def add_main_layout(self):
     def handleNewSignals(self):
         self.apply()
 
+    def _intercept_changed(self):
+        self.apply()
+
     def _reg_type_changed(self):
         self.controls.alpha_index.setEnabled(self.reg_type != self.OLS)
         self.l2_ratio_slider.setEnabled(self.reg_type == self.Elastic)
@@ -116,7 +125,8 @@ def _l2_ratio_changed(self):
     def create_learner(self):
         alpha = self.alphas[self.alpha_index]
         preprocessors = self.preprocessors
-        args = {"preprocessors": preprocessors}
+        args = dict(preprocessors=preprocessors,
+                    fit_intercept=self.fit_intercept)
         if self.reg_type == OWLinearRegression.OLS:
             learner = LinearRegressionLearner(**args)
         elif self.reg_type == OWLinearRegression.Ridge:
@@ -134,9 +144,11 @@ def update_model(self):
         if self.model is not None:
             domain = Domain(
                 [ContinuousVariable("coef")], metas=[StringVariable("name")])
-            coefs = [self.model.intercept] + list(self.model.coefficients)
-            names = ["intercept"] + \
-                    [attr.name for attr in self.model.domain.attributes]
+            coefs = list(self.model.coefficients)
+            names = [attr.name for attr in self.model.domain.attributes]
+            if self.fit_intercept:
+                coefs.insert(0, self.model.intercept)
+                names.insert(0, "intercept")
             coef_table = Table.from_list(domain, list(zip(coefs, names)))
             coef_table.name = "coefficients"
         self.Outputs.coefficients.send(coef_table)
@@ -155,7 +167,10 @@ def get_learner_parameters(self):
                               .format(self.alphas[self.alpha_index],
                                       self.l2_ratio,
                                       1 - self.l2_ratio))
-        return (("Regularization", regularization), )
+        return (
+            ("Regularization", regularization),
+            ("Fit intercept", ["No", "Yes"][self.fit_intercept])
+        )
 
 
 if __name__ == "__main__":  # pragma: no cover


Test patch: