Repository: Qiskit/qiskit

Issue:
`FutureWarning` issued by `scikit-learn` in `opflow`
<!-- ⚠️ If you do not respect this template, your issue will be closed -->
<!-- ⚠️ Make sure to browse the opened and closed issues -->

### Information

- **Qiskit Terra version**: 0.18+
- **Python version**: any
- **Operating system**: any

### What is the current behavior?

With the new(ish) `scikit-learn` version 1.0, there is a `FutureWarning` issued when using the `normalize` parameter of `LinearModel`, which `opflow` does when calculating gradients. This doesn't fail our tests (though perhaps it should), because `FutureWarning` doesn't count as a type of `DeprecationWarning`.

### Steps to reproduce the problem

Run the test suite, especially `test.python.opflow.test_gradients.TestGradients.test_natural_gradient`.

### What is the expected behavior?

No future warning.


### Suggested solutions

Fix `opflow` to handle fitting with `StandardScaler` rather than the now-deprecated `normalize` parameter.  This has existed for several minor releases of `scikit-learn`, so should be safe for us to do.


Discussion:
Hi, I would like to work on this as my first issue. If someone wouldn't mind assigning me that would be great. Thanks!
No problem, I'll assign it to you.
See also #6904 where I suggest erroring on all (unexpected) warnings.
Yeah, absolutely agreed on that. It's something I want too - I had started to work on reducing the number of allowed deprecation warnings as a precursor to this, but then other stuff took over.
Hi @st4eve, how's it going?  Do you need any help on getting this working?
Thanks for following up with me. I am new to contributing and I was having trouble getting the tox test file to run properly. I am on a windows machine and for some reason the test file always throws:

ERROR:   py38: InvocationError for command 'C:\Users\coshc\qiskit-terra\.tox\py38\Scripts\pip.EXE' install '-cC:\Users\coshc\qiskit-terra/constraints.txt' -U --exists-action w -e . (exited with code 1)

When running the entire test sweet. If I try to run the specific test_gradients.py test file then I am met with the error:

ModuleNotFoundError: No module named 'qiskit.quantum_info.states.cython.exp_value'

Finally, when I attempt to build the project from source after following the steps on the Qiskit website I end up with:

ERROR: Command errored out with exit status 1: 'C:\Users\coshc\anaconda3\envs\qiskitDevEnv\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\coshc\\qiskit-terra\\setup.py'"'"'; __file__='"'"'C:\\Users\\coshc\\qiskit-terra\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command out

I will likely have more questions when I am able to get this working.

Because of this I have been unable to see the specific deprecation warning that the issue was referring to. Apologies for the delays on my end as well. Been a busy few weeks of school for me.
We would probably need to see the full logs that those commands allude to, but most likely the problem is that you don't have a correctly working C compiler on your system.  There's a little intro to the "building Terra from source" section on the website that says:

> On Windows, it is easiest to install the Visual C++ compiler from the [Build Tools for Visual Studio 2017](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2017). You can instead install Visual Studio version 2015 or 2017, making sure to select the options for installing the C++ compiler.

Have you got those installed?  In particular, since that link seems to be slightly out of date now, you want the thing that's now marked "[Build Tools for Visual Studio 2022](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2022)", and if it ever gives you an option (I don't think it does), make sure that you install the C++ tools.
I'd also recommend double checking you've followed all the steps in the [Installing terra from source](https://qiskit.org/documentation/contributing_to_qiskit.html#installing-terra-from-source), including the final `pip install -e .` step 😄 
How is that going @st4eve ? I also would recommend you to ask for help in the channel `#qiskit-pr-help` in our slack workspace: https://join.slack.com/share/enQtMzAzNDA0MDQzMDMyMy0yZWY4MGU4NDJmM2UzYjc1YjQ4OTg0YmY3Y2FlYzgzMmVhMDU2MDgwMDM5MjM0ZDQyYTA2YjA2OWIzMDc5MDEz
Is this issue actually still present? I just ran the opflow tests and did not encounter any `FutureWarning` anymore. 🙂 
Yep - for example https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=34312&view=logs&jobId=8fcf860c-8d5a-5d3a-546e-59924dc71ae1&j=8fcf860c-8d5a-5d3a-546e-59924dc71ae1&t=d677ba53-4fc9-5372-ced6-0b4a8cddaca6 from line 2753 onwards.

Perhaps you didn't have scikit-learn installed, so the tests were skipped?
Hi, there is a difference between the deprecated `normalize` parameter and the `StandardScaler` outlined [here](https://stackoverflow.com/a/70093848) Will that be of any concern, maybe in the tests?
@Cryoris, @woodsp-ibm: perhaps you know the answer to this question?
Hi, following up on this if there's any updates?
Since the `normalize` argument is only used in an empirical regularization method, i.e., `Ridge`, it should be perfectly fine to implement minor changes such as changing to `StandardScaler`.
Thanks, Christa.  @prakharb10: the comment above is from the person who originally wrote the code, so we can go ahead with that.  Since it _may_ affect the exact results achieved from a given seed / set of initial points, we should include a one-line "upgrade" release note with the change mentioning that, but it should be sufficiently internal that we don't need any deprecation cycles, I think.

Resolving patch:
diff --git a/qiskit/opflow/gradients/natural_gradient.py b/qiskit/opflow/gradients/natural_gradient.py
--- a/qiskit/opflow/gradients/natural_gradient.py
+++ b/qiskit/opflow/gradients/natural_gradient.py
@@ -354,7 +354,7 @@ def _ridge(
             lambda4: right starting point for L-curve corner search
             tol_search: termination threshold for regularization parameter search
             fit_intercept: if True calculate intercept
-            normalize: deprecated if fit_intercept=False, if True normalize A for regression
+            normalize: ignored if fit_intercept=False, if True normalize A for regression
             copy_a: if True A is copied, else overwritten
             max_iter: max. number of iterations if solver is CG
             tol: precision of the regression solution
@@ -369,11 +369,11 @@ def _ridge(
 
         """
         from sklearn.linear_model import Ridge
+        from sklearn.preprocessing import StandardScaler
 
         reg = Ridge(
             alpha=lambda_,
             fit_intercept=fit_intercept,
-            normalize=normalize,
             copy_X=copy_a,
             max_iter=max_iter,
             tol=tol,
@@ -383,7 +383,10 @@ def _ridge(
 
         def reg_method(a, c, alpha):
             reg.set_params(alpha=alpha)
-            reg.fit(a, c)
+            if normalize:
+                reg.fit(StandardScaler().fit_transform(a), c)
+            else:
+                reg.fit(a, c)
             return reg.coef_
 
         lambda_mc, x_mc = NaturalGradient._reg_term_search(
@@ -425,7 +428,7 @@ def _lasso(
             lambda4: right starting point for L-curve corner search
             tol_search: termination threshold for regularization parameter search
             fit_intercept: if True calculate intercept
-            normalize: deprecated if fit_intercept=False, if True normalize A for regression
+            normalize: ignored if fit_intercept=False, if True normalize A for regression
             precompute: If True compute and use Gram matrix to speed up calculations.
                                              Gram matrix can also be given explicitly
             copy_a: if True A is copied, else overwritten
@@ -444,11 +447,11 @@ def _lasso(
 
         """
         from sklearn.linear_model import Lasso
+        from sklearn.preprocessing import StandardScaler
 
         reg = Lasso(
             alpha=lambda_,
             fit_intercept=fit_intercept,
-            normalize=normalize,
             precompute=precompute,
             copy_X=copy_a,
             max_iter=max_iter,
@@ -461,7 +464,10 @@ def _lasso(
 
         def reg_method(a, c, alpha):
             reg.set_params(alpha=alpha)
-            reg.fit(a, c)
+            if normalize:
+                reg.fit(StandardScaler().fit_transform(a), c)
+            else:
+                reg.fit(a, c)
             return reg.coef_
 
         lambda_mc, x_mc = NaturalGradient._reg_term_search(


Test patch: