Repository: scikit-learn/scikit-learn
Pull request: [MRG+2] Add sklearn show_versions() method
PR that adds a `show_versions()` method convenient for debugging.

#### Reference Issues/PRs
Fixes #11522 

#### What does this implement/fix? Explain your changes.
Fork of the `pandas.show_versions()`.
It displays optionally the system BLAS information.

Problem statement:
Add sklearn.show_versions() similar to pandas.show_versions (with numpy blas binding info)
Some numeric issues are related to the specific blas that numpy is using. I'm wondering if it makes sense to add the relevant ``system_info`` invocations to the template to make it easier for people to report.

Hints from issue discussion:
I don't see why not!

@lesteve suggested we could add something like `pandas.show_versions()` that would print all the relevant information for debugging. For instance, on my laptop, I get,

<details>

```
>>> pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Linux
OS-release: 4.11.6-gentoo
machine: x86_64
processor: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.23.0
pytest: 3.5.1
pip: 10.0.1
setuptools: 39.1.0
Cython: 0.27.3
numpy: 1.14.3
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.4.0
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
```
</details>

we certainly don't care about all the dependencies that pandas might, but I agree that for scikit-learn, having  e.g.
 - BLAS information
 - whether "conda" is in path
 - with https://github.com/scikit-learn/scikit-learn/pull/11166 whether the bundled or unbundled joblib is used
 
would be definitely useful for debuging. It's more practical to have small function for those, than a copy passable snippet (particularly if it becomes more complex).

Tagging this for v0.20 as this would be fairly easy to do, and help with the maintenance after the release..

+10 about `sklearn.show_versions()` I edited the issue title.
I think what we want to do:
* add sklearn.show_versions with similar info as pandas.show_versions and whatever we ask for in the ISSUE_TEMPLATE.md
* modify ISSUE_TEMPLATE.md
* amend the docs, this git grep can be handy:
```
❯ git grep platform.platform
CONTRIBUTING.md:  import platform; print(platform.platform())
ISSUE_TEMPLATE.md:import platform; print(platform.platform())
doc/developers/contributing.rst:     import platform; print(platform.platform())
doc/developers/tips.rst:        import platform; print(platform.platform())
sklearn/feature_extraction/dict_vectorizer.py:            " include the output from platform.platform() in your bug report")
```
This might be of help
```py
from sklearn._build_utils import get_blas_info
```
The problem with doing this is that it won't be runnable before 0.20!​

Hi guys, I'm looking at this right now.

Few questions to help me get this done:

- what are the relevant information from the `get_blas_info` that need to be printed ? 
  unfortunately the compilation information is printed (probably through `cythonize`) but not returned.
- shall I restrict the printed python libraries to the main ones ? 
  I'd suggest `numpy`, `scipy`, `pandas`, `matplotlib`, `Cython`, `pip`, `setuptools`, `pytest`.
- is `sklearn/utils` the right place to put it ?
> The problem with doing this is that it won't be runnable before 0.20!​

Good point we can modify only the rst doc for now and delay the changes in .md until the release.

Not an expert, but I think all the `get_blas_info` is useful. About the dependencies, I am not sure look at what pandas is doing and do something similar (they have things about sys.executable, 32bit vs 64bit, bitness which may be useful). It would be good to keep it as short as possible. For example I am not convinced `pytest` makes sense.

> is sklearn/utils the right place to put it ?

You can probably put the code in `sklearn/utils`. I would be in favour of making it accessible at from the root namespace so that you can do `from sklearn import show_versions`




+1 for adding show_versions.
Maybe optionally include the blas stuff?
> Good point we can modify only the rst doc for now and delay the changes in .md until the release.

Will do.

> I would be in favour of making it accessible at from the root namespace so that you can do from `sklearn import show_versions`

Agreed. I would have suggested to add `from sklearn.utils import show_versions` in `sklearn.__init__.py` instead of putting the file there. Does that sound ok ?

> Maybe optionally include the blas stuff?

I wrote it as an option for now.

Commit message: added to the function list in classes.rst

Patch to review:
diff --git a/doc/developers/contributing.rst b/doc/developers/contributing.rst
index 1b43400f4a7a0..4b218e542c115 100644
--- a/doc/developers/contributing.rst
+++ b/doc/developers/contributing.rst
@@ -112,7 +112,7 @@ following rules before submitting:
    `new algorithm requirements
    <http://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms>`_.
 
--  If you are submitting a bug report, we strongly encourage you to follow the guidelines in 
+-  If you are submitting a bug report, we strongly encourage you to follow the guidelines in
    :ref:`filing_bugs`.
 
 .. _filing_bugs:
@@ -140,11 +140,16 @@ feedback:
   your **Python, scikit-learn, numpy, and scipy versions**. This information
   can be found by running the following code snippet::
 
-     import platform; print(platform.platform())
-     import sys; print("Python", sys.version)
-     import numpy; print("NumPy", numpy.__version__)
-     import scipy; print("SciPy", scipy.__version__)
-     import sklearn; print("Scikit-Learn", sklearn.__version__)
+     import sklearn; sklearn.show_versions()
+
+  .. note::
+
+    This utility function is only available in scikit-learn v0.20+. For all
+    previous versions, one has to manually dig such information e.g. snippet::
+
+      import numpy; print('numpy:', numpy.__version__)
+      import scipy; print('scipy:', scipy.__version__)
+      import sklearn; print('scikit-learn:', sklearn.__version__)
 
 - Please ensure all **code snippets and error messages are formatted in
   appropriate code blocks**.  See `Creating and highlighting code blocks
diff --git a/doc/developers/tips.rst b/doc/developers/tips.rst
index 9e66174a7eff7..9369b650fbc59 100644
--- a/doc/developers/tips.rst
+++ b/doc/developers/tips.rst
@@ -121,15 +121,11 @@ Issue: Self-contained example for bug
 Issue: Software versions
     ::
 
-        To help diagnose your issue, could you please paste the output of:
+        To help diagnose your issue, please paste the output of:
         ```py
-        import platform; print(platform.platform())
-        import sys; print("Python", sys.version)
-        import numpy; print("NumPy", numpy.__version__)
-        import scipy; print("SciPy", scipy.__version__)
-        import sklearn; print("Scikit-Learn", sklearn.__version__)
+        import sklearn; sklearn.show_versions()
         ```
-        ? Thanks.
+        Thanks.
 
 Issue: Code blocks
     ::
diff --git a/doc/modules/classes.rst b/doc/modules/classes.rst
index 1753bf9b404bb..44ff2ff0aefa3 100644
--- a/doc/modules/classes.rst
+++ b/doc/modules/classes.rst
@@ -48,6 +48,7 @@ Functions
    config_context
    get_config
    set_config
+   show_versions
 
 .. _calibration_ref:
 
diff --git a/doc/whats_new/v0.20.rst b/doc/whats_new/v0.20.rst
index 6b199e9e2bac1..b2b8c516da33d 100644
--- a/doc/whats_new/v0.20.rst
+++ b/doc/whats_new/v0.20.rst
@@ -217,6 +217,12 @@ Misc
   exposed in :mod:`sklearn.utils`.
   :issue:`11166`by `Gael Varoquaux`_
 
+- A utility method :func:`sklearn.show_versions()` was added to print out
+  information relevant for debugging. It includes the user system, the
+  Python executable, the version of the main libraries and BLAS binding
+  information.
+  :issue:`11596` by :user:`Alexandre Boucaud <aboucaud>`
+
 Enhancements
 ............
 
diff --git a/sklearn/__init__.py b/sklearn/__init__.py
index 4de0df5ed1f95..80a35e201df05 100644
--- a/sklearn/__init__.py
+++ b/sklearn/__init__.py
@@ -62,6 +62,8 @@
 else:
     from . import __check_build
     from .base import clone
+    from .utils._print_versions import show_versions
+
     __check_build  # avoid flakes unused variable error
 
     __all__ = ['calibration', 'cluster', 'covariance', 'cross_decomposition',
@@ -74,7 +76,8 @@
                'preprocessing', 'random_projection', 'semi_supervised',
                'svm', 'tree', 'discriminant_analysis', 'impute', 'compose',
                # Non-modules:
-               'clone', 'get_config', 'set_config', 'config_context']
+               'clone', 'get_config', 'set_config', 'config_context',
+               'show_versions']
 
 
 def setup_module(module):
diff --git a/sklearn/utils/_print_versions.py b/sklearn/utils/_print_versions.py
new file mode 100644
index 0000000000000..7ff01bd40175d
--- /dev/null
+++ b/sklearn/utils/_print_versions.py
@@ -0,0 +1,117 @@
+"""
+Utility methods to print system info for debugging
+
+adapted from :func:`pandas.show_versions`
+"""
+# License: BSD 3 clause
+
+import platform
+import sys
+import importlib
+
+
+def _get_sys_info():
+    """System information
+
+    Return
+    ------
+    sys_info : dict
+        system and Python version information
+
+    """
+    python = sys.version.replace('\n', ' ')
+
+    blob = [
+        ("python", python),
+        ('executable', sys.executable),
+        ("machine", platform.platform()),
+    ]
+
+    return dict(blob)
+
+
+def _get_deps_info():
+    """Overview of the installed version of main dependencies
+
+    Returns
+    -------
+    deps_info: dict
+        version information on relevant Python libraries
+
+    """
+    deps = [
+        # (MODULE_NAME, f(mod) -> mod version)
+        ("pip", lambda mod: mod.__version__),
+        ("setuptools", lambda mod: mod.__version__),
+        ("sklearn", lambda mod: mod.__version__),
+        ("numpy", lambda mod: mod.__version__),
+        ("scipy", lambda mod: mod.version.version),
+        ("Cython", lambda mod: mod.__version__),
+        ("pandas", lambda mod: mod.__version__),
+    ]
+
+    deps_blob = []
+
+    for modname, ver_func in deps:
+        try:
+            if modname in sys.modules:
+                mod = sys.modules[modname]
+            else:
+                mod = importlib.import_module(modname)
+            ver = ver_func(mod)
+            deps_blob.append((modname, ver))
+        except ImportError:
+            deps_blob.append((modname, None))
+
+    return dict(deps_blob)
+
+
+def _get_blas_info():
+    """Information on system BLAS
+
+    Uses the `scikit-learn` builtin method
+    :func:`sklearn._build_utils.get_blas_info` which may fail from time to time
+
+    Returns
+    -------
+    blas_info: dict
+        system BLAS information
+
+    """
+    from .._build_utils import get_blas_info
+
+    cblas_libs, blas_dict = get_blas_info()
+
+    macros = ['{key}={val}'.format(key=a, val=b)
+              for (a, b) in blas_dict.get('define_macros', [])]
+
+    blas_blob = [
+        ('macros', ', '.join(macros)),
+        ('lib_dirs', ':'.join(blas_dict.get('library_dirs', ''))),
+        ('cblas_libs', ', '.join(cblas_libs)),
+    ]
+
+    return dict(blas_blob)
+
+
+def show_versions():
+    "Print useful debugging information to the terminal"
+
+    sys_info = _get_sys_info()
+    deps_info = _get_deps_info()
+    blas_info = _get_blas_info()
+
+    print('\nSystem')
+    print('------')
+    for k, stat in sys_info.items():
+        print("{k:>10}: {stat}".format(k=k, stat=stat))
+
+    print('\nBLAS')
+    print('----')
+    for k, stat in blas_info.items():
+        print("{k:>10}: {stat}".format(k=k, stat=stat))
+
+    print('\nPython deps')
+    print('-----------')
+    for k, stat in deps_info.items():
+        print("{k:>10}: {stat}".format(k=k, stat=stat))
diff --git a/sklearn/utils/tests/test_print_versions.py b/sklearn/utils/tests/test_print_versions.py
new file mode 100644
index 0000000000000..a22bb9206ad44
--- /dev/null
+++ b/sklearn/utils/tests/test_print_versions.py
@@ -0,0 +1,32 @@
+
+from sklearn.utils._print_versions import _get_sys_info
+from sklearn.utils._print_versions import _get_deps_info
+from sklearn.utils._print_versions import show_versions
+
+
+def test_get_sys_info():
+    sys_info = _get_sys_info()
+
+    assert 'python' in sys_info
+    assert 'executable' in sys_info
+    assert 'machine' in sys_info
+
+
+def test_get_deps_info():
+    deps_info = _get_deps_info()
+
+    assert 'pip' in deps_info
+    assert 'setuptools' in deps_info
+    assert 'sklearn' in deps_info
+    assert 'numpy' in deps_info
+    assert 'scipy' in deps_info
+    assert 'Cython' in deps_info
+    assert 'pandas' in deps_info
+
+
+def test_show_versions_with_blas(capsys):
+    show_versions()
+    out, err = capsys.readouterr()
+    assert 'python' in out
+    assert 'numpy' in out
+    assert 'BLAS' in out


Code review comments:
diff_hunk: @@ -0,0 +1,117 @@
+"""
+Utility methods to print system info for debugging
+
+adapted from :func:`pandas.show_versions`
+"""
+# License: BSD 3 clause
+
+import platform
+import sys
+import importlib
+
+
+def _get_sys_info():
+    """System information
+
+    Return
+    ------
+    sys_info : dict
+        system and Python version information
+
+    """
+    python = sys.version.replace('\n', ' ')
+
+    blob = [
+        ("python", python),
+        ('executable', sys.executable),
+        ("machine", platform.platform()),
+    ]
+
+    return dict(blob)
+
+
+def _get_deps_info():
+    """Overview of the installed version of main dependencies
+
+    Returns
+    -------
+    deps_info: dict
+        version information on relevant Python libraries
+
+    """
+    deps = [
+        # (MODULE_NAME, f(mod) -> mod version)
+        ("pip", lambda mod: mod.__version__),
+        ("setuptools", lambda mod: mod.__version__),
+        ("sklearn", lambda mod: mod.__version__),
+        ("numpy", lambda mod: mod.__version__),
+        ("scipy", lambda mod: mod.version.version),
+        ("Cython", lambda mod: mod.__version__),
+        ("pandas", lambda mod: mod.__version__),
+    ]
+
+    deps_blob = []
+
+    for modname, ver_func in deps:
+        try:
+            if modname in sys.modules:
+                mod = sys.modules[modname]
+            else:
+                mod = importlib.import_module(modname)
+            ver = ver_func(mod)
+            deps_blob.append((modname, ver))
+        except ImportError:
+            deps_blob.append((modname, None))
+
+    return dict(deps_blob)
+
+
+def _get_blas_info():
+    """Information on system BLAS
+
+    Uses the `scikit-learn` builtin method
+    :func:`sklearn._build_utils.get_blas_info` which may fail from time to time
+
+    Returns
+    -------
+    blas_info: dict
+        system BLAS information
+
+    """
+    from .._build_utils import get_blas_info
+
+    cblas_libs, blas_dict = get_blas_info()
+
+    macros = ['{key}={val}'.format(key=a, val=b)
+              for (a, b) in blas_dict.get('define_macros', [])]
+
+    blas_blob = [
+        ('macros', ', '.join(macros)),
+        ('lib_dirs', ':'.join(blas_dict.get('library_dirs', ''))),
+        ('cblas_libs', ', '.join(cblas_libs)),
+    ]
+
+    return dict(blas_blob)
+
+
+def show_versions():
+    "Print useful debugging information to the terminal"
path: sklearn/utils/_print_versions.py
text: @user1:
remove "to the terminal"

diff_hunk: @@ -0,0 +1,117 @@
+"""
+Utility methods to print system info for debugging
+
+adapted from :func:`pandas.show_versions`
+"""
+# License: BSD 3 clause
+
+import platform
+import sys
+import importlib
+
+
+def _get_sys_info():
+    """System information
+
+    Return
+    ------
+    sys_info : dict
+        system and Python version information
+
+    """
+    python = sys.version.replace('\n', ' ')
+
+    blob = [
+        ("python", python),
+        ('executable', sys.executable),
+        ("machine", platform.platform()),
+    ]
+
+    return dict(blob)
+
+
+def _get_deps_info():
+    """Overview of the installed version of main dependencies
+
+    Returns
+    -------
+    deps_info: dict
+        version information on relevant Python libraries
+
+    """
+    deps = [
+        # (MODULE_NAME, f(mod) -> mod version)
+        ("pip", lambda mod: mod.__version__),
+        ("setuptools", lambda mod: mod.__version__),
+        ("sklearn", lambda mod: mod.__version__),
+        ("numpy", lambda mod: mod.__version__),
+        ("scipy", lambda mod: mod.version.version),
path: sklearn/utils/_print_versions.py
text: @user1:
I still don't get why we can't use `__version__` here and hence in all cases

@user2:
We can and should...

@author:
It slipt off my head. I changed it to avoid the `lambda` function.