Repository: scikit-learn/scikit-learn
Pull request: [MRG] Change default dataset for `plot_johnson_lindenstrauss_bound.py`
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md#pull-request-checklist
-->

#### Reference Issues/PRs

Closes #12649 


#### What does this implement/fix? Explain your changes.

Change the default dataset used in the `examples/plot_johnson_lindenstrauss_bound.py`  to the  twenty news groups. Use the digits dataset only if the flag `---use-digits-dataset` is specified. 

Also, changed the hexbin plots to have square heatmaps.

#### Any other comments?


<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
`plot_johnson_lindenstrauss_bound.py` projects a 64 dim data into 300, 1000, and 10000
Trying to fix warnings in our examples, not sure what to do with this one:

`examples/plot_johnson_lindenstrauss_bound.py` projects the data which has 64 features into a 300, 1000, and 10000 dimensional space, which results in the following warning:

```
DataDimensionalityWarning: The number of components is higher than the number of
features: n_features < n_components (64 < 10000).The dimensionality of the problem
will not be reduced.
```

Which one should we do?
1. Ignore the warning (`warnings.filterwarnings`)
2. Use a sample data with higher dimensions
3. Project into lower dimensions than 64
4. Do nothing

Hints from issue discussion:
We could decide to use the twenty news groups dataset by default. It requires a data fetch. This is why it's not enabled by default. However there are many other examples that already use this dataset so +1 for using 20 newsgroups by default in this example. We change the handling of the command line flags to use the digits dataset only if the users pass the `---use-digits-dataset` flag. The docstring will need to be updated.

If someone fixes this, it would be great to also set xlim and ylim for the hexbin plots to have square heatmaps. A perfectly distance preserving projection would lie on the diagonal.

```python
max_dist = max(projected_dists.max(), dists.max())
plt.xlim(0, max_dist)
plt.ylim(0, max_dist)
```
I will be working on this

Commit message: fix doctsring

Patch to review:
diff --git a/examples/plot_johnson_lindenstrauss_bound.py b/examples/plot_johnson_lindenstrauss_bound.py
index 988de0fe2735b..931aad82ca93f 100644
--- a/examples/plot_johnson_lindenstrauss_bound.py
+++ b/examples/plot_johnson_lindenstrauss_bound.py
@@ -102,27 +102,26 @@
 # Empirical validation
 # ====================
 #
-# We validate the above bounds on the digits dataset or on the 20 newsgroups
-# text document (TF-IDF word frequencies) dataset:
-#
-# - for the digits dataset, some 8x8 gray level pixels data for 500
-#   handwritten digits pictures are randomly projected to spaces for various
-#   larger number of dimensions ``n_components``.
+# We validate the above bounds on the 20 newsgroups text document
+# (TF-IDF word frequencies) dataset or on the digits dataset:
 #
 # - for the 20 newsgroups dataset some 500 documents with 100k
 #   features in total are projected using a sparse random matrix to smaller
 #   euclidean spaces with various values for the target number of dimensions
 #   ``n_components``.
 #
-# The default dataset is the digits dataset. To run the example on the twenty
-# newsgroups dataset, pass the --twenty-newsgroups command line argument to
-# this script.
+# - for the digits dataset, some 8x8 gray level pixels data for 500
+#   handwritten digits pictures are randomly projected to spaces for various
+#   larger number of dimensions ``n_components``.
+#
+# The default dataset is the 20 newsgroups dataset. To run the example on the
+# digits dataset, pass the ---use-digits-dataset command line argument to this
+# script.
 
-if '--twenty-newsgroups' in sys.argv:
-    # Need an internet connection hence not enabled by default
-    data = fetch_20newsgroups_vectorized().data[:500]
-else:
+if '---use-digits-dataset' in sys.argv:
     data = load_digits().data[:500]
+else:
+    data = fetch_20newsgroups_vectorized().data[:500]
 
 ##########################################################
 # For each value of ``n_components``, we plot:
@@ -158,7 +157,10 @@
         projected_data, squared=True).ravel()[nonzero]
 
     plt.figure()
-    plt.hexbin(dists, projected_dists, gridsize=100, cmap=plt.cm.PuBu)
+    min_dist = min(projected_dists.max(), dists.min())
+    max_dist = max(projected_dists.max(), dists.max())
+    plt.hexbin(dists, projected_dists, gridsize=100, cmap=plt.cm.PuBu,
+               extent=[min_dist, max_dist, min_dist, max_dist])
     plt.xlabel("Pairwise squared distances in original space")
     plt.ylabel("Pairwise squared distances in projected space")
     plt.title("Pairwise distances distribution for n_components=%d" %


Code review comments:
diff_hunk: @@ -102,27 +102,26 @@
 # Empirical validation
 # ====================
 #
-# We validate the above bounds on the digits dataset or on the 20 newsgroups
-# text document (TF-IDF word frequencies) dataset:
-#
-# - for the digits dataset, some 8x8 gray level pixels data for 500
-#   handwritten digits pictures are randomly projected to spaces for various
-#   larger number of dimensions ``n_components``.
+# We validate the above bounds on the 20 newsgroups text document
+# (TF-IDF word frequencies) dataset or on the digits dataset:
 #
 # - for the 20 newsgroups dataset some 500 documents with 100k
 #   features in total are projected using a sparse random matrix to smaller
 #   euclidean spaces with various values for the target number of dimensions
 #   ``n_components``.
 #
-# The default dataset is the digits dataset. To run the example on the twenty
-# newsgroups dataset, pass the --twenty-newsgroups command line argument to
-# this script.
+# - for the digits dataset, some 8x8 gray level pixels data for 500
+#   handwritten digits pictures are randomly projected to spaces for various
+#   larger number of dimensions ``n_components``.
+#
+# The default dataset is the 20 newsgroups dataset. To run the example on the
+# digits dataset, pass the ---use-digits-dataset command line argument to this
path: examples/plot_johnson_lindenstrauss_bound.py
text: @user1:
```suggestion
# digits dataset, pass the ``--use-digits-dataset`` command line argument to this
```

diff_hunk: @@ -102,27 +102,26 @@
 # Empirical validation
 # ====================
 #
-# We validate the above bounds on the digits dataset or on the 20 newsgroups
-# text document (TF-IDF word frequencies) dataset:
-#
-# - for the digits dataset, some 8x8 gray level pixels data for 500
-#   handwritten digits pictures are randomly projected to spaces for various
-#   larger number of dimensions ``n_components``.
+# We validate the above bounds on the 20 newsgroups text document
+# (TF-IDF word frequencies) dataset or on the digits dataset:
 #
 # - for the 20 newsgroups dataset some 500 documents with 100k
 #   features in total are projected using a sparse random matrix to smaller
 #   euclidean spaces with various values for the target number of dimensions
 #   ``n_components``.
 #
-# The default dataset is the digits dataset. To run the example on the twenty
-# newsgroups dataset, pass the --twenty-newsgroups command line argument to
-# this script.
+# - for the digits dataset, some 8x8 gray level pixels data for 500
+#   handwritten digits pictures are randomly projected to spaces for various
+#   larger number of dimensions ``n_components``.
+#
+# The default dataset is the 20 newsgroups dataset. To run the example on the
+# digits dataset, pass the ---use-digits-dataset command line argument to this
+# script.
 
-if '--twenty-newsgroups' in sys.argv:
-    # Need an internet connection hence not enabled by default
-    data = fetch_20newsgroups_vectorized().data[:500]
-else:
+if '---use-digits-dataset' in sys.argv:
path: examples/plot_johnson_lindenstrauss_bound.py
text: @user1:
```suggestion
if '--use-digits-dataset' in sys.argv:
```