Issue 1:
Before I start I'm not sure if this is an issue of interest, however from my perspective it is a documentation concern at least.

Additionally I could be using this incorrectly. Please inform me if that's the case.

Description

I'm working with multi-label data and I've been receiving results from an json api.

I got this process working in a jupyter notebook, so I wasn't certain why it didn't appear to be working when I was calling this api with more data.

Steps/Code to Reproduce

import numpy as np
from sklearn import metrics

targets = [[0, 0, 1, 0],
[1, 0, 1, 0],
[1, 0, 0, 1],
[0, 1, 0, 1],
[0, 0, 1, 0],
[0, 1, 0, 1],
[0, 0, 1, 1],
[1, 1, 0, 0],
[0, 1, 0, 1],
[1, 0, 0, 0],
[0, 0, 1, 1],
[1, 1, 0, 0],
[0, 1, 1, 0],
[0, 1, 0, 1],
[1, 0, 1, 0]]

pred_classes = [[0, 0, 1, 0],
[1, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 0, 1],
[0, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 1, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 1, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0]]

target_names = ['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']

Expected Results

print(metrics.classification_report(targets, pred_classes, target_names=target_names))

precision recall f1-score support
alt.atheism 0.50 0.17 0.25 6
comp.graphics 0.75 0.43 0.55 7
sci.med 0.60 0.43 0.50 7
soc.religion.christian 0.75 0.43 0.55 7
avg / total 0.66 0.37 0.47 27

Actual Results

print(metrics.classification_report(targets, pred_classes, target_names=target_names))

Traceback (most recent call last):
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2885, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-34-ba0a51c823fd>", line 1, in <module>
print(metrics.classification_report(targets, pred_classes, target_names=target_names))
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1391, in classification_report
labels = unique_labels(y_true, y_pred)
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/utils/multiclass.py", line 98, in unique_labels
raise ValueError("Unknown label type: %s" % repr(ys))
ValueError: Unknown label type: ([[0, 0, 1, 0], [1, 0, 1, 0], [1, 0, 0, 1], [0, 1, 0, 1], [0, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 0, 1], [1, 0, 0, 0], [0, 0, 1, 1], [1, 1, 0, 0], [0, 1, 1, 0], [0, 1, 0, 1], [1, 0, 1, 0]], [[0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0]])

print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))

Traceback (most recent call last):
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2885, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-35-f56c95fbee5e>", line 1, in <module>
print(metrics.classification_report(targets, pred_classes, labels=[0, 1, 2, 3], target_names=target_names))
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1415, in classification_report
sample_weight=sample_weight)
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 1003, in precision_recall_fscore_support
y_type, y_true, y_pred = _check_targets(y_true, y_pred)
File "/Users/folcon/.virtualenvs/thoughtriver/lib/python3.5/site-packages/sklearn/metrics/classification.py", line 89, in _check_targets
raise ValueError("{0} is not supported".format(y_type))
ValueError: multiclass-multioutput is not supported

Solution

targets = np.vstack(targets)
pred_classes = np.vstack(pred_classes)

print(metrics.classification_report(targets, pred_classes, target_names=target_names))
precision recall f1-score support
alt.atheism 0.50 0.17 0.25 6
comp.graphics 0.75 0.43 0.55 7
sci.med 0.60 0.43 0.50 7
soc.religion.christian 0.75 0.43 0.55 7
avg / total 0.66 0.37 0.47 27

Suggestions

A possible improvement would be a warning that the datastructure could be the issue.

Alternatively this is outside the scope of the library and that you expect users to be aware of the data they are working with.

Versions

Darwin-15.6.0-x86_64-i386-64bit

Python 3.5.2 (default, Oct 11 2016, 05:05:28)

[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)]

NumPy 1.12.0b1

SciPy 0.18.1

Scikit-Learn 0.18

Issue 2:
Please note that this is a follow up issue to #4701 and #4197 not a duplicate.

Calling sklearn.metrics.silhouette_samples with a large data set will cause a memory MemoryError in numpy.dot. This is not an issue with numpy.dot but with the implementation of sklearn.metrics.silhouette_samples. For a data set of length N it computes the full NxN pairwise distance matrix which can clearly take a lot of memory. Mathematically all that is needed in memory is a list of the same length as the largest cluster and the rest can be done by appropriate looping.

This issue is to propose a new feature where a new parameter is added to the sklearn.metrics.silhouette_samples method. This would be a boolean to indicate if one wanted to use the faster but more memory consumptive existing method or the alternative method which is better on memory but likely slower.

A working version of the the new method was posted in an answer on StackOverflow