scikit-learnのpipelineにsparse codingが組み込めない


# pipeline.Pipelineが便利です

scikit-learn において，特徴次元を削減した後に分類・回帰などをする，など，複数の手法を直列に接続し，それぞれの手法のパラメータを model_selection.GridSearchCV を用いてグリッドサーチで最適化するには，pipeline.Pipeline を用いるのが良さそうです．

Scikit-learnのpipeleine.Pipelineが便利
http://tkzs.hatenablog.com/entry/2016/06/26/093502

# でも，sparse codingが使えない．

しかし，この pipeline.Pipeline は，スパースコーディング decomposition.SparseCoder を要素として利用することができません．以下がそのテストコードです．

```sparse_coding_test.py
from sklearn.decomposition import SparseCoder
from sklearn.svm import LinearSVC
from sklearn.pipeline import Pipeline
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import Normalizer
from sklearn.datasets import load_iris

# parameters
n_samples_dict = 5

# training samples
iris = load_iris()
# Take the first two features. We could avoid this by using a two-dim dataset
X_train = iris.data
y_train = iris.target

# dictionary
rand_dict = Normalizer(norm='l2').transform(X_train[:n_samples_dict])

# models
sp = SparseCoder(dictionary=rand_dict, transform_algorithm='threshold')
svm = LinearSVC()
pipeline = Pipeline([ ('sp', sp), ('svm', svm) ])

# grid search
grid = {'sp__transform_alpha': [0.1, 1, 10], 'svm__C': [0.1, 1, 10]}
model = GridSearchCV(estimator=pipeline, param_grid=grid, scoring='accuracy')

# check
print('Components: \n', model.estimator.named_steps['sp'].components_)
print('Sparse coder: \n', model.estimator.named_steps['sp'])
print('Sparse coder: \n', sp)
print('Sparse coder: \n', SparseCoder(dictionary=rand_dict, transform_algorithm='threshold'))

# test
X_transformed = sp.transform(X_train)
print('original: \n', X_train[5:10])
print('transformed: \n', X_transformed[5:10])

# fit
model.fit(X_train, y_train)
print(model.best_estimator_)
```

実行すると以下のようになります．

```
# python sparse_coding_test.py

Components: 
 [[ 0.80377277  0.55160877  0.22064351  0.0315205 ]
 [ 0.82813287  0.50702013  0.23660939  0.03380134]
 [ 0.80533308  0.54831188  0.2227517   0.03426949]
 [ 0.80003025  0.53915082  0.26087943  0.03478392]
 [ 0.790965    0.5694948   0.2214702   0.0316386 ]]
Sparse coder: 
 SparseCoder(dictionary=None, n_jobs=1, split_sign=False,
      transform_algorithm='threshold', transform_alpha=None,
      transform_n_nonzero_coefs=None)
Sparse coder: 
 SparseCoder(dictionary=None, n_jobs=1, split_sign=False,
      transform_algorithm='threshold', transform_alpha=None,
      transform_n_nonzero_coefs=None)
Sparse coder: 
 SparseCoder(dictionary=None, n_jobs=1, split_sign=False,
      transform_algorithm='threshold', transform_alpha=None,
      transform_n_nonzero_coefs=None)
original: 
 [[ 5.4  3.9  1.7  0.4]
 [ 4.6  3.4  1.4  0.3]
 [ 5.   3.4  1.5  0.2]
 [ 4.4  2.9  1.4  0.2]
 [ 4.9  3.1  1.5  0.1]]
transformed: 
 [[ 5.87934932  5.86505251  5.87960063  5.88026013  5.88139547]
 [ 4.89118162  4.8746732   4.89092577  4.8889183   4.89427115]
 [ 5.23160303  5.22620715  5.23190722  5.23153995  5.22964031]
 [ 4.45147063  4.45215643  4.45227627  4.45585845  4.4481669 ]
 [ 4.98259107  4.98790769  4.9834534   4.98631329  4.97653151]]
Traceback (most recent call last):
  File "sparse_coder_test.py", line 51, in <module>
    model.fit(X_train, y_train)
  File ".../site-packages/sklearn/model_selection/_search.py", line 624, in fit
    base_estimator = clone(self.estimator)
  File ".../site-packages/sklearn/base.py", line 61, in clone
    new_object_params[name] = clone(param, safe=False)
  File ".../site-packages/sklearn/base.py", line 49, in clone
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File ".../site-packages/sklearn/base.py", line 49, in clone
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File ".../site-packages/sklearn/base.py", line 62, in clone
    new_object = klass(**new_object_params)
  File ".../site-packages/sklearn/decomposition/dict_learning.py", line 916, in __init__
    self._set_sparse_coding_params(dictionary.shape[0],
AttributeError: 'NoneType' object has no attribute 'shape'
```

# なぜでしょう？

上記の状況をまとめると，たぶんこんな感じです．

* GridSearchCV で定義した model を fit する際に，パラメータを固定した model を新しく作って fit する仕組みになっている．（"model_selection/_search.py", line 624, in fit）
* この「model を新しく作る」ステップにおいて，SparseCoder の（deepcopyを作れば十分なはずなのに）インスタンスを新しく作ろうとする．（"base.py", line 62, in clone）
* SparseCoder の新しいインスタンスを作るときに，参照元の SparseCoder のインスタンスから（self.components_ を引き継げれば良いはずが） dictionary を引き継ぐこと（で対応しようとしているがそれ）はできない．（途中経過の Sparse coder の print 出力）
* なので， \_\_init\_\_ が正常にできず，エラーで終了．（"decomposition/dict_learning.py", line 916, in \_\_init\_\_）

（追記 2017.8.23： やはり SparseCoder のバグだったようです． 参照： Problem with SparseCoder Object, dictionary not assigned #8675
 https://github.com/scikit-learn/scikit-learn/issues/8675 ）

# 解決方法

残念なことに，今のところ本質的な解決策はありません．暫定的な解決策としては，自分で GridSearchCV まがいの方法を実装する方法です．

```sparse_coding_test_tmp.py
from sklearn.decomposition import SparseCoder
from sklearn.svm import LinearSVC
from sklearn.pipeline import Pipeline
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import Normalizer
from sklearn.datasets import load_iris

# parameters
n_samples_dict = 5

# training samples
iris = load_iris()
# Take the first two features. We could avoid this by using a two-dim dataset
X_train = iris.data
y_train = iris.target

# dictionary
rand_dict = Normalizer(norm='l2').transform(X_train[:n_samples_dict])

# models
sp = SparseCoder(dictionary=rand_dict, transform_algorithm='threshold')
svm = LinearSVC()

# grid search
grid_sp  = {'transform_alpha': [0.1, 1, 10]}
grid_svm = {'C': [0.1, 1, 10]}
model = GridSearchCV(estimator=svm, param_grid=grid_svm, scoring='accuracy')

# for each transform_alpha
best_score = 0.0
for alpha in grid_sp['transform_alpha']:
	sp.transform_alpha = alpha
	X_train_trans = sp.transform(X_train)
	smodel.fit(X_train_trans, y_train)
	if model.best_score_ > best_score:
    	best_estimator = model.best_estimator_
    	best_score = model.best_score_

# print
print(best_estimator)
```

本質的な解決策が何か見つかったら，この記事を更新したいと思います．