理解したい！カーネル主成分分析まとめ

# このスライドについて
以前にカーネル法について自分なりにまとめたものです。カーネル法の概念は大変難しく、専門でない自分にはかなり苦労しました。間違っているところがあったら教えていただけたら嬉しいです。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/061b9593-b10c-a6f5-869d-82b9c6dcfd66.png)
# 目次
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/98130bce-3002-1020-a203-c9980ab486b3.png)
# PCA 主成分分析の概要
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/92259f6c-2e3b-f183-8b52-bdbe381bb559.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/01818ffa-af90-0d2f-184f-394478a75b7d.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/a33ebdff-699b-b099-68a3-7b10eaea6cb6.png)
# カーネルPCAの概要
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/e76aa771-6f2e-8721-7f0d-32022b975c0f.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/b1c2819f-9048-e08d-68c9-8ff88a843170.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/c877f5ea-aea3-e007-7f71-965686e0fc0a.png)
# カーネルPCAの導出
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/da60422f-c276-cb4f-cc6f-3d0a373b150a.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/51f63e9e-271f-ef0a-a01e-b9fd6954ee0a.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/2e730721-11ec-bc81-32f2-87b355880b12.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/3b708e33-3af8-b808-4a1e-50012dd7c33e.png)
# カーネルトリックについて
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/99504599-6448-f0c1-4a14-96533dda1477.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/2d9b266f-93c3-2929-12a1-4e47b38152ee.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/61fb494d-3b82-0d89-7619-1f6d6801ccda.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/1acd5b17-c0e5-72a9-4634-30ecdc01095e.png)
# カーネルPCAの実装
ここでは簡単にRBFカーネルを```scipy```などを用いて実装してみる。
実際に使う際は絶対```sklearn```がいい！
```python
from scipy.spatial.distance import pdist, squareform
from scipy import exp
from scipy.linalg import eigh
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.datasets import make_moons

X,y = make_moons(n_samples=100)
X1 = X[:,0]
X2 = X[:,1]

plt.scatter(x=X1, y=X2, c=y)
plt.title('make_moos')
plt.show()
```
![make_moons.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/425534e2-56a8-0a01-7848-982c2f6890bd.png)

```python
def rbf_kernel_pca(X, gamma, n_components):
    #M N 次元のデータセットでペアごとのユークリッド距離の二乗を計算
    sq_dists = pdist(X, 'sqeuclidean')
    #ペアごとの距離を正方行列に変換
    mat_sq_dists = squareform(sq_dists)
    #対称カーネル行列に変換
    K = exp(-gamma * mat_sq_dists)

    #カーネル行列の中心化
    N = K.shape[0]
    one_n = np.ones((N,N)) / N
    K = K - one_n.dot(K) - K.dot(one_n) + one_n.dot(K).dot(one_n)

    #固有値固有ベクトルの取得
    eigvals, eigvecs = eigh(K)
    eigvals, eigvecs = eigvals[::-1], eigvecs[:, ::-1]

    #上位の固有ベクトルの収集
    X_pc = np.column_stack((eigvecs[:, i] for i in range(n_components)))

    return X_pc
```
```python
X_kpca = rbf_kernel_pca(X=X, gamma=50, n_components=2)

X1_kpca = X_kpca[:,0]
X2_kpca = X_kpca[:,1]


plt.scatter(x=X1_kpca, y=X2_kpca, c=y)
plt.title('make_moos KernelPCA')
plt.savefig('kernel_pca')
```
![kernel_pca.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/502f6983-7ed2-5220-4c14-4dc8f3f76845.png)


# make_moon datasetのカーネルPCA
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/2755c7ae-9aab-1846-5948-ffa4e7fc6ca0.png)
# まとめ
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2730459/d3a34c75-a770-f0dc-0a33-47657137b879.png)