<科目> 機械学習 第五章:アルゴリズム1(k近傍法(kNN))

#<科目> 機械学習

目次
[第一章:線形回帰モデル](https://qiita.com/matsukura04583/items/8892d8a41f7ccbcfe739)
[第二章:非線形回帰モデル]
(https://qiita.com/matsukura04583/items/baa3f2269537036abc57)
[第三章:ロジスティク回帰モデル]
(https://qiita.com/matsukura04583/items/0fb73183e4a7a6f06aa5)
[第四章:主成分分析]
(https://qiita.com/matsukura04583/items/b3b5d2d22189afc9c81c)
[第五章:アルゴリズム1(k近傍法(kNN))]
(https://qiita.com/matsukura04583/items/543719b44159322221ed)
[第六章:アルゴリズム2(k-means)]
(https://qiita.com/matsukura04583/items/050c98c7bb1c9e91be71)
[第七章:サポートベクターマシン]
(https://qiita.com/matsukura04583/items/6b718642bcbf97ae2ca8)


##第五章:アルゴリズム1(k近傍法(kNN))
###k近傍法(kNN)
+ 分類問題のための機械学習手法
    + 最近傍のデータを個取ってきて、それらがもっとも多く所属するクラスに識別
![kNN1.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/f14d6e58-b855-d1b0-e516-261e67a5ae46.jpeg)
+ kを変化させると結果も変わる
![kNN2.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/5ac7b66f-f666-6948-2cb4-81f369417538.jpeg)
+ kを大きくすると決定境界は滑らかになる
![kNN3.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/d3d6ce22-8e41-d003-1a34-485f8ab2210f.jpeg)

###(演習5) 人口データと分類結果をプロット
+ 設定 人口データを分類
+ 課題 人口データと分類結果をプロットしてください

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```
####訓練データ生成
```python
def gen_data():
    x0 = np.random.normal(size=50).reshape(-1, 2) - 1
    x1 = np.random.normal(size=50).reshape(-1, 2) + 1.
    x_train = np.concatenate([x0, x1])
    y_train = np.concatenate([np.zeros(25), np.ones(25)]).astype(np.int)
    return x_train, y_train

X_train, ys_train = gen_data()
plt.scatter(X_train[:, 0], X_train[:, 1], c=ys_train)

```

<img width="533" alt="スクリーンショット 2019-12-13 17.15.47.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/f830bbe2-cf6d-f5bd-7c15-df479776062f.png">

####学習 ステップはない
####予測 
予測するデータ点との、距離が最も近い 𝑘 個の、訓練データのラベルの最頻値を割り当てる

```python
def distance(x1, x2):
    return np.sum((x1 - x2)**2, axis=1)

def knc_predict(n_neighbors, x_train, y_train, X_test):
    y_pred = np.empty(len(X_test), dtype=y_train.dtype)
    for i, x in enumerate(X_test):
        distances = distance(x, X_train)
        nearest_index = distances.argsort()[:n_neighbors]
        mode, _ = stats.mode(y_train[nearest_index])
        y_pred[i] = mode
    return y_pred

def plt_resut(x_train, y_train, y_pred):
    xx0, xx1 = np.meshgrid(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100))
    xx = np.array([xx0, xx1]).reshape(2, -1).T
    plt.scatter(x_train[:, 0], x_train[:, 1], c=y_train)
    plt.contourf(xx0, xx1, y_pred.reshape(100, 100).astype(dtype=np.float), alpha=0.2, levels=np.linspace(0, 1, 3))
```

```python
n_neighbors = 3

xx0, xx1 = np.meshgrid(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100))
X_test = np.array([xx0, xx1]).reshape(2, -1).T

y_pred = knc_predict(n_neighbors, X_train, ys_train, X_test)
plt_resut(X_train, ys_train, y_pred)
```
<img width="417" alt="スクリーンショット 2019-12-13 17.21.24.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/8904f8e5-62fb-e55c-4283-d394fb91d8ca.png">

####numpy実装でも試してみる

```python
xx0, xx1 = np.meshgrid(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100))
xx = np.array([xx0, xx1]).reshape(2, -1).T

from sklearn.neighbors import KNeighborsClassifier
knc = KNeighborsClassifier(n_neighbors=n_neighbors).fit(X_train, ys_train)
plt_resut(X_train, ys_train, knc.predict(xx))
```
<img width="417" alt="スクリーンショット 2019-12-13 17.23.56.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/a89d7b12-a589-8a21-c81f-6ec5dd106755.png">

```python
n_neighbors = 15

xx0, xx1 = np.meshgrid(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100))
X_test = np.array([xx0, xx1]).reshape(2, -1).T

y_pred = knc_predict(n_neighbors, X_train, ys_train, X_test)
plt_resut(X_train, ys_train, y_pred)
```
![kNN4.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/357717/7d60f889-9cbb-ebb8-84f0-fe56d3e5383c.jpeg)
**n_neighborsの値を3から15に上げてみると、大分滑らかになった。**

関連サイト
[第一章:線形回帰モデル](https://qiita.com/matsukura04583/items/8892d8a41f7ccbcfe739)
[第二章:非線形回帰モデル]
(https://qiita.com/matsukura04583/items/baa3f2269537036abc57)
[第三章:ロジスティク回帰モデル]
(https://qiita.com/matsukura04583/items/0fb73183e4a7a6f06aa5)
[第四章:主成分分析]
(https://qiita.com/matsukura04583/items/b3b5d2d22189afc9c81c)
[第五章:アルゴリズム1(k近傍法(kNN))]
(https://qiita.com/matsukura04583/items/543719b44159322221ed)
[第六章:アルゴリズム2(k-means)]
(https://qiita.com/matsukura04583/items/050c98c7bb1c9e91be71)
[第七章:サポートベクターマシン]
(https://qiita.com/matsukura04583/items/6b718642bcbf97ae2ca8)