Pythonで実装 PRML 4章 パーセプトロンアルゴリズムによる分類

パターン認識と機械学習、第4章「線形識別モデル」の図4.7の再現です。

この図はパーセプトロンアルゴリズムでの分類が収束する様子を表したもので、右下の図で収束完了となってます。本当は特徴空間$(\phi_1,\phi_2)$上での話なのですが、作図の都合上、$\phi({\bf x}) = {\bf x}$となる恒等関数ということにしてしまってます。


##実装の大まかな流れ
①求めたい識別境界は(4.52)

```math
  y({\bf x}) = f ({\bf x}^{\bf T} \phi({\bf x}))(4.52)
```

②(4.54)の誤差を最小にするパラメータ[tex:w]を確率的最急降下法で(4.55)を更新しながら求める。

```math
  E_p({\bf w}) = -\sum_{n=1}^N {\bf w}^{\bf T} \phi({\bf x}_n){\bf t}_n(4.54)
```

```math
{\bf w}^{\rm \tau+1} = {\bf w}^{\rm \tau} - \mu \nabla  E_p({\bf{w}}) = {\bf w}^{\rm \tau}-  \mu \phi({\bf x}_n) {\bf t}_n (4.55)
```

##コード
```python
import matplotlib.pyplot as plt
from pylab import *
import numpy as np
import random
%matplotlib inline

def func(train_data, k):
    def perceptron(train_data, k):
        x = 0
        w = array([1.0,1.0, 1.0])
        eta = 0.1
        converged = False
    
        while not converged:
            x += 1
            if x == k:
                converged = True
                
            for i in range(N):
                w_t_phi_x = np.dot(w, [1,train_data[i,0], train_data[i,1]])
                
                if w_t_phi_x > 0:
                    d = 1
                else:
                    d = 2
             
                if d == train_data[i,2]:
                    continue
                else:
                    if d == 1:
                        t = 1
                        w += -eta * array([1, train_data[i,0], train_data[i,1]]) * t
                    else:
                        t = -1
                        w += -eta * array([1, train_data[i,0], train_data[i,1]]) * t
        return w
    
    w = perceptron(train_data, k)

    #Plot
    plt.scatter(x_red,y_red,color='r',marker='x')
    plt.scatter(x_blue,y_blue,color='b',marker='x')
    x_line = linspace(-1.0, 1.0, 1000)
    y_line = -w[1]/w[2] * x_line - w[0]/w[2]
    plt.plot(x_line, y_line, 'k')
    xlim(-1.0, 1.0)
    ylim(-1.0, 1.0)
    title("Figure 7.2")
    show()

if __name__ == "__main__":
    #Generate sumple data
    #Mean
    mu_blue = [-0.5,0.3]
    mu_red = [0.5,-0.3]
    #Covariance
    cov = [[0.08,0.06], [0.06,0.08]]

    #Data Blue
    x_blue = [-0.85, -0.8, -0.5, -0.4, -0.1]
    y_blue = [0.6,0.95, 0.62, 0.7, -0.1]
    blue_label = np.ones(5)

    #Data Red
    x_red = [0.2, 0.4, 0.45, 0.6, 0.95]
    y_red = [0.7, -0.5, 0.4, -0.9, 0.97]
    red_label = np.ones(5)*2

    Data_blue = vstack((x_blue, y_blue, blue_label)).T
    Data_red = vstack((x_red, y_red, red_label)).T
    Data_marge = vstack((Data_blue,Data_red))
    
    N = len(Data_marge)
    for k in (N-9, N-6,N-3, N):
        func(Data_marge, k)
```

##結果
![Screen Shot 2015-09-27 at 00.59.17.png](https://qiita-image-store.s3.amazonaws.com/0/62779/b1b87ca3-99ba-2ed2-915c-27c44d9d2b3d.png)

![Screen Shot 2015-09-27 at 00.59.34.png](https://qiita-image-store.s3.amazonaws.com/0/62779/ea915b92-2244-860f-0c7c-a1861a57da94.png)

![Screen Shot 2015-09-27 at 00.59.53.png](https://qiita-image-store.s3.amazonaws.com/0/62779/72eaf5eb-e5c0-442a-58af-f76bc4554948.png)

![Screen Shot 2015-09-27 at 01.00.02.png](https://qiita-image-store.s3.amazonaws.com/0/62779/a2efbeb3-f6cb-2fb8-5de9-e2537f51023f.png)