Jetbot & PyTorch CNN（３）衝突回避用のデモプログラム

ついに衝突回避モデルのデモプログラムです。
live_demo_resnet18.ipynbを修正し、CNNモデルで動くようにします。写真のサイズも小さくしています（56x112）。

IMPORT（追加あり）

```ruby:live_demo_resnet18.ipynb
import torch
import torchvision
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

from PIL import Image
import numpy as np
```

ネットワークの定義（新規）

```ruby:live_demo_resnet18.ipynb
class miniNet(nn.Module):

    def __init__(self, num_classes=2):
        super(miniNet, self).__init__()

        self.conv1 = nn.Conv2d(3, 6, 3)
        self.pool1 = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 3)
        self.pool2 = nn.MaxPool2d(2, 2)
        self.conv3 = nn.Conv2d(16, 32, 3)
        self.pool3 = nn.MaxPool2d(2, 2)

        #self.fc1 = nn.Linear(32 * 26 * 26, 1024) # 写真サイズ変更
        self.fc1 = nn.Linear(4160, 1024)          # 写真サイズ変更
        self.fc2 = nn.Linear(1024, 512)
        self.fc3 = nn.Linear(512, num_classes)


    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.pool1(x)
        x = F.relu(self.conv2(x))
        x = self.pool2(x)
        x = F.relu(self.conv3(x))
        x = self.pool3(x)

        #x = x.view(-1, 32 * 26 * 26)      # 写真サイズ変更
        x = x.view(-1, 4160)               # 写真サイズ変更
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)

        return x
```

モデルのロード（新規）

```ruby:live_demo_resnet18.ipynb
trained_net = miniNet()
trained_net.load_state_dict(torch.load('best_model_CNN.pth'))
print(trained_net)
```

GPUに渡す（修正）

```ruby:live_demo_resnet18.ipynb
device = torch.device('cuda')
trained_net.to(device)                       # モデル変更
trained_net = trained_net.eval().half()      # モデル変更
```

Create the preprocessing function（修正）

```ruby:live_demo_resnet18.ipynb
import torchvision.transforms as transforms
import torch.nn.functional as F
import cv2
import PIL.Image
import numpy as np

mean = torch.Tensor([0.485, 0.456, 0.406]).cuda().half()
std = torch.Tensor([0.229, 0.224, 0.225]).cuda().half()

normalize = torchvision.transforms.Normalize(mean, std)

def preprocess(image):
    image2 = np.copy(image)             # 写真サイズ変更
    image2 = image2[168:224, 0:224]     # 写真サイズ変更
    image = PIL.Image.fromarray(image2)
    image = transforms.functional.to_tensor(image).to(device).half()
    image.sub_(mean[:, None, None]).div_(std[:, None, None])
    return image[None, ...]
```

スライダーとカメラの表示（修正なし）

```ruby:live_demo_resnet18.ipynb
import traitlets
from IPython.display import display
import ipywidgets.widgets as widgets
from jetbot import Camera, bgr8_to_jpeg

camera = Camera.instance(width=224, height=224)
image = widgets.Image(format='jpeg', width=224, height=224)



blocked_slider = widgets.FloatSlider(description='blocked', min=0.0, max=1.0, orientation='vertical')

camera_link = traitlets.dlink((camera, 'value'), (image, 'value'), transform=bgr8_to_jpeg)

display(widgets.HBox([image, blocked_slider]))
```

 robot instance（修正なし）

```ruby:live_demo_resnet18.ipynb
from jetbot import Robot
robot = Robot()
```

neural network execution function（修正なし）

```ruby:live_demo_resnet18.ipynb
import torch.nn.functional as F
import time

def update(change):
    global blocked_slider, robot
    x = change['new'] 
    x = preprocess(x)
    y = trained_net(x)
    
    # we apply the `softmax` function to normalize the output vector so it sums to 1 (which makes it a probability distribution)
    y = F.softmax(y, dim=1)
    
    prob_blocked = float(y.flatten()[0])
    
    blocked_slider.value = prob_blocked
    
    if prob_blocked < 0.5:
        robot.forward(0.2)
    else:
        robot.forward(0.0)
    
    time.sleep(0.001)
        
update({'new': camera.value})  # we call the function once to intialize
```

observe function（修正なし）

```ruby:live_demo_resnet18.ipynb
camera.observe(update, names='value')  # this attaches the 'update' function to the 'value' traitlet of our camera

```

stop this behavior,（修正なし）

```ruby:live_demo_resnet18.ipynb
import time

camera.unobserve(update, names='value')

time.sleep(0.1)  # add a small sleep to make sure frames have finished processing

robot.stop()
```

close the camera conneciton（修正なし）

```ruby:live_demo_resnet18.ipynb
camera.stop()
```