Rubyで物体検出(ONNXRuntime + YOLOv3)

RubyでDeepLearningネタがめずらしくない気もする今日この頃ですが、気にせずにyoloをやりました。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/d7554f51-0865-da3b-b3cc-81d6d17c2362.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/9b2d4c7f-4370-c4d9-20b2-d2e10b9baeae.png)
https://commons.wikimedia.org/wiki/File:Arimatsushibori.JPG


## ONNXRuntime
推論エンジン

* マイクロソフト公式
    * https://github.com/microsoft/onnxruntime
* ankane氏によるRubyバインディング
    * https://github.com/ankane/onnxruntime

## yolov3
* ONNXのモデル
    * https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/yolov3
* 公式
    * https://pjreddie.com/darknet/yolo/

事前準備、ダウンロード

```bash
# model
wget https://onnxzoo.blob.core.windows.net/models/opset_10/yolov3/yolov3.onnx
# coco-labels
wget https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt
# gem
gem instal numo-narray mini_magick
```

yolov3.rb

```ruby
require 'mini_magick'
require 'numo/narray'
require 'onnxruntime'

SFloat = Numo::SFloat

input_path = ARGV[0]
output_path = ARGV[1]

model = OnnxRuntime::Model.new('yolov3.onnx')
labels = File.readlines('coco-labels-2014_2017.txt')

# preprocessing
img = MiniMagick::Image.open(input_path)
image_size = [[img.height, img.width]]
img.combine_options do |b|
  b.resize '416x416'
  b.gravity 'center'
  b.background 'transparent'
  b.extent '416x416'
end
img_data = SFloat.cast(img.get_pixels)
img_data /= 255.0
image_data = img_data.transpose(2, 0, 1)
                     .expand_dims(0)
                     .to_a # NArray -> Array

# inference
output = model.predict(input_1: image_data, image_shape: image_size)

# postprocessing
boxes, scores, indices = output.values
results = indices.map do |idx|
  { class: idx[1],
    score: scores[idx[0]][idx[1]][idx[2]],
    box: boxes[idx[0]][idx[2]] }
end

# visualization
img = MiniMagick::Image.open(input_path)
img.colorspace 'gray'
results.each do |r|
  hue = r[:class] * 100 / 80.0
  label = labels[r[:class]]
  score = r[:score]
  # draw box
  y1, x1, y2, x2 = r[:box].map(&:round)
  img.combine_options do |c|
    c.draw        "rectangle #{x1}, #{y1}, #{x2}, #{y2}"
    c.fill        "hsla(#{hue}%, 20%, 80%, 0.25)"
    c.stroke      "hsla(#{hue}%, 70%, 60%, 1.0)"
    c.strokewidth (score * 3).to_s
  end
  # draw text
  img.combine_options do |c|
    c.draw "text #{x1}, #{y1 - 5} \"#{label}\""
    c.fill 'white'
    c.pointsize 18
  end
end

img.write output_path
```

## 実行
```bash
ruby yolov3.rb a.jpg b.jpg
```

ゾウさん
![elephant.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/e432f1f7-d8e3-670d-8236-959aa491a052.png)
![elephant2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/8da0bc44-3922-fdec-a568-5ccf8a31513e.png)

![cat2.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/b2b7864f-a1e1-d0b9-cb96-20ad806790eb.jpeg)
![cat3.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/0f656ea8-c8ca-f878-38e6-b493fedb9dda.jpeg)

犬がコンピュータに認識されていなくてかわいそうである。

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/55f9444c-a798-5f50-42a4-ef4773254544.png)

こういうときは、やっぱり人間がなんとかしないとね。

![dog.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/144608/097de39d-1d81-2311-f2aa-a2cf28caecd1.jpeg)

これでよし！

AIの時代でも人間らしい心を失わないようにしたいものですね。

この記事は以上です。

## 画像
* https://www.pexels.com/ja-jp/