YOLO object detection using ONNXRuntime with Ruby
Tags: ruby, machinelearning, deeplearning, onnxruntime
Published: 2019-09-02 04:44:26

---
title: YOLO object detection using ONNXRuntime with Ruby
published: true
description: YOLO object detection using ONNXRuntime with Ruby
tags: ruby, machinelearning, deeplearning, onnxruntime
cover_image: https://thepracticaldev.s3.amazonaws.com/i/qm6x81igt3f7285wxgzv.png
---

## ONNXRuntime
Scoring engine for ML models.

* Microsft
    * https://github.com/microsoft/onnxruntime
* Ruby binding to ONNXRuntime created by Andrew Kane
    * https://github.com/ankane/onnxruntime

## YOLOv3
* ONNX Model Zoo
    * https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/yolov3
* YOLOv3
    * https://pjreddie.com/darknet/yolo/

## Download

```bash
# model
wget https://onnxzoo.blob.core.windows.net/models/opset_10/yolov3/yolov3.onnx
# coco-labels
wget https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt
# gems
gem instal numo-narray mini_magick
```

yolov3.rb

```ruby
require 'mini_magick'
require 'numo/narray'
require 'onnxruntime'

SFloat = Numo::SFloat

input_path = ARGV[0]
output_path = ARGV[1]

model = OnnxRuntime::Model.new('yolov3.onnx')
labels = File.readlines('coco-labels-2014_2017.txt')

# preprocessing
img = MiniMagick::Image.open(input_path)
image_size = [[img.height, img.width]]
img.combine_options do |b|
  b.resize '416x416'
  b.gravity 'center'
  b.background 'transparent'
  b.extent '416x416'
end
img_data = SFloat.cast(img.get_pixels)
img_data /= 255.0
image_data = img_data.transpose(2, 0, 1)
                     .expand_dims(0)
                     .to_a # NArray -> Array

# inference
output = model.predict(input_1: image_data, image_shape: image_size)

# postprocessing
boxes, scores, indices = output.values
results = indices.map do |idx|
  { class: idx[1],
    score: scores[idx[0]][idx[1]][idx[2]],
    box: boxes[idx[0]][idx[2]] }
end

# visualization
img = MiniMagick::Image.open(input_path)
img.colorspace 'gray'
results.each do |r|
  hue = r[:class] * 100 / 80.0
  label = labels[r[:class]]
  score = r[:score]
  # draw box
  y1, x1, y2, x2 = r[:box].map(&:round)
  img.combine_options do |c|
    c.draw        "rectangle #{x1}, #{y1}, #{x2}, #{y2}"
    c.fill        "hsla(#{hue}%, 20%, 80%, 0.25)"
    c.stroke      "hsla(#{hue}%, 70%, 60%, 1.0)"
    c.strokewidth (score * 3).to_s
  end
  # draw text
  img.combine_options do |c|
    c.draw "text #{x1}, #{y1 - 5} \"#{label}\""
    c.fill 'white'
    c.pointsize 18
  end
end

img.write output_path
```

## Run
```bash
ruby yolov3.rb a.jpg b.jpg
```

Elephants.
![elephant.png](https://thepracticaldev.s3.amazonaws.com/i/hjfco4tcztv67p1h8fz8.png)
Ok. Easy.
![elephant2.png](https://thepracticaldev.s3.amazonaws.com/i/tysphvmk8v9r1fnu53rc.png)

Person, horse, dog.
![dog2.jpg](https://thepracticaldev.s3.amazonaws.com/i/6ygeuvvsyl4pyl513iax.jpeg)
![dog3.jpg](https://thepracticaldev.s3.amazonaws.com/i/xzbmmncjnlft8uvug5f1.jpeg)

Where is a dog?
It is a pity that the dog is not recognized by my computer.

![dog.png](https://thepracticaldev.s3.amazonaws.com/i/grqkuedodjtlkc6n99h1.png)

Now It's time for a human being to do something. 

![inu.jpg](https://thepracticaldev.s3.amazonaws.com/i/fqk0nsuvu8swff9x69hq.jpeg)

That's it! 
(*inu* means dog in the Japanese dialect of human language.)

I don't want to lose the human mind even in the age of AI.

## Images
* https://www.pexels.com/
* https://commons.wikimedia.org/wiki/File:Arimatsushibori.JPG
![image.png](https://thepracticaldev.s3.amazonaws.com/i/t7ruhh9cq4qh7jzjj8vw.png)
![image.png](https://thepracticaldev.s3.amazonaws.com/i/qm6x81igt3f7285wxgzv.png)

Thank you for reading.
Have a nice day!