YOLOv7 object detection in Ruby in 10 minutes
Tags: ruby, onnx, deeplearning
Published: 2022-07-27 06:30:00

Young people these days don't do Ruby.

So I am writing this note about YOLOv7 in Ruby. Since there is ONNX Runtime, almost anything works if you just want to infer.

{% embed https://github.com/ankane/onnxruntime-ruby %}

I wrote ↓ three years ago, so I will reuse code.

{% embed https://dev.to/kojix2/yolo-object-detection-using-onnxruntime-with-ruby-1ap0 %}

The YOLOv7 ONNX model exists. ↓

{% embed https://github.com/ibaiGorordo/ONNX-YOLOv7-Object-Detection %}

Download the ONNX model from here ↓ : [307_YOLOv7](https://github.com/PINTO0309/PINTO_model_zoo/tree/main/307_YOLOv7)

{% embed https://github.com/PINTO0309/PINTO_model_zoo %}

The key point here is to use `post-process_merged` to download a model with post-process implemented, otherwise you will have to implement post-process yourself, which is troublesome.

```sh
git clone https://github.com/PINTO0309/PINTO_model_zoo
cd PINTO_model_zoo/307_YOLOv7
chmod +x download_single_batch_post-process_merged.sh
./download_single_batch_post-process_merged.sh
```

Now the models are downloaded. Use `yolov7_post_640x640.onnx`, which seems to be the main one. 

Install the Ruby version of onnxruntime.

```sh
gem install onnxruntime
```

[Those who want to use GPUs need to set up their own](https://github.com/ankane/onnxruntime-ruby#gpu-support).

Prepare Netron to check the ONNX model.

{% embed https://github.com/lutzroeder/netron %}

It is a good idea to open the model in Netron to see what the inputs and outputs look like.

![Netron](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/meq96g2i9o2sf3q6mn50.png)

Very complicated. All I know is the shape of the INPUT and OUTPUT matrices. Let's Write throwaway code!

```rb
require 'mini_magick'
require 'numo/narray'
require 'onnxruntime'

SFloat = Numo::SFloat

input_path = ARGV[0]
output_path = ARGV[1]

model = OnnxRuntime::Model.new('yolov7_post_640x640.onnx')

labels = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train',
          'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter',
          'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant',
          'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
          'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite',
          'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket',
          'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana',
          'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
          'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table',
          'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
          'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock',
          'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']

# preprocessing
img = MiniMagick::Image.open(input_path)
img.combine_options do |b|
  b.resize '640x640!'
  b.gravity 'center'
  b.background 'transparent'
# b.extent '640x640'
end
img_data = SFloat.cast(img.get_pixels)
img_data /= 255.0
image_data = img_data.transpose(2, 0, 1)
                     .expand_dims(0)
                     .to_a # NArray -> Array

# inference
output = model.predict({ images: image_data })

# postprocessing
scores, indices = output.values

# visualization
img = MiniMagick::Image.open(input_path)
img.colorspace 'gray'
scores.zip(indices).each do |score, i|
  cl = i[1] # cl is class
  hue = cl * 100 / 80.0
  label = labels[cl]
  score = score[0]
  p "draw box"
  y1 = i[2] * img.height / 640
  x1 = i[3] * img.width  / 640
  y2 = i[4] * img.height / 640
  x2 = i[5] * img.width  / 640
  img.combine_options do |c|
    c.draw        "rectangle #{x1}, #{y1}, #{x2}, #{y2}"
    c.fill        "hsla(#{hue}%, 20%, 80%, 0.25)"
    c.stroke      "hsla(#{hue}%, 70%, 60%, 1.0)"
    c.strokewidth (score * 3).to_s
  end
  # draw text
  img.combine_options do |c|
    c.draw "text #{x1}, #{y1 - 5} \"#{label}\""
    c.fill 'white'
    c.pointsize 18
  end
end

img.write output_path
```

Then run `ruby yolo.rb A.png B.png`

![Before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d5fop142oooy89ciw3rw.png)

↓

![After](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3oib8fdlb01ik0z9jqdb.png)

I can see that the accuracy is even better than it was in [YOLOv3](https://dev.to/kojix2/yolo-object-detection-using-onnxruntime-with-ruby-1ap0).

mini_magick is much slower than YOLO. 
I hear that [rmagick](https://github.com/rmagick/rmagick) is well maintained these days, so you may want to use that.

If ONNX bindings work, any minor language should be able to do the same. I think this would mean that if you create a machine learning model in Python, you can convert it to ONNX and then deploy it in any language.

Thank you for your reading.
Have a nice day!

[Original Japanese version](https://qiita.com/kojix2/items/97a55365e6908225d178)