Node-RedでOpenCV メモ

備忘録です。随時更新
#使うまではこっちら
https://qiita.com/NWLab/items/0b7769a1289994dc53cf

#直接MATをmsgで渡すと不可解な状態になった。
msgでMATを中継すると一つは成功するが、2個目以降はエラーが出る。
フロースコープのコンテキストを使ってMATを渡すといけそう。
##エラーが出る例（うまくいかない方)
###フロー
下の方が画像が出ず、MATじゃないよって怒らた例。最初にフローをつないだ方だけうまくいきます。3つにしても最初の1個だけしかうまくいきませんでした。
![Screenshot from 2022-02-16 16-42-11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/10970477-1988-35f3-97dc-5033b8413d61.png)

###GetUSBCameraのコード
![Screenshot from 2022-02-16 16-43-24.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/f34432f1-9d5b-f49f-787e-5ff263e7c159.png)

###ImageToStringのコード
上下のファンクションノードとも一緒です
![Screenshot from 2022-02-16 16-43-39.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/057bda45-6301-18db-37eb-5a3913850b45.png)

##成功した例（うまくいった方）
ただし、自分でフローコンテキストは管理する必要あり。
###フロー
![Screenshot from 2022-02-16 16-46-25.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/9c379469-10ce-35fb-fdd3-82939ec1eafa.png)

###GetUSBCameraのコード
![Screenshot from 2022-02-16 16-46-36.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/e41f74f4-ee0a-aa66-1eef-edd757b7fb9f.png)

###ImageToStringのコード
上下のノードとも一緒のコードです
![Screenshot from 2022-02-16 16-47-26.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/9ecfe61d-4cf1-33c6-c277-0be61839f0d0.png)

#MATのクローン
MATのクローンをつくる方法　(cloneは無い？)

```
image=msg.image;
let image2=new cv.Mat(image.rows,image.cols,image.type);
image.copyTo(image2);
```
#赤色抽出のフロー
赤色の抽出です。

![Screenshot from 2022-02-16 17-12-19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/7b01576c-4057-fa5a-46ca-3c8f8cf57a48.png)
bitwiseAndはOpenCVでエラーが発生してNode-Redが落ちたのでコメントしています。（使い方が悪いのかも）
※```msg.image="image";```を忘れていますね。
![Screenshot from 2022-02-16 17-19-51.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/0ce360f3-cde2-b57a-3e91-84df0084e617.png)


```
[{"id":"637930b1e9479668","type":"tab","label":"Flow 1","disabled":false,"info":"","env":[]},{"id":"c8fbdf0ef5c7195a","type":"inject","z":"637930b1e9479668","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":240,"y":120,"wires":[["82e4670c942ea6f1"]]},{"id":"82e4670c942ea6f1","type":"function","z":"637930b1e9479668","name":"Get UsbCamera","func":"const cap=new cv.VideoCapture(0);\nimage=cap.read();\nmsg.image=\"image\";\nflow.set(\"image\",image);\ncap.release();\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"cv","module":"opencv4nodejs"}],"x":400,"y":120,"wires":[["7304776300c36499","b5e39fb9e9f8f947"]]},{"id":"7304776300c36499","type":"function","z":"637930b1e9479668","name":"ImageToString","func":"image=flow.get(msg.image);\nstr=cv.imencode(\".jpeg\",image).toString(\"base64\");\nmsg.payload=Buffer.from(str,\"base64\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"cv","module":"opencv4nodejs"}],"x":640,"y":120,"wires":[["12f223cfc9e19c0a"]]},{"id":"12f223cfc9e19c0a","type":"image","z":"637930b1e9479668","name":"","width":160,"data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":false,"outputs":0,"x":820,"y":120,"wires":[]},{"id":"e182aa2325399962","type":"image","z":"637930b1e9479668","name":"","width":160,"data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":false,"outputs":0,"x":840,"y":320,"wires":[]},{"id":"97d5de62aaf075e1","type":"function","z":"637930b1e9479668","name":"Image2ToString","func":"image=flow.get(msg.image);\nstr=cv.imencode(\".jpeg\",image).toString(\"base64\");\nmsg.payload=Buffer.from(str,\"base64\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"cv","module":"opencv4nodejs"}],"x":640,"y":320,"wires":[["e182aa2325399962"]]},{"id":"b5e39fb9e9f8f947","type":"function","z":"637930b1e9479668","name":"RedExtraction","func":"image=flow.get(msg.image);\nbgrLower=new cv.Vec(0,0,80);\nbgrUpper=new cv.Vec(100,100,255);\nimage_mask=image.inRange(bgrLower,bgrUpper);\n//image2=image.bitwiseAnd(image_mask);\nimage2=image_mask;\nflow.set(\"image\",image2);\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"cv","module":"opencv4nodejs"}],"x":400,"y":320,"wires":[["97d5de62aaf075e1"]]}]
```
#HTTP Inとつなげた
いい感じです。
##フロー
![Screenshot from 2022-02-16 18-53-36.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/9d538da9-b596-4549-1454-0d011843c992.png)
##HTTP In部分(M5 TimerCameraXからPost)
![Screenshot from 2022-02-16 18-55-49.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/5e5c06da-7b87-e72b-380e-90027f069b00.png)

##MATに変換の部分
![Screenshot from 2022-02-16 18-56-13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/cc952615-5f92-d175-5c77-b10d408877cc.png)

##赤色の抽出
RGBでは難しかったのでHSVで。
![Screenshot from 2022-02-16 18-56-39.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/7ae11854-1398-21bf-2d32-7bb879ed2fe3.png)
参考にしたホームページ

https://pystyle.info/opencv-inrange/

```
[{"id":"b5e39fb9e9f8f947","type":"function","z":"637930b1e9479668","name":"RedExtraction","func":"image=flow.get(msg.image);\n//bgrLower=new cv.Vec(50,50,100);\n//bgrUpper=new cv.Vec(150,80,255);\n//image_mask=image.inRange(bgrLower,bgrUpper);\n//image2=image.bitwiseAnd(image_mask);\nhsv=image.cvtColor(cv.COLOR_BGR2HSV);\nbgrLower=new cv.Vec(0,0,0);\nbgrUpper=new cv.Vec(20,255,255);\nimage_mask=hsv.inRange(bgrLower,bgrUpper);\n//image2=image.bitwiseAnd(image_mask);\n\nimage2=image_mask;\nflow.set(\"image2\",image2);\nmsg.image=\"image2\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"cv","module":"opencv4nodejs"}],"x":400,"y":520,"wires":[["97d5de62aaf075e1"]]}]
```
##面積の計算とか
少し、ドキュメントの間違い等があって苦労した

```
image=flow.get(msg.image);
contours=image.findContours(cv.RETR_EXTERNAL,cv.CHAIN_APPROX_SIMPLE);

image2=new cv.Mat(image.rows,image.cols,cv.CV_8UC3);
image2.drawRectangle(new cv.Point2(0,0),new cv.Point2(image.cols,image.rows),new cv.Vec3(0,0,0),-1);
let maxArea=0;
let index=0;
let xx=0;
let yy=0;
for(let i=0;i<contours.length;i++){
    console.log("contour["+i+"]:numPoints="+contours[i].numPoints+
    ":area="+contours[i].area);
    mu=contours[i].moments();
    x=Math.round(mu.m10/mu.m00)-image.cols/2;
    y=Math.round(mu.m01/mu.m00)-image.rows/2;
    console.log("  x="+x+",y="+y);
    if(contours[i].area>50){
        image2.drawContours([contours[i].getPoints()],0,new cv.Vec(0,0,255));
        if(maxArea<contours[i].area){
            maxArea=contours[i].area;
            index=i;
            xx=x;
            yy=y;
        }
    }
}
if(index>-1){
    msg.theta=Math.atan2(xx,yy)*180/Math.PI;
    msg.area=maxArea;
}else{
    msg.theta=0;
    msg.area=0;
}

flow.set("image4",image2);
msg.image="image4";
return msg;
```

#作ったもの
![Screenshot from 2022-02-18 10-00-41.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/2784cb86-3b49-78ee-8dc4-4933350b4258.png)
##結果
2台からの結果を表示。

![Screenshot from 2022-02-18 10-33-00.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/97061/41efbafe-3a11-af8b-b412-ba4a0cf97284.png)