人工知能－基礎と応用
システム知能学講座
趙研究グループ
北側１階１01E,F

研究室紹介• 博士後期：１人• 修士：12人• 学部：10人• 研究生：１人

A Panel Code Suitable forLong Distance Meter Reading
Kaito Suzuki and Jumpei Kanno Supervisor: Qiangfu Zhao
BackgroundIn many Japanese factories, there is still aneed to read analog meters manually. To
replace thousands of analog meters withdigital ones is too expensive. Therefore, in ajoint research with a private company, we
try to automate the meter reading process
to reduce both human labor and humanerrors. Here, we introduce a panel codethat can be used to encode thousands ofmeters and to identify the meters from longdistance.

The proposed method

Figure 1: Examples of panel codes

The panel code is a 2D-Code consisting offour colored markers (i.e. the findermarkers) and multiple code sections. The
four colored markers are used to correct
the image, and the code sections are used
to identify the meter by embedding themeter information (e.g. the meter ID).
The panel code can be adjusted according
to the shape of the meter, such asrectangular, triangular, or circular. The
color and shape of the colored markers will
also be adjusted to match the environment
for easy recognition.
Compared with QR code, the panel code ishas an advantage in reading distance andis suitable for “normalizing” the meter
images to improve the reading accuracy.

Encode and Decode
The colored markers and meter
identification numbers converted tobinary codes are placed around themeter. The markers can be detectedusing color detection or object detection
methods such as SSD. A single type ofpanel code (e.g. the squared one) canbe used to encode various meters.
Therefore, it is not necessary to design
many models for detecting variousmeters. In addition, using homographictransformation based on the coloredmarkers, we can normalize the meter
images and make meter reading easierand more accurate.

Flow of meter reading

Image normalization

ServerGet the meter information
by decoding the panel code

Meter reading
method

特願2020-189720

Meter ValueAutomatic Reading of Flow Meters
Based on Object Detection
Jumpei Kanno and Kaito Suzuki Supervisor: Qiangfu Zhao
Background
Various types of analog meters are
installed in Japanese factories to monitor
conditions and detect abnormalities in themanufacturing process. Therefore, humaninspectors have to go around the factoryand visually read the values of the meters,
record the data, and take proper actions ifneeded. This work is done manually, whichputs a heavy burden on the inspector andis prone to human error. The purpose ofthis study is to automate the manual
process by using deep learning models ofobject detection and image processing.

The Proposed Method
Our proposed method consists of six
steps using the object detection method
SSD (Single Shot Multi-Box Detector) andimage processing. SSD is trained to detect
flow and float. Image processing is used tonormalize the image and improve thereading accuracy.
Below are the basic steps for automatic
reading (see Fig. 1):
1. Detect the flow meter using SSD
2. Image normalization
3. Detection of flow meter scale range byimage processing
4. Detect the float using SSD

5. Estimate the coordinate of the float byimage processing
6. Estimate the value of the flow meter

Experimental Results
We performed experiments to measure
the accuracy of our reading method. We
performed the reading process on 100images of each scale (1〜10) and theresults are shown in Table 1.
The error for each value is basicallywithin the range [-0.2, 0.2], which iswithin the tolerance for practicalapplication. The brightness in the image
could cause image processing to fail anderrors to increase. Therefore, it is better
to use the average or median value as
the reading result in multiple frames.
Table 1: Reading experiment result

Schematic diagram of the proposal

step4 & 5
Capture an image

step1

step2
step3
特願2021-092181

Fig.1.Reading flow

6. Calculate
the scalevalue andoutput theresult

Generative Model-Based Anomaly Detection
for Symmetric Industrial Products
Shota Nakada and Takumi Meguro Supervisor: Qiangfu Zhao
BackgroundAt factories or manufacturing sites, visualinspections are conducted to ensure thequality of industrial products. In recentyears,
variousdeep
learning-based
methods have been investigated foranomaly detection. In this research, westudy anomaly detection for symmetricproducts. Using the symmetric property ofthe product images, we propose several
image ”normalization” methods that areuseful to reduce the diversity of the dataand obtain more effective generative deeplearning models for anomaly detection.

The Proposed Method
Our proposed anomaly detection method
consists of five steps using generativemodels such as variational auto-encoder(VAE) and generative adversarial network(GAN). Below are the basic steps for ouranomaly detection method.

4. Binarization. Create black-white image
from the difference image.
5. Connectedcomponentextraction.
Calculate the size of “defect”.

Experimental Results
We trained VAE and GAN using 3000
normal images. Then we tested ouranomaly detection method using 64
abnormal images and 500 normalimages. Table 1 shows the results of theproposed method. Compared withexisting method, these results are muchbetter. However, in product anomaly
detection, It is important to get Recall
close to 100%. From Table 1 we cansee that the proposed method is still notgood enough for practical use. We areinvestigating methods to improve theperformance.

Table 1: Anomaly detection results. Thresholdmeans the size of defect.

1. Image rotation. Rotate the images tomake them “orientation invariant”.
2. Image translation. Shift the images
parallel to the x-axis and/or y-axis tomake them “position invariant”.
3. Creating models. Train the models only
for “edges” and “corners”.

Schematic diagram of the proposal

Fig.1.Binarization

Fig.2.Creating models

Learn what is good, and know what is bad!

Product Defect Detection Based on
Multi-Class Classification
Takumi Meguro and Shota Nakada Supervisor: Qiangfu Zhao
BackgroundExperimental Results

In anomaly detection tasks, missing
anomalies is a problem that must be
avoided as far as possible. It has very
serious consequences for companies that
manufacture products if products withanomalies are delivered to the users. On
the other hand, we should also minimize
the over-detection rate while keeping thenumber of missed anomalies to zero or
almost zero. To achieve these objectives,
we propose an image-based multi-classclassification method that can not only
detect but also localize the anomalies.

The following table shows the resultsof the existing and proposed methods,
where FN indicates a missed anomaly
and a threshold value should be used to
set FN to 0.
Comparing the two results, theproposed method is able to reduce FN
to zero while suppressing overdetection. In addition, the number ofrejections is very small. It can be said
that the proposed method clearly
distinguishes
abnormaldata
fromnormal data.

The Proposed Method

Table 1: Prior method result

The proposed method can pay more
attention to the anomalies, images of thenormal parts will provide a less negative
effect on the performance. Interestingly, by
setting a proper threshold for one of theoutputs, we can separate normal andabnormal data more clearly compared withthe straightforward two-class approach.
Fig. 1. How to label

Label : 1100

Table 2: Proposed method result

We speculate that two factors contributed
to the above results. The first is that therise in the relevant class caused a large
difference in the output of the normal class
between normal and abnormal data. The
second is that the training was conductedusing multi-class classification, which
resulted in a large difference between
normal and abnormal data in the feature
space.
Thiswasconfirmed
whendimensionality reduction was performed.

Schematic diagram of the proposal
Classification Model

Look closer and we can gain more!

You Only Look at Interested Cells –An Efficient Way for Detecting On-road Risks
Kai Su Supervisors: Yoichi Tomioka and Qiangfu ZHAO
Background

On-road risk detection

On-road risk detection is one of theprevailing applications in object detection
and recognition, and it is also a crucial partof the driving assistance system. Only atiny amount of research has recently been
conducted to create a lightweight road risk
detection system for low-cost vehicles
such as mobility scooters. However, roadriskdetectiontasks
requirestrictrequirements for real-time and highdetection accuracy. Furthermore, most of
the detection methods in use today need
expensive computational power. Therefore,
this is necessary to find a more efficientand effective way to solve the above
problems.

In our on-road risk detection system, adepth camera is used and mounted on amobility scooter. Based on the scooter
speed and the human response time, weregard the road within 6 meters from themobility scooter as the detection area. As
shown in Fig. 1, the first three rows (i.e.,cells 1 to 32) are the notification area.
The danger area is the last four rows (i.e.,cells 33 to 96). The white tapes on thefloor are the reference lines. We set 96
cells to detect on-road risks and eight todetect traffic signs.

Our proposal
Our proposed approach is called You
Only Look at Interested Cell (YOLIC). The
detection model leverages both boundingbox and semantic segmentation methods.
In this way, a detected object can be
represented by one or more small cells (orblocks) instead of one big bounding box
or pixel area where each cell will focus on
the different parts of the object. Fig. 1
illustrates interested cells for on-road risk
detection. The main idea is to focus onroad and traffic signal cells. The cell sizes
can be different depending on thedistances and other factors.

Experimental Results
In our experiment, we have defined 11
types of objects. The following results canbe obtained if we use MobileNet v2 as
the backbone network for detection. To
know the detection performance more
directly. Please scan the QR code to seean actual test demo.

Figures to illustrate the proposed method

Road risk detection demo
Fig. 1: Interested cells for on-road risk detection

Pay attention to important things!

他の研究テーマ• Modular selective neuralnetwork (Takahashi)• Image annotation based on
unsupervised learning (Ogata)• Deep learning-based Automatic
documentation (Hasegawa)• Attention recognition (Liao)• Deep learning-based Chinese
painting generation (Wang)• Deep learning-based Xrecognition (Hamauzu)My research• Conditional computation:– NNTree: Neural network trees(combination of specialist neuralnetworks using a tree structure)– MS-Net: Modular selective neuralnetworks (combination ofspecialist neural networks using ageneralist network)• Awareness computation:– Sensor array-based situation
awareness -> senior-care– Three-value logic-basedawareness -> sparse computing.

No.10