令和2年度電気・電子・情報関係学会四国支部連合大会 講演論文集 (愛媛大学)
2020 SHIKOKU-SECTION JOINT CONVENTION RECORD OF THE INSTITUTES OF ELECTRICAL AND RELATED ENGINEERS (EHIME UNIV.)

1-7

Classification of 3D Models Made from Images from Overheadby Using Spherical Convolutional Neural Networks
Shu SUMIMOTO Yoko UWATE Yoshifumi NISHIO( Tokushima University )1. Introduction

3. Results

Drones are recently infiltrating various fields, for example delivery, rescue, guard and so on. It is necessary thatdrones fly safely. Then image recognition by deep learning
is becoming important for that drones fly safely. However,standing people in overhead images from the view of drones
are not able to be recognized by using YOLOv3 which is anobject detection algorithm.
In this study, we make 3D models, such a human andcars, in overhead images with Pixel2Mesh[1]. It is the neural network algorithm which can make 3D models from 2D
images to extract features of object apexes and reshape aelliptical sphere to a 3D model.
We classify the 3D models of a human and a car by using
Spherical Convolutional Neural Networks on Unstructured
Grids (UGSCNN)[2]. It has spherical convolutional layers
which are icosahedral spherical meshes. It can classify 3Dmodels to project 3D plots to a sphere and extract 3D features.
We aim at diﬀerentiating human or other objects to make3D models of them from 2D images.

We compare the learning and test accuracies of 3 data
sets of images or 3D models of a human and a car. The firstdata set has only RGB images which are 100 human images
and 100 car images taken from overhead. The second data
set has 3D models, which are 100 human models and 100
car models with scenery made from the first data set byusing Pixel2Mesh. The third data set has 3D models aswell as second data set, but they are made from first data
set images without scenery.
The first data set is classified into a human or a car byusing Convolutional Neural Networks which has 2 convolutional layers. The other data sets are classified by using
UGSCNN.
The test accuracy of the classification for the first data
set is 0.50. This low score is contrasted by higher clasification accuracy values for the second and third dataset,
which respectively contain 3D models data with and without scenery. It is considered that 2D images have only RGB
information of a head or a shoulder. On the other hands,
3D models have more detailed information than 2D images.

2. Proposed system
We propose to classify objects of a human and a car takenfrom overhead.
First, we process RGB images from overhead into 3Dmodels by using Pixel2Mesh in Fig. 1. It learned ShapeNetdata set for 3D models and our data sets are made by transfer learning. One of our data sets have each 100 3D models
of a human and a car were made from images with scenery.
Another data set of 3D models made from images withoutscenery.
Second, we classify 3D models of a human and a car with
the UGSCNN. It has 7 spherical convolutional layers. They
are made with 20 meshes, and 3D plots are projected tothem. Convolutional processes work on sphere and featuresof 3D models are extracted.

Table 1: Test accuracies.
data set
RGB imaegs
3D models with scenery3D models without scenery

Test accuracy
0.50
0.60
0.70

4. Conclusion
In this study, we have investigated comparison of theclassification into a human or a car taken from overheadwith the 3 data sets. The first data set has only 2D images
taken from overhead. The other data sets has 3D models
made from the first data set.
From these simulation results, we consider it is eﬀective for classifying objects taken from overhead to make3D models from 2D images.
However, the test accuracies are still low, so we will let
UGSCNN learn another 3D data made by the sensors.

References
(a)

(b)

(c)

Figure 1: RGB images and 3D models.
(a) A RGB image with scenery.
(b) A 3D model with scenery.
(c) A RGB image without scenery.
(d) A 3D model without scenery.

(d)[1] Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu,Wei Liu, and Yu-Gang Jiang, “Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images”, ECCV2018.
[2] ChiyuMaxJiang, Jingwei Huang, Karthik
Kashinath, Prabhat, Philip Marcus, Matthias NieBner,“Spherical CNNs on Unstructured Grids”, ICLR 2019.