DeepHolo: Recognizing 3D Objects using a Binary-weighted
Computer-Generated Hologram
Naoya Muramatsu

Chun Wei Ooi

Yuta Itoh

University of Tsukuba
Pixie Dust Technologies, Inc.
sh.mn.nat@gmail.com

University of Tsukuba
made3102@gmail.com

University of Tsukubas1711428@s.tsukuba.ac.jp

Yoichi Ochiai
University of Tsukuba
Pixie Dust Technologies, Inc.
wizard@slis.tsukuba.ac.jp

Figure 1: Framework of DeepHolo network.

ABSTRACTCCS CONCEPTS

Three-dimensions (3D) models contain a wealth of information
about every object in our universe. However, it is difficult to semantically recognize the media forms, even when they featured insimplest form of objects. We propose the DeepHolo network using
binary-weighted computer-generated holograms (CGHs) reconstructed from point cloud models. This neural network facilitates
manipulation of 3D point cloud form, and allows it to be processed
as Two-dimensions (2D) data. We construct the network using
hologram data, which is simpler and contains smaller volume ofinformation than their point cloud data(PCD) counterpart, leading
to a smaller number of parameters required. The Deep Neural Network (DNN) is trained to recognize holograms of 3D objects, sothat it resembles a point attributed to a 3D model of a similar object
to the one depicted in the hologram. DeepHolo network allows
high precision object recognition, as well as processing 3D datausing much lesser computer resources. We evaluate our method on
a recognition task space efficiency, and outperforming state-of-theart methods.

• Computing methodologies → Classification and regression
trees;

SA ’17 Posters, November 27-30, 2017, Bangkok, Thailand© 2017 Copyright held by the owner/author(s).
This is the author’s version of the work. It is posted here for your personal use. Notfor redistribution. The definitive Version of Record was published in Proceedings of SA’17 Posters, https://doi.org/10.1145/3145690.3145725.

KEYWORDS
object recognition, convolutional neural network, computer-generated
hologram
ACM Reference Format:
Naoya Muramatsu, Chun Wei Ooi, Yuta Itoh, and Yoichi Ochiai. 2017.
DeepHolo: Recognizing 3D Objects using a Binary-weighted ComputerGenerated Hologram. In Proceedings of SA ’17 Posters. ACM, New York, NY,USA, 2 pages. https://doi.org/10.1145/3145690.3145725

1INTRODUCTION

The understanding of 3D environment is an important element of
the latest computer vision research ranging from automotive toautonomous robotic application scenarios. In order to handle 3D
information, however, large amount of memory is required.
In this paper, we propose a simple and effective neural network
using computer-generated holograms for 3D object recognition andreconstruction. In conventional studies, various deep learning approaches to 3D recognition and construction have been attempted.
VoxNet [Maturana and Scherer 2015] uses high number of parameters. Also, these methods require much complex networks.
We use computer generated holograms, they are the result ofsimulation of light interference between a coherent light source andpoint clouds of a 3D object. The resulted holograms are a 2D interference pattern with depth information which can be reconstructedSA ’17 Posters, November 27-30, 2017, Bangkok, Thailand

N. Muramatsu et. al.

Table 1: Classification results on ModelNet40.

3DShapeNets [Wu et al. 2015]
VoxNet [Maturana and Scherer 2015]
Subvolume [Qi et al. 2016b]
LFD [Wu et al. 2015]
MVCNN [Su et al. 2015]
PointNet [Qi et al. 2016a]
Ours Network

input#params#viewsvolumevolumevolume
image
image
point
image

16.6M
16.6M
3.5M
0.67M

11220
1080
1
1

into a 3D hologram when illuminated with a suitable light source.
Our neural network takes these interference patterns as input, andlearn the feature of input objects in 2D hologram patterns.
Instead of learning 3D objects as they are, we thought that it as
possible to handle information densely by dropping the data intoa 2D image. Also, in this method, DeepHolo can be more space
efficient because we are able to reduce the parameters expressing
the features.

2APPROACH

accuracy
avg. class
77.3
83.0
86.0
75.5
90.1
86.2
80.2

accuracy
overall
84.7
85.9
89.2
89.2
83.3

proposing a CGH recognizing network based on some inception
modules [Szegedy et al. 2015] that consist of three convolutional
layers. The modules can be expected to have fewer spatially expanded clusters that can be covered by convolution over larger
patches and that the number of patches will decrease over larger
areas. DeepHolo has a simple structure of six inception modules
followed by average pooling layers. At the end of the network,
there are three fully connected layers and a sigmoid layer placed
consecutively.

3RESULTS

The main idea of DeepHolo is to build up an end-to-end fashion
deep neural network. As illustrated in Figure 1, our DeepHolo network mainly consists of two parts: CGH and 2D CNN. We first
convert a 3D PCD into a hologram with a 128 × 240 resolution.
The 2D hologram data is then fed into the 2D CNN. For 2D CNN,
we use inception modules [Szegedy et al. 2015] to classify the 2D
hologram image of the original 3D PCD. Key to DeepHolo is theimplementation of the CGH program. The CGH program can generate binary hologram correctly, and can be trained with standard
back-propagation, allowing for end-to-end fashion.

To evaluate the performance of view selection, we compare someDNNs against two alternative methods. The recognition network istrained on ModelNet40 [Wu et al. 2015] shape classification benchmark. Table 1 shows some of the recognition results and summarizes
space (number of parameters in the network) complexity of our classification network. We also compare DeepHolo to a representative
set of volumetric, multiview and point cloud based architectures inprevious works. DeepHolo is much more space efficient than theothers in terms of #params in the network.

2.1

REFERENCES

Computer Generated Hologram

We used a simple gradation representation method using a binaryweighted computer-generated hologram (CGH) [Fujiwara et al.
2017] to convert 3D PCD into binary hologram. This method uses
multiple bit planes including binary weighted CGH with various
pulse widths. The object point of the 3D object is assigned to multiple bit planes according to the gray level. Bit planes are sequentiallydisplayed in a time division multiplexing manner. Therefore, themethod realizes a gradation representation of the reconstructed 3Dobject.

2.2

Compression of binary hologram

The size of our CGH is full HD(1024 × 1920) that is large file capacity for DNNs. We need to compress the size of binary hologram
and replaced 8 × 8 as one pixel of 64 bit expression. To solve thisproblem, we compressed the size of binary hologram by dividing
every hologram into 128 × 240 parts of 8 × 8 pixels and regarding
each of them as one pixel of 64bit expression.

2.3

2D CNN Architecture

Figure 1 shows the framework of 2D CNN. Our primary goal isto recognize a 3D point cloud data. We tackle the complication by

Masato Fujiwara, Naoki Takada, Hiromitsu Araki, Chun Wei Ooi, Shohei Ikawa, Yuki
Maeda, Hiroaki Niwase, Takashi Kakue, Tomoyoshi Shimobaba, and Tomoyoshi
Ito. 2017. Gradation representation method using a binary-weighted computergenerated hologram based on pulse-width modulation. Chinese Optics Letters 15, 6(2017), 060901.
Daniel Maturana and Sebastian Scherer. 2015. Voxnet: A 3d convolutional neuralnetwork for real-time object recognition. In Intelligent Robots and Systems (IROS),
2015 IEEE/RSJ International Conference on. IEEE, 922–928.
Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2016a. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. CoRR
abs/1612.00593 (2016). http://arxiv.org/abs/1612.00593
Charles R Qi, Hao Su, Matthias Nießner, Angela Dai, Mengyuan Yan, and Leonidas JGuibas. 2016b. Volumetric and multi-view cnns for object classification on 3d data.
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
5648–5656.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. 2015. MultiView Convolutional Neural Networks for 3D Shape Recognition. In The IEEE International Conference on Computer Vision (ICCV).
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir
Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015.
Going deeper with convolutions. In Proceedings of the IEEE conference on computervision and pattern recognition. 1–9.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang,and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes.
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
1912–1920.