  Despite the recent successes in computer vision, there remain new avenues to
explore. In this work, we propose a new dataset to investigate the effect of
self-occlusion on deep neural networks. With TEOS (The Effect of
Self-Occlusion), we propose a 3D blocks world dataset that focuses on the
geometric shape of 3D objects and their omnipresent challenge of
self-occlusion. We designed TEOS to investigate the role of self-occlusion in
the context of object classification. Even though remarkable progress has been
seen in object classification, self-occlusion is a challenge. In the
real-world, self-occlusion of 3D objects still presents significant challenges
for deep learning approaches. However, humans deal with this by deploying
complex strategies, for instance, by changing the viewpoint or manipulating the
scene to gather necessary information. With TEOS, we present a dataset of two
difficulty levels (L1 and L2 ), containing 36 and 12 objects, respectively. We
provide 738 uniformly sampled views of each object, their mask, object and
camera position, orientation, amount of self-occlusion, as well as the CAD
model of each object. We present baseline evaluations with five well-known
classification deep neural networks and show that TEOS poses a significant
challenge for all of them. The dataset, as well as the pre-trained models, are
made publicly available for the scientific community under
https://nvision2.data.eecs.yorku.ca/TEOS.
