# Depth Estimation of a Deformable Object via a Monocular Camera
## 1 Introduction  
In the case of human-robot cooperation, deep reinforcement learning (DRL) is used to train the robot to undertake the task. For an instance of the bolt screwing task, the human partner's arms might be obstacles for the robot during the working process (Figure 1). In order to train the capability of avoiding obstacles for the robot through DRL method, one has to prepare a huge number of samples, which are usually hard to generate. In this case, it can be done by reconstructing the 3D imaging [1; 2; 3] of a human worker executing the task. The reconstruction sequence of the human's arms can be used as moving obstacles to train the obstacle avoidance capability of the robot in a virtual environment (Figure 2). In the scenarios above, a common prerequisite is the accurate pose information of a human or a robot. However, when an object is projected onto the camera plane, its depth information along the optical axis is lost, which possibly makes two actually far separated objects look close to each other [4]. This results in incorrect estimation of the pose without correct depth information. Although lasers are used to provide depth information, they are usually expensive. Low-cost equipment such as a Kinect are also prevailing alternatives, but their accuracy is not high enough (Kinect has an error of at least 4 mm and has a dead zone within 0.5 m. Additionally, the farther the distance is, the more inaccurate the measuring result is). Many previous studies propose different approaches to produce depth estimation from images. Among them, a few researchers used optimization methods to handle the problem. [PERSON] et al. propose a novel motion segmentation method to produce dense depth map from two consecutive framesthrough a single monocular camera. They segment the optical flow field into a set of motion models, with which the scene is reconstructed by minimizing a convex program [5]. [PERSON] et al. present a method which estimates depth from a single polarisation image by solving a large, sparse system of linear equations [6]. [PERSON] proposes a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling [7]. Optimization methods need the manually defined constraints to guarantee the accuracy of the resulted depth estimation. Thus, these methods require human experiences to give effective constraints. Additionally, approaches using learning-based methods to model manually extracted features are other promising alternatives as well. [PERSON] et al. propose a Markov Random Field (MRF) learning algorithm to handle monocular cues like texture gradients and variations, defocus, etc. They incorporate these cues into a stereo system to obtain depth estimation results [8]. [PERSON] et al. improve the ResNet-50 network by transfer learning to tackle the depth estimation problem through a single image [9]. [PERSON] et al. propose a phase-coded aperture camera for depth estimation. They equip the camera with an optical phase mask to produce unambiguous depth-related color characteristics for the captured image [10]. [PERSON] et al. present a convolutional neural network architecture that pays more attention to the relationships of different image locations and incorporates the absolute and relative features [11]. These methods depend on the extracted features. In some scenarios, these features might not incorporate sufficient cues of the depth information. Recently, approaches using deep learning to generate depth maps from images become prevailing. [PERSON] et al. propose a space-increasing discretization (SID) strategy to discretize depth and recast depth network learning as an ordinal regression problem [12]. [PERSON] et al. propose an approach to handle the depth estimation and semantic labeling task simultaneously. They present a concept called the attention-driven loss for the network supervision, and a synergy network to learn the relevance between the two tasks [13]. [PERSON] et al.