Query: depth estimation deep learning semiconductor

Candidate papers:

abstract: Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end to end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.
author_names: Chaoqiang Zhao

Qiyu Sun

Chongzhen Zhang

Yang Tang

Feng Qian
title: Monocular depth estimation based on deep learning: An overview
venue: Science China Technological Sciences
year: 2020

abstract: Monocular depth estimation from Red Green Blue (RGB) images is a well studied ill posed problem in computer vision which has been investigated intensively over the past decade using Deep Learning (DL) approaches. The recent approaches for monocular depth estimation mostly rely on Convolutional Neural Networks (CNN) Estimating depth from two dimensional images plays an important role in various applications including scene reconstruction, 3D object detection, robotics and autonomous driving. This survey provides a comprehensive overview of this research topic including the problem representation and a short description of traditional methods for depth estimation. Relevant datasets and 13 state of the art deep learning based approaches for monocular depth estimation are reviewed, evaluated and discussed. We conclude this paper with a perspective towards future research work requiring further investigation in monocular depth estimation challenges.
author_names: Faisal Khan

Saqib Salahuddin

Hossein Javidnia
title: Deep Learning Based Monocular Depth Estimation Methods A State of the Art Review
venue: Sensors
year: 2020

abstract: Estimating depth from RGB images is a long standing ill posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo based depth estimation has been addressed through matching hand crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo based depth estimation has attracted growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this article, we provide a comprehensive survey of this new and continuously growing field of research, summarize the commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we conjecture what the future may hold for deep learning based stereo for depth estimation research.
author_names: Hamid Laga

Laurent Valentin Jospin

Farid Boussaid

Mohammed Bennamoun
title: A Survey on Deep Learning Techniques for Stereo based Depth Estimation
venue: IEEE transactions on pattern analysis and machine intelligence
year: 2020

abstract: Abstract In a single bevel GMAW (gas metal arc welding) with gap fluctuation, a deep learning model was constructed using the monitoring image during the welding to predict the welding quality. We utilized Python and the library Keras and created a CNN (Convolutional neural network) model using the top surface image including the molten pool as an input. The classification model was used to predict the burn through, and the regression model was used to estimate the penetration depth. As a result, the excessive penetration and burn through could be predicted in advance and more than 95 of estimated results of penetration depth were less 1 mm error for stepped and tapered sample shapes.
author_names: Kazufumi Nomura

Koki Fukushima

Takumi Matsumura

Satoru Asai
title: Burn through prediction and weld depth estimation by deep learning model monitoring the molten pool in gas metal arc welding with gap fluctuation
year: 2020

abstract: We introduce SUW Learn: A framework for deep learning with joint supervised learning (S) unsupervised learning (U) and weakly supervised learning (W) We deploy SUWLearn for deep learning of the monocular depth from images and video sequences. The supervised learning module optimizes a depth estimation network by knowledge of the ground truth depth. In contrast, the unsupervised learning module has no knowledge of the ground truth depth, but optimizes the depth estimation network by predicting the current frame from the estimated 3D geometry. The weakly supervised module optimizes the depth estimation by evaluating the consistency between the estimated depth and weak labels derived from other information, such as the semantic information. SUW Learn trains the deep learning networks end to end with joint optimization of the desired SUW objectives. We benchmark SUW Learn on the commonly used KITTI driving scene and achieve the state of the art performance. To demonstrate the capacity of SUW Learn in learning the depth of scenes with people from different sources with different domain knowledge, we construct the M&M dataset from the Megadepth and Mannequin Challenge datasets.
author_names: Haoyu Ren

Aman Raj

Mostafa El-Khamy

Jungwon Lee
title: SUW Learn: Joint Supervised, Unsupervised, Weakly Supervised Deep Learning for Monocular Depth Estimation
venue: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
year: 2020

abstract: Abstract. Depth is an essential component for various scene understanding tasks and for reconstructing the 3D geometry of the scene. Estimating depth from stereo images requires multiple views of the same scene to be captured which is often not possible when exploring new environments with a UAV. To overcome this monocular depth estimation has been a topic of interest with the recent advancements in computer vision and deep learning techniques. This research has been widely focused on indoor scenes or outdoor scenes captured at ground level. Single image depth estimation from aerial images has been limited due to additional complexities arising from increased camera distance, wider area coverage with lots of occlusions. A new aerial image dataset is prepared specifically for this purpose combining Unmanned Aerial Vehicles (UAV) images covering different regions, features and point of views. The single image depth estimation is based on image reconstruction techniques which uses stereo images for learning to estimate depth from single images. Among the various available models for ground level single image depth estimation, two models, 1) a Convolutional Neural Network (CNN) and 2) a Generative Adversarial model (GAN) are used to learn depth from aerial images from UAVs. These models generate pixel wise disparity images which could be converted into depth information. The generated disparity maps from these models are evaluated for its internal quality using various error metrics. The results show higher disparity ranges with smoother images generated by CNN model and sharper images with lesser disparity range generated by GAN model. The produced disparity images are converted to depth information and compared with point clouds obtained using Pix4D. It is found that the CNN model performs better than GAN and produces depth similar to that of Pix4D. This comparison helps in streamlining the efforts to produce depth from a single aerial image.
author_names: L Madhuanand

Francesco Nex

M Ying Yang
title: DEEP LEARNING FOR MONOCULAR DEPTH ESTIMATION FROM UAV IMAGES
year: 2020

abstract: Monocular depth estimation relied on RGB images is an important ill posed problem in ithe system of computer vision. Recently, people use the method of deep learning to discuss this problem Most of the existing monocular depth estimation algorithms relied on convolution neural network. Depth estimation based on 2D images has important applications in image segmentation, 3D object detection, robot navigation, object tracking and autonomous driving. This paper gives a brief overview of this problem, reviews, evaluates and discusses the monocular depth estimation algorithms relied on deep learning, and looks forward to the direction of further research in the face of some challenges.
author_names: Ruan Xiaogang

Yan Wenjing

Huang Jing

Guo Peiyuan

Guo-Qing Wei
title: Monocular Depth Estimation Based on Deep Learning:A Survey
venue: 2020 Chinese Automation Congress (CAC)
year: 2020

abstract: Offset Pixel Aperture (OPA) camera has been recently proposed to estimate disparity of a scene with a single shot. Disparity is obtained in the image by offsetting the pixels by a fixed distance. Previously, correspondence matching schemes have been used for disparity estimation with OPA. To improve disparity estimation we use a data oriented approach. Specifically, we use unsupervised deep learning to estimate the disparity in OPA images. We propose a simple modification to the training strategy which solves the vanishing gradients problem with the very small baseline of the OPA camera. Training degenerates to poor disparity maps if the OPA images are used directly for left right consistency check. By using images obtained from displaced cameras at training, accurate disparity maps are obtained. The performance of the OPA camera is significantly improved compared to previously proposed single shot cameras and unsupervised disparity estimation methods. The approach provides 8 frames per second on a single Nvidia 1080 GPU with 1024x512 OPA images. Unlike conventional approaches, which are evaluated in controlled environments, our paper shows the utility of deep learning for disparity estimation with real life sensors and low quality images. By combining OPA with deep learning, we obtain a small depth sensor capable of providing accurate disparity at usable frame rates. Also the ideas in this work can be used in small baseline stereo systems for short range depth estimation and multi baseline stereo to increase the depth range.
author_names: Saad Ali Imran

Sikander Bin Mukarram

Muhammad Umar Karim Khan

C M Kyung
title: Unsupervised deep learning for depth estimation with offset pixels.
venue: Optics express
year: 2020

abstract: Light field imaging is one of the most promising methods to capture realistic 3D scenes. In this paper, we propose a deep learning network composed of two sub networks performing depth estimation and light field image reconstruction, respectively. We simultaneously train the two sub networks by employing a loss function combining the reconstruction loss of the reconstruction network and the estimation loss of the depth estimation network. Experimental results demonstrate that the proposed method accurately estimates the disparity maps of light field images and also faithfully reconstructs light field images.
author_names: Jae-Seong Yun

Jae-Young Sim
title: Deep Learning Based Depth Estimation and Reconstruction of Light Field Images
venue: 2020 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
year: 2020

abstract: Image depth estimation is an important technology for obtaining scene depth for 3D images, and it has developed rapidly in the field of computer vision. In this paper, convolutional neural networks and conditional random fields are unified into a deep learning framework to build a computer vision model. First, based on the architecture of multiscale CNN and CRF, information of the scene image is obtained from computer video using depth learning, and then, a new frame model is built to predict the depth of images in a computer video. Second, three international standard datasets are used for training, and the results show that the CNN CRF model can use a small number of samples to complete high precision training and has a better estimation effect on images outside the dataset. The experimental results show that the CNN CRF model can recover the depth and speculate the 3D structure of the scene, which has predictable application and development value for improving production efficiency.
author_names: Runchu Xu
title: Depth Estimation of Computer Video Images Based on Deep Learning
venue: Circuits Syst. Signal Process.
year: 2020