Imagine deploying deep neural networks for safety critical applications, such as autonomous driving or space robotics for planetary exploration. For these applications, it is crucial to estimate well-calibrated uncertainty, but it is hard to do so with deep neural networks. As an example, consider a task for a drone which is to classify and locate two objects in an image. The objects are a space rover and a space lander. We show the real world performance of the standard state of the R2D object detector for this task, using deep neural networks. As you can see in the video, it detects the objects successfully with 100% certainty. Now let us change the evaluation scenario by modifying the configurations of the objects. Instead of the normal space rover, we expose the electronics. Also, we put the solar panels of the lander up. The trained object detector has never seen such configurations of the objects. Therefore, the natural answer should be, I do not know. In this new evaluation scenario, we see that the object detector fails. This is expected as we have violated the assumption that the training data and the testing data should come from the same distribution. Here, the more important problem is the issue of overconfidence. The neural network wrongly predicts with very high confidence. Whenever the predictions are wrong, the confidence should also be low and vice versa. This problem of uncertainty estimation and deep learning is what we address in this work, with a focus on being faster and accurate. Our main idea is, to perform uncertainty predictions, with scalable Gaussian processes, and deep neural networks. We illustrate the proposed predictive model here. To explain, our approach has a deep neural network module, that computes a single, most likely prediction, for a given input. We also have an uncertainty estimation module, that is comprised of a variant of scalable Gaussian processes, called mixtures of Gaussian process experts, or GP experts. The mixtures of experts model has a gating function, and GP experts. The role of gating function is, to partition the input data, and assign it to an expert or learner. Then, the assigned expert learn and make uncertainty predictions on this partition data. Here, each GP expert uses the Jacobian of deep neural networks, to estimate the network's uncertainty. The main advantages, that we inherit the benefits of deep neural networks for the predictions, and Gaussian processes for uncertainty estimates, which is a widely known, golden standard for probabilistic machine learning. Importantly, our approach does not require, combining multiple predictions of neural networks, as opposed to many existing methods, such as deep ensembles, and Bayesian neural networks, and so, it can be fast and accurate. In our paper, we theoretically show how deep neural networks, and mixtures of GP experts are connected, and demonstrate practical ways, to exploit the derived duality between these two models in practice. Now, we revisit the problem of overconfidence, and show that we can obtain well calibrated uncertainty estimates. To this end, we show the first integration of our method to an aerial robot. For the results, we see that our method confidently classifies the known configurations of the objects. We see that our method largely corrects, the overconfidence of the object detector, and also computes the uncertainty about their locations.