  Integration of reinforcement learning with unmanned aerial vehicles (UAVs) to
achieve autonomous flight has been an active research area in recent years. An
important part focuses on obstacle detection and avoidance for UAVs navigating
through an environment. Exploration in an unseen environment can be tackled
with Deep Q-Network (DQN). However, value exploration with uniform sampling of
actions may lead to redundant states, where often the environments inherently
bear sparse rewards. To resolve this, we present two techniques for improving
exploration for UAV obstacle avoidance. The first is a convergence-based
approach that uses convergence error to iterate through unexplored actions and
temporal threshold to balance exploration and exploitation. The second is a
guidance-based approach using a Domain Network which uses a Gaussian mixture
distribution to compare previously seen states to a predicted next state in
order to select the next action. Performance and evaluation of these approaches
were implemented in multiple 3-D simulation environments, with variation in
complexity. The proposed approach demonstrates a two-fold improvement in
average rewards compared to state of the art.
