Paper:
Asynchronous Advantage Actor Critic (A3C) is an effective Reinforcement Learning (RL) algorithm for a wide range of tasks, such as Atari games and robot control. The agent learns policies and value function through trial-and-error interactions with the environment until converging to an optimal policy. Robustness and stability are critical in RL; however, neural network can be vulnerable to noise from unexpected sources and is not likely to withstand very slight disturbances. We note that agents generated from mild environment using A3C are not able to handle challenging environments. Learning from adversarial examples, we proposed an algorithm called Adversary Robust A3C (AR-A3C) to improve the agent's performance under noisy environments. In this algorithm, an adversarial agent is introduced to the learning process to make it more robust against adversarial disturbances, thereby making it more adaptive to noisy environments. Both simulations and real-world experiments are carried out to illustrate the stability of the proposed algorithm. The AR-A3C algorithm outperforms A3C in both clean and noisy environments.

Review:
The authors propose an extension of adversarial reinforcement learning to A3C. The proposed technique is of modest contribution and the experimental results do not provide sufficient validation of the approach.  
The authors propose extending A3C to produce more robust policies by training a zero-sum game with two agents: a protagonist and an antagonist. The protagonist is attempting to achieve the given task while the antagonist's goal is for the task to fail. 
The contribution of this work, AR-A3C, is extending adversarial reinforcement learning, namely robust RL (RRL) and robust adversarial RL (RARL), to A3C. In the context of this prior work the novelty is extending the family of adversarial RL methods. However, the proposed method is still within the same family methods as demonstrated by RARL.
The authors state that AR-A3C requires half as many rollouts as compared to RARL. However, no empirical comparison between the two methods is performed. The paper only performs analysis against the A3C and no other adversarial baseline and on only one environment: cartpole.  While they show transfer to the real world cartpole with this technique, there is not sufficient analysis to satisfactorily demonstrate the benefits of the proposed technique. 
The paper reads well. There are a few notational issues in the paper that should be addressed. The authors mislabel the value function V as the  action value, or Q function. The action value function is action dependent where the value function is not.  As a much more minor issue, the authors introduce y as the discount factor, which deviates from the standard notation of \gamma without any obvious reason to do so.
Double blind was likely compromised with the youtube video, which was linked to a real name account instead of an anonymous account.
Overall, the proposed technique is of modest contribution and the experimental results do not provide sufficient validation of the approach.