text: 162,61,954018861208109056,881959726958862337,Yuhuai (Tony) Wu,"If you use K-FAC you only need to do 1 update (ACKTR), but if you use first order optimizer, you need to do 320 updates (PPO). AND 1 update by K-FAC still wins. This is what we (with @baaadas) find by comparing ACKTR vs. PPO vs. PPOKFAC. <LINK> <LINK>",https://arxiv.org/abs/1801.05566,"In this technical report, we consider an approach that combines the PPO objective and K-FAC natural gradient optimization, for which we call PPOKFAC. We perform a range of empirical analysis on various aspects of the algorithm, such as sample complexity, training speed, and sensitivity to batch size and training epochs. We observe that PPOKFAC is able to outperform PPO in terms of sample complexity and speed in a range of MuJoCo environments, while being scalable in terms of batch size. In spite of this, it seems that adding more epochs is not necessarily helpful for sample efficiency, and PPOKFAC seems to be worse than its A2C counterpart, ACKTR. ","An Empirical Analysis of Proximal Policy Optimization with