Research authors: Aleksei Petrenko (Intel Labs, University of Southern California) Zhehui Huang (University of Southern California), Tushar Kumar (University of Southern California), Gaurav Sukhatme (University of Southern California), Vladlen Koltun (Intel Labs)
Abstract: Increasing the scale of reinforcement learning experiments has allowed researchers to achieve unprecedented results in both training sophisticated agents for video games, and in sim-to-real transfer for robotics, but it is cost prohibitive. Researchers optimized the efficiency and resource utilization of reinforcement learning algorithms instead of relying on distributed computation. Using the Sample Factory, a high-throughput training system optimized for a single-machine setting, the architecture combined a highly efficient, asynchronous, GPU-based sampler with off-policy correction techniques. Researchers achieved throughput higher than 105 environment frames/second on non-trivial control problems in 3D without sacrificing sample efficiency.