Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 97 \% reduction in the number of network weights with an average reduction of 64 \% over multiple algorithms on multiple tasks. Given the practical benefits of reducing actor complexity  , we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently  .
Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 99 \% reduction in the number of network weights with an average reduction of 77 \% over multiple actor-critic algorithms on 9 independent tasks. Given that reducing actor complexity results in a direct reduction of run-time inference cost , we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently , particularly in resource-constrained applications or when deploying multiple actors simultaneously .
R
97
99
R
64
77
R
algorithms on multiple
actor-critic algorithms on 9 independent
R
the practical benefits of
that
A
results in a direct reduction of run-time inference cost
A
, particularly in resource-constrained applications or when deploying multiple actors simultaneously