Before revision: In this paper, we propose actor-critic approaches by introducing an actor policy on QMIX 9 , which can remove the monotonicity constraint of QMIX and implement a non-monotonic value function factorization for  joint action-value . We evaluate our actor-critic methods on StarCraft II micromanagement tasks , and show that it has a stronger performance on maps with heterogeneous agent types  .
After revision: In this paper, we propose actor-critic approaches by introducing an actor policy on QMIX 9 , which can remove the monotonicity constraint of QMIX and implement a non-monotonic value function factorization for joint action-value function. In addition, NQMIX takes the state-value as the learning target, which overcomes the problem in QMIX that the learning target is overestimated. Furthermore, NQMIX can be extended to continuous action space settings by introducing deterministic policy gradient on itself. Finally, we evaluate our actor-critic methods on StarCraft II micromanagement tasks , and show that it has a stronger performance on maps with heterogeneous agent types .