  When solving a complex task, humans will spontaneously form teams and to
complete different parts of the whole task, respectively. Meanwhile, the
cooperation between teammates will improve efficiency. However, for current
cooperative MARL methods, the cooperation team is constructed through either
heuristics or end-to-end blackbox optimization. In order to improve the
efficiency of cooperation and exploration, we propose a structured
diversification emergence MARL framework named {\sc{Rochico}} based on
reinforced organization control and hierarchical consensus learning.
{\sc{Rochico}} first learns an adaptive grouping policy through the
organization control module, which is established by independent multi-agent
reinforcement learning. Further, the hierarchical consensus module based on the
hierarchical intentions with consensus constraint is introduced after team
formation. Simultaneously, utilizing the hierarchical consensus module and a
self-supervised intrinsic reward enhanced decision module, the proposed
cooperative MARL algorithm {\sc{Rochico}} can output the final diversified
multi-agent cooperative policy. All three modules are organically combined to
promote the structured diversification emergence. Comparative experiments on
four large-scale cooperation tasks show that {\sc{Rochico}} is significantly
better than the current SOTA algorithms in terms of exploration efficiency and
cooperation strength.
