###### Abstract
In the paper, the authors deploy local learning models (LLMs) and global learning models (GLMs) in distributed nodes (UE) and centralized controllers (APs), respectively. Every UE integrates locally updated value in a federated ACK (FACK) message, which is fed to APs for training GLM. UE trains LLM combining the feedback from the local exploration and the Q-value estimated by GLM. FRL has advantages of high privacy protection and wide application scenarios, but to the best knowledge of the authors, almost all FRL solutions cause additional signaling and data transmission. This is because FRL based solutions require a centralized server to control the interaction of distributed nodes. Considering our motivation for designing a decentralized solution: avoiding additional signaling and data transmission. Learning cooperation is a suitable MARL method. Each agent makes policy decisions based on local observations without information of other agent. Furthermore, we need to solve two problems in learning cooperation: (1) The non-stationarity of agent learning environment [24] caused by lack of global state information; (2) Relative over-generalization: a sub-optimal policy in joint action space is preferred over an optimal policy for distributed agent [25]. Consider in the field of wireless communication, for example, each BS (seen as an agent) needs to configure UL/DL slots ratio (seen as an action) according to service buffer and channel conditions of the intra-BS UE (seen as local observations). For the lack of system configuration information, it is difficult for a BS to learn the impact of inter-BS interference on data rate (seen as a local reward) through exploration. In other words, from the perspective of distributed agents, the agent's decision-making process no longer meets the Markov decision-making conditions [24]. This is the non-stationarity that distributed MARL encounters. On the other hand, each BS can only obtain transmission rate information itself, and cannot obtain the influence itself's D-TDD configuration on the overall system rate (sum of each BS's UL/DL transmission rate.). This leads to a BS making a resource allocation decision which is most beneficial to itself, thereby reducing the overall system rate. This is the relative over-generalization that MARL needs to face. Even though several standard activities and related studies exist in the literature, providing efficient D-TDD configuration in 5G and B5G network still remains an open research area. In fact, the problem is complex and multifaceted, and, therefore, no single technology seems to be able to provide an effective solution. To the best of our knowledge, none of the existing studies proposed a fully decentralized solution to solve the problem of D-TDD configuration. Therefore, in this paper, we apply leniency control based MARL (LC-MARL) and propose a fully decentralized D-TDD configuration scheme to effectively accommodate different traffic demands of uplink/downlink service. Our contribution is:  
* We formulate D-TDD configuration problem as a dynamic programming problem, and design a distributed MARL based decentralized solution;
* In order to obtain a higher overall system gain of distributed MARL, we apply a leniency control based learning cooperation method. Leniency controller reduces the influence of non-stationary learning environment on agent by effectively controlling the Q value estimation in MARL according exploration level and reward (buffer length under D-TDD configuration);* BLSTM based binary auto-encoder is designed to extra the features of high dimensional time series and generate binary hashing-code, which increases the ability of leniency control;
* The proposed fully decentralized D-TDD configuration scheme requires no information interactions between BSs, which save the resource caused by transmission of control signaling and data.