# Cooperative Attention-Based Learning between Diverse Data Sources
## 4 Methodology
### Learning-Based Multi-Agent Cooperation
Suppose we have a state \(s\) that represents a person's health status, and the possible actions are to take a medicine or not. The reward function \(\gamma\left(s_{t}\times a^{n,t},s_{t+1}\right)\) could represent the improvement or worsening of the person's health depending on the action taken. The trade-off factor \(\omega\) could represent the importance of immediate relief versus potential long-term side effects. \[\beta(s)=E\left[\sum_{m>-1}^{t\geq 0}\omega\gamma\left(s_{t}\times a^{n,t},\,s_ {t+1}\right),\,s_{0}=s\right] \tag{1}\]  
Hence, for multiple nodes,  
\[\beta^{i}(s)=E\left[\sum_{m>-1}^{t\geq 0}\omega^{t}\gamma^{j,n}\left(s_{t} \times a^{n,t},\,s_{t+1}\right),\,s_{0}=s\right] \tag{2}\]  
Now, to have an optimal reward distribution policy in a cooperative setting \(\gamma^{j,n}=\gamma^{j,1}\dots\dots\dots\gamma^{j,N}\), and to obtain global optima, Nash equilibrium is utilized by averaging the reward, \(\sum_{i\in N,t\geq 0,n>-1}\gamma\left(s_{t}\times a^{n,t},\,s_{t+1}\right)\). Thus, a proper distribution and link formation policy is created by applying a competitive setting \(\sum_{i\in N,t\geq 0,n>-1}\gamma\left(s_{t}\times a^{n,t},\,s_{t+1}\right)=0\) to define the link reaction policy (\(\pi\)). Suppose we have a multi-agent system where each node represents a different person, and state \(s\) represents the current weather conditions. The possible actions could be to go outside or stay indoors. The joint policy function \(\pi(a|s)\) would then represent the probability of each person going outside or staying indoors based on the current weather conditions. The probabilities \(\pi^{(i,j)(\frac{a^{n,j-i}}{\eta_{i}})}\) would represent  
Figure 2: Action and reward cooperation. the likelihood of each pair of people \((i,j)\) going outside or staying indoors together based on the weather conditions. \[\pi(a|s):=\cap_{i\in N,n>-1,i\geq 0}\pi^{i,j}\left(\frac{a^{n,i-j,i}}{s_{t}}\right) \tag{3}\]  
where \(i\) and \(j\) are node links and \(x_{i}\) is the neighborhood bond. Hence, the optimal reward link is,  
\[\beta_{\pi^{i}}^{i,j}(s)=E\Big{[}\sum_{n>-1}^{t\geq 0,\ i,j\geq 0}\omega^{t} \gamma^{i,n}\big{(}s_{t}\times a^{n,t},\ s_{t+1}\big{)}|a^{n,i-j,t}\sim\pi(Event |s_{t}),\ s_{0}=s\Big{]} \tag{4}\]  
Now optimal joint cooperative policy \(\pi^{i,j}:s_{t}^{i,j}\rightarrow\Delta a^{n,t}\) for \(i,j\) will be dependent on actions history \(h^{i,j}\),  
\[\tau_{\pi}^{t}=\prod_{\
u:\
u^{t}\in\mathbb{Z}}\ \pi^{p(\
u^{t})}(a|M(h \to S)):\ \tau_{\pi^{i,j}}^{t}=\prod_{N\cup p(\
u^{t})}^{i,j>0}\ \prod_{\
u:\
u^{t}\in\mathbb{Z}}^{n}\ \pi^{i,j}\,p^{(\
u^{t}\in i,j)}\left(a^{i} \Big{|}M(h\to S)\right) \tag{5}\]  
where, \(p(h)\sim p(h^{\prime})=P\) is the probability of an action user node takes for a policy \(\pi^{p}\). \[\beta_{\pi^{i}}^{i,j}(s)=E[\sum_{n>-1}^{t\geq 0,\ i,j\geq 0}\omega^{t}\gamma^{i,n} (s_{t}\times a^{n,t},\ s_{t+1}\big{)}|a^{n,i-j,t},x_{i}(t)\sim\tau_{\pi}^{t},\ s_{0}=s] \tag{6}\]  
The optimal joint cooperative policy \(\pi^{i,j}\) for nodes \(i\) and \(j\), is dependent on the actions history \(h^{i,j}\). For example, suppose node \(i\) represents a hospital and node \(j\) represents a government health department. The actions history \(h^{i,j}\) might include information on the number of COVID-19 patients being treated at the hospital, the availability of medical supplies, and the current policies being implemented by the health department. Based on this information, the joint policy \(\pi^{i,j}\) can be determined to optimize the reward distribution for both nodes \(i\) and \(j\). To calculate the optimal reward distribution, Equation (5) is utilized, where \(\tau_{\pi}^{t}\) represents the probability of a user node taking a certain action for policy \(\pi^{p}\), and \(\beta_{\pi^{i}}^{i,j}(s)\) represents the expected reward for nodes \(i\) and \(j\).