Patent ID: 11948079
Assignee: nan
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A multi-agent coordination apparatus, comprising:
N agents, each agent comprising a sensor, an executor and a neural network, and N being an integer greater than 1; and
a central controller comprising a memory, a random label generator, N local discriminators, a global discriminator, and a trainer,
wherein the random label generator is configured to randomly select a first coordination pattern label from a plurality of prior probability distributions, the first coordination pattern label indicating a first coordination pattern for coordination among the N agents;
sensors of the N agents are configured to obtain N observations xt(1), xt(2), . . . , xt(N) of an environment under a state variable st of the N agents in the first coordination pattern;
neural networks of the N agents are configured to determine action vectors ut(1), ut(2), . . . , ut(N) based on the N observations xt(1), xt(2), . . . , xt(N) under the state variable st and the first coordination pattern label, t being any integer from 0 to N−1;
executors of the N agents are configured to apply actions to the environment based on the action vectors ut(1), ut(2), . . . , ut(N);
the sensors of the N agents are further configured to obtain N observations xt+1(1), xt+1(2), xt+1(N) of the environment under a state variable st+1 at a next moment of the N agents in the first coordination pattern;
each of the N local discriminators is configured to determine a first probability that a current coordination pattern is the first coordination pattern based on a corresponding observation in the N observations xt+1(1), xt+1(2), . . . , xt+1(N);
the global discriminator is configured to determine a second probability that the current coordination pattern is the first coordination pattern based on a set of the N observations xt+1(1), xt+1(2), . . . , xt+1(N);
the central controller is configured to determine a pseudo reward based on the first probability and the second probability;
the memory is configured to store the first coordination pattern label, the N observations xt(1), xt(2), . . . , xt(N), the N observations xt+1(1), xt+1(2), . . . , xt+1(N), the action vectors ut(1), ut(2), . . . , ut(N), and the pseudo reward; and
the trainer is configured to: extract from the memory the first coordination pattern label, observations at a current moment, action vectors at the current moment, the pseudo reward at the current moment, and observations at the next moment; optimize the neural networks of the N agents using reinforcement learning based on the first coordination pattern label, the observations at the current moment, the action vectors at the current moment, the pseudo reward at the current moment, and the observations at the next moment from the memory; and optimize the N local discriminators and the global discriminator using supervised learning based on the first coordination pattern label and the observations at the current moment,
wherein the N agents are N robot agents, the environment is a real-world environment, and the N robot agents apply, based on action vectors determined by the optimized neural networks, actions corresponding to the action vectors to the real-world environment, and
said determining the pseudo reward based on the first probability and the second probability comprises:
calculating the pseudo reward based on:

rz=log qø(z|f(x))−1/NΣi=1N log qøi(z|f(x(i))),

where z represents the first coordination pattern, rz represents the pseudo reward, x represents a set of the N observations, x(i) represents an-ith observation among the N observations, f( ) represents a feature extraction function, qøi(z|f(x(i))) represents the first probability that the current coordination pattern is the first coordination pattern z based on the i-th observation x(i), and qø(z|f(x)) represents the second probability that the current coordination pattern is the first coordination pattern z based on the set of the N observations x.