TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: This paper considers multi-agent reinforcement learning (MARL) in networked system control. Specifically, each agent learns a decentralized control policy based on local observations and messages from connected neighbors. We formulate such a networked MARL (NMARL) problem as a spatiotemporal Markov decision process and introduce a spatial discount factor to stabilize the training of each local agent. Further, we propose a new differentiable communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL. Based on experiments in realistic NMARL scenarios of adaptive traffic signal control and cooperative adaptive cruise control, an appropriate spatial discount factor effectively enhances the learning curves of non-communicative MARL algorithms, while NeurComm outperforms existing communication protocols in both learning efficiency and control performance.

SOLUTION: This paper proposes a new formulation and a new communication protocol for networked multi-agent control problems

PROBLEM: Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, an attacker is not usually able to directly modify another agent's observations. This might lead one to wonder: is it possible to attack an RL agent simply by choosing an adversarial policy acting in a multi-agent environment so as to create natural observations that are adversarial? We demonstrate the existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents. The adversarial policies reliably win against the victims but generate seemingly random and uncoordinated behavior. We find that these policies are more successful in high-dimensional environments, and induce substantially different activations in the victim policy network than when the victim plays against a normal opponent. Videos are available at https://attackingrl.github.io.

SOLUTION: Deep RL policies can be attacked by other agents taking actions so as to create natural observations that are adversarial.

PROBLEM: Generating visualizations and interpretations from high-dimensional data is a
 common problem in many fields. Two key approaches for tackling this problem 
 are clustering and representation learning. There are very performant deep
 clustering models on the one hand and interpretable representation learning techniques, 
 often relying on latent topological structures such as self-organizing maps,
 on the other hand. However, current methods do not yet successfully combine
 these two approaches. We present a new deep architecture for probabilistic clustering, 
 VarPSOM, and its extension to time series data, VarTPSOM, composed of VarPSOM 
 modules connected by LSTM cells. We show that they achieve superior 
 clustering performance compared to current deep clustering methods on static 
 MNIST/Fashion-MNIST data as well as medical time series, while inducing an
 interpretable representation. Moreover, on the medical time series, VarTPSOM
 successfully predicts future trajectories in the original data space.

SOLUTION:
We present a new deep architecture, VarPSOM, and its extension to time series data, VarTPSOM,  which achieve superior clustering performance compared to current deep clustering methods on static and temporal data.