Efficient Multi-agent Communication via Self-supervised Information Aggregation
Anonymous Author(s)
Abstract
Utilizing messages from teammates can improve coordination in cooperative Multiagent Reinforcement Learning (MARL).
To obtain meaningful information fordecision-making, previous works typically combine raw messages generated byteammates with local information as inputs for policy.
However, neglecting theaggregation of multiple messages poses great inefficiency for policy learning.
Motivated by recent advances in representation learning, we argue that efficientmessage aggregation is essential for good coordination in MARL.
In this paper, wepropose M ulti-
A gent communication via S elf-supervised I nformation A ggregation(MASIA), with which agents can aggregate the received messages into compactrepresentations with high relevance to augment the local policy.
Specifically, wedesign a permutation invariant message encoder to generate common informationaggregated representation from raw messages and optimize it via reconstructingand shooting future information in a self-supervised manner.
Each agent wouldutilize the most relevant parts of the aggregated representation for decision-makingby a novel message extraction mechanism.
Empirical results demonstrate that ourmethod significantly outperforms strong baselines on multiple cooperative MARLtasks for various task settings.
Introduction
Multi-Agent Reinforcement Learning (MARL) [15] has attracted widespread attention [15, 7] recently,achieving remarkable success in many complex domains [1], such as traffic signal control [8],autonomous driving [49] and droplet control [22].
For better coordination on further applications,some issues remain to be solved.
To solve the non-stationarity caused by the concurrent learning ofmultiple policies and scalability as the agent number increases, most recent works on MARL adoptthe Centralized Training and Decentralized Execution (CTDE) [18, 24] paradigm which includesboth value-based methods [37, 32, 40] and policy gradient methods [12, 23, 43].
Under the CTDEparadigm, however, the coordination ability of the learned policies can be fragile due to the partialobservability in the multi-agent environment, which is a common challenge in many multi-agenttasks [26].
While recurrent neural networks could in principle relieve this issue by conditioning thepolicy on action-observation history [14], the uncertainty of other agents (e.g., states and actions) atexecution time can result in catastrophic miscoordination and even sub-optimality [42, 5].
Communication shows great potential in solving these problems [45, 50], with which agents can shareinformation such as observations, intentions, or experiences to stabilize the learning process, leadingto a better understanding of the environment (or the other agents) and better coordination as a result.
Previous communication methods either focus on generating meaningful information [42, 17, 47] forthe message senders, or design techniques such as attention mechanism [3, 28], message gate [25, 5]to filter the most relevant information on raw received messages.
These approaches treat the receivedinformation as a black box and tacitly assume that policy networks can automatically extract the mostcritical information from multiple raw messages during policy learning.
On this occasion with theonly signal given by reinforcement learning, the extraction process may be reasonably inefficientespecially in complex scenarios.
Motivated by recent advances in state representation learning [34, 20], which reveals that auxiliaryrepresentation objectives could facilitate policy learning [10], we aim at efficiently aggregatinginformation as compact representations for policy by designing a novel communication framework
M ulti-
A gent communication via S elf-supervised I nformation A ggregation (MASIA).
Specifically,representations are optimized through self-supervised objectives, which encourages the representations to be both abstract of the true states and predictive on the future information.
Since agents areguided towards higher cumulative rewards during policy learning, correlating representations withtrue states and future information could intensify the learning signals in policy learning.
In this way,the efficiency of policy learning could be encouraged.
Also, considering that permutation invarianceand member invariance of representations can also promote efficiency, we design a self-attentionmechanism to maintain the invariance of obtained representations.
We also design a network whichcould weight the aggregated representation for individual agents to derive unique and highly relevant representation to augment local policies.
For evaluation of the proposed method, we conductextensive experiments on various cooperative multi-agent benchmarks, including Hallway [42],
Level-Based Foraging [30], Traffic Junction [3], and two maps from StarCraft Multi-Agent Challenge(SMAC)
The experimental results show that MASIA outperforms previous approaches, strongbaselines, and ablations of our method.
Our main contributions are:
• We propose a novel framework message aggregation network to extract the multiple mes- sages from various teammates, with which we acquire a permutation and member invariantinformation aggregation representation.
Agents can then use a novel focusing network toextract the most relevant information for decision-making.•
Two representation objectives are introduced to make the information representation compactand sufficient , including the true state reconstruction and multi-step prediction on futurestates.• Sufficient experimental results on various benchmarks and communication conditionsdemonstrate that our proposed approach significantly improves the communication performance, and visualization results further reveal why it works.
Problem Formulation
This paper considers a fully cooperative MARL communication problem, which can be modeled as
Decentralised Partially Observable Markov Decision Process under Communication (Dec-POMDPCom) [29] and formulated as a tuple ⟨N , S , A , P, Ω , O, R, γ, M⟩ , where N = { 1 , . .
. , n } is theset of agents, S is the set of global states, A is the set of actions, Ω is the set of observations, Ois the observation function, R represents the reward function, γ ∈ [0 , 1) stands for the discountedfactor, and M indicates the set of message.
At each time step, due to partial observability, eachagent i ∈ N can only acquire the observation o i ∈ Ω drawn from the observation function
O ( s, i )with s ∈ S , each agent holds an individual policy π ( a i | τ i , m i ) , where τ i represents the history(
o 1 i , a 1 i , . .
, o t − 1 i , a t − 1 i , o ti ) of agent i at current timestep t , and m i ∈ M is the message receivedby the agent i .
The joint action a = ⟨ a 1 , . .
. , a n ⟩ leads to next state s ′ ∼ P ( s ′ | s, a ) and
theglobal reward R ( s, a ) .
The formal objective is to find a joint policy π ( τ , a ) to maximize the globalvalue function Q π tot ( τ , a )
=
E s, a [ (cid:80) ∞ t =
γ t R ( s, a ) | s
=
s, a 0 = a , π ] , with τ = ⟨ τ 1 , . .
, τ n ⟩ .
As each agent can behave as a message sender as well as a message receiver, this paper considers
𝑄 !"!
(𝝉, 𝒂)
[Figure]
StateDecoder
[Figure]
ℒ ./
[Figure] [Figure]
learning useful message representation in the received end, and agents only use local information o ias message to share within the team.
We optimize the policy by value-based MARL, where deep Q-learning [27] implements the actionvalue function Q ( s, a ) with a deep neural network Q ( τ , a ; θ ) parameterized by θ .
This paper followsthe CTDE paradigm.
In the centralized training phase, deep Q-learning uses a replay memory Dto store the transition tuple ⟨ τ , a , r, τ ′ ⟩ .
We use Q ( τ , a ; θ ) to approximate Q ( s, a ; θ ) to relievethe partial observability.
Thus, the parameters θ are learnt by minimizing the expected Temporal
Difference (TD) error:
[Equation]
, a ; θ ) (cid:1) 2 (cid:105) , where V (cid:0) τ ′ ; θ − (cid:1) = max a ′ Q (cid:0) τ ′ , a ′ ; θ − (cid:1) is the expected future return of the TD target and θ − areparameters of the target network, which will be periodically updated with θ .
Method
In this paper, we propose efficient Multi-Agent communication via Self-supervised Information
Aggregation (MASIA), which is a new multi-agent communication mechanism for promoting cooperation performance.
Redundant communications could increase the burden of information processingfor each agent to make decisions and pose new challenges for information extraction since plenty ofirrelevant information is contained in raw message.
To design an efficient communication mechanism,we believe two properties are of vital importance - compactness and sufficiency .
The Compactnesscalls for higher information density, while the sufficiency calls for a rich amount of information.
To meet the standard of sufficiency, a global encoder, which we call Information Aggregation Encoder(IAE), is shared among agents to aggregate the information broadcasted by agents into a commonrepresentation.
With proper training, this representation could reflect the global observation so thateach agent could obtain sufficient information from it to make decisions.
As for compactness, we firstdesign an auxiliary loss on the global representation to correlate it with the policy learning process,and make each agent only focus on the part of the representation related to its performance andcoordination by the designed focusing network through excluding the unrelated parts.
The entireframework of our method is shown in Fig.
We introduce the aggregation and extraction process in
Sec. 3.1, and the objectives for compactness and sufficiency in Sec. 3.2.
3.1 Information Aggregation and Extraction
Information Aggregation.
Believing that the true state should be reflected from combined messages,we design the aggregation encoder to be capable of subsuming all the messages sent from agents.
Also, the communication system in multi-agent systems are flexible and permutation invariant innature, which calls for a permutation invariant structure for the aggregation encoder.
Based onthese beliefs, we apply a self-attention mechanism to aggregate multiple messages from differentteammates:
[Equation]
where matrices Q , K , and V transform the perception from all agents into the correspondingquery Q , key K , and value V .
Specifically, each row vector of H can be seen as a querying result of one agent for all available information, and the hidden state H will be fed intothe subsequent integration network to finally obtain the output aggregated representation z t .
Adetailed discussion about the design of the integration network can be found in Appendix A.2.
[Figure]
On the centralized training phase, we use the aggregatedrepresentation z t as extra inputs to feed the values function.
Since z t contains the information required to determinethe true state, taking z t as extra inputs could reduce theuncertainty about the environment states and produce better estimations on the Q-value for value functions underany value-based policy learning algorithm.
Information Extraction.
Similar to the decision processof human beings, global messages are usually redundantfor an individual agent to make good coordination in communication systems.
For example, on the task of Traffic
Junction [3], one natural idea is that the information ofneighbouring cars are more important for agents to perceive than those distant ones, and the unrelated information in global message may sometimes even confuse the agents and impede the learning when themap is large.
A toy experiment shown in Fig.
2 supports this idea.
We apply the QMIX algorithmin Traffic Junction tasks with different sight settings.
The results show that the agents learn betterpolicies when in a small-sight setting (sight-1) than both in a super-limited-sight setting (sight-0) andfull-sight setting (sight-full).
To make each agent capable of deciding its own perceptive area, we employ the focusing networkto weight the aggregated representations for each agent.
The focusing network is designed as a
Multi-Layer Perception (MLP) with the Sigmoid output activation function to ensure that eachdimension of w ti is bounded between 0 and 1 .
By taking element-wise multiplication with z t , aunique representation could be distilled for individual agents.
In this way, if the focusing networkproduces higher weights on certain dimensions, changes in aggregated representations on thesedimensions would be more significant and thus, the agent would be more sensitive to aggregatedrepresentation on these parts.
On the contrary, if some near-zero weights are outputted on somedimensions, information on those dimensions would be filtered out.
3.2 Information Aggregation Representation Optimization
As for the learning process of aggregated representation, we consider two typical objectives inglobal encoder training: reconstruction and multi-step prediction, which constrain the representationsproduced by the global encoder to be sufficient and compact respectively.
For the reconstructionobjective, we employ an additional decoder, which aims to reconstruct the global state by theaggregated representation, to allow self supervision on the global encoder.
Specifically, the decoder are optimized together with aggregation encoder by reconstructing global states s t from messages o t :
[Equation]
[Equation]
where f θ , g η denote the encoder network parameterized by θ and the decoder network parameterizedby η respectively.
This loss term resembles a classical auto-encoder loss, while the decoder here isnot to reconstruct the input, but to recover the global state from representations instead.
By utilizingthis loss, we guide the encoder to extract observational features that can help infer the global stateand let z t
= f θ ( o t ) be a sufficient representation.
As for the multi-step prediction objective, we constrain the produced representation to be predictiveon future information.
Specifically, we design a transition model h ψ : Z ×A n → Z parameterized byψ as auxiliary model, which predicts the aggregated representation z t +1 on next step t +1 through theaggregated representation z t and joint action a t on current step t .
We regress the predicted aggregatedrepresentation after k -step rollout on the actual aggregated representation of future messages o t + k ,updating both the aggregation encoder and the auxiliary model via the multi-step prediction loss:
[Equation]
To further stabilize the learning process, we apply the double network technique, which employs twonetworks with the same architecture but different update frequencies, for the aggeration encoder.
Thetarget network is updated via Exponential Moving Average (EMA) like in SPR [34].
By forcing theaggregated representation to be predictive on its future states, the aggregated representation could bemore correlated with the information required for its decision-making, which meets the compactnessstandard.
By combining these two objectives, the aggregation encoder could extract more helpfulinformation for agents to coordinate better.
To improve the capability of information extraction onindividual agents, we also the learning process of these components with a RL objective.
Specifically,we consider minimizing the TD loss:
[Equation]
where Q tot is computed with individual Q -values.
The computation of Q-values is actually dependenton the specific valued-based learning algorithm.
We apply it to prevalent methods, including
VDN [37], QMIX [32], and QPLEX [40].
Moreover, the updating of the focusing network is coupledto the RL objective, making the weights produced by the focusing network could be task-sensitive,which could also facilitate policy learning.
Experiment
We conduct experiments on various benchmarks with different communication request levels.
We aimto answer the following questions in this section: 1) How does our method perform when comparedwith multiple baselines in various scenarios?
(Sec. 4.1) 2)
What kind of knowledge has been learnedby the information aggregation encoder?
(Sec. 4.2) 3)
How can the information extraction moduleextract the most relevant information for the individual from the learned embedding space?
(Sec.
Can MASIA be applied to different value decomposition baselines to improve their coordinationability and robustness in various communication conditions?
(Sec. 4.4)
We compare MASIAagainst a variety of baselines, including communication-free methods and some state-of-the-artcommunication approaches.
QMIX [32] is a strong communication-free baseline, and we use theimplementation by PyMARL 1 for comparison which has shown excellent performance on diverse
⋯ ⋯ ⋯ " ! "
" # ! # # $ ! $ !
$
[Figure] [Figure]
e d i a n T e s t W i n R a t
[Figure]
multi-agent benchmarks [33].
TarMAC utilizes an attention mechanism to select messages accordingto their relative importance.
The implementation we used is provided by [42], denoted as TarMAC+ QMIX.
NDQ [42] aims at learning nearly decomposable Q functions via generating meaningfulmessages and communication minimization.
TMC [47] applies a temporal smoothing technique atthe message sender end to drastically reduce the amount of information exchanged between agents.
For the ablation study, we design a baseline only different in the communication protocol, whichadopts a full communication paradigm, where each agent gets message from all other teammates ateach timestep, denoted as FullComm.
We evaluate our proposed method on multiple benchmarks shown in Fig 3.
Hallway [42] is acooperative environment under partial observability, where m agents are randomly initialized atdifferent positions and required to arrive at the goal g simultaneously.
We consider two scenarioswith various agents and different groups, and different groups have to arrive at different times.
Level
Based Foraging [30] is another cooperative partially observable grid world game, where agents shouldcoordinate to collect food concurrently.
Traffic Junction [3] is a popular benchmark used to testcommunication ability, where many cars move along two-way roads with one or more road junctionsfollowing the predefined routes, and we test on the medium and hard maps.
Two maps named1o2r_vs_4r and 1o10b_vs_1r from SMAC [42] require the agents to cooperate and communicate toget the position of the enemies.
For evaluation, all results are reported with median performance with95% confidence interval on 5 random seeds.
Details about benchmarks, network architecture andhyper-parameter choices of our method are all presented in Appendix A.1, and A.2, respectively.
4.1 Communication Performance
We first compare MASIA against multiple baselines to investigate the communication efficiency onvarious benchmarks.
As illustrated in Fig. 4, MASIA achieves the best performance with low variance
[Figure] [Figure]
on all benchmarks, indicating MASIA’s strong applicability in scenarios with various difficulties.
In
Hallway (Fig. 4a & 4e), where frequent communications are required for good performance (methodwithout communication such as QMIX fails), other communication methods such as TarMAC, NDQ,and TMC achieve low performance or even fail in this environment.
This indicates that inappropriatemessage generation or message selection would injury the learning process.
We believe the reason why FullComm succeeds is that there is hardly any redundancy in agents’ observations in Hallway.
Our MASIA also succeeds in this environment, showing superiority over others.
The dominatingperformance of MASIA is even more significant in extended Hallway (Fig. 4e), where agents areseparated into different groups.
In this environment, MASIA can help agents extract informationabout their teammates and learn a coordination pattern more efficiently.
In LBF (Fig. 4b & 4f),existing communication-based MARL methods like NDQ, Fullcomm, and TarMAC struggle dueto the sparsity of rewards, especially when the foods are more sparsely distributed (Fig. 4f).
Incontrast to the performance of QMIX in Hallway, QMIX performs well in LBF, which is attributedto the fact that agents can observe the grids near them, and the mixing network of QMIX can helpimprove the coordination ability of the fixed group of agents in the training phase.
Our methodachieves comparable performance with QMIX and TarMAC, showing its strong coordination abilityeven in sparse reward scenarios.
In Traffic Junction (Fig.
4c & 4g), TarMAC and NDQ have highvariance due to the instability of their messages, while MASIA gains high sample efficiency andcan generate steady messages since it aims to reconstruct the state.
On the SMAC benchmarks(Fig. 4d & 4h), we test on two complex scenarios requiring communication to succeed, where oneoverseer is in active service to get the information of the enemies.
Messages are demanded sincethe agents have limited sights so that other teammates need the overseer’s messages to identify thepositions of enemies.
Our method MASIA can maintain the high efficiency of learning and alwayshave competitive performance when converged, which is superior to other baselines.
4.2 Insights into Information Aggregation Encoder
To determine what kind of knowledge the encoder has learned through training, we conduct avisualization analysis on the map 1o2r_vs_4r from SMAC to demonstrate the information contained in the aggregation representation z t .
We project the aggregation representation vectors into twodimensional plane by t-SNE [38] in Fig 5a.
We take trajectories from 3 scenarios of different types ofinitialization under a task where agents have to seek the enemy at the start, discover the enemy, andfinally battle with it for better performance.
It can be observed that (1) the aggregated representationscould be well distinguished by phases.
Projected representations in seeking phase is far from those inbattle phase and closing phase.
This implies that our learned representations could well reflect the truestates.
(2) the aggregated representations are first divergent in seeking phase when enemies have beeninitialized, but become increasingly interlaced later until the closing phase, when enemies have beenwiped out after fierce battle.
Since enemies are highly related to decision making, such result verifiesthat the aggregated representations exploits also reward information.
To sum up, the visualizationresults shows that MASIA can extract valuable global information with these representations.
4.3 Study about Individual Information Extraction
[Figure]
where s n means the sight range is
To demonstrate the effectiveness of our information extraction module, we analyse the weights computed by focusing network.
Specifically, we select the Traffic Junction (medium) task for evaluation,and compare the weights produced by two different agent.
In thisenvironment gents could dynamically enter or leave the plane, making the agents staying in the environment flexible through time.
Itcan be observed that agent 3 and agent 6 put focus on similar areas of the aggregated representation.
Especially after timestep 15 ,when agents 3 and 6 are in similar situations and distant from theintersection, their focuses are nearly the same.
This verifies thatthe global state information have been successfully extracted to individual agents.
Also, on the top of the figure, we draw a figureto measure the relationship between cosine similarity of weight vectors of different agents againsttimesteps.
It reveals that the similarity of their focus rises after these two agents begin to proceed in the same line (indicated by the render images posted on the lower parts of Fig. 5).
Thisalso conforms to the intuition that similar messages should be extracted for similar observations.
4.4 Generality of Our Method
QMIX
[Figure]
We aim to verify that the proposed approach is agnostic to varioussight ranges and applied value-based MARL methods.
We first conduct experiments on the map 1o10b_vs_1r to show that MASIAcould also generalize well on agents with limited observations.
Theresults in Sec.
4.1 show the performance of MASIA when the agentshave a sight range of 9 .
When we narrow the agents’ sight ranges, asshown in Fig.
6, by receiving and aggregating messages from teammates, the performance of MASIA does not suffer from a significantdrop.
Our information aggregation and extraction modules preventthe agent from forfeiting knowledge about the state when the sightrange is further limited.
To show the generality of the MASIA framework, we also carry out experiments to integrate MASIAwith current baselines, including VDN, QMIX, and QPLEX.
As illustrated in Fig.
7, when integratingwith MASIA, the performance of the baselines can be vastly improved on the map 1o10b_vs_1r from
SMAC.
In this scenario, one overseer is in service to monitor the situations of the enemies.
Withoutcommunication, the other agents have to exhaustively search the map for the enemies.
With reliablecommunications, they could communicate with each others and the overseer for better coordination.
The results demonstrate that MASIA can efficiently aggregate the messages and improve the agents’coordination ability for these value-based MARL methods.
Related Work
Multi-agent reinforcement learning MARL has made prominent progress these years.
Havingemerged under the CTDE paradigm, many methods are designed to relieve the non-stationarity issue,and have made noticeable progress these years.
Most of them can be roughly divided into policybased and value-based methods.
Typical policy gardient-methods involves MADDPG [23], COMA[12], MAAC [16], SQDDPG [41], FOP [48], and HAPPO [19] which explore the optimization ofmulti-agent policy gradient methods, while value-based methods mainly focus on the factorization ofthe global value function [39, 6].
VDN [37] applies a simple additive factorization to decompose thejoint value function into agent-wise value functions.
QMIX [32] structurally enforces the learnedjoint value function to be monotonic with respect to the agent’s utilities, which can represent a richerclass of value functions.
QPLEX [40] further takes a duplex dueling network architecture to factorizethe joint value function, achieving a full expressiveness power of IGM [35].
Multi-agent Communication Communication plays a promising role in multi-agent coordinationunder partial observability [50].
Extensive researches have been made on learning communicationprotocols to improve performance on cooperative tasks [13, 11, 21, 44, 9].
Previous works canbe divided into two categories.
One focuses on generating a meaningful message for the messagesenders.
The simplest way is to treat the raw local observation, or the local information history asmessage [11, 36].
VBC [46] and TMC [47] apply techniques, such as variance-based control andtemporal smoothing, in the sender end to make the generated messages meaningful and valuablefor policy learning.
NDQ [42] generates minimized messages for different teammates to learnnearly decomposable value functions, and optimize the message generator based on two differentinformation-theory-based regularizers to achieve expressive communication.
On the contrary, otherworks try to learn efficiently extract the most useful message on the receiver end, and they designmechanisms to differentiate the importance of messages.
I2C [5] and ACML [25] employ gatemechanism to be selective on received messages.
There are also works inspired by the broadapplication of the attention mechanism [2, 4].
TarMAC [3] achieves targeted communication viaa simple signature-based soft-attention mechanism, where the sender broadcasts a key encodingthe properties of the agents, then the receiver attends to all received messages for a weighted sumof messages for decision marking.
SARNet [31] and MAGIC [28] further remove the signature in
TarMAC and leverage attention-based networks to learn efficient and interpretable relations betweenentities, decide when and whom to communicate with.
To the best of our knowledge, none of the existing MARL communication methods explicitly considerhow the multiple received messages can be optimized for efficient policy learning.
Agents may beconfused by redundant information from teammates, and simply augmenting the local policy with theraw message may burden the learning.
Our proposed method applies message aggregating module tolearn a compact information representation and extracts the most relevant part for decision-making.
Conclusion and Future Work
In this paper, we investigate on the information representation for multi-agent communication.
Previous works either focus on generating a meaningful message or designing a mechanism toselect the most relevant message in a raw way, ignoring the aggregation of the message, resultingin low sample efficiency in complex scenarios.
Our approach improves communication efficiencyby learning a compact information representation to ground the true state and optimizing it in aself-supervised way.
Also, we apply a focusing network to extract the most relevant part for decisionmaking.
We conduct sufficient experiments in various benchmarks to verify the efficiency of theproposed methods, and more visualization results further reveal why our approach works.
For futurework, more results on image input and solving the scalability issue when facing environments withhundreds or thousands of agents by techniques like grouping would be of great interest.
Besides, wedo not see it has obvious negative societal impacts.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection 6.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes] Our paper conforms to them.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] See our
supplemental material.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Appendix A.2.
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See the results in Section 4.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix A.2.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Our new assets are included in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]