prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the main methodology used in the research?

Read the following text and answer briefly.

Context 1: Paper Info

Title: On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning
Publish Date: Unkown
Author List: Seth Karten, Siva Kailas, Huao Li, Katia Sycara

Figure

Figure1 By using contrastive learning, our method seeks similar representations between the state-message pair and future states while creating dissimilar representations with random states Thus satisfying the utility objective of the information bottleneck The depicted agents are blind and cannot see other cars 
Figure 2 An example of two possible classes, person and horse, from a single observation in the Pascal VOC game 
Figure 3  Blind Traffic Junction Left: Our method uses compositional complexity and contrastive utility to outperform other baselines in terms of performance and sample complexity The legend provides the mean ± variance of the best performance Right: Top: success, contrastive, and complexity losses for our method Right, Bottom: success, autoencoder loss for ae-comm with supervised pretraining 
Figure 4  Pascal VOC Game Representing compositional concepts from raw pixel data in images to communicate multiple concepts within a single image Our method significantly outperforms ae-comm and no-comm due to our framework being able to learn composable, independent concepts 
Figure 5  Blind Traffic Junction Social shadowing enables significantly lower sample complexity when compared to traditional online MARL 
Beta ablation: Messages are naturally sparse in bits due to the complexity loss Redundancy measures the capacity for a bijection between the size of the set of unique tokens and the enumerated observations and intents Min redundancy is 1 0 (a bijection) Lower is better 

abstract

Explicit communication among humans is key to coordinating and learning  Social learning, which uses cues from experts, can greatly benefit from the usage of explicit communication to align heterogeneous policies, reduce sample complexity, and solve partially observable tasks  Emergent communication, a type of explicit communication, studies the creation of an artificial language to encode a high task-utility message directly from data 
However, in most cases, emergent communication sends insufficiently compressed messages with little or null information, which also may not be understandable to a third-party listener  This paper proposes an unsupervised method based on the information bottleneck to capture both referential complexity and task-specific utility to adequately explore sparse social communication scenarios in multi-agent reinforcement learning (MARL) 
We show that our model is able to i) develop a natural-language-inspired lexicon of messages that is independently composed of a set of emergent concepts, which span the observations and intents with minimal bits, ii) develop communication to align the action policies of heterogeneous agents with dissimilar feature models, and iii) learn a communication policy from watching an expert's action policy, which we term 'social shadowing' 

INTRODUCTION

Social learning agents analyze cues from direct observation of other agents (novice or expert) in the same environment to learn an action policy from others  However, observing expert actions may not be sufficient to coordinate with other agents  Rather, by learning to communicate, agents can better model the intent of other agents, leading to better coordination 
In humans, explicit communication for coordination assumes a common communication substrate to convey abstract concepts and beliefs directly , which may not be available for new partners  To align complex beliefs, heterogeneous agents must learn a message policy that translates from one theory of mind to another to synchronize coordination 
Especially when there is complex information to process and share, new agent partners need to learn to communicate to work with other agents  Emergent communication studies the creation of artificial language  Often phrased as a Lewis game, speakers and listeners learn a set of tokens to communicate complex observations  
However, in multi-agent reinforcement learning (MARL), agents suffer from partial observability and non-stationarity (due to unaligned value functions) , which aims to be solved with decentralized learning through communication  In the MARL setup, agents, as speakers and listeners, learn a set of tokens to communicate observations, intentions, coordination, or other experiences which help facilitate solving tasks  
Agents learn to communicate effectively through a backpropagation signal from their task performance   This has been found useful for applications in human-agent teaming , multirobot navigation , and coordination in complex games such as StarCraft II   Communication quality has been shown to have a strong relationship with task performance , leading to a multitude of work attempting to increase the representational capacity by decreasing the convergence rates  
Yet these methods still create degenerate communication protocols , which are uninterpretable due to joined concepts or null (lack of) information, which causes performance degradation  In this work, we investigate the challenges of learning a arXiv:2302 14276v1 LG] 28 Feb 2023 messaging lexicon to prepare emergent communication for social learning (EC4SL) scenarios 
We study the following hypotheses: H1) EC4SL will learn faster through structured concepts in messages leading to higher-quality solutions, H2) EC4SL aligns the policies of expert heterogeneous agents, and H3) EC4SL enables social shadowing, where an agent learns a communication policy while only observing an expert agent's action policy 
By learning a communication policy, the agent is encouraged to develop a more structured understanding of intent, leading to better coordination  The setting is very realistic among humans and many computer vision and RL frameworks may develop rich feature spaces for a specific solo task, but have not yet interacted with other agents, which may lead to failure without alignment 
We enable a compositional emergent communication paradigm, which exhibits clustering and informativeness properties  We show theoretically and through empirical results that compositional language enables independence properties among tokens with respect to referential information  Additionally, when combined with contrastive learning, our method outperforms competing methods that only ground communication on referential information 
We show that contrastive learning is an optimal critic for communication, reducing sample complexity for the unsupervised emergent communication objective  In addition to the more human-like format, compositional communication is able to create variable-length messages, meaning that we are not limited to sending insufficiently compressed messages with little information, increasing the quality of each communication 
In order to test our hypotheses, we show the utility of our method in multi-agent settings with a focus on teams of agents, high-dimensional pixel data, and expansions to heterogeneous teams of agents of varying skill levels  Social learning requires agents to explore to observe and learn from expert cues 
We interpolate between this form of social learning and imitation learning, which learns action policies directly from examples  We introduce a 'social shadowing' learning approach where we use first-person observations, rather than third-person observations, to encourage the novice to learn latently or conceptually how to communicate and develop an understanding of intent for better coordination 
The social shadowing episodes are alternated with traditional MARL during training  Contrastive learning, which works best with positive examples, is apt for social shadowing  Originally derived to enable lower complexity emergent lexicons, we find that the contrastive learning objective is apt for agents to develop internal models and relationships of the task through social shadowing 
The idea is to enable a shared emergent communication substrate (with minimal bandwidth) to enable future coordi-nation with novel partners  Our contributions are deriving an optimal critic for a communication policy and showing that the information bottleneck helps extend communication to social learning scenarios 
In real-world tasks such as autonomous driving or robotics, humans do not necessarily learn from scratch  Rather they explore with conceptually guided information from expert mentors  In particular, having structured emergent messages reduces sample complexity, and contrastive learning can help novice agents learn from experts 
Emergent communication can also align heterogeneous agents, a social task that has not been previously studied 

Multi-Agent Signaling

Implicit communication conveys information to other agents that is not intentionally communicated   Implicit signaling conveys information to other agents based on one's observable physical position   Implicit signaling may be a form of implicit communication such as through social cues or explicit communication such as encoded into the MDP through "cheap talk"  
Unlike implicit signaling, explicit signaling is a form of positive signaling that seeks to directly influence the behavior of other agents in the hopes that the new information will lead to active listening  Multi-agent emergent communication is a type of explicit signaling which deliberately shares information 
Symbolic communication, a subset of explicit communication, seeks to send a subset of pre-defined messages  However, these symbols must be defined by an expert and do not scale to particularly complex observations and a large number of agents  Emergent communication aims to directly influence other agents with a learned subset of information, which allows for scalability and interpretability by new agents 

Emergent Communication

Several methodologies currently exist to increase the informativeness of emergent communication  With discrete and clustered continuous communication, the number of observed distinct communication tokens is far below the number permissible   As an attempt to increase the emergent "vocabulary" and decrease the data required to converge to an informative communication "language", work has added a bias loss to emit distinct tokens in different situations  
More recent work has found that the sample efficiency can be further improved by grounding communication in observation space with a supervised reconstruction loss   Information-maximizing autoencoders aim to maximize the state reconstruction accuracy for each agent  How-ever, grounding communication in observations has been found to easily satisfy these input-based objectives while still requiring a myriad more samples to explore to find a task-specific communication space  
Thus, it is necessary to use task-specific information to communicate informatively  This will enable learned compression for task completion rather than pure compression for input recovery  Other work aims to use the information bottleneck to decrease the entropy of messages   In our work, we use contrastive learning to increase representation similarity with future goals, which we show optimally optimizes the Q-function for messages 

Natural Language Inspiration

The properties of the tokens in emergent communication directly affect their informative ability  As a baseline, continuous communication tokens can represent maximum information but lack human-interpretable properties  Discrete 1-hot (binary vector) tokens allow for a finite vocabulary, but each token contains the same magnitude of information, with equal orthogonal distance to each other token 
Similar to word embeddings in natural language, discrete prototypes are an effort to cluster similar information together from continuous vectors   Building on the continuous word embedding properties, VQ-VIB , an information-theoretic observation grounding based on VQ-VAE properties , uses variational properties to provide word embedding properties for continuous emergent tokens 
Like discrete prototypes, they exhibit a clustering property based on similar information but are more informative  However, each of these message types determines a single token for communication  Tokens are stringed together to create emergent "sentences" 

Preliminaries

We formulate our setup as a decentralized, partially observable Markov Decision Process with communication (Dec-POMDP-Comm)  Formally, our problem is defined by the tuple, S, A, M, T , R, O, Ω, γ   We define S as the set of states, A i , i ∈ [1, N ] as the set of actions, which includes task-specific actions, and M i as the set of communications for N agents 
T is the transition between states due to the multi-agent joint action space T : S × A 1 ,    , A N → S  Ω defines the set of observations in our partially observable setting  Partial observability requires communication to complete the tasks successfully  O i : M 1 ,    , M N × Ŝ → Ω maps the communications and local state, Ŝ, to a distribution of observations for each agent 
R defines the reward function and γ defines the discount factor 

Architecture

The policy network is defined by three stages: Observation Encoding, Communication, and Action Decoding  The best observation encoding and action decoding architecture is task-dependent, i e , using multi-layer perceptrons (MLPs), CNNs , GRUs , or transformer layers are best suited to different inputs 
The encoder transforms observation and any sequence or memory information into an encoding H  The on-policy reinforcement learning training uses RE-INFORCE or a decentralized version of MAPPO as specified by our experiments  Our work focuses on the communication stage, which can be divided into three substages: message encoding, message passing (often considered sparse communication), and message decoding 
We use the message passing from   For message decoding, we build on a multiheaded attention framework, which allows an agent to learn which messages are most important   Our compositional communication framework defines the message encoding, as described in section 4 

Objective

Mutual information, denoted as I(X; Y ), looks to measure the relationship between random variables, which is often measured through Kullback-Leibler divergence , I(X; Y ) = D KL (p(x, y)||p(x) ⊗ p(y))  The message encoding substage can be defined as an information bottleneck problem, which defines a tradeoff between the complexity of information (compression, I(X, X)) and the preserved relevant information (utility, I( X, Y )) 
The deep variational information bottleneck defines a trade-off between preserving useful information and compression   We assume that our observation and memory/sequence encoder provides an optimal representation H i suitable for sharing relevant observation and intent/coordination information  We hope to recover a representation Y i , which contains the sufficient desired outputs 
In our scenario, the information bottleneck is a trade-off between the complexity of information I(H i ; M i ) (representing the encoded information exactly) and representing the relevant information I(M j =i ; Y i ), which is signaled from our contrastive objective  In our setup, the relevant information flows from other agents through communication, signaling a combination of the information bottleneck and a Lewis game 
We additionally promote complexity through our compositional independence objective, This is formulated by the following Lagrangian, where the bounds on mutual information Î are defined in equations 1, 2, and 10  Overall, our objective is,

Complexity through Compositional Communication

We aim to satisfy the complexity objective, I(H i , M i ), through compositional communication  In order to induce complexity in our communication, we want the messages to be as non-random as possible  That is, informative with respect to the input hidden state h  In addition, we want each token within the message to share as little information as possible with the preceding tokens 
Thus, each additional token adds only informative content  Each token has a fixed length in bits W   The total sequence is limited by a fixed limit, L l W l ≤ S, of S bits and a total of L tokens  We use a variational message generation setup, which maps the encoded hidden state h to a message m; that is, we are modeling the posterior, π i m (m l |h) 
We limit the vocabulary size to K tokens, e j ∈ R D , j ∈ [1, K] ⊂ N, where each token has dimensionality D and l ∈ [1, L] ⊂ N  Each token m l is sampled from a categorical posterior distribution, 0 otherwise such that the message m l is mapped to the nearest neighbor e j   A set of these tokens makes a message m 
To satisfy the complexity objective, we want to use m i to well-represent h i and consist of independently informative m i l  

Independent Information

We derive an upper bound for the interaction information between all tokens  Proposition 4 1  For the interaction information between all tokens, the following upper bound holds: The proof is in Appendix A 1  Since we want the mutual information to be minimized in our objective, we minimize,

Input-Oriented Information

In order to induce complexity in the compositional messages, we additionally want to minimize the mutual information I(H; M ) between the composed message m and the encoded information h  We derive an upper bound on the mutual information that we use as a Lagrangian term to minimize  Proposition 4 2  For the mutual information between the composed message and encoded information, the following upper bound holds:
The proof is in Appendix A 1  Thus, we have our Lagrangian term, Conditioning on the input or observation data is a decentralized training objective 

Sequence Length

Compositional communication necessitates an adaptive limit on the total length of the sequence  Corollary 4 3  Repeat tokens, w, are redundant and can be removed  Suppose one predicts two arbitrary tokens, w k and w l   Given equation 1, it follows that there is low or near-zero mutual information between w k and w l  
A trivial issue is that the message generator will predict every available token as to follow the unique token objective  Since the tokens are imbued with input-oriented information (equation 2), the predicted tokens will be based on relevant referential details  Thus, it follows that tokens containing irrelevant information will not be chosen 
A nice optimization objective that follows from corollary 4 3 is that one can use self-supervised learning with an end-ofsequence (EOS) token to limit the variable total length of compositional message sequences  (3) Algorithm 1 Compositional Message Gen (h t ) m i ∼ N ( ĥ; µ, σ) 9: end for 10: return m

Message Generation Architecture

Now, we can define the pipeline for message generation  The idea is to create an architecture that can generate features to enable independent message tokens  We expand each compressed token into the space of the hidden state h (1-layer linear expansion) since each token has a natural embedding in R |h|  
Then, we perform attention using a softmin to help minimize similarity with previous tokens and sample the new token from a variational distribution  See algorithm 1 for complete details  During execution, we can generate messages directly due to equation 1, resolving any computation time lost from sequential compositional message generation 

Utility through Contrastive Learning

First, note that our Markov Network is as follows: H j → M j → Y i ← H i   Continue to denote i as the agent identification and j as the agent ID such that j = i  We aim to satisfy the utility objective of the information bottleneck, I(M j ; Y i ), through contrastive learning as shown in figure 1  Proposition 5 1 
Utility mutual information is lower bounded by the contrastive NCE-binary objective, The proof is in Appendix A 1  This result shows a need for gradient information to flow backward across agents along communication edge connections 

Experiments and Results

We condition on inputs, especially rich information (such as pixel data), and task-specific information  When evaluating an artificial language in MARL, we are interested in referential tasks, in which communication is required to complete the task  With regard to intent-grounded communication, we study ordinal tasks, which require coordination information between agents to complete successfully 
Thus, we consider tasks with a team of agents to foster messaging that communicates coordination information that also includes their observations  To test H1, structuring emergent messages enables lower complexity, we test our methodology and analyze the input-oriented information and utility capabilities 
Next, we analyze the ability of heterogeneous agents to understand differing communication policies (H2))  Finally, we consider the effect of social shadowing (H3), in which agents solely learn a communication policy from an expert agent's action policy  We additionally analyze the role of offline reinforcement learning for emergent communication in combination with online reinforcement learning to further learn emergent communication alongside an action policy 
We evaluate each scenario over 10 seeds 

Environments

Blind Traffic Junction We consider a benchmark that requires both referential and ordinal capabilities within a team of agents  The blind traffic junction environment requires multiple agents to navigate a junction without any observation of other agents  Rather, they only observe their own state location 
Ten agents must coordinate to traverse through the lanes without colliding into agents within their lane or in the junction  Our training uses REINFORCE   Pascal VOC Game We further evaluate the complexity of compositional communication with a Pascal VOC   This is a two-agent referential game similar to the Cifar game but requires the prediction of multiple classes 
During each episode, each agent observes a random image from the Pascal VOC dataset containing exactly two unique labels  Each agent must encode information given only the raw pixels from the original image such that the other agent can recognize the two class labels in the original image  An agent receives a reward of 0 25 per correctly chosen class label and will receive a total reward of 1 if both agents guess all labels correctly 
See figure 2  Our training uses heterogeneous agents trained with PPO (modified from MAPPO repository)  For simplicity of setup, we consider images with exactly two unique labels from a closed subset of size five labels of the original set of labels from the Pascal VOC data  Furthermore, these images must be of size 375 × 500 pixels 
Thus, the resultant dataset comprised 534 unique images from the Pascal VOC dataset 

Baselines

To evaluate our methodology, we compare our method to the following baselines: (1) no-comm, where agents do not communicate; (2) rl-comm, which uses a baseline communication method learned solely through policy loss ; (3) ae-comm, which uses an autoencoder to ground communication in input observations ; (4) VQ-VIB, which uses a variational autoencoder to ground discrete communication in input observations and a mutual information objective to ensure low entropy communication  
We provide an ablation of the loss parameter β in table 1 in the blind traffic junction scenario  When β = 0, we use our compositional message paradigm without our derived loss terms  We find that higher complexity and independence losses increase sample complexity  When β = 1, the model was unable to converge 
However, when there is no regularization loss, the model performs worse (with no guarantees about referential representation)  We attribute this to the fact that our independence criteria learns a stronger causal relationship  There are fewer spurious features that may cause an agent to take an incorrect action 
In order to understand the effect of the independent concept representation, we analyze the emergent language's capacity for redundancy  A message token m l is redundant if there exists another token m k that represents the same information  With our methodology, the emergent 'language' converges to the exact number of observations and intents required to solve the task 
With a soft discrete threshold, the independent information loss naturally converges to a discrete number of tokens in the vocabulary  Our β ablation in table 1 yields a bijection between each token in the vocabulary and the possible emergent concepts, i e , the enumerated observations and intents  Thus for β = 0 1, there is no redundancy 
Sparse Communication In corollary 4 3, we assume that there is no mutual information between tokens  In practice, the loss may only be near-zero  Our empirical results yield independence loss around 1e − 4  In table 1, the size of the messages is automatically compressed to the smallest size to represent the information 
Despite a trivially small amount of mutual information between tokens, our compositional method is able to reduce the message size in bits by 2 3x using our derived regularization, for a total of an 8x reduction in message size over non-compositional methods such as ae-comm  Since the base unit for the token is a 32-bit float, we note that each token in the message may be further compressed 
We observe that each token uses three significant digits, which may further compress tokens to 10 bits each for a total message length of 20 bits 

Communication Utility Results

Due to coordination in MARL, grounding communication in referential features is not enough  Finding the communication utility requires grounding messages in ordinal information  Overall, figure shows that our compositional, contrastive method outperforms all methods focused on solely input-oriented communication grounding 
In the blind traffic junction, our method yields a higher average task success rate and is able to achieve it with a lower sample complexity  Training with the contrastive update tends to spike to high success but not converge, often many episodes before convergence, which leaves area for training improvement 
That is, the contrastive update begins to find aligned latent spaces early in training, but it cannot adapt the methodology quickly enough to converge  The exploratory randomness of most of the early online data prevents exploitation of the high utility f + examples  This leaves further room for improvement for an adaptive contrastive loss term 
Regularization loss convergence After convergence to high task performance, the autoencoder loss increases in order to represent the coordination information  This follows directly from the information bottleneck, where there exists a tradeoff between utility and complexity  However, communication, especially referential communication, should have an overlap between utility and complexity 
Thus, we should seek to make the complexity loss more convex  Our compositional communication complexity loss does not converge before task performance convergence  While the complexity loss tends to spike in the exploratory phase, the normalized value is very small  Interestingly, the method eventually converges as the complexity loss converges below a normal- ized 0 3 
Additionally, the contrastive loss tends to decrease monotonically and converges after the task performance converges, showing a very smooth decrease  The contrastive f − loss decreases during training, which may account for success spikes prior to convergence  The method is able to converge after only a moderate decrease in the f + loss 
This implies empirical evidence that the contrastive loss is an optimal critic for messaging  See figure 3 

Heterogeneous Alignment Through Communication

In order to test the heterogeneous alignment ability of our methodology to learn higher-order concepts from highdimensional data, we analyze the performance on the Pascal VOC game  We compare our methodology against ae-comm to show that concepts should consist of independent information directly from task signal rather than compression to reconstruct inputs 
That is, we show an empirical result on pixel data to verify the premise of the information bottleneck  Our methodology significantly outperforms the observation-grounded ae-comm baseline, as demonstrated by figure 4  The ae-comm methodology, despite using autoencoders to learn observation-grounded communication, performs only slightly better than no-comm 
On the other hand, our methodology is able to outperform both baselines significantly  It is important to note that based on figure 4, our methodology is able to guess more than two of the four labels correctly across the two agents involved, while the baseline methodologies struggle to guess exactly two of thew four labels consistently 
This can be attributed to our framework being able to learn compositional concepts that are much more easily discriminated due to mutual independence 

Social Shadowing

Critics of emergent communication may point to the increased sample complexity due to the dual communication and action policy learning  In the social shadowing scenario, heterogeneous agents can learn to generate a communication policy without learning the action policy of the watched expert agents  To enable social shadowing, the agent will alternate between a batch of traditional MARL (no expert) and (1st-person) shadowing an expert agent performing the task in its trajectory 
The agent only uses the contrastive objective to update its communication policy during shadowing  In figure , the agent that performs social shadowing is able to learn the action policy with almost half the sample complexity required by the online reinforcement learning agent  Our results show that the structured latent space of the emergent communication learns socially benevolent coordination 
This tests our hypothesis that by learning communication to understand the actions of other agents, one can enable lower sample complexity coordination  Thus, it mitigates the issues of solely observing actions 

Discussion

By using our framework to better understand the intent of others, agents can learn to communicate to align policies and coordinate  Any referential-based setup can be performed with a supervised loss, as indicated by the instant satisfaction of referential objectives  Even in the Pascal VOC game, which appears to be a purely referential objective, our results show that intelligent compression is not the only objective of referential communication 
The emergent communication paradigm must enable an easy-to-discriminate space for the game  In multi-agent settings, the harder challenge is to enable coordination through communication  Using contrastive communication as an optimal critic aims to satisfy this, and has shown solid improvements  Since contrastive learning benefits from good examples, this method is even more powerful when there is access to examples from expert agents 
In this setting, the communication may be bootstrapped, since our optimal critic has examples with strong signals from the 'social shadowing' episodes  Additionally, we show that the minimization of our independence objective enables tokens that contain minimal overlapping information with other tokens 
Preventing trivial communication paradigms enables higher performance  Each of these objectives is complementary, so they are not trivially minimized during training, which is a substantial advantage over comparative baselines  Unlike prior work, this enables the benefits of training with reinforcement learning in multi-agent settings 
In addition to lower sample complexity, the mutual information regularization yields additional benefits, such as small messages, which enables the compression aspect of sparse communication  From a qualitative point of view, the independent information also yields discrete emergent concepts, which can be further made human-interpretable by a post-hoc analysis  
This is a step towards white-box machine learning in multi-agent settings  The interpretability of this learned white-box method could be useful in human-agent teaming as indicated by prior work   The work here will enable further results in decision-making from high-dimensional data with emergent concepts 
The social scenarios described are a step towards enabling a zero-shot communication policy  This work will serve as future inspiration for using emergent communication to enable ad-hoc teaming with both agents and humans 

Appendix

A 1  Proofs Proposition 4 1 For the interaction information between all tokens, the following upper bound holds: Proof  Starting with the independent information objective, we want to minimize the interaction information, which defines the conditional mutual information between each token and, Let π i m (m l |h) be a variational approximation of p(m l |h), which is defined by our message encoder network 
Given that each token should provide unique information, we assume independence between m l   Thus, it follows that our compositional message is a vector, m = [m 1 ,       , m L ], and is jointly Gaussian  Moreover, we can define q( m|h) as a variational approximation to p(m|h) = p(m 1 ;       , m L |h) 
We can model q with a network layer and define its loss as || m − m|| 2   Thus, transforming equation 4 into variational form, we have, it follows that q( m|h) log q( m|h)d m ≥ q( m|h) log Thus, we can bound our interaction information, Proposition 4 2 For the mutual information between the composed message and encoded information, the following upper bound holds:
Proof  By definition of mutual information between the composed messages M and the encoded observations H, we have, Substituting q( m|h) for p( m|h), the same KL Divergence identity, and defining a Gaussian approximation z( m) of the marginal distribution p( m), it follows that, In expectation of equation 1, we have,
This implies that, for m = [m 1 ,       , m L ], there is probabilistic independence between m j , m k , j = k  Thus, expanding, it follows that, where z(m l ) is a standard Gaussian  Proposition 5 1  Utility mutual information is lower bounded by the contrastive NCE-binary objective, Proof  We suppress the reliance on h since this is directly passed through 
By definition of mutual information, we have, Our network model learns π R + (y|m) from rolled-out trajectories, R + , using our policy  The prior of our network state, π R − (y), can be modeled from rolling out a random trajectory, R−  Unfortunately, it is intractable to model π R + (y|m) and π R − (y) directly during iterative learning, but we can sample y + ∼ π R + (y|m) and y − ∼ π R − (y) directly from our network during training 
It has been shown that log p(y|m) provides a bound on mutual information , with the expectation over l p(m l , y l )  However, we need a tractable understanding of the information Y   In the information bottleneck, Y represents the desired outcome  In our setup, y is coordination information that helps create the desired output, such as any action a −  
This implies, y =⇒ a −   Since the transition is known, it follows that a − =⇒ s − f , a random future state  Thus, we have, π This is similar to the proof for lemma A 5, but requires assumptions on messages m from the emergent language  We note that when m is random, the case defaults to lemma A 5  Thus, we assume we have at least input-oriented information in m given sufficiently satisfying equation 2  Given a sufficient emergent language, it follows that y =⇒ a + , where a + is an intention action based on m 
Similarly, since the transition is known, a + =⇒ s + f , a desired goal state along the trajectory  Thus, we have, π R + (y|m) = p(s = s + f |y, m)  Recall the following (as shown in ), which we have adapted to our communication objective, Proposition A 3 (rewards → probabilities)  The Q-function for the goal-conditioned reward function r g (s t , m t ) = (1 − γ)p(s = s g |y t ) is equivalent to the probability of state s g under the discounted state occupancy measure:
and Lemma A 4  The critic function that optimizes equation 8 is a Q-function for the goal-conditioned reward function up to a multiplicative constant 1 The critic function f (s, m, s f ) = y enc(s f ) represents the similarity between the encoding y = enc(s, m) and the encoding of the future rollout s f  
Given lemmas A 5 A 6 A 8 and proposition A 7, it follows that equation 8 is the NCE-binary (InfoMAX ) objective, Î(M j , Y i ) = log σ(f (s, m, s + f )) + log 1 − σ(f (s, m, s − f )) which lower bounds the mutual information, I(M j , Y i ) ≥ Î(M j , Y i )  The critic function is unbounded, so we constrain it to [0, 1] with the sigmoid function, σ( * ) 
We suppress the reliance on h since this is directly passed through  By definition of mutual information, we have, Our network model learns π R + (y|m) from rolled-out trajectories, R + , using our policy  The prior of our network state, π R − (y), can be modeled from rolling out a random trajectory, R− 
Unfortunately, it is intractable to model π R + (y|m) and π R − (y) directly during iterative learning, but we can sample y + ∼ π R + (y|m) and y − ∼ π R − (y) directly from our network during training  It has been shown that log p(y|m) provides a bound on mutual information , with the expectation over l p(m l , y l ) 
However, we need a tractable understanding of the information Y   Lemma A 5  π R − (y) = p(s = s − f |y)  In the information bottleneck, Y represents the desired outcome  In our setup, y is coordination information that helps create the desired output, such as any action a −   This implies, y =⇒ a −   Since the transition is known, it follows that a − =⇒ s − f , a random future state 
Thus, we have, π R − (y) = p(s = s − f |y)  Lemma A 6  π R + (y|m) = p(s = s + f |y, m)  This is similar to the proof for lemma A 5, but requires assumptions on messages m from the emergent language  We note that when m is random, the case defaults to lemma A 5  Thus, we assume we have at least input-oriented information in m given sufficiently satisfying equation 2  Given a sufficient emergent language, it follows that y =⇒ a + , where a + is an intention action based on m 
Similarly, since the transition is known, a + =⇒ s + f , a desired goal state along the trajectory  Thus, we have, π R + (y|m) = p(s = s + f |y, m)  Recall the following (as shown in ), which we have adapted to our communication objective, Proposition A 7 (rewards → probabilities)  The Q-function for the goal-conditioned reward function r g (s t , m t ) = (1 − γ)p(s = s g |y t ) is equivalent to the probability of state s g under the discounted state occupancy measure:
and Lemma A 8  The critic function that optimizes equation 8 is a Q-function for the goal-conditioned reward function up to a multiplicative constant 1 p(s f ) : exp(f * (s, m, s f ) = 1 p(s f ) Q π s f (s, m)  The critic function f (s, m, s f ) = y enc(s f ) represents the similarity between the encoding y = enc(s, m) and the encoding of the future rollout s f  
Given lemmas A 5 A 6 A 8 and proposition A 7, it follows that equation 8 is the NCE-binary (InfoMAX ) objective, which lower bounds the mutual information, I(M j , Y i ) ≥ Î(M j , Y i )  The critic function is unbounded, so we constrain it to [0, 1] with the sigmoid function, σ( * ) 
Context 2: Hey folks! Here is the shiny new Changelog thread  We're including the archived patch notes from the old forums, so that they are preserved for anyone that would like to reference back to them  We will continue to update this thread with new notes as the patches are released 
Chat is now accessible from the quest board, upgrade screen, and many other menus 
Tapping on objects and menus may reveal helpful hints about that object 
Team PI is now colored red if lower than recommended for the quest 
Many text fixes and consistency improvements 
• A new Basic Catalyst found in Special Events is used in every recipe!
Several heroes have received improvements to their base stats 
The abilities of all Champions have increased in effectiveness 
A new Critical Boost buff has been introduced 
Iron Fist and Spiderman now have the ability to Armor Break with their Critical Hits 
Deadpool’s ability to Regenerate is more powerful, but only triggers once per fight 
Scarlet Witch now has a chance to trigger Nullify off of any Critical Hit 
Juggernaut and Rhino now have a layer of Armor 
Punisher and Winter Solider now may also trigger Fury in addition to Bleed 
Colossus now further increases his base Armor with the Armor Up ability 
Thor and Ronan no longer Armor Break; instead, base stats and Stun durations have improved 
We reduced the effectiveness of the Revive items in order to give away more as rewards 
A bonus of 50% for using ISO-8 matching your Champion’s Class can now be previewed on the Upgrade screen 
It’s now possible sell Champions in exchange for ISO-8 and Gold  The amount received increases proportionately to the Rank and Level of the sold Champion 
-You can now skip dialogue on the quest map by pressing ‘SKIP’ 
-Added a ‘Quit’ button directly on the quest interface 
-The Back button on the Top Bar now returns the player to the Home screen 
-Various game balance and cosmetic improvements to the available quests 
• PVP energy has been replaced with Hero Stamina  Each Hero has their own Stamina values, meaning the more Heroes you have the more you can play in PVP 
• Each Hero has 1 Stamina and takes 2 hours to recharge 
We have removed the Next Quest button for a much more favorable and flavorful approach to teaching and informing people about Marvel : Contest of Champions  In the Main Menu(Bottom Right Corner) you will now see an image of the Collector showing you what the best or recommended actions that you should preform  This can be anything from opening Crystals, Continuing a Quest, Ranking Up Champions if the difficulty is too hard, Tips where to obtain items, and Playing Versus/Arenas 
• Adjusted the PI calculation for Power Burn and Power Drain abilities to improve accuracy 
• Significantly increased the Power Burn multiplier as well as the amount of Power burned  Prior to this change, Vision's Special Attack damage output was far below the curve  Vision's Special Damage is distinct from other heroes in that the dependency on opponents' Power levels cause the damage dealt to be highly variable, and sometimes quite low; however, when striking an opponent with high Power levels, Vision has the potential to deal very high amounts of direct, Armor-ignoring damage 
• Slightly adjusted the Armor Break trigger to be less punishing to opponents with the Armor Up ability without sacrificing PI or damage output 
• Slightly increased his base Health and, in turn, the amount of Health recovered by Regeneration  This improvement is reflected by an increase to PI of about 1% 
• Slightly reduced the damage from Bleeding, but slightly increased the amount of Power drained by E M P  Arrow to compensate  This added utility strengthens the choice between whether to offensively Bleed the enemy or defensively drain their Power  These changes may modify PI by +/-1% 
• Slightly reduced the frequency of Nullify for basic attacks, but slightly increased the chance a Special Attack is critical  Chaotic Bombardment no longer has a chance to critical, and instead has a 100% chance to Nullify the target  This is less punishing to opponents with beneficial effects, while providing a more reliable source of Nullify  Overall, her PI has decreased by about 2% 
• Decreased base Health and Attack by 2% each to bring his PI in line with other Champions without compromising Special Attack effectiveness 
• Slightly increased base Health by 2% to bring his PI in line with other Champions  This change may result in a PI increase of up to 1% 
• Fixed a bug with her Bleed ability scaling incorrectly  This has no effect on PI 
• User's on iPhone 4 devices will no longer encounter a progression blocker after fighting Iron Man in the tutorial 
• Fixed an issue where player's Hero would disappear after using a special move 
• Fixed an issue where very rarely a character would lose all functionality when dashing 
• Added additional Network support to better diagnose disconnects  The game should resolve and recover much more gracefully than in previous updates 
• Adjusted some of the touch sensitivity while fighting  Heroes moves should feel more responsive  This is something that is going to be an ongoing process  Please let us know how you think it feels 
• Fixed various issues with Chat 
• We have updated Open GL versions/drivers for iOS devices that support Open GL 3 0 
• User's will no longer receive delayed Game Center notifications  This caused some weirdness to occur while opening Crystals in the Crystal Vault 
• The Crystal Vault has received another polish pass and should now feel much more responsive, thank you for all your feedback on this feature!
• Many more minor bug fixes were included in this update 
• Special Attack 1 base damage increased by +25% Attack Rating 
• Heavy Attack base Power gained reduced to 63 points 
We recently improved the functionality of Heavy Attacks, so they’re easier to use  Their base Power has been reduced to normal levels – previously, they generated Power at a higher rate to compensate for their difficult execution  Special Attacks have been adjusted to give the unlucky recipients more of a fighting chance  These changes bring these attacks in line with existing damage-to-power ratios 
*NOTE: Special Attacks only generate Power for the target struck, not for the user; this prevents infinite loops and helps serve as a comeback mechanic 
Versus Crystal prizes have been adjusted due to the Champion Stamina changes 
Arena Crystal prizes have been increased to help balance the adjustments to the Versus Crystal 
Payouts have significantly increased when receiving a duplicate Champion with a Star rating of two or more  The boosted amount increases based on Star rating  We apologize for any inconvenience caused by delivering each reward individually, and are working to get a fix to you as soon as possible  In the meantime, using the “Skip” button avoids the inconvenience 
• We fixed a bug where finding a new match could cost a player Units 
• Spending Units to find a new opponent will now return opponents with lower ratings 
• Chapters 3 and 4 of Act 2 Story Quests are now available  A mysterious opponent awaits you at the end of Act 2!
*NOTE: This caused some players' progress to reset for a brief time, but that issue should now be corrected 
• Event Quest difficulty has been adjusted to match Catalyst availability 
• Rank-Up Recipes have been adjusted to be more accessible across all ranks 
• Bosses for the Monday through Saturday Daily Events now have a small chance to drop a Class Catalyst  This is in addition to the drop chance from Chests 
• Ambush Rates have been adjusted on all Event Quests 
• Increased Catalyst drops for the Collector Free-For-All Event Quest 
• Alpha Catalysts now have a chance to drop from Chests in Medium and Hard difficulties of The Collector Free-For-All event 
• The unobtainable chest in Act 1, Chapter 1, Quest 6 has been removed from the Battlerealm 
Increased the amount of Gold awarded by the Arena Crystal 
Slightly reduced the cost to level-up a 3-Star Champion at Rank 1 to cleanly align with ISO-8 chunk values 
Fixed a bug with Billion-Dollar Punch not triggering Armor Break 
• Duplicate 2-Star, 3-Star, and 4-Star Champions now awaken a brand new ability unique to that Champion in addition to the rare ISO8 they currently give  Duplicates thereafter continue to level-up this ability to make it stronger  When a Champion is awakened, their Stars turn bright and glow, making them easy to identify (and look pretty cool too)  These new abilities can be quite powerful, so please fight responsibly!
• Various other improvements, including rank and level information for opponents, find match options in team select, and animation tuning 
• There is now a chance to encounter the elusive Treasure Adaptoid, who divulges his hoard of ISO8 and Gold to those able to defeat him in battle 
• Class Relationships can be viewed by tapping “Enemy Classes” before entering a quest, and preview the number of enemies in that quest for each class type 
• You can also now see rewards for completion and exploration on the Edit Team screen 
• Opponents are more aware of the distance between you and them, improving their interaction with knockback effects, such as that from Heavy Attacks 
Mutant Champions are now effective against Skill Champions 
• The high Special Attack damage and regenerative abilities of Mutant Champions are effective against Skill Champions, which typically rely on Bleed damage from their weaponry  We think of this relationship as if the X-Gene grants Mutant Champions superpowers that evolved to be stronger than Champions that are merely “Skilled” 
Skill Champions are now effective against Science Champions 
• While scientists fiddle in their cute little laboratories to create flasks full of serums to turn even frail young men into super-soldiers, Skill Champions were just born that way baby  Often donning sharp weaponry to make their opponents Bleed, Skill Champions enjoy watching the high base attributes of Science Champions just melt away 
Cosmic Champions are now effective against Tech Champions 
• Tech Champions construct durable robots and thick suits of Armor to outlast their opponents in battles of tank-the-nuke   which gives Cosmic Champions extra time to build up stacks of beneficial effects to overrun Tech Champions using their peculiar alien enhancements 
• Tech Champions are still effective against Mutant Champions 
Tech Champions typically excel at Armor, Resistance, and Power manipulation, which is effective against the high Special Attack damage of Mutant Champions  Think of the robotic Sentinels adapting for tactical advantages in the war against Mutantkind!
Science Champions are still effective against Mystic Champions 
• Science Champions – a Class of behemoths like Hulk and super-soldiers like Captain America – typically have above average base attributes like Health, Attack, and Armor  These raw stats cannot be affected by pesky Mystics and their removal abilities: Nullify and Purge 
Mystic Champions are still effective against Cosmic Champions 
• Cosmic Champions explore strange new beneficial effects to seek out new power and new abilities, to boldly take their attributes where no class has gone before  Well, not if Mystic Champions – who are fully capable of stripping Cosmic Champions of their beneficial effects – have anything to say about it! Maybe it’s the Mystic Agenda to protect the secrets of the universe?
These changes ensure that having a Class Bonus always gives you the advantage it promises, as it now also reflects ability trends for a particular Class  Please keep in mind that these are generalizations, and some Champions abilities may not always strictly align with these relationships  Learn more about Champions’ abilities by viewing their profiles and tapping on features for detailed information 
• When you attack someone, you charge up their Power in addition to yours  This meant they would reach a full three bars while you only reached one and a half  We've reduced the amount defenders receive such that you'll be at two bars when they're at three  This change maintains the underdog functionality to give defenders a chance to comeback while being less punishing to players earning high Combos 
New damage types for attacks now play a larger role in the abilities of Champions  For example, some heroes power-up by successfully blocking magical damage, while others’ abilities may harm anyone that makes physical contact with them 
New Resistances and Immunities have found their way to the Battlerealm  Some heroes are completely immune to specific status effects based on either lore from the comics or logic  For example, the android Vision has no blood, and is therefore fully immune to Bleed conditions  We’ve also strengthened the effectiveness of certain status effects, so be careful who you choose to bring into battle! Could you guess who might be immune to the new “Poison” condition?
• Poison: Inflicts damage over time and reduces healing and regeneration effectiveness 
• Unstoppable: A buff to shrug off the impact from attacks, but still take the damage 
• Weakness: A debuff that reduces Attack attributes 
• Heal Block: Fully prevents the target from gaining health in any way 
• Power Lock: Seals the target, preventing them from gaining any Power 
• When fighting, you may notice that many status effects are now able to stack  This also changes how certain beneficial “buffs” and detrimental “debuffs” interact with one another  For example, it's now possible to have both Armor Up and Armor Break effects on you simultaneously  Let the tug-o-war begin, and may the strongest effects win!
• Black Bolt's Corkscrew: +25% damage, but at the cost of minor recoil damage 
• Punisher's “Wrath” has been replaced by "Payback"  Payback deals additional damage based on the total damage dealt to Frank 
• Colossus' “Unbreakable” now deal bonus damage based on his armor level at the time of activation 
• All of Black Panther’s special attacks now deal bonus damage based on the number of Bleeds on the target 
• Spider-Man’s Web-Slinger now has a chance to inflict Weakness 
• Vision’s Physical Disruption: Added a minor Power Burn effect due to “his” use of his Infrared Beam  “He” also now purges all status effects while phasing through the ground 
• Scarlet Witch: Increased the Critical Hit Chance for Hex Bolt and Hex Sphere 
• Many knockback effects have been adjusted to improve consistency 
We’ve tested the Signature Abilities quite extensively before releasing them, but there have been a few abilities that we have been keeping an eye on  We’ve compared our notes with the feedback you’ve been sending us and are making some balance changes to them  Thanks for your feedback!
Slightly reduced the frequency and duration of Juggernaut’s “Unstoppable” ability 
• He was indeed a bit too   unstoppable  We’ve toned down the frequency this ability triggers, as well as reduced the duration it’s active for when it does trigger  We feel Juggernaut is still a powerful Champion despite these revisions  Take care!
Slightly reduced the starting values of Wolverine's “Cellular Regeneration” 
• We found that Cellular Regeneration was too strong at lower levels where fewer counters to Regeneration exist 
Re-scaled Gamora's “Assassination” to start higher but scale slower 
• At lower levels, Special Attacks were used too infrequently, giving this powerful ability little visibility  We’ve adjusted the scaling to better match Special Attack usage at all levels 
Increased the frequency that Black Bolt’s “Provocation” triggers 
• Due to the varying Critical Hit rates across all Champions, in some cases Provocation would trigger rarely or not at all within a fight  We’ve increased the frequency to ensure you’ll see it every match – but especially so against opponents with high Critical Hit rates 
We’ll continue to follow the effect of these new abilities on gameplay  Please keep your feedback coming!
Hey everyone! We have been hard at work on improving the game and have prepared a big update inspired in part by your great community feedback  Please keep letting us know what you think!
• Fixed many Dash, Medium, Heavy and Special Attacks missing or failing to execute 
• Added Alliances and a new Alliance Crystal 
• Rocket Raccoon and Unstoppable Colossus join The Contest 
• Temporary Boosts to Attack, Health, and XP are now available from the Alliance Crystal 
• Rewards for completing and exploring Chapters and Acts  Earn a guaranteed 3-Star hero crystal for each fully explored Act! This is retroactive, just complete any quest to claim them 
• A new Fight Menu combines The Arenas, Story Quests and Event Quest menus 
• Updated Summoner Profiles with new information  Inspect other players’ Profiles and brag about your achievements!
• A list of blocked users has been added to Chat windows  The option to unblock these users is found in this new menu  The power is in your hands now!
• We fixed Dash and Medium Attack issues for many heroes that sometimes missed or did not activate 
• We fixed issues to Drax and Colossus Light and Medium Attacks where they would not connect 
• Fixed an issue where the camera would stop moving after a level 3 special sequence 
• Fixed an issue where the player’s heavy attack would get stuck in charge even after the player has released input 
• Fixed a rare bug where Champions were still able to deal damage after they died, resulting in tied fights 
Form Alliances with your Friends!
What is better than playing? Playing with your friends! Create a new Alliance or join an existing one through the new Alliance Menu 
• Invite other players to your Alliance 
• Search for an Alliance by name or join a Recommended Alliance 
• Receive rewards for entering your first Alliance 
• Alliance News Feed  The news feed celebrates your Alliance member’s achievements 
• Alliance Chat  Chat with other members of your Alliance in a private channel all to yourself



ground_truth: An unsupervised method based on the information bottleneck and contrastive learning.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: How can players skip dialogue on the quest map?

Read the following text and answer briefly.

Context 2: 
• Help Allies  Players can ask for help when out of Energy or Stamina  Alliance members help each other as much as they can to earn Loyalty Points  Loyalty points have a daily limit to how many can be earned 
• Alliance Crystal  Access a new Alliance Crystal while part of any Alliance  Use new Loyalty Points for purchasing Alliance Crystals 
He may start out slow, but watch out for his immense power at high ranks!
• Adjusted the range of many Heavy Attacks, including Hulk and Drax, to ensure they correctly connect with enemies 
• Many Special Attacks, including those for Wolverine, Iron Fist, Winter Soldier, Punisher, Black Panther, and many others have had their range adjusted to ensure they correctly connect with enemies even if activated immediately after a combo that knocked the enemy back 
• Payback and Unbreakable now display their maximum potential damage bonus 
• Added detailed descriptions for Bleed Immunity and Poison Immunity 
• Gamora: We’ve adjusted the scaling of her base Special Attack damage to ensure they scale up more similarly to other heroes  This also makes Gamora more reliant on her high Bleed damage, and improves the chances of opponents able to deal with her high Bleed 
Vital Strike and Jade Assassin damage decreased by 10% 
Godslayer damage increased by 10% 
• Magik: Rewind is a game-changer for Magik that allows her to go up against foes like Gamora and Rewind off big Critical Hits and Bleed damage; however, the frequency of Rewind triggering was too low to be there when she needed it 
Increased the likelihood Rewind triggers by +20% at all levels 
Rewind now heals over one second instead of instantly 
Fixed a bug allowing Magik to break out of an enemy combo using Rewind  It now only removes Status Effects 
• Hulk: Given the riskiness of losing Health in certain game modes, Hulk’s anger-management provided too little help too late in the game  We’ve increased the Attack boost to ensure he’s appropriately scary in all game modes – as long as he’s angry!
Increased Hulk Rage by +20% Attack at all ability levels 
Arc Overload no longer causes Armor Break when it expires 
• Vision: Added Poison Immunity to our robot friend 
Arena tuning is an ongoing process  The team is continually making adjustments to Arenas to improve the experience 
Ultron has infected The Contest!
Many new Champions join the battle against Ultron 
Quest through the new Ultron’s Assault Event 
Wield new power with Summoner Masteries 
Grow your Friend’s List with the new Social Hub 
Team up with your Alliance in new Events, Arenas, and more!
Filter and sort your Stash 
Fights have been optimized for performance improvements on all devices 
Users can now filter through the items in their Stash 
Fixed several issues where Hero Rating would fluctuate 
Fixed a bug with Rhino and Juggernaut having 11-20% more Armor than intended 
Fixed a bug with Rocket Raccoon’s Dash attack being slower than intended 
Added a confirmation popup when spending Units on stamina recharges and unlocking arenas 
Regeneration no longer displays green Health values if you’re at full Health 
Several new improvements to how status effects are displayed 
AI opponents are no longer able to perform one unavoidable attack in response to a Special Attack 3 
A new and improved look for all Health Potions in the Battlerealm 
All Revive Potions now revive your Champions with +10% more Health 
We’re adding so many new Champions, they could form their own Alliance!
Some of your favourite heroes of the Marvel Cinematic Universe join The Contest!
Summoner Mastery is on the horizon!
Masteries provide beneficial effects for your Champions 
Access Masteries through your Summoner Profile 
Earn Mastery Points when you level up 
Choose your Masteries wisely and strategically customize your benefits 
Recover your points to try a new specialization as often as you’d like 
Keep an eye on in-game messaging for more information 
The daily loyalty limit has been set to refresh at 08:00UTC for all players 
A timer has been added to show when the daily loyalty limit resets 
Loyalty balance is now displayed in the Alliance menus 
Ask for Versus help with a single tap on the ‘Help’ icon in Team Select 
New Alliance Events are coming very soon!
Work together with your Alliance to complete objectives and receive rewards!
Muster your might, Alliance Arenas will soon open their gates!
Competing in Alliance Arenas shares your points across your whole Alliance; work together to reach milestones and top ranks!
Work together to amass a huge score, and defeat your competition in classic Arena combat! No slackers here either - if you don’t contribute to win the competition, you’re not eligible for the goods!
All social features (Chat, Mail, and Friends) can now be accessed through the new Social Hub 
Search for and add friends, and send private messages to Summoners on your Friends List 
Redesigned chat and mail screens 
Take on other Summoners’ top Champions for bragging rights and prizes in 1-on-1 Duels!
A new series of special Ultron quests are available, starting with the first Chapter  Fight back against Ultron’s infection alongside the Summoner, and team up with some of Marvel’s finest! New quests unlock each week!
The Spider-Man Champion gate has been removed from Act 1, Chapter 1, Quest 5 
• Fixed an issue where chat snapped to the most recent message 
• Fixed several issues where Hero Rating would fluctuate 
• Various improvements to the Summoner Mastery screens and descriptions 
• Increased the ISO8 awarded by duplicate 2-Star Champions 
Quest through the new single-player campaign, Ant-Man’s Adventure!
In addition to Ant-Man and Yellowjacket feuding throughout the Battlerealm, additional new Champions will be joining The Contest!
Access more Masteries in the new Utility Mastery tree!
Please note, these changes may result in a loss of Hero Rating as incorrect effects are restored back to normal levels 
Improved and polished combat mechanics to reduce the amount of stutters and lost input 
Fixed and optimized rendering related issues with Metal enabled devices 
Team up with Ant-Man, and put a stop to Yellowjacket’s mysterious mission!
All Alliance Quests only last for a specified amount of time, defeat the boss with your Alliance before it expires!
New Prestige System - A dynamic difficulty and score setting that adjusts as you and your Alliance succeed in harder quests  The better you do and the tougher your Alliance is, the higher the prestige  The higher the prestige, the better the rewards!
Choose your teams carefully as Champions within Alliance Quests cannot be used in other Story or Event Quests 
Act 4 has been released! Play Chapter 1 now!
Summoner level maximum has been increased to level 60!
5-Star Champions are coΩming to The Contest! These are the most powerful Champions yet!
Additional improvements have been made to the UI, Versus Arenas, Synergy Bonuses, the Stash & Items Store 
Act 4 - Chapter 1 released!
New challenges - more path variation and features to challenge the strongest Summoners!
Greater challenge means greater rewards! Earn 4 Star Crystals and Mastery Points!
The Summoner Level cap has been increased by ten levels to level 60!
Champion Items will be coming soon! These allow you to apply items and buffs to a specific Champion, keep an eye out for updates on these new Champion Items!
Synergy Bonuses have updated iconography and the calculation has been updated to a distinct, additive bonus - What you see is what you get!
Alliance class distribution is now displayed on team select - Choose the right class!
Your Catalysts now have their own inventory, and will no longer appear in the Upgrade Item inventory 
The Stash is now separated into three tabs: Catalysts, Rewards and ISO, allowing you to sort and view your Stash much faster!
The UI flow for both Quests and Arenas have been greatly improved  You can now skip through fight victory and reward animations!
Here is the rundown of patch 5 1 0, filled with various bug fixes and optimizations  The important ones to note are below 
New Champions, new theme, and a new arena!
To celebrate our one year anniversary AND the holidays, we’ll be running a special event quest! Battle through the history of The Contest, and test your mettle against familiar faces both old and new!
A special reward will be available to those who master every quest!
Our Anniversary Celebration will be happening very soon; stay tuned for more info!
More Act 4 quests are coming very soon!
Opponents in Story Quests now have the ability to use their Special 3 attack! Note that we are not changing previous quest opponents to have this special attack (Act 1-3, Proving Grounds, Realm of Legends will not change); this will be in effect starting with the soon-to-be-released Act 4 content 
As with our previous major build releases (3 0’s Ultron, 4 0’s Ant Man, and 5 0’s Battlerealm), the Contest has been reskinned with a new theme!
The Road to Knowhere map is here! Fight in a new level inspired by Guardians of the Galaxy!
A new button in your Alliance Chat to take you directly to Alliance Quests!
You can now collect Catalyst Fragments in Event Quests, Proving Grounds, and Alliance Quests; these can be pieced together into a Catalyst!
Selling Items is now a thing! Sell any items in your inventory for gold!
Level 3 and Level 4 Health Potions have arrived! These are powerful instruments to help you tackle all the new Act 4 content!
Over 400 bugs were fixed in this patch!
This patch is a fix for the missing Champions during the Special 3 animation on Android devices 
This issue occurred during our upload process to the Google Play Store  This was an odd edge case scenario that we could not have caught during our internal tests, as it began appearing once we uploaded to the Google Play Store  This hotfix will be out by tomorrow, and will put Android at version 6 0 1  As this issue does not occur on iOS devices, iOS will remain at version 6 0 
3:30pm PST: We have started slow-rolling this patch out to Android devices, beginning with about 20% of users  We expect this to be available for 100% of users within the next 24 hours 
We have a few new Champions that you will see within the next couple of months (including one of my personal favorites)!
Over 200 total bugs squashed in this patch!
An artifact left over from the early days of the contest was Black Panther’s ability to gain a Critical Hit Rate boost during Special 3 attacks  As many might know, Critical Hits aren’t possible during a Special 3 anymore, making this effect   unhelpful  We’ve switched it out with a new ability to stack up even more Bleed effects on the opponent based on how many Bleeds are already active 
Example: The opponent has 4 stacks (instances) of Bleed on them when you launch a Special 3  With this new ability, you have a chance to add an additional 0 - 4 more stacks (instances) of Bleed 
Previously, a bug existed that allowed champions with Evade to continue to dodge Black Widow’s attacks, even if her Signature Ability was maxed out  This issue has been fixed 
Captain America WW2 has started to become outpaced by his non-WW2 counterpart and while we want the two to feel different and each have their own specific uses, we also want to ensure they are kept within range of each other in terms of balance  To accomplish this, we’ve given WW2 Cap the ability to Stun with his Special 1 and Special 3 attacks, but kept his Bleed on Special 2 the same, giving him options during combat against non-bleeding champions 
A bug that prevented Daredevil from triggering Armor Breaks from Heavy Attacks has been fixed and is now working as intended 
Against non-bleeding champions: Critical Hits have a chance to Armor Break on Special Attacks 
Increase range of Signature to 25% from 20% 
Many players found Elektra’s signature ability lacked enough opportunities to use it  To remedy this, we’ve increased the range from 20% to 25%  Additionally, to help make Elektra unique from other skill champions, we’ve given her the ability to deal with naturally Bleed Immune champions  Note: This Armor Break only applies to champions naturally immune to bleed, such as Colossus and Ultron, and not to champions granted Bleed Immunity from Local or Link Nodes 
Guillotine’s Bleed effect used to have a chance to activate from any given attack, meaning that it had to be kept quite weak to compensate for the frequency of triggers  We’ve made the switch to have her Bleed behave closer to existing champions, and in doing so have boosted the strength of the Bleed and have allowed it to stack 
Norman Osborn overloads the Arc Reactor in his chest if Health drops below 10%, granting a large burst of power, with (18% - 48% ) Armor, Regeneration, and Power Gain  After that, his suit burns out and cannot trigger Armor Up, Armor Break or Stun and loses all base Armor 
Many players didn’t like Iron Patriot’s old signature ability, feeling that due to the lack of Regeneration, it was considerably weaker than Iron Man’s  While we agreed, we didn’t want to just copy and paste his signature ability, but rather give him his own unique twist on the ability  This “all or nothing version” feels more like Norman Osborn, pushing his suit to the limit to get a larger boost but at the cost of damaging the suit  The addition of Power Gain allows Iron Patriot a large attack before the suit burns out, if timed correctly 
Heavy Attacks: 90% chance to Stagger the enemy for 8 seconds  A Staggered enemy cannot gain their next beneficial effect 
All versions of Juggernaut, even those who haven’t been awakened, now gain the 2 second Unstoppable ability at the start of the fight when they hit Rank 2 
We wanted to add some new functionality to Juggernaut, while also keeping him true to his Mystic class assignment  To accomplish this, we added this “buff smasher” effect which keeps an opponent from gaining their next beneficial effect  Additionally, we wanted to make non-awakened versions of Juggernaut more fun to play, without adding more power to the awakened variations  As a result, we gave all versions of Juggernaut the ability to become Unstoppable at the start of the fight 
While many players liked the new functionality of Star-Lord’s Element Gun effect, they found it to be a little too random, specifically when it would Heal Block a champion incapable of Healing  We’ve now added in some contingencies that will make Heal Block appear less unless the opposing champion shows that he / she can Heal during the fight  This includes both activated healing effects, such as Wolverine or Ultron’s Heal, or passive healing effects gained from Masteries, such as Salve or Willpower 
It’s been a bit weird that Bucky wasn’t friends with his most famous friend  Well, he is now  This affects 3 Star and above versions 
We’ve increased the overall speed of this attack, allowing quick players to use this ability after a four or five hit combo 
It seems the Marvel’s have gotten tired of their beams being dodged so easily and have decided to angle it a little better, increasing the overall range of the attack and making it harder to dodge away from  We’ve also increased the speed of both special attacks to allow them to better flow into combat 
In order to allow this attack to better flow in combat, we’ve shaved off a few frames from the beginning, allowing players to chain this attack into 4 and 5 hit combos 
Alliance Wars have arrived! It’s Alliance versus Alliance in a war for Battlerealm supremacy!
Enter the NEW Loyalty Store to buy Alliance Potions, Mastery items, or other EXCLUSIVE items 
Gain Power back from Special Attacks, enhance or defend against Special Attacks, OR gain a temporary Arena Point Boosts with hoards of new Summoner Boost items!
Additional changes and improvements are listed below 
This patch will be released February 24th 
A new area of the Battlerealm has been opened! Compete with your Alliance-mates for pride, glory, and PRIZES!
Matchmake to find a rival Alliance, then combine strategy and teamwork to dominate them 
Setup the ultimate defensive team to fortify your Battlerealm, then take your offensive team on the assault!
Watch your War Rating skyrocket as your Alliance works together to defeat rivals!
Load up on Crystal Shards, Loyalty, and brand new exclusive rewards!
Note that this will be slow-rolled to Alliances in phases, similar to the introduction of Alliance Quests (to ensure server stability and gather your feedback on the new mode)  Expect tuning changes throughout these phases, as well as into Season 1 
Use Loyalty instead of Units to obtain items for Alliance Quests & Wars!
Items will rotate daily, similar to how the Mastery cores in the current Store change 
Store contents will be randomly chosen from a pool of categories/items; a select few items will be persistent and always be available for purchase 
A 5-Star version of Unstoppable Colossus will be available in the Loyalty Store (keep in mind, this is an expensive Champion due to his exclusivity; this will require winning quite a few Alliance Wars and saving up!) 
This is accessible from the “Store” section of the pop-down menu, and will be available at a later date after the initial 7 0 launch; there will be advance notice through forums and in-game before we release the Loyalty Store 
New Summoner Boosts have arrived in the Loyalty Store; NEW Boost types, purchasable with Loyalty Points 
Class specific Boosts, such as Mystic Champions restoring power after using Special Attacks 2 and 3, or Skill Champions boosting their Special Attack Damage 
Defensive Boosts, where your Champions take reduced incoming Special 3 Attack Damage 
Gain a temporary Arena Point boost with new Arena Boost items!
Fixed an issue where, after Parrying certain Champion’s Special Attacks, your Champion would be stuck in a blocking state until the Special Attack finished 
Fixed an issue where 90s Cyclops’ Armor Breaks would not remove Armor Ups 
Fixed an issue with Scarlet Witch’s Signature Ability proc rate (previously, the % chance displayed did not match in-game functionality; this is now fixed) 
(Netflix) Daredevil’s Heavy Attack now has a chance to apply 2 stacks of Armor Break, instead of the previous 1 stack 
When spending Battlechips to enter an Arena (such as the Tier 4 Basic or Alpha Catalyst Arena), there is now a confirmation popup 
The Alliance Crystal now has a purchase limit that resets daily 
Permanently increased the Alliance Crystal’s points in Summoner Advancement (from 30 to 300) 
Updates to Champion Special Attack animations, flow, and timing 
7 0 1 will be released within the next few days 
A celebration message is sent to the War Room when an Alliance War battlegroup is cleared 
Players can now tap directly on another node icon while the tile info popup is open (previously, the popup had to be closed before selecting another node) 
Alliance’s reward tier position is now highlighted in the Alliance War tier breakdown 
In Attack Phase, players can view the score breakdown for both the battlegroup and overall 
The “Place Your Defenders” text now disappears much faster after tapping on the screen 
Mail messages now display the date they were sent 
It should be much harder to accidentally tap the Units Store when closing a screen 
Players can tap to skip the point animation in Versus mode again 
Resolved an issue with Class Masteries (specifically Mystic Dispersion) not functioning 
The Juggernaut issue with his linked nodes not appearing in Act 4, Chapter 3, Quest 3 (4 3 3) has been fixed 
Fixed a crash that occurs when a player who is not in an Alliance enters Alliance Wars through an outside link 
Fixed a text issue where Alliance War specific descriptions would appear on the Alliance Quest “Select a Battlegroup” screen 
Resolved ~20 various rare crashes and additional minor issues in different game modes 
Fixed and optimized performance on the new Samsung S7 
Fixed an Unknown Error that occurred rarely after a device was woken after going to sleep 
Improved Performance(Frames Per Second) tracking per fight to help diagnose hitches/pauses/lag spikes during gameplay 
Improved gesture tracking(Swipe, Tap, Hold) during low performance moments in combat 
Fixed a rare crash that would sometimes occur when receiving a phone call while in combat 
Tuned and updated many Champion Special Attack animations to improve timing and combat flow  Please see the expanded forum post HERE for a full list 
Fixed She-Hulk’s Special Attacks being marked as a projectile (allowing Daredevil to evade them) 
Fixed an issue where the player would be stuck in place after parrying Captain America’s Special 1 
Fixed an issue where chaining 2 medium attacks into Old Man Logan’s Special 2 would cause the first 2 strikes to miss opponents 
Fixed an issue with Daredevil or Spider-man missing with a dash attack if Vision charges a heavy attack during the dash 
Fixed an issue where some hidden information in Alliance Wars was visible 
Fixed a display issue where Defender Placement percentage was not displaying all placed Alliance members 
Resolved minor issue with the total Alliance’s score being displayed on the War Progress widget (now only displays the score of the battlegroup being viewed) 
Multiple minor Alliance War issues have also been fixed in this patch 
Fixed a display issue where Shard amounts provided by defeating a boss displayed as double 
Fixed a display issue where opponent PI values would display differently between the map, prefight screen, and in combat 
Boss power is now correctly displayed after removing Global and Linked boosts 
Fixed an issue where a player in Alliance Quests would lose input ability on the quest board after sleeping the device 
Fixed an issue where a player enters Alliance Quests and gets stuck after viewing the linked node or buff node tutorial 
Fixed an issue where sending an Alliance invite to a player would cause the “Add Friend” button to become greyed out 
Fixed a text issue that appears when viewing Featured Hero information from the Home Screen 
Join The Iron or fight for The Blue with new events, quests, Champions, and special Shards; inspired by Marvel’s Captain America: Civil War!
Solo Events: constantly-evolving events that vary in length, requirements, and prizes!
Compare statistics against other players and Alliances with the new Leaderboards!

Context 3: Paper Info

Title: CONTOUR COMPLETION USING DEEP STRUCTURAL PRIORS
Publish Date: 9 Feb 2023
Author List: Ali Shiraee, Morteza Rezanejad, Mohammad Khodadad, Dirk Walther, Hamidreza Mahyar

Figure

Figure 1: Just by looking at subfigure (a), we, as humans, can easily perceive a shape like the one in subfigure (b) This is an extraordinary capability of our human brain and in this paper, we tried to see whether convolutional neural networks can show such capabilities 
Figure 2: The trajectory from random noise X N to the incomplete image X I in image space The network will pass a completed version of the image, X C , throughout this trajectory 
Figure 4: This figure shows our iterative process to complete the fragmented contours of an image given as input to our pipeline 
Figure 5: This example shows how different scores change throughout a single run All three scores change in the range of [0, 100] Our goal is to maximize reconstruction_score and minimize the overfit_score, but we should consider that the minimization lower bound is data dependent and is not zero 
Figure 6: Evolutionary process of the deep structure prior The right column shows the incomplete shapes given to the model and the rest of the columns show how the model is overfitting gradually to produce the incomplete shapes In each column, we are showing an intermediate iteration of this process The loss-term setup enables our pipeline to let the completed image appears during this iterative process 
Average MSE and IoU values between the incomplete (Raw) images, the output of DIP and DSP methods, and ground truth for each image are provided in this table 
For this experiment, we ran the model over a subset of complex dataset with 500 incomplete images at various levels of alpha for 250 iterations After the image completion is done, we compared the evaluation metrics between the completed image and the ground truth to examine the performance of the model for different values of alpha 
In this table, we show the effect of the receptive filter size on our algorithm's capability to fill in bigger gap sizes The numbers in this table are showing the percentage of the time that DIP was successful to complete shapes with each gap size and corresponding receptive field size As predicted, the bigger the filter size, the more successful the algorithm is in filling in the gaps 

abstract

Humans can easily perceive illusory contours and complete missing forms in fragmented shapes  This work investigates whether such capability can arise in convolutional neural networks (CNNs) using deep structural priors computed directly from images  In this work, we present a framework that completes disconnected contours and connects fragmented lines and curves 
In our framework, we propose a model that does not even need to know which regions of the contour are eliminated  We introduce an iterative process that completes an incomplete image and we propose novel measures that guide this to find regions it needs to complete  Our model trains on a single image and fills in the contours with no additional training data 
Our work builds a robust framework to achieve contour completion using deep structural priors and extensively investigate how such a model could be implemented 

Introduction

The human visual system is used to seeing incomplete outlines  Our brains can effortlessly group visual elements and fragmented contours that seem to be connected to each other  This power enables us to make shapes, organize disconnected visual features, and even properties of 3D surfaces when projected on 2D planes 
demonstrated how early vision may quickly complete partially-occluded objects using monocular signals  This capability of perceptual grouping has been studied in vision science for decades   Although there has been some work on perceptual grouping in the past couple of years, it has been less studied in the past decade due to the enormous progress of deep neural networks and their success in dealing with the pixel-by-pixel inference of images 
Different types of lines and curves have been studied to maximize the connectivity of two broken ends in the planer contour completion problem   Different types of lines and curves have been studied to maximize the connectivity of two broken ends in the planer contour completion problem  Geometry-based constraints can be utilized to address some challenges of contour completion problems, such as smoothness and curvature consistency  
However, such approaches only work for simple, smooth contours and usually fail in more complex settings  On the other hand, we currently have deep models that could easily take an incomplete image and complete the missing regions using enough training data   The amazing capability of such models especially those that are trained on different modalities with millions or billions of training data raises the question of whether we need such a large amount of training to perceive all the visual cues that are present in an image, which underlies visual perception by humans 
In human vision, Gestalt psychology suggests that our brain is designed to perceive structures and patterns that are grouped by some known rules  In this work, we show that some perceptual structures can also be learned from the image itself directly using architectures that enable such learning  Earlier work has shown  This is an extraordinary capability of our human brain and in this paper, we tried to see whether convolutional neural networks can show such capabilities 
that some forms of perceptual grouping can be achieved using computational models, such as stochastic completion fields   This type of learning resonates with some of the Gestalt perceptual grouping principles including "proximity", "good continuation" and "similarity"  In scenarios where color and/or texture are present, the cue of "similarity" helps us group regions with consistent patterns  
When color and texture are present, they provide a collection of rich information for such cues  In the present article, we probe convolutional neural networks in a scenario where both are absent, and the neural network is dealing with just forms and shapes  Specifically, we explore whether the convolutional neural network architecture itself can give rise to some of these grouping cues when they are fed just contours and shapes alone 
For years, neural networks have been treated as black boxes that can generalize images very well to multiple classes when there are enough training exemplars  One of the reasons that neural networks are trained on many exemplars is to avoid the problem of overfitting  On the other hand, we know that CNNs that generalize well to large classes of exemplars can easily overfit when those class labels are randomly permuted  
Inspired by this observation, suggest that image priors can be learned to a large extent through a generator network architecture that is solely trained on a single image  This encouraged us to take a deeper look at what structural information can be learned from a single-shape image and whether we can reconstruct some of those perceptual grouping capabilities using a generator network 
Inspired by , in this work, we adopt a novel training regime to complete shapes and contours where we use a UNet architecture with random initial weights and try to complete the contours within a single image without any training data  In our model, we imagine that the input image (i e , the only image used to update the model's weights) is an image of fragmented contours 
In this work, instead of training the model on multiple images fetched from a big image dataset, we imagine a random fixed tensor noise image as input to this model  At each iteration, the random noise tensor is inferred through our generative network and the network produces an outcome image  We introduce a novel loss function that enables this network to complete contours 
This process repeats, and the weights of our network are updated gradually based on this loss function, which is an energy term defined based on the input image and the output of the network  The model will reconstruct the missing structures i e , group fragmented contours that perceptually seem to be connected, before it fully overfits to the incomplete input image 
Contributions of our work are summarized as follows: 1  In our pipeline, we propose a novel algorithm that enables us to complete contours that appear to be connected to each other in an illusory form  2  Our model is trained on just one single query image and does not need any training data  3  Our model does not need to know which regions of the image are masked or occluded, i e , we remove the dependency of the algorithm on the guiding mask (a guiding mask is a mask that informs the model on where the missing regions are located at) 
We also introduce two metrics to produce a stopping criterion to know when to stop training before the model fully overfits to the incomplete image, i e , we guide the model to stop when the completed image is produced 

Methods

Our eyes are trained to predict a missing region of an occluded object within a scene  We can easily perceive or make guesses about parts of objects or shapes that we do not necessarily see  Even when we are looking at an image, we might guess about the shape, property, or other attributes of an unknown object within a scene 
Such capability extends beyond just known objects or shapes  We can look at a disconnected set of contours and guess what the connected form may look like  This capability is rooted in our prior knowledge about the world  (see Figure )  In this work, we aim to achieve a similar capability using deep generative networks 
Most neural networks that we work with these days are trained with a massive amount of data and one might think that this is the only way that a neural network can obtain prior information  Authors of Deep Image Prior (DIP) suggest that the convolutional architecture can capture a fair amount of information about image distribution 
They show that the hourglass architectures like UNet can show some good performances in some inverse problems such as image denoising, super-resolution, and inpainting  In this work, we focus on completing fragmented contours end-to-end just by using a single image  To be able to address this problem, we first look at a similar problem in image editing, known as image inpainting 
Image inpainting is the task of completing an image where some regions of that image are covered or filtered by a mask  In image inpainting, the generative model receives a masked image with the mask that guides the algorithm to fill in those missing regions  Although in the problem of contour completion, we have a very similar goal, the additional challenge that we suffer from is that we do not necessarily have a mask that covers the regions of interest for us 
For example, when we look at Figure (left), we are not provided that which regions of the image are incomplete by a guiding mask  Our brain figures this out just by looking at the form and predicting those missing regions  Inspired by the image inpainting work of DIP , we propose a novel algorithm for the contour completion problem (see Figure ), where, unlike DIP, we do not have a guiding mask to know where to fill in the missing regions of our disconnected contours 
Let us assume that we are given a degraded image x I containing a fragmented contour  We propose an iterative process (see Figure ) that can connect those discontinuities and glue those fragmented pieces together as follows  We propose an hour-glass model structure (f ) that is initially set up with completely random parameters (θ 0 ) at first 
Through an iterative process, we start feeding our network with a fixed random tensor noise z signal and obtain the inferred output (f (z)) from that network  We then back-propagate the difference between the inferred output and the incomplete image to the network  We then repeat this process until the difference between the generated outcome of the network (f θ (z)) and the incomplete image (x I ) gets smaller and smaller and finally overfits the incomplete image (x I ) 
In this work, we propose a novel error metric to backpropagate in the model and update its weights  we set the metric in a way that enables us to complete the incomplete image before it overfits the incomplete image  This is where the magic of our algorithm happens  We also propose a stopping criterion, so that when the image is complete, we no longer overfit the outcome of the model and instead produce a plausible connected set of fragmented contour pieces 
As illustrated in Figure , this trajectory will pass through a complete version of the image in image space, which is close to the actual connected ground truth x gt , which we do not have access to directly 

Energy Function

We can model this iterative process mathematically by maximizing a posterior distribution  Let us assume that the optimal image x * that we want to achieve is on a path that connects a random tensor noise z to the incomplete image x I   With this assumption, we can eventually overfit any random tensor noise to the incomplete image x I , and we can formulate the posterior distribution of our wanted optimal image x * as follows:
No Prior To better recapitulate what we want to achieve using our generative model, we solve an energy minimization problem on the parameter space of the model, rather than explicitly working with probability distributions and optimizing on x (image space)  Thus, we solve an energy minimization problem that incorporates the incomplete image (x I ) and model parameters (f θ (z)):
As shown in Figure , the pipeline starts from a random initialized set of parameters θ and updates those weights until it reaches a local minimum θ *   The only information provided for the network is the incomplete image x I   When we reach the optimal θ * , the completed image is obtained as x * = f θ * (z) where z is random tensor noise 
In this work, we use a U-Net architecture with skip connections as the generator model  " As we mentioned previously, in this work we were inspired by an earlier work known as Deep Image Prior (DIP)   In this work, the authors suggested a mean-squared-error loss term that enables the network to compare the output of the generator to the incomplete input image:
where x I is the incomplete image with missing pixels in correspondence of a binary mask m ∈ {0, 1} H×W and operator is for point-wise multiplication of two image matrices  In the inpainting tasks, the existence of a mask is essential as the algorithm needs to know where to fill in the missing area, whereas, in our work, we wanted to know whether the network can perform completion on its own without the need for the mask 
In other words, is it possible for the network to predict where to fill in at the same time that it is trying to reconstruct the incomplete image through the iterative process? To answer this question, we tried to solve a much harder problem in which the mask is not provided to the model and the model is agnostic to it 
To better understand how a solution could be hypothesized for this problem, we first imagine that we want to consider all the available regions in our image that could be potential places to fill in, i e , we set the mask in the previous formula 1 to be equal to the incomplete image x I   This is problematic as the model quickly tries to fill in all white space and quickly reconstructs the incomplete image by doing so 
On the other hand, we can take the inverse problem of the current problem, where the model tries to just fill in the regions that fragmented contour lives in  Taking these two at the same time, we came up with a novel loss term for energy minimization term that helps us remove the need for the mask in the case of the contour completion problem:
In this term, we introduce a linear combination of the two loss terms, where one focuses on reconstructing the missing regions in the foreground, and one focuses on avoiding inpainting regions in the background  The logic behind this is that, if we assume the original image to be representative of the mask, then the model tries to reconstruct in all white regions (the foreground), and in the inverse problem we just want to reconstruct the regions that are already part of the ground truth 

Stopping Criteria

As shown in Figure , knowing when to stop iterating to the over-fitted model is a key to obtaining a completed shape  Therefore, we equipped our model with a criterion that uses two individual novel terms to know when to stop and output the result of the network  These two metrics expand the capability of the generator network beyond what it does currently and achieve a full end-to-end contour completion model that trains and infers on a single image of divided contour fragments 
These new terms are: reconstruction_score (ρ) and overfit_score (ω) 

Reconstruction Score

The first score that this paper suggests is the reconstruction score, i  e , we have to make sure that the model is trained enough that it can reconstruct at least the entire set of fragmented contours within the image  This is a trivial score and to compute the reconstruction_score (ρ), we apply a k-dimensional tree (KDTree) nearest-neighbor lookup to find the ratio of points in the original incomplete image (x 0 ) 
This score ranges from [0 − 100] 

Overfit Score

It is evident that the model overfits the fragmented contours  This is due to the fact that the error in our loss term is minimized as the x overfits to x I , i  e , replacing x with x I in the loss term would give us zero  As we hypothesize iterative process also produces the complete image before it overfits to the incomplete image, we can imagine that at some point the image is complete (x C ) and does not need to be fine-tuned any more to overfit to x I  
We suggest a new score called overfit_score  overfit_score determines how much of the reconstructed outcome is over the number of pixels that are already in the incomplete image (x I )  To compute the overfit_score (ω), we apply a k-dimensional tree (KDTree) nearest-neighbor lookup of points in the outcome of the input image and see what portions of those points are novel and not already in the incomplete image (x I ) 
Similar to reconstruction_score, the overfit_score also ranges from [0 − 100]  Our goal is to maximize reconstruction_score and minimize the overfit_score, but we should consider that the minimization lower bound is data dependent and is not zero 

Combined Score

To be able to find the best possible set of completed contours, we combine the two and have a loop that tries to achieve close to full reconstruction and avoids over-fitting at the same time  This is what we call an "ideal" stopping point in the contour completion problem  In each run throughout all iterations, we pick an output of the network that minimizes a dissimilarity term:
where δ represents our dissimilarity score  The reconstruction_score and overfit_score are obtainable given network output and the incomplete image  Ideally, we want to achieve an output image that has a reconstruction_score equal to 100 and an overfit_score of γ which is a hyperparameter that is dependent on the image and complexity of the shape 
Empirically, we observed that this value is highly correlated with the distance between gaps that are observed in fragmented contours, i  e , the larger the gap results in a larger γ value  We will discuss this in more detail in the next section (see Section 3)  For one sample image, we computed the two metrics reconstruction_score and overfit_score and the combined value of dissimilarity (δ) and showed how these values change (see Figure ) 
Our initial observations show that the reconstruction_score will increase to 100 quickly for the incomplete image indicating that the already existing fragments of the contours have been reconstructed in the output  However, as mentioned previously, we cannot solely rely on this score since we also want to minimize the overfitting 
Remember that our goal is to produce an output that: a) preserves the original contours in the incomplete image and b) fills in the gaps between the fragmented contours  It is evident that overfit_score decreases throughout an iterative run of our process until it reaches zero  The dissimilarity will also decrease along with the overfit to a point, then it will increase, as the model tries to reproduce the incomplete image 
This is where an ideal γ value can be picked, i e , where to stop when the reconstruction is good but we have not done a full overfit to the incomplete image  Thus, one should pick the value of γ empirically in the scenario that the ground truth is not available, whereas, assuming that the ground truth is available, we can easily compute the best γ value 
In our experiments, we tried two datasets of images with different gap sizes  We observed that the best the γ for one set of samples is ∼ 5 (the set with shorter gaps) while it is ∼ 23 for samples from the other set, i  e, the set with longer gaps (see Figure for some completed examples) 

Experiments and Results

Performing unsupervised contour completion is a difficult task to benchmark as one can never know what fragments exactly are connected to each other in a real-world scenario  This makes the problem of contour completion a hard problem to solve  In this paper, we tried to create artificial shapes that are occluded by some masks and then tried to see if our model can regenerate the missing pieces and glue those divided contours together 
To demonstrate our model's behavior, we will conduct experiments on datasets created for this task and will report on them in this section  To compare network results in different settings, we will use pixel-wise Mean Squared Error (MSE) and Intersection over Union (IoU) between the produced result of the network and unmasked ground truth data and the reconstructed image on black pixels (where contours live) 

Data

We prepared two datasets, one labeled "Simple" and one "Complex", in accordance with the number of gaps in each shape  Both datasets contain nine different categories of shapes  In order to generate the Complex dataset, we used FlatShapeNet which is a dataset for the educational game Ariga  The dataset includes the following categories: Circle, Kite, Parallelogram, Rectangle, Rhombus, Square, Trapezoid, Triangle and Overlap 
The "overlap" category contains images that are made as a mixture of two shapes that are overlapping from the previous categories  These are some standard shapes with a few gaps in simple dataset, while the complex dataset has some hand-drawn shapes with fragmented lines and more gaps that produce more variety in general 
For each instance, a ground truth image is available for comparison  Most of our experiments have been conducted using the complex dataset in order to evaluate the generalization of our approach  For the analysis of how γ values should be set for each shape, we used the simple dataset as a reference 

Evaluation

In this section, we compare our model to the original Deep Image Prior (DIP) inpainting model  DIP's inpainting module accepts a degraded image and a binary mask corresponding to that image  In order to make a fair comparison, instead of providing a binary mask, we used the incomplete images both as input and as a mask in order to see whether it can produce a result similar to ours 
For DIP, we run the iterative process for a maximum number of 2500 iterations with the U-net backbone  We used the exact same architecture and setting in our model for a fair comparison  Using our ground truth dataset images, we calculate the MSE loss between the network output and ground truth during each iteration instead of relying on our stopping mechanism described in the previous section 
We then store the output with minimal loss throughout all the iterations  Finally, we select the best output among all iterations, report the MSE and IoU with the ground truth, and save the iteration number which resulted in the lowest MSE  Table compares the results that are obtained using the DIP method, the DSP method (ours), and the difference between raw images and the ground truth 
We have presented the average MSE-loss, average IoU, and the average number of iterations for the best output for different methods  As can be seen from the table, our model improves both MSE and IoU between the incomplete image and ground truth in fewer iterations  The DIP method can neither generate a better result than the raw image nor provide stopping criteria to prevent overfitting 
We provide a more detailed analysis of this result in Figure   As results show, our algorithm not only provides a much faster convergence but also consistently provides a better-completed image (consistently less MSE loss and better IoU), whereas it is challenging for the DIP method to accomplish better results without a guiding mask 
We compare MSE loss between the degraded raw images that these algorithms started with (shown in blue) (a) Mean Squared Error Loss: we clearly see that for almost all images, DSP (green) achieves a lower MSE than the incomplete images (blue) whereas, the DIP completed images either do not improve the MSE or even worsen that for the incomplete images 
Note that, the MSE is computed to an available ground truth image hidden from our methods (the lower is better)  (b) Intersection Over Union: here, we are looking at the IoU metric that specifies the amount of intersection over the union between the obtained images and the ground truth data  Again, we see that DSP produces images that are much closer to the ground truth (in most cases) whereas the DIP can not achieve a similar result 
While we see few DIP completed images produce a better IoU than the degraded images (in terms of IoU), most of them are worse than the starting image (the higher is better)  (c) The number of iterations that are needed for each algorithm to obtain its best results  Here, we see that DSP can quickly produce the best outcome with the least MSE loss whereas the DIP algorithm's best results are when we run the iterative process for more iterations (the lower is better) 

Correlation of γ with the Gap Size

To better understand the γ parameter of our combined score, we conducted the following experiment  A total of 12000 samples were obtained by merging all of our images from the two datasets, simple and complex  As we have access to the ground truth for each degraded image in the combined dataset, we can easily calculate reconstruction_score and overfit_score for each degraded-ground truth pair 
As expected, we obtain a reconstruction_score of 100 for all samples, but the overfit_score varies among them  Intuitively, we hypothesized that an optimal value of overfit score should be intertwined with the total area of gaps  To test this hypothesis, we did the following experiment  We first define a function φ(x) which takes a binary, black and white image x and returns the number of black pixels in it 
Then we define a gap term as follows: where x I is the incomplete image and x gt is the ground truth  In this case, gap indicates the total area of the gap with respect to the whole shape  We found out that this term and the best result have a correlation of 97 43%  This indicates that the value of γ is highly correlated with the gap size, that is something expected in a way 

Effect of α

We conducted additional experiments concerning how α affects the quality of reconstruction  In the previous section, we defined Equation 2 as the loss term that guides our iterative process  The term α specifies the amount of emphasis the model should place on reconstructing missing regions, rather than filling in fragmented contours 
A lower α indicates a better grouping quality, as shown in Equation   However, we will not achieve completion if we remove the first term completely from the loss by setting α = 0  Therefore, the first term should be kept, but its weight should be very low in order to achieve a good completion  On the other hand, if we set α = 1 and omit the second term, we lose the contour completion regularization term and obtain the same output as a vanilla deep image prior, which does not complete shapes 

Effect of Receptive Field Size

To better understand the effect of receptive field size on our algorithm, we test the following hypothesis: can models with bigger receptive field size complete shapes with bigger gaps? In Table , we report showing the results of this experiment  As we can see, the bigger the receptive field size, the more complete shapes we can reconstruct using DSP 

Implementation Details

As shown in Figure , we use a model with 5 layers and 128 channels for downsampling and upsampling convolutions, and 64 channels for skip convolutions  The upsampling and downsampling modules use 3 × 3 filters, while the skip module uses 1 × 1 filters  In the upsample part of the network, the nearest neighbor algorithm is used 
We used 256 × 256 images with three channels in all of our experiments  In training, we use the MSE loss between the degraded image and the output of the network, and we optimize the loss using the ADAM optimizer and a learning rate equal to 0 01   In our experiments, we also used α = 0 15 as an optimal proportion coefficient for reconstruction loss 

Conclusion

In this work, we introduced a novel framework for contour completion using deep structure priors (DSP)  This work offers a novel notion of a maskless grouping of fragmented contours  In our proposed framework, we introduced a novel loss metric that does not require a strict definition of the mask  Instead, it lets the model learn the perceivable illusory contours and connects those fragmented pieces using a generator network that is solely trained on just the single incomplete input image 
Our model does not require any pre-training which demonstrates that the convolutional architecture of the hour-glass model is able to connect disconnected contours  We present an extended set of experiments that show the capability of our algorithm  We investigate the effect of each parameter introduced in our algorithm separately and show how one could possibly achieve the best result for their problem using this model 
In future work, we plan to extend this model and try to see how it performs with real images  In particular, we want to determine whether we can inpaint real-world photographs while retaining perceptually aware scene structures  The importance of shape in perception by deep neural networks has been highlighted in many adversarial examples to appearance-based networks  
The outcome of this work has strong potential to impact the designing and implementation of models that are robust to such perturbations 



ground_truth: Players can skip dialogue on the quest map by pressing the 'SKIP' button.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: How does the receptive field size affect the completion of shapes?

Read the following text and answer briefly.

Context 4:  The process is repeated for all the couplings obtaining then a fully connected adjacency matrix  The results here presented are obtained with $\lambda = 0 01$ 
   For the minimization we have used the MATLAB routine \emph{minFunc\_2012}\cite{min_func}  
      
      \begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{Jor11_2D_l2_JR_soJR_TPJR}
    	\caption{Top panels: instances of single site coupling reconstruction for the case of $N=64$ XY spins on a 2D lattice with ordered $J$ (left column) and bimodal distributed $J$ (right column)  
    	
    	Bottom panels: sorted couplings }
    	\label{PL-Jor1}
    \end{figure}

   
To produce the data by means of numerical Monte Carlo simulations a system with $N=64$  spin variables  is considered on a deterministic 2D lattice with periodic boundary conditions  
Each spin has then connectivity $4$, i e , we expect to infer an adjacency matrix with $N c = 256$ couplings different from zero  
The dynamics of the simulated model is based on the Metropolis algorithm and parallel tempering\cite{earl05} is used to speed up the thermalization of the system 
The thermalization is tested looking at the average energy over logarithmic time windows and
the acquisition of independent configurations
starts only after the system is well thermalized 

   For the values of the couplings we considered two cases: an ordered case, indicated in the figure as  $J$ ordered (e g , left column of Fig  \ref{PL-Jor1}) where the couplings can take values $J_{ij}=0,J$, with $J=1$, 
   and a quenched disordered case, indicated in the figures as  $J$ disordered (e g , right column of Fig  \ref{PL-Jor1})
   where the couplings can take also  negative values, i e , 
    $J_{ij}=0,J,-J$, with a certain probability   The results here presented were obtained with bimodal distributed $J$s: 
   
    $P(J_{ij}=J)=P(J_{ij}=-J)=1/2$   The performances of the PLM have shown not to depend on $P(J)$  
   
    We recall that in Sec  \ref{sec:plm} we used the temperature-rescaled notation, i e , $J_{ij}$ stands for $J_{ij}/T$  
   
    To analyze the performances of the PLM, in Fig  \ref{PL-Jor1} the inferred couplings, $\mathbb{J}^R_{\rm inf}$, are shown on top of the original couplings,  $\mathbb{J}^R_{\rm true}$ 
     The first figure (from top) in the left column shows  the $\mathbb{J}^R_{\rm inf}$ (black) and the $\mathbb{J}^R_{\rm tru}$ (green) for a given spin
     at temperature $T/J=0 7$ and number of samples  $M=1024$  PLM appears to reconstruct the correct couplings, though zero couplings are always given a small inferred non-zero value  
     In the left column of Fig   \ref{PL-Jor1},  both the $\mathbb{J}^R_{\rm{inf}}$ and the $\mathbb{J}^R_{\rm{tru}}$ are sorted in decreasing order and plotted on top of each other  
     We can clearly see that $\mathbb{J}^R_{\rm inf}$ reproduces the expected step function  Even though the jump is smeared, the difference between inferred couplings corresponding to the set of non-zero couplings 
     and to the set of zero couplings can be clearly appreciated 
     Similarly, the plots in the right column of Fig  \ref{PL-Jor1} show the results obtained for the case with  bimodal disordered couplings, for the same working temperature and number of samples  
     In particular, note that the algorithm infers half positive and half negative couplings, as expected 
     
     
\begin{figure}
\centering
\includegraphics[width=1\linewidth]{Jor11_2D_l2_errJ_varT_varM}
\caption{Reconstruction error $\mbox{err}_J$, cf  Eq  (\ref{eq:errj}), plotted as a function of temperature (left) for three values of the number of samples $M$ and  as a function $M$ (right) for three values of temperature in the ordered system, i e , $J_{ij}=0,1$  
The system size is $N=64$ }
\label{PL-err-Jor1}
\end{figure}

In order to analyze the effects of the number of samples and of the temperature regimes, we plot in Fig  \ref{PL-err-Jor1} the reconstruction error, Eq  (\ref{err}), as a function of temperature for three different sample sizes $M=64,128$ and $512$  
The error is seen to sharply rise al low temperature, incidentally, in the ordered case, for  $T<T_c \sim 0 893$, which is the Kosterlitz-Thouless transition temperature of the 2XY model\cite{Olsson92}  
 However, we can see that if only $M=64$ samples are considered, $\mbox{err}_J$ remains high independently on the working temperature  
 In the right plot of Fig  \ref{PL-err-Jor1},  $\mbox{err}_J$ is plotted as a function of $M$ for three different working temperatures $T/J=0 4,0 7$ and $1 3$  As we expect, 
 $\mbox{err}_J$  decreases as $M$ increases  This effect was observed also with mean-field inference techniques on the same model\cite{Tyagi15} 

To better understand the performances of the algorithms, in Fig  \ref{PL-varTP-Jor1} we show several True Positive (TP) curves obtained for various values of $M$ at three different temperatures $T$  As $M$ is large and/or temperature is not too small,  we are able to reconstruct correctly all the couplings present in the system (see bottom plots) 
The True Positive curve displays how many times the inference method finds a true link of the original network  as a function of the index of the vector of sorted absolute value of reconstructed couplings $J_{ij}^{\rm inf}$  
The index $n_{(ij)}$ represents the related spin couples $(ij)$  The TP curve is obtained as follows: 
first  the values $|J^{\rm inf}_{ij}|$ are sorted in descending order and  the spin pairs $(ij)$ are ordered according to the sorting position of $|J^{\rm inf}_{ij}|$  Then,
     	a cycle over the ordered set of pairs $(ij)$, indexed by $n_{(ij)}$, is performed, comparing with the original network coupling $J^{\rm true}_{ij}$ and verifying whether it is zero or not  The true positive curve is computed as
\begin{equation}
\mbox{TP}[n_{(ij)}]= \frac{\mbox{TP}\left[n_{(ij)}-1\right] (n_{ij}-1)+ 1 -\delta_{J^{\rm true}_{ij},0}}{n_{(ij)}}
\end{equation}
As far as $J^{\rm true}_{ij} \neq 0$, TP$=1$  As soon as the true coupling of a given $(ij)$ couple in the sorted list is zero, the TP curve departs from one  
In our case, where the connectivity per spin of the original system is  $c=4$ and there are $N=64$ spins, we know that we will have  $256$ non-zero couplings   
     	If the inverse problem is successful, hence, we expect a steep decrease of the TP curve when  $n_{ij}=256$ is overcome 

In Fig  \ref{PL-varTP-Jor1}
it is  shown that,  almost independently of $T/J$, the TP score improves as $M$ increases  Results are plotted for three different temperatures, $T=0 4,1$ and $2 2$, with increasing number of samples $M = 64, 128,512$ and $1024$ (clockwise)  
We can clearly appreciate the improvement in temperature if the size of the data-set is not very large: for small $M$, $T=0 4$ performs better  
When $M$ is high enough (e g , $M=1024$), instead, the TP curves do not appear to be strongly influenced by the temperature 

\begin{figure}[t!]
	\centering
	\includegraphics[width=1\linewidth]{Jor11_2D_l2_TPJR_varT_varM}
	\caption{TP  curves  for 2D short-range ordered $XY$ model with $N=64$ spins at three different values of $T/J$ with increasing - clockwise from top - $M$ }
	\label{PL-varTP-Jor1} 
\end{figure} 

\subsection{$XY$ model with complex-valued couplings}
For the complex $XY$ we have to contemporary  infer $2$ apart coupling matrices,  $J^R_{i j}$ and $J^I_{i j}$   As before, a system of $N=64$ spins is considered on a 2D lattice 
For the couplings we have considered both ordered and bimodal disordered cases 
In Fig  \ref{PL-Jor3}, a single row of the matrix $J$ (top) and the whole sorted couplings (bottom) are displayed for the ordered model (same legend as in Fig  \ref{PL-Jor1}) for the real, $J^R$ (left column), and the imaginary part, $J^I$   

\begin{figure}[t!]
	\centering
\includegraphics[width=1\linewidth]{Jor3_l2_JRJI_soJRJI_TPJRJI}
	\caption{Results related to the ordered complex XY model with $N=64$ spins on a  2D lattice  Top: instances of  single site reconstruction for the real, JR (left column), and
		the imaginary, JI (right column), part of $J_{ij}$  Bottom: sorted values of JR (left) and JI (right) }
		
	\label{PL-Jor3}
\end{figure}
 
 
  \section{PLM with Decimation}
 \label{sec:res_dec}
 

\begin{figure}[t!]
   	\centering
   	\includegraphics[width=1\linewidth]{Jor1_dec_tPLF_varT_varM}
    	\caption{Tilted Pseudolikelyhood, ${\cal L}_t$, plotted as a function of decimated couplings  Top: Different ${\cal L}_t$ curves obtained for different values of $M$ plotted on top of each other  Here $T=1 3$  The black line indicates the expected number of decimated couplings, $x^*=(N (N-1) - N c)/2=1888$  As we can see, as $M$ increases, the maximum point of ${\cal L}_t$ approaches $x^*$  Bottom: Different ${\cal L}_t$ curves obtained for different values of T with $M=2048$  We can see that, with this value of $M$, no differences can be appreciated on the maximum points of the different  ${\cal L}_t$ curves }
   	\label{var-$t$PLF}
   \end{figure}

\begin{figure}[t!]
    \centering
    \includegraphics[width=1\linewidth]{Jor1_dec_tPLF_peak_statistics_varM_prob eps}
    	\caption{Number of most likely decimated couplings, estimated by the maximum point of $\mathcal{L}_t$, as a function of the number of samples $M$  We can clearly see that the maximum point of $\mathcal{L}_t$ tends toward $x^*$, which is the right expected number of zero couplings in the system }  
    	\label{PLF_peak_statistics}
    \end{figure}
      
      For the ordered real-valued XY model we show in Fig  \ref{var-$t$PLF}, top panel, the outcome on the tilted pseudolikelyhood, $\mathcal{L}_t$ Eq  \eqref{$t$PLF}, of the progressive decimation: from a fully connected lattice  down to an empty lattice  The figure shows the behaviour of $\mathcal{L}_t$ for three different data sizes $M$  A clear data size dependence of the maximum point of  $\mathcal{L}_t$, signalling the most likely value for decimation, is shown  For small $M$ the most likely number of  couplings is overestimated and for increasing $M$ it tends to the true value, as displayed in Fig  \ref{PLF_peak_statistics}   In the bottom panel of Fig  \ref{var-$t$PLF} we display instead different 
 $\mathcal{L}_t$ curves obtained for three different values of $T$ 
  Even though the values of $\mathcal{L}_t$ decrease with increasing temperature, the value of the most likely number of decimated couplings appears to be quite independent on $T$ with $M=2048$ number of samples 
In Fig  \ref{fig:Lt_complex} we eventually display the tilted pseudolikelyhood for a 2D network with complex valued ordered couplings, where the decimation of the real and imaginary coupling matrices proceeds in parallel, that is, 
when a real coupling is small enough to be decimated its imaginary part is also decimated, and vice versa 
One can see that though the apart errors for the real and imaginary parts are different in absolute values, they display the same dip, to be compared with the maximum point of $\mathcal{L}_t$ 
     
       \begin{figure}[t!]
    	\centering
      	\includegraphics[width=1\linewidth]{Jor3_dec_tPLF_new}
      	\caption{Tilted Pseudolikelyhood, ${\cal L}_t$, plotted with the reconstruction errors for the XY model with $N=64$ spins on a 2D lattice  These results refer to the case of  ordered and complex valued couplings  The full (red) line indicates ${\cal L}_t$  The dashed (green) 
      		and the dotted (blue) lines show the reconstruction errors (Eq  \eqref{eq:errj}) obtained for the real and the imaginary couplings respectively  We can see that both ${\rm err_{JR}}$ and ${\rm err_{JI}}$ have a minimum at $x^*$ }
          	\label{fig:Lt_complex}
    \end{figure}

\begin{figure}[t!]
   	\centering
   	\includegraphics[width=1\linewidth]{Jor1_dec_JR_soJR_TPJR}
   	\caption{XY model on a 2D lattice with $N=64$ sites and real valued couplings  The graphs show the inferred (dashed black lines) and true couplings (full green lines) plotted on top of each other  The left and right columns refer to the
   		 cases of ordered and bimodal disordered couplings, respectively  Top figures: single site reconstruction, i e , one row of the matrix $J$  Bottom figures: couplings are plotted sorted in descending order }  
   	\label{Jor1_dec}
   \end{figure}
   
\begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{Jor3_dec_JRJI_soJRJI_TPJRJI}
    	\caption{XY model on a 2D lattice with $N=64$ sites and ordered complex-valued couplings 
    		The inferred and true couplings are plotted on top of each other  The left and right columns show the real and imaginary parts, respectively, of the couplings  Top figures refer to a single site reconstruction, i e , one row of the matrix $J$  Bottom figures report the couplings sorted in descending order }
    	\label{Jor3_dec}
    \end{figure}
    
       







       \begin{figure}[t!]
     	\centering
     	\includegraphics[width=1\linewidth]{MF_PL_Jor1_2D_TPJR_varT}
     	\caption{True Positive curves obtained with the three techniques: PLM with decimation, (blue) dotted line,  PLM with $l_2$ regularization, (greed) dashed line, and mean-field, (red) full line   These results refer to real valued ordered couplings with $N=64$ spins on a 2D lattice  The temperature is here $T=0 7$ while the four graphs refer to different sample sizes: $M$ increases clockwise }
     	\label{MF_PL_TP}
     \end{figure}
    
    \begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{MF_PL_Jor1_2D_errJ_varT_varM}
    	\caption{Variation of reconstruction error, ${\rm err_J}$, with respect to temperature as obtained with the three different techniques, see Fig  \ref{MF_PL_TP}, for four different sample size:  clockwise from top   $M=512,1024, 2048$ and $4096$ } 
    	\label{MF_PL_err}
    \end{figure}
    
     Once the most likely network has been identified through the decimation procedure, we perform the same analysis displayed in Fig  \ref{Jor1_dec}  for ordered and then quenched disordered real-valued couplings
and in Fig  \ref{Jor3_dec} for  complex-valued ordered couplings   In comparison to the results shown in Sec  \ref{sec:res_reg},
  the PLM with decimation leads to rather cleaner results  In Figs  \ref{MF_PL_err} and \ref{MF_PL_TP} we compare the performances of the PLM with decimation in respect to ones of the PLM with $l_2$-regularization  These two techniques are also analysed in respect to a mean-field technique previously implemented on the same XY systems\cite{Tyagi15} 
  
    For what concerns the network of connecting links, in Fig  \ref{MF_PL_TP} we compare the TP curves obtained with the three techniques  The results refer to the case of ordered and real valued couplings, but similar behaviours were obtained for the other cases analysed  
  The four graphs are related to different sample sizes, with $M$ increasing clockwise  When $M$ is high enough, all techniques reproduce the true network  
  However, for lower values of $M$ the performances of the PLM with $l_2$ regularization and with decimation drastically overcome those ones of the previous mean field technique  
  In particular, for $M=256$ the PLM techniques still reproduce the original network while the mean-field method fails to find more than half of the couplings  
  When $M=128$, the network is clearly reconstructed only through the PLM with decimation while the PLM with $l_2$ regularization underestimates the couplings  
  Furthermore, we notice that the PLM method with decimation is able to clearly infer the network of interaction even when $M=N$ signalling that it could be considered also in the under-sampling regime $M<N$   
 
 
In Fig  \ref{MF_PL_err} we compare the temperature behaviour of the reconstruction error 
In can be observed that for all temperatures and for all sample sizes  the reconstruction error, ${\rm err_J}$, (plotted here in log-scale) obtained with the PLM+decimation is always smaller than 
that one obtained with the other techniques  The temperature behaviour of ${\rm err_J}$ agrees with the one already observed for Ising spins in \cite{Nguyen12b} and for XY spins  in \cite{Tyagi15} with a mean-field approach:  ${\rm err_J}$ displays a minimum around $T\simeq 1$ and then it increases for very lower $T$; however,
 the error obtained with the PLM with decimation is several times smaller  than the error estimated by the other methods 



 
 

     
     \section{Conclusions}
     \label{sec:conc}


Different statistical inference methods have been applied to the inverse problem of the XY model 
After a short review of techniques based on pseudo-likelihood and their formal generalization to the model we have tested their performances against data generated by means of Monte Carlo numerical simulations of known instances
with diluted, sparse, interactions 

The main outcome is that the best performances are obtained by means of the  pseudo-likelihood method combined with decimation  Putting to zero (i e , decimating) very weak bonds, this technique turns out to be very precise for  problems whose real underlying interaction network is sparse, i e , the number of couplings per variable does not scale with number of variables 
The PLM + decimation method is compared to the PLM + regularization method, with $\ell_2$ regularization and to a mean-field-based method  The behavior of the quality of the network reconstruction is analyzed by looking at the overall sorted couplings and at the single site couplings, comparing them with the real network, and at the true positive curves in all three approaches  In the PLM +decimation method, moreover, the identification of the number of decimated bonds at which the tilted pseudo-likelihood is maximum allows for a precise estimate of the total number of bonds  Concerning this technique, it is also shown that the network with the most likely number of bonds is also the one of least reconstruction error, where not only the prediction of the presence of a bond is estimated but also its value 

The behavior of the inference quality in temperature and in the size of data samples is also investigated, basically confirming the low $T$ behavior hinted by Nguyen and Berg \cite{Nguyen12b} for the Ising model  In temperature, in particular, the reconstruction error curve displays a minimum at a low temperature, close to the critical point in those cases in which a critical behavior occurs, and a sharp increase as temperature goes to zero  The decimation method, once again, appears to enhance this minimum of the reconstruction error of almost an order of magnitude with respect to other methods 
 
The techniques displayed and the results obtained in this work can be of use in any of the many systems whose theoretical representation is given by Eq  \eqref{eq:HXY} or Eq  \eqref{eq:h_im}, some of which are recalled in Sec  \ref{sec:model}  In particular, a possible application can be the field of light waves propagation through random media and the corresponding problem of the  reconstruction of an object seen through an opaque medium or a disordered optical fiber \cite{Vellekoop07,Vellekoop08a,Vellekoop08b, Popoff10a,Akbulut11,Popoff11,Yilmaz13,Riboli14} 

 	



ground_truth: Bigger receptive field size leads to more successful shape completion.