prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: Who was Brooksley Elizabeth's first husband?

Read the following text and answer briefly.

Context 1: Brooksley Elizabeth Born (born August 27, 1940) is an American attorney and former public official who, from August 26, 1996, to June 1, 1999, was chair of the Commodity Futures Trading Commission (CFTC), the federal agency which oversees the U S  futures and commodity options markets  During her tenure on the CFTC, Born lobbied Congress and the President to give the CFTC oversight of off-exchange markets for derivatives, in addition to its role with respect to exchange-traded derivatives, but her warnings were ignored or dismissed, and her calls for reform resisted by other regulators <ref name="nytimes">Goodman, Peter S  The Reckoning - Taking Hard New Look at a Greenspan Legacy, The New York Times, October 9, 2008 </ref> Born resigned as chairperson on June 1, 1999, shortly after Congress passed legislation prohibiting her agency from regulating derivatives 

In 2009, Born received the John F  Kennedy Profiles in Courage Award, along with Sheila Bair of the Federal Deposit Insurance Corporation, in recognition of the "political courage she demonstrated in sounding early warnings about conditions that contributed" to the 2007-08 financial crisis 

Early life and education
Born graduated from Abraham Lincoln High School (San Francisco, California) at the age of 16  She then attended Stanford University, where she majored in English and was graduated with the class of 1961   She initially wanted to become a doctor, but a guidance counsellor at Stanford advised her against medicine, so she majored in English literature instead 

She then attended Stanford Law School, one of only seven women in her class   She was the first female student ever to be named president of the Stanford Law Review  She received the "Outstanding Senior" award and graduated as valedictorian of the class of 1964 

Legal career
Immediately after law school Born was selected as a law clerk to judge Henry Edgerton of the U S  Court of Appeals for the District of Columbia Circuit  It was during this time that she met her first husband, Jacob C  Landau, who was a journalist covering the Federal courts at the time  Following her clerkship, she became an associate at the Washington, D C -based international law firm of Arnold & Porter  Born was attracted to Arnold & Porter because it was one of the few major law firms to have a woman partner at that time, Carolyn Agger, who was the head of the tax practice  Born took a two-year leave of absence from Arnold & Porter to accompany her first husband to Boston, where he had received a fellowship  During that time she worked as a research assistant to law professor Alan Dershowitz 

Born's early career at Arnold & Porter focused on international trade law, in which she represented a number of Swiss industries and the government of Switzerland  She developed a practice representing clients in numerous complex litigation and arbitration cases involving financial market transactions  Among her high-profile cases was the matter of the Hunt Brothers attempt to corner the market in silver in the 1970s  She made partner at Arnold & Porter, after moving to a three-day schedule to help raise her second child, and eventually rose to be the head of the firm's derivatives practice 

Born was among the first female attorneys to systematically address inequities regarding how the laws treated women  Born and another female lawyer, Marna Tucker, taught what is considered to have been the first "Women and the Law" course at Catholic University’s Columbus School of Law  The class exclusively concerned prejudicial treatment of women under the laws of the United States, past and present   Born and Tucker were surprised to discover that there was no textbook on the issue at the time   Born is also one of the co-founders of the National Women's Law Center  Born also helped rewrite the American Bar Association rules to make it possible for more women and minorities to sit on federal bench 

During her long legal career, and into her retirement, Born did much pro bono and other types of volunteer work  She was active in the American Bar Association, the largest professional organization of lawyers in the United States   Initially Born was named a member of the governing council of the ABA's Individual Rights Section, eventually becoming chairperson   Born and Tucker founded the ABA Women's Caucus, the first organization of female lawyers in the ABA   She held several other senior positions in the ABA, including being named the first woman member of the ABA's Standing Committee on the Federal Judiciary   As a member of the Judiciary Committee, Born provided testimony and opinion on persons nominated for federal judgeships  In 1980 she was named chair of the committee   As chair of the committee, Born was invited to address the U S  Congress regarding the nomination of Judge Sandra Day O'Connor to the U S  Supreme Court 

In 1993, Born's name was floated as a possible candidate for Attorney General of the United States, but Janet Reno was nominated 

In July 2009, Nancy Pelosi appointed Brooksley Born as a commissioner to the Financial Crisis Inquiry Commission (FCIC) 

Born and the OTC derivatives market
Born was appointed to the CFTC on April 15, 1994, by President Bill Clinton  Due to litigation against Bankers Trust Company by Procter and Gamble and other corporate clients, Born and her team at the CFTC sought comments on the regulation of over-the-counter derivatives, a first step in the process of writing CFTC regulations to supplement the existing regulations of the Federal Reserve System,  the Options Clearing Corporation, and the National Association of Insurance Commissioners  Born was particularly concerned about swaps, financial instruments that are traded over the counter between banks, insurance companies or other funds or companies, and thus have no transparency except to the two counterparties and the counterparties' regulators, if any   CFTC regulation was strenuously opposed by Federal Reserve chairman Alan Greenspan, and by Treasury Secretaries Robert Rubin and Lawrence Summers  On May 7, 1998, former SEC Chairman Arthur Levitt joined Rubin and Greenspan in objecting to the issuance of the CFTC's concept release  Their response dismissed Born's analysis and focused on the hypothetical possibility that CFTC regulation of swaps and other OTC derivative instruments could create a "legal uncertainty" regarding such financial instruments,  hypothetically reducing the value of the instruments  They argued that the imposition of regulatory costs would "stifle financial innovation" and encourage financial capital to transfer its  transactions offshore  The disagreement between Born and the Executive Office's top economic policy advisors has been described not only as a classic Washington turf war, but also a war of ideologies,  insofar as it is possible to argue that Born's actions were consistent with Keynesian and neoclassical economics while Greenspan, Rubin, Levitt, and Summers consistently espoused neoliberal, and neoconservative policies 

In 1998, a trillion-dollar hedge fund called Long Term Capital Management (LTCM) was near collapse   Using mathematical models to calculate debt risk, LTCM used derivatives to leverage $5 billion into more than $1 trillion, doing business with fifteen of Wall Street's largest financial institutions   The derivative transactions were not regulated, nor were investors able to evaluate LTCM's exposures   Born stated, "I thought that LTCM was exactly what I had been worried about"   In the last weekend of September 1998, the President's working group was told that the entire American economy hung in the balance   After intervention by the Federal Reserve, the crisis was averted   In congressional hearings into the crisis, Greenspan acknowledged that language had been introduced into an agriculture bill that would prevent CFTC from regulating the derivatives which were at the center of the crisis that threatened the US economy   U S  Representative Maurice Hinchey (D-NY) asked "How many more failures do you think we'd have to have before some regulation in this area might be appropriate?"  In response, Greenspan brushed aside the substance of Born's warnings with the simple assertion that "the degree of supervision of regulation of the over-the-counter derivatives market is quite adequate to maintain a degree of stability in the system"  Born's warning was that there wasn't any regulation of them   Born's chief of staff, Michael Greenberger summed up Greenspan's position this way: "Greenspan didn't believe that fraud was something that needed to be enforced, and he assumed she probably did  And of course, she did "  Under heavy pressure from the financial lobby, legislation prohibiting regulation of derivatives by Born's agency was passed by the Congress   Born resigned on June 1, 1999 

The derivatives market continued to grow yearly throughout both terms of George W  Bush's administration  On September 15, 2008, the bankruptcy of Lehman Brothers forced a broad recognition of a financial crisis in both the US and world capital markets   As Lehman Brothers' failure temporarily reduced financial capital's confidence, a number of newspaper articles and television programs suggested that the failure's possible causes included the conflict between the CFTC and the other regulators Faiola, Anthony, Nakashima, Ellen and Drew, Jill  The Crash: Risk and Regulation - What Went Wrong, The Washington Post, October 15, 2008 

Born declined to publicly comment on the unfolding 2008 crisis until March 2009, when she said: "The market grew so enormously, with so little oversight and regulation, that it made the financial crisis much deeper and more pervasive than it otherwise would have been " She also lamented the influence of Wall Street lobbyists on the process and the refusal of regulators to discuss even modest reforms 

An October 2009 Frontline documentary titled "The Warning"  described Born's thwarted efforts to regulate and bring transparency to the derivatives market, and the continuing opposition thereto  The program concluded with an excerpted interview with Born sounding another warning: "I think we will have continuing danger from these markets and that we will have repeats of the financial crisis -- may differ in details but there will be significant financial downturns and disasters attributed to this regulatory gap, over and over, until we learn from experience "

In 2009 Born, along with Sheila Bair of the FDIC, was awarded the John F  Kennedy Profiles in Courage Award in recognition of the "political courage she demonstrated in sounding early warnings about conditions that contributed" to the 2007-08 financial crisis   According to Caroline Kennedy, "Brooksley Born recognized that the financial security of all Americans was being put at risk by the greed, negligence and opposition of  powerful and well connected interests     The catastrophic financial events of recent months have  proved them [Born and Sheila Bair] right "  One member of the President's working group had a change of heart about Brooksley Born   SEC Chairman Arthur Levitt stated "I've come to know her as one of the most capable, dedicated, intelligent and committed public servants that I have ever come to know", adding that "I could have done much better  I could have made a difference" in response to her warnings 

In 2010, a documentary film Inside Job further alleged that derivatives regulation was ineffective from the Clinton administration on  Along with fellow whistleblower, former IMF Chief Economist Raghuram Rajan, who was also scorned by the economic establishment, Brooksley Born was cited as one of the authorities arguing that financial derivatives increase economic risk 

 Personal life 
Born is married to Alexander E  Bennett (also retired from Arnold & Porter)   She has five adult children - two from a previous marriage to Jacob Landau and three stepchildren  Notably, Born was named a partner at Arnold & Porter while working part-time so she could raise her two young children   When both of her children were school-age, Born returned to practice full-time 

References

External links
Attorney profile at Arnold & Porter
Brooksley Born (2009 Winner) of the Profiles in Courage Award, with acceptance speech transcript and NECN video

Profile at MarketsWiki
Speeches and statements
"Testimony Of Brooksley Born Chairperson of the CFTC Concerning The Over-The-Counter Derivatives Market", before the House Committee On Banking And Financial Services, July 24, 1998 
"The Lessons of Long Term Capital Management L P ", Remarks of Brooksley Born, Chairperson of the CFTC, Chicago-Kent-IIT Commodities Law Institute, Chicago, Illinois, October 15, 1998 
 Interview: Brooksley Born for "PBS Frontline: The Warning", PBS, (streaming VIDEO 1 hour), October 20, 2009 
Articles
Manuel Roig-Franzia  "Credit Crisis Cassandra:Brooksley Born's Unheeded Warning Is a Rueful Echo 10 Years On", The Washington Post, May 26, 2009
 Taibbi, Matt  "The Great American Bubble Machine", Rolling Stone'', July 9–23, 2009

1940 births
American women lawyers
Arnold & Porter people
Clinton administration personnel
Columbus School of Law faculty
Commodity Futures Trading Commission personnel
Heads of United States federal agencies
Lawyers from San Francisco
Living people
Stanford Law School alumni
21st-century American women
Stanford University alumni 
Context 2: Paper Info

Title: On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning
Publish Date: Unkown
Author List: Seth Karten, Siva Kailas, Huao Li, Katia Sycara

Figure

Figure1 By using contrastive learning, our method seeks similar representations between the state-message pair and future states while creating dissimilar representations with random states Thus satisfying the utility objective of the information bottleneck The depicted agents are blind and cannot see other cars 
Figure 2 An example of two possible classes, person and horse, from a single observation in the Pascal VOC game 
Figure 3  Blind Traffic Junction Left: Our method uses compositional complexity and contrastive utility to outperform other baselines in terms of performance and sample complexity The legend provides the mean ± variance of the best performance Right: Top: success, contrastive, and complexity losses for our method Right, Bottom: success, autoencoder loss for ae-comm with supervised pretraining 
Figure 4  Pascal VOC Game Representing compositional concepts from raw pixel data in images to communicate multiple concepts within a single image Our method significantly outperforms ae-comm and no-comm due to our framework being able to learn composable, independent concepts 
Figure 5  Blind Traffic Junction Social shadowing enables significantly lower sample complexity when compared to traditional online MARL 
Beta ablation: Messages are naturally sparse in bits due to the complexity loss Redundancy measures the capacity for a bijection between the size of the set of unique tokens and the enumerated observations and intents Min redundancy is 1 0 (a bijection) Lower is better 

abstract

Explicit communication among humans is key to coordinating and learning  Social learning, which uses cues from experts, can greatly benefit from the usage of explicit communication to align heterogeneous policies, reduce sample complexity, and solve partially observable tasks  Emergent communication, a type of explicit communication, studies the creation of an artificial language to encode a high task-utility message directly from data 
However, in most cases, emergent communication sends insufficiently compressed messages with little or null information, which also may not be understandable to a third-party listener  This paper proposes an unsupervised method based on the information bottleneck to capture both referential complexity and task-specific utility to adequately explore sparse social communication scenarios in multi-agent reinforcement learning (MARL) 
We show that our model is able to i) develop a natural-language-inspired lexicon of messages that is independently composed of a set of emergent concepts, which span the observations and intents with minimal bits, ii) develop communication to align the action policies of heterogeneous agents with dissimilar feature models, and iii) learn a communication policy from watching an expert's action policy, which we term 'social shadowing' 

INTRODUCTION

Social learning agents analyze cues from direct observation of other agents (novice or expert) in the same environment to learn an action policy from others  However, observing expert actions may not be sufficient to coordinate with other agents  Rather, by learning to communicate, agents can better model the intent of other agents, leading to better coordination 
In humans, explicit communication for coordination assumes a common communication substrate to convey abstract concepts and beliefs directly , which may not be available for new partners  To align complex beliefs, heterogeneous agents must learn a message policy that translates from one theory of mind to another to synchronize coordination 
Especially when there is complex information to process and share, new agent partners need to learn to communicate to work with other agents  Emergent communication studies the creation of artificial language  Often phrased as a Lewis game, speakers and listeners learn a set of tokens to communicate complex observations  
However, in multi-agent reinforcement learning (MARL), agents suffer from partial observability and non-stationarity (due to unaligned value functions) , which aims to be solved with decentralized learning through communication  In the MARL setup, agents, as speakers and listeners, learn a set of tokens to communicate observations, intentions, coordination, or other experiences which help facilitate solving tasks  
Agents learn to communicate effectively through a backpropagation signal from their task performance   This has been found useful for applications in human-agent teaming , multirobot navigation , and coordination in complex games such as StarCraft II   Communication quality has been shown to have a strong relationship with task performance , leading to a multitude of work attempting to increase the representational capacity by decreasing the convergence rates  
Yet these methods still create degenerate communication protocols , which are uninterpretable due to joined concepts or null (lack of) information, which causes performance degradation  In this work, we investigate the challenges of learning a arXiv:2302 14276v1 LG] 28 Feb 2023 messaging lexicon to prepare emergent communication for social learning (EC4SL) scenarios 
We study the following hypotheses: H1) EC4SL will learn faster through structured concepts in messages leading to higher-quality solutions, H2) EC4SL aligns the policies of expert heterogeneous agents, and H3) EC4SL enables social shadowing, where an agent learns a communication policy while only observing an expert agent's action policy 
By learning a communication policy, the agent is encouraged to develop a more structured understanding of intent, leading to better coordination  The setting is very realistic among humans and many computer vision and RL frameworks may develop rich feature spaces for a specific solo task, but have not yet interacted with other agents, which may lead to failure without alignment 
We enable a compositional emergent communication paradigm, which exhibits clustering and informativeness properties  We show theoretically and through empirical results that compositional language enables independence properties among tokens with respect to referential information  Additionally, when combined with contrastive learning, our method outperforms competing methods that only ground communication on referential information 
We show that contrastive learning is an optimal critic for communication, reducing sample complexity for the unsupervised emergent communication objective  In addition to the more human-like format, compositional communication is able to create variable-length messages, meaning that we are not limited to sending insufficiently compressed messages with little information, increasing the quality of each communication 
In order to test our hypotheses, we show the utility of our method in multi-agent settings with a focus on teams of agents, high-dimensional pixel data, and expansions to heterogeneous teams of agents of varying skill levels  Social learning requires agents to explore to observe and learn from expert cues 
We interpolate between this form of social learning and imitation learning, which learns action policies directly from examples  We introduce a 'social shadowing' learning approach where we use first-person observations, rather than third-person observations, to encourage the novice to learn latently or conceptually how to communicate and develop an understanding of intent for better coordination 
The social shadowing episodes are alternated with traditional MARL during training  Contrastive learning, which works best with positive examples, is apt for social shadowing  Originally derived to enable lower complexity emergent lexicons, we find that the contrastive learning objective is apt for agents to develop internal models and relationships of the task through social shadowing 
The idea is to enable a shared emergent communication substrate (with minimal bandwidth) to enable future coordi-nation with novel partners  Our contributions are deriving an optimal critic for a communication policy and showing that the information bottleneck helps extend communication to social learning scenarios 
In real-world tasks such as autonomous driving or robotics, humans do not necessarily learn from scratch  Rather they explore with conceptually guided information from expert mentors  In particular, having structured emergent messages reduces sample complexity, and contrastive learning can help novice agents learn from experts 
Emergent communication can also align heterogeneous agents, a social task that has not been previously studied 

Multi-Agent Signaling

Implicit communication conveys information to other agents that is not intentionally communicated   Implicit signaling conveys information to other agents based on one's observable physical position   Implicit signaling may be a form of implicit communication such as through social cues or explicit communication such as encoded into the MDP through "cheap talk"  
Unlike implicit signaling, explicit signaling is a form of positive signaling that seeks to directly influence the behavior of other agents in the hopes that the new information will lead to active listening  Multi-agent emergent communication is a type of explicit signaling which deliberately shares information 
Symbolic communication, a subset of explicit communication, seeks to send a subset of pre-defined messages  However, these symbols must be defined by an expert and do not scale to particularly complex observations and a large number of agents  Emergent communication aims to directly influence other agents with a learned subset of information, which allows for scalability and interpretability by new agents 

Emergent Communication

Several methodologies currently exist to increase the informativeness of emergent communication  With discrete and clustered continuous communication, the number of observed distinct communication tokens is far below the number permissible   As an attempt to increase the emergent "vocabulary" and decrease the data required to converge to an informative communication "language", work has added a bias loss to emit distinct tokens in different situations  
More recent work has found that the sample efficiency can be further improved by grounding communication in observation space with a supervised reconstruction loss   Information-maximizing autoencoders aim to maximize the state reconstruction accuracy for each agent  How-ever, grounding communication in observations has been found to easily satisfy these input-based objectives while still requiring a myriad more samples to explore to find a task-specific communication space  
Thus, it is necessary to use task-specific information to communicate informatively  This will enable learned compression for task completion rather than pure compression for input recovery  Other work aims to use the information bottleneck to decrease the entropy of messages   In our work, we use contrastive learning to increase representation similarity with future goals, which we show optimally optimizes the Q-function for messages 

Natural Language Inspiration

The properties of the tokens in emergent communication directly affect their informative ability  As a baseline, continuous communication tokens can represent maximum information but lack human-interpretable properties  Discrete 1-hot (binary vector) tokens allow for a finite vocabulary, but each token contains the same magnitude of information, with equal orthogonal distance to each other token 
Similar to word embeddings in natural language, discrete prototypes are an effort to cluster similar information together from continuous vectors   Building on the continuous word embedding properties, VQ-VIB , an information-theoretic observation grounding based on VQ-VAE properties , uses variational properties to provide word embedding properties for continuous emergent tokens 
Like discrete prototypes, they exhibit a clustering property based on similar information but are more informative  However, each of these message types determines a single token for communication  Tokens are stringed together to create emergent "sentences" 

Preliminaries

We formulate our setup as a decentralized, partially observable Markov Decision Process with communication (Dec-POMDP-Comm)  Formally, our problem is defined by the tuple, S, A, M, T , R, O, Ω, γ   We define S as the set of states, A i , i ∈ [1, N ] as the set of actions, which includes task-specific actions, and M i as the set of communications for N agents 
T is the transition between states due to the multi-agent joint action space T : S × A 1 ,    , A N → S  Ω defines the set of observations in our partially observable setting  Partial observability requires communication to complete the tasks successfully  O i : M 1 ,    , M N × Ŝ → Ω maps the communications and local state, Ŝ, to a distribution of observations for each agent 
R defines the reward function and γ defines the discount factor 

Architecture

The policy network is defined by three stages: Observation Encoding, Communication, and Action Decoding  The best observation encoding and action decoding architecture is task-dependent, i e , using multi-layer perceptrons (MLPs), CNNs , GRUs , or transformer layers are best suited to different inputs 
The encoder transforms observation and any sequence or memory information into an encoding H  The on-policy reinforcement learning training uses RE-INFORCE or a decentralized version of MAPPO as specified by our experiments  Our work focuses on the communication stage, which can be divided into three substages: message encoding, message passing (often considered sparse communication), and message decoding 
We use the message passing from   For message decoding, we build on a multiheaded attention framework, which allows an agent to learn which messages are most important   Our compositional communication framework defines the message encoding, as described in section 4 

Objective

Mutual information, denoted as I(X; Y ), looks to measure the relationship between random variables, which is often measured through Kullback-Leibler divergence , I(X; Y ) = D KL (p(x, y)||p(x) ⊗ p(y))  The message encoding substage can be defined as an information bottleneck problem, which defines a tradeoff between the complexity of information (compression, I(X, X)) and the preserved relevant information (utility, I( X, Y )) 
The deep variational information bottleneck defines a trade-off between preserving useful information and compression   We assume that our observation and memory/sequence encoder provides an optimal representation H i suitable for sharing relevant observation and intent/coordination information  We hope to recover a representation Y i , which contains the sufficient desired outputs 
In our scenario, the information bottleneck is a trade-off between the complexity of information I(H i ; M i ) (representing the encoded information exactly) and representing the relevant information I(M j =i ; Y i ), which is signaled from our contrastive objective  In our setup, the relevant information flows from other agents through communication, signaling a combination of the information bottleneck and a Lewis game 
We additionally promote complexity through our compositional independence objective, This is formulated by the following Lagrangian, where the bounds on mutual information Î are defined in equations 1, 2, and 10  Overall, our objective is,

Complexity through Compositional Communication

We aim to satisfy the complexity objective, I(H i , M i ), through compositional communication  In order to induce complexity in our communication, we want the messages to be as non-random as possible  That is, informative with respect to the input hidden state h  In addition, we want each token within the message to share as little information as possible with the preceding tokens 
Thus, each additional token adds only informative content  Each token has a fixed length in bits W   The total sequence is limited by a fixed limit, L l W l ≤ S, of S bits and a total of L tokens  We use a variational message generation setup, which maps the encoded hidden state h to a message m; that is, we are modeling the posterior, π i m (m l |h) 
We limit the vocabulary size to K tokens, e j ∈ R D , j ∈ [1, K] ⊂ N, where each token has dimensionality D and l ∈ [1, L] ⊂ N  Each token m l is sampled from a categorical posterior distribution, 0 otherwise such that the message m l is mapped to the nearest neighbor e j   A set of these tokens makes a message m 
To satisfy the complexity objective, we want to use m i to well-represent h i and consist of independently informative m i l  

Independent Information

We derive an upper bound for the interaction information between all tokens  Proposition 4 1  For the interaction information between all tokens, the following upper bound holds: The proof is in Appendix A 1  Since we want the mutual information to be minimized in our objective, we minimize,

Input-Oriented Information

In order to induce complexity in the compositional messages, we additionally want to minimize the mutual information I(H; M ) between the composed message m and the encoded information h  We derive an upper bound on the mutual information that we use as a Lagrangian term to minimize  Proposition 4 2  For the mutual information between the composed message and encoded information, the following upper bound holds:
The proof is in Appendix A 1  Thus, we have our Lagrangian term, Conditioning on the input or observation data is a decentralized training objective 

Sequence Length

Compositional communication necessitates an adaptive limit on the total length of the sequence  Corollary 4 3  Repeat tokens, w, are redundant and can be removed  Suppose one predicts two arbitrary tokens, w k and w l   Given equation 1, it follows that there is low or near-zero mutual information between w k and w l  
A trivial issue is that the message generator will predict every available token as to follow the unique token objective  Since the tokens are imbued with input-oriented information (equation 2), the predicted tokens will be based on relevant referential details  Thus, it follows that tokens containing irrelevant information will not be chosen 
A nice optimization objective that follows from corollary 4 3 is that one can use self-supervised learning with an end-ofsequence (EOS) token to limit the variable total length of compositional message sequences  (3) Algorithm 1 Compositional Message Gen (h t ) m i ∼ N ( ĥ; µ, σ) 9: end for 10: return m

Message Generation Architecture

Now, we can define the pipeline for message generation  The idea is to create an architecture that can generate features to enable independent message tokens  We expand each compressed token into the space of the hidden state h (1-layer linear expansion) since each token has a natural embedding in R |h|  
Then, we perform attention using a softmin to help minimize similarity with previous tokens and sample the new token from a variational distribution  See algorithm 1 for complete details  During execution, we can generate messages directly due to equation 1, resolving any computation time lost from sequential compositional message generation 

Utility through Contrastive Learning

First, note that our Markov Network is as follows: H j → M j → Y i ← H i   Continue to denote i as the agent identification and j as the agent ID such that j = i  We aim to satisfy the utility objective of the information bottleneck, I(M j ; Y i ), through contrastive learning as shown in figure 1  Proposition 5 1 
Utility mutual information is lower bounded by the contrastive NCE-binary objective, The proof is in Appendix A 1  This result shows a need for gradient information to flow backward across agents along communication edge connections 

Experiments and Results

We condition on inputs, especially rich information (such as pixel data), and task-specific information  When evaluating an artificial language in MARL, we are interested in referential tasks, in which communication is required to complete the task  With regard to intent-grounded communication, we study ordinal tasks, which require coordination information between agents to complete successfully 
Thus, we consider tasks with a team of agents to foster messaging that communicates coordination information that also includes their observations  To test H1, structuring emergent messages enables lower complexity, we test our methodology and analyze the input-oriented information and utility capabilities 
Next, we analyze the ability of heterogeneous agents to understand differing communication policies (H2))  Finally, we consider the effect of social shadowing (H3), in which agents solely learn a communication policy from an expert agent's action policy  We additionally analyze the role of offline reinforcement learning for emergent communication in combination with online reinforcement learning to further learn emergent communication alongside an action policy 
We evaluate each scenario over 10 seeds 

Environments

Blind Traffic Junction We consider a benchmark that requires both referential and ordinal capabilities within a team of agents  The blind traffic junction environment requires multiple agents to navigate a junction without any observation of other agents  Rather, they only observe their own state location 
Ten agents must coordinate to traverse through the lanes without colliding into agents within their lane or in the junction
Context 3: '无锡速芯微电子有限公司是一家集芯片 研发，销售和服务于一体的国家高新技 术企业，为客户提供高性能，高集成 度，极致体验的全协议快充芯片。 无锡速芯微电子有限公司 FastSOC Microelectronics Co ,Ltd  销售联系方式： 联系人：顾先生 手机：1800 185 3071 邮箱：gpp@fastsoc com 网址：www fastsoc com 地址：无锡市新吴区菱湖大道200号中国物联网国际创新园E-503室 顾工微信号 速芯微公众号 免责声明：本文所述方法、方案均供客户参考，用于提示或者展示芯片应用的一种或者多种方式，不作为最终产品的实际方案。文中所描述的功能和性能指标在实 验室环境下测试得到，部分可以提供第三方测试报告，但是不保证客户产品上能获得相同的数据。本文信息只作为芯片使用的指导，不授权用户使用本公司或者其 他公司的知识产权。本文信息只作为芯片使用的指导，不承担因为客户自身应用不当而造成的任何损失。 **文中信息仅供参考，详情请联系我司获取最新资料” 无锡速芯微电子有限公司 FastSOC Microelectronics Co ,Ltd  产品手册 2023年 
新品快览 FS312A：PD3 0 诱骗- FS312A支持PD2 0/PD3 0最高诱骗电压：20V - FS312AE支持PD2 0/PD3 0 最高诱骗电压：20V支持Emarker模拟功能 - 封装：SOT23-5 VBUS CC1 CC2 DM DP 用电电路 4 7K 0 47uF R C C 1 V D D F U N C C C 2F S 3 1 2 B D M D P EP GND 应用图 FS8628：A+C快充协议CC2 CC1 VBUS CC2 CC1 FS312A FUNC GND VDD 4 7K GND R 用电电路 1uF GND 应用图 多口极简方案 FS8611SP*2+CCM-8611SP-A+7533B-T 双C智能降功率方案 FS8611S USB-C AC-DC 双变压器 7533B-T CCM-8611SP-A FS8611S USB-C 采用2颗FS8611SP搭配CCM-8611SP-A （MCU），7533B-T配合工作 - 支持多种协议 - 支持I2C控制 - 任意单 C 的为 35W - 双 插 降 功 率 ， 三 档 功 率 智 能 配 置：27 4W+7 4W；17 4W+17 4W； 27 4W - BOM极简，成本低 FS312B：PD3 1 诱骗FS8611K*2+CCM-8611K-A+7550B-T 双C方案 - FS312BL支持PD2 0/PD3 0/PD3 1/第三方协议最高诱骗电压：20V - FS312BLE支持PD2 0/PD3 0/PD3 1/第三方协议最高诱骗电压：20V支持Emarker模拟功能 - FS312BH支持PD2 0/PD3 0/PD3 1/第三方协议最高诱骗电压：48V - FS312BHE支持PD2 0/PD3 0/PD3 1/第三方协议最高诱骗电压：48V 支持Emarker模拟功能 - 封装：DFN2x2-6L - 兼容兼容BC1 2、Apple2 4A、 QC2 0 Class A、QC3 0 Class A/B、 FCP、SCP、AFC、低压直充等 - 兼容Type-C PD2 0、Type-C PD3 0、 Type-C PD3 0 PPS、QC4 0协议 - 支持两路DP/DM - 支持CV/CC（分段CC）功能 - 支持定制PDO - 支持A+C双口工作，电压自动回5V - 支持FB/OPTO反馈 - 封装：QFN3x3-20L VPWR FB PowerSystem 100K GND R1 GND 19 VIN 17 FB FUNC1 FUNC2 20 15 18 13 PLUGIND VFB FS8628 QFN3x3-20L AGATE 47K 7 5K 47K 7 5K 1 16 8 7 3 4 5 6 10 9 11 CGATE CVBUS CC2 CC1 CDP CDM AVBUS DM DP ISP ISN 12 应用图 2 V3P3 100Ω 1u EP GND GND CVBUS TYPE- C CC2 CC1 CDP CDM CGND TYPE-A AVBUS DM DP 10n 200 AGND 5mΩ GND FS8611K USB-C AC-DC DC-DC 7550B-T CCM-8611K-A FS8611K USB-C 采用2颗FS8611K搭配CCM-8611K-A （MCU）工作，7550B-T配合工作 - 支持PD2 0/PD3 0/QC2 0/AFC/FCP - 支持PDO定制 - 任意单 C 的为 35W(可定制) - 双插18W（可定制15W/20W） - BOM极简，成本低 FS212C+ACM-212C-A+7550B-T 双C方案 FS212C USB-C AC-DC DC-DC 7550B-T ACM-212C-A FS8623B-A+C方案 AC-DC DC-DC FS8623B USB-A USB-C USB-A 采 用 1 颗 F S 2 1 2 C 搭 配 ACM-212C-A 工 作，7550B-T配合工作 - 支持PD2 0/PD3 0 - 支持PDO定制 - 任意单 C 的为20W - 双插7 5W回5V - BOM极简，成本低 采用一颗FS8623B实现A+C方案 - 兼容兼容Apple2 4A/低压直充 QC2 0 Class A/QC3 0 Class A/B/ FCP/SCP等 - 兼 容Type -C PD2 0 / PD3 0 / PD3 0PPS/QC4 0协议 - 支持PDO定制 - 双插回5V 
多口方案选型 产品选型 受电端芯片选型 速芯微现有多种多口的方案选择：A+C，C+C，C+C+A，C+C+C，C+C+A+A等方案。对于 A+C的方案，可使用1颗芯片实现，也可用多颗芯片来实现。 速芯微现有多种受电端诱骗芯片，客户可根据应用需求进行选择。 受电端诱骗芯片应用领域 筋膜枪 无线充 线材 无人机 产品型号 PD2 0 PD3 0 PD3 1 第三方协议 诱骗电压(V) 控制方式 内置Emarker 定制 封装 FS312A √ √ 5/9/12/15/20 电阻阻值 可变电压策略 SOT23-5 FS312AE √ √ 5/9/12/15/20 电阻阻值 √ (公头专用) 可变电压策略 SOT23-5 FS312BL √ √ √ √ 5/9/12/15/20 电阻阻值 可变电压策略 DFN2x2-6 FS312BLE √ √ √ √ 5/9/12/15/20 电阻阻值 √ (公头专用) 可变电压策略 DFN2x2-6 FS312BH √ √ √ √ 5/20/28/36/48 电阻阻值 可变电压策略 DFN2x2-6 FS312BHE √ √ √ √ 5/20/28/36/48 电阻阻值 √ (公头专用) 可变电压策略 DFN2x2-6 FS312LC √ √ √ 5/9/12 电阻阻值 可变第三方 协议 SSOP10 FS312HC √ √ √ 5/9/12/15/20 电阻阻值 可变第三方 协议 SSOP10 FS2711Q √ √ √ 任意设置 I2C √ QFN3x3-16 FS2711P √ √ √ 任意设置 I2C √ QFN3x3-16 FS2711PA √ √ 全协议 任意设置 I2C √ SSOP10 FS2711SW √ √ 全协议 SSOP10 FS512 √ √ 全协议 任意设置 I2C √ SSOP10 方案 类型 产品型号 单C 单A 双插 A+C方案 FS8623 20W（PPS）（可定制） A口全协议18w 5V共享3A FS8623B 20W（PPS）(可定制) A口全协议18w 5V共享3A FS8628 20W（PPS）(可定制) A口全协议18w 5V共享3A FS8611RPC+FS116DB 65W（PPS）(可定制) A口全协议18w A口：5V/2 4A C口：45W FS8628RC+FS116DB 35W(可定制) A口全协议18w A口：5V（BC1 2，Apple 2 4） C口：20W 方案类型 产品型号 单C1 单C2 C1/C2 C+C方案 FS8611RPB*2 30W(可定制) 30W(可定制) C1/C2：5V/3A(或5V/2 4A) FS8611GH*2 35W（可定制） 35W（可定制） C1/C2:18W（可定制） FS8628P*2 35W（可定制） 35W（可定制） C1/C2：17 4W可定制） FS8611KL*2 20W（可定制） 20W（可定制） C1/C2：5V/1 5 A FS8611PC*2 35W 35W C1/C2：18W FS8611BH*2 65W（可定制） 65W（可定制） C1：45W（可定制）C2：20W（可定制） FS8628RPC+FS8611RB 45W（可定制）） 36W （可定制）） C1：30W（可定制）C2：5V/1 5A（可定制） 方案类型 产品型号 单C1 单C2 单A C1+C2 C1/C2+A C1+C2+A C+C+A FS8611S*2+FS116DB 65W（可定制） 65W（ 可定制）） A口全协议18w 智能分配功率 45W+18W C1/C2：智能分配功率 A：18W（或5V1 5A） FS8612C+FS8628P 100W（可定制） 35W （可定制）） 20W C1：65W C2：20W C1+A：65W+20W C2+A：7 5W+7 5W C1：65W C2：7 5W A：7 5W 其他 
Source-TYPE C协议芯片选型 Source-TYPE A协议芯片选型 速芯微现有多种TYPE-C的快充协议芯片，支持多种协议，支持客户定制，多样化，满 足客户对TYPE C的各种快充需求。 速芯微现有多种TYPE A快充协议芯片，支持全协议，支持定制，满足客户对A口协议的各种需 求。速芯微的TYPE-A快充协议芯片的协议丰富，FS112系列拥有多种的型号；FS116D 系列带插入指示，可搭配TYPE-C快充协议芯片，实现A+C，A+C+C，A+A+C+C等多口方 案，协议丰富，其中FS116A一般用于插入指示使用 Source-TYPE A协议芯片引脚封装图 D+ VSS FB 1 2 3 FS112 6 5 4 D- VDD FUNC GATE VIN FUNC FB LED/PLUG_IN 1 2 3 4 5 FS116D 10 DM 9 8 7 6 DP CSP CSN VSS速芯微的各TYPE-C快充协议芯片之间可搭配使用，实现多口方案，更多详情请咨 询我司工作人员。 多口降功率专用快充协议芯片：FS8611RB，FS8611RC，FS8611RPB，FS8611RPC， FS8612CP。 带I2C快充协议芯片：FS8611S，FS8611SP 产品型号 BC1 2 Apple 2 4 QC2 0 QC3 0 AFC FCP SCP HISCP 大电流直充 封装 FS112 √ √ √ √ √ √ √ SOT23-6 FS112H √ √ √ √ √ √ √ √ √ SOT23-6 FS113 √ v √ √ √ √ √ √ √ SOT23-6 FS116DP √ √ √ √ √ √ √ √ SSOP10 FS116DB √ √ √ √ √ √ √ √ SSOP10 FS116E √ √ √ √ √ √ √ √ √ SSOP10 FS116A √ √ SSOP10 其他 可定制 PD2 0 PD3 0 PD3 0 PPS 第三方协议 反馈方式 MOS CV/CC 定制 封装 FS212C √ √ FB √ SOT23-6 FS212CM √ √ FB PMOS(可省) √ SOT23-6 FS212D √ √ √ FB √ SOT23-6 FS212DH √ √ √ FB √ SOT23-6 FS212DP √ √ √ FB PMOS √ SOT23-6 FS212DG √ √ √ FB PMOS √ SOT23-6 FS8611G √ √ FB PMOS(可省) √ SOP-8 FS8611K √ √ QC2 0/AFC/FCP FB PMOS(可省) √ SOP8 FS8611J √ √ √ 全协议 FB PMOS(可省) √ SOP8 FS8611B √ √ √ 全协议 FB PMOS(可省) √ SSOP10 FS8611RB √ √ 全协议 FB PMOS √ SSOP10 FS8611RC √ √ 全协议 FB PMOS √ SSOP10 FS8611S √ √ √ 全协议 FB PMOS √ SSOP10 FS8611PP √ √ √ 全协议 FB PMOS √ SSOP10 FS8611BP √ √ √ 全协议 FB PMOS(可省) √ SSOP10 FS8611RPB √ √ √ 全协议 FB PMOS √ SSOP10 FS8611RPC √ √ √ 全协议 FB PMOS √ SSOP10 FS8611SP √ √ √ 全协议 FB PMOS(可省) SSOP10 FS8612 √ √ √ 全协议 OPTO PMOS √ √ SSOP16 FS8612B √ √ √ 全协议 FB PMOS √ √ SSOP16 FS8612BP √ √ √ 全协议 FB PMOS √ √ SSOP16 FS8612C √ √ √ 全协议 FB/OPTO PMOS √ √ QFN4x4-16 FS8612CP √ √ √ 全协议 FB/OPTO PMOS √ √ QFN4x4-16 
'



ground_truth: Jacob C. Landau.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the main methodology used in the research?

Read the following text and answer briefly.

Context 2:  Our training uses REINFORCE   Pascal VOC Game We further evaluate the complexity of compositional communication with a Pascal VOC   This is a two-agent referential game similar to the Cifar game but requires the prediction of multiple classes 
During each episode, each agent observes a random image from the Pascal VOC dataset containing exactly two unique labels  Each agent must encode information given only the raw pixels from the original image such that the other agent can recognize the two class labels in the original image  An agent receives a reward of 0 25 per correctly chosen class label and will receive a total reward of 1 if both agents guess all labels correctly 
See figure 2  Our training uses heterogeneous agents trained with PPO (modified from MAPPO repository)  For simplicity of setup, we consider images with exactly two unique labels from a closed subset of size five labels of the original set of labels from the Pascal VOC data  Furthermore, these images must be of size 375 × 500 pixels 
Thus, the resultant dataset comprised 534 unique images from the Pascal VOC dataset 

Baselines

To evaluate our methodology, we compare our method to the following baselines: (1) no-comm, where agents do not communicate; (2) rl-comm, which uses a baseline communication method learned solely through policy loss ; (3) ae-comm, which uses an autoencoder to ground communication in input observations ; (4) VQ-VIB, which uses a variational autoencoder to ground discrete communication in input observations and a mutual information objective to ensure low entropy communication  
We provide an ablation of the loss parameter β in table 1 in the blind traffic junction scenario  When β = 0, we use our compositional message paradigm without our derived loss terms  We find that higher complexity and independence losses increase sample complexity  When β = 1, the model was unable to converge 
However, when there is no regularization loss, the model performs worse (with no guarantees about referential representation)  We attribute this to the fact that our independence criteria learns a stronger causal relationship  There are fewer spurious features that may cause an agent to take an incorrect action 
In order to understand the effect of the independent concept representation, we analyze the emergent language's capacity for redundancy  A message token m l is redundant if there exists another token m k that represents the same information  With our methodology, the emergent 'language' converges to the exact number of observations and intents required to solve the task 
With a soft discrete threshold, the independent information loss naturally converges to a discrete number of tokens in the vocabulary  Our β ablation in table 1 yields a bijection between each token in the vocabulary and the possible emergent concepts, i e , the enumerated observations and intents  Thus for β = 0 1, there is no redundancy 
Sparse Communication In corollary 4 3, we assume that there is no mutual information between tokens  In practice, the loss may only be near-zero  Our empirical results yield independence loss around 1e − 4  In table 1, the size of the messages is automatically compressed to the smallest size to represent the information 
Despite a trivially small amount of mutual information between tokens, our compositional method is able to reduce the message size in bits by 2 3x using our derived regularization, for a total of an 8x reduction in message size over non-compositional methods such as ae-comm  Since the base unit for the token is a 32-bit float, we note that each token in the message may be further compressed 
We observe that each token uses three significant digits, which may further compress tokens to 10 bits each for a total message length of 20 bits 

Communication Utility Results

Due to coordination in MARL, grounding communication in referential features is not enough  Finding the communication utility requires grounding messages in ordinal information  Overall, figure shows that our compositional, contrastive method outperforms all methods focused on solely input-oriented communication grounding 
In the blind traffic junction, our method yields a higher average task success rate and is able to achieve it with a lower sample complexity  Training with the contrastive update tends to spike to high success but not converge, often many episodes before convergence, which leaves area for training improvement 
That is, the contrastive update begins to find aligned latent spaces early in training, but it cannot adapt the methodology quickly enough to converge  The exploratory randomness of most of the early online data prevents exploitation of the high utility f + examples  This leaves further room for improvement for an adaptive contrastive loss term 
Regularization loss convergence After convergence to high task performance, the autoencoder loss increases in order to represent the coordination information  This follows directly from the information bottleneck, where there exists a tradeoff between utility and complexity  However, communication, especially referential communication, should have an overlap between utility and complexity 
Thus, we should seek to make the complexity loss more convex  Our compositional communication complexity loss does not converge before task performance convergence  While the complexity loss tends to spike in the exploratory phase, the normalized value is very small  Interestingly, the method eventually converges as the complexity loss converges below a normal- ized 0 3 
Additionally, the contrastive loss tends to decrease monotonically and converges after the task performance converges, showing a very smooth decrease  The contrastive f − loss decreases during training, which may account for success spikes prior to convergence  The method is able to converge after only a moderate decrease in the f + loss 
This implies empirical evidence that the contrastive loss is an optimal critic for messaging  See figure 3 

Heterogeneous Alignment Through Communication

In order to test the heterogeneous alignment ability of our methodology to learn higher-order concepts from highdimensional data, we analyze the performance on the Pascal VOC game  We compare our methodology against ae-comm to show that concepts should consist of independent information directly from task signal rather than compression to reconstruct inputs 
That is, we show an empirical result on pixel data to verify the premise of the information bottleneck  Our methodology significantly outperforms the observation-grounded ae-comm baseline, as demonstrated by figure 4  The ae-comm methodology, despite using autoencoders to learn observation-grounded communication, performs only slightly better than no-comm 
On the other hand, our methodology is able to outperform both baselines significantly  It is important to note that based on figure 4, our methodology is able to guess more than two of the four labels correctly across the two agents involved, while the baseline methodologies struggle to guess exactly two of thew four labels consistently 
This can be attributed to our framework being able to learn compositional concepts that are much more easily discriminated due to mutual independence 

Social Shadowing

Critics of emergent communication may point to the increased sample complexity due to the dual communication and action policy learning  In the social shadowing scenario, heterogeneous agents can learn to generate a communication policy without learning the action policy of the watched expert agents  To enable social shadowing, the agent will alternate between a batch of traditional MARL (no expert) and (1st-person) shadowing an expert agent performing the task in its trajectory 
The agent only uses the contrastive objective to update its communication policy during shadowing  In figure , the agent that performs social shadowing is able to learn the action policy with almost half the sample complexity required by the online reinforcement learning agent  Our results show that the structured latent space of the emergent communication learns socially benevolent coordination 
This tests our hypothesis that by learning communication to understand the actions of other agents, one can enable lower sample complexity coordination  Thus, it mitigates the issues of solely observing actions 

Discussion

By using our framework to better understand the intent of others, agents can learn to communicate to align policies and coordinate  Any referential-based setup can be performed with a supervised loss, as indicated by the instant satisfaction of referential objectives  Even in the Pascal VOC game, which appears to be a purely referential objective, our results show that intelligent compression is not the only objective of referential communication 
The emergent communication paradigm must enable an easy-to-discriminate space for the game  In multi-agent settings, the harder challenge is to enable coordination through communication  Using contrastive communication as an optimal critic aims to satisfy this, and has shown solid improvements  Since contrastive learning benefits from good examples, this method is even more powerful when there is access to examples from expert agents 
In this setting, the communication may be bootstrapped, since our optimal critic has examples with strong signals from the 'social shadowing' episodes  Additionally, we show that the minimization of our independence objective enables tokens that contain minimal overlapping information with other tokens 
Preventing trivial communication paradigms enables higher performance  Each of these objectives is complementary, so they are not trivially minimized during training, which is a substantial advantage over comparative baselines  Unlike prior work, this enables the benefits of training with reinforcement learning in multi-agent settings 
In addition to lower sample complexity, the mutual information regularization yields additional benefits, such as small messages, which enables the compression aspect of sparse communication  From a qualitative point of view, the independent information also yields discrete emergent concepts, which can be further made human-interpretable by a post-hoc analysis  
This is a step towards white-box machine learning in multi-agent settings  The interpretability of this learned white-box method could be useful in human-agent teaming as indicated by prior work   The work here will enable further results in decision-making from high-dimensional data with emergent concepts 
The social scenarios described are a step towards enabling a zero-shot communication policy  This work will serve as future inspiration for using emergent communication to enable ad-hoc teaming with both agents and humans 

Appendix

A 1  Proofs Proposition 4 1 For the interaction information between all tokens, the following upper bound holds: Proof  Starting with the independent information objective, we want to minimize the interaction information, which defines the conditional mutual information between each token and, Let π i m (m l |h) be a variational approximation of p(m l |h), which is defined by our message encoder network 
Given that each token should provide unique information, we assume independence between m l   Thus, it follows that our compositional message is a vector, m = [m 1 ,       , m L ], and is jointly Gaussian  Moreover, we can define q( m|h) as a variational approximation to p(m|h) = p(m 1 ;       , m L |h) 
We can model q with a network layer and define its loss as || m − m|| 2   Thus, transforming equation 4 into variational form, we have, it follows that q( m|h) log q( m|h)d m ≥ q( m|h) log Thus, we can bound our interaction information, Proposition 4 2 For the mutual information between the composed message and encoded information, the following upper bound holds:
Proof  By definition of mutual information between the composed messages M and the encoded observations H, we have, Substituting q( m|h) for p( m|h), the same KL Divergence identity, and defining a Gaussian approximation z( m) of the marginal distribution p( m), it follows that, In expectation of equation 1, we have,
This implies that, for m = [m 1 ,       , m L ], there is probabilistic independence between m j , m k , j = k  Thus, expanding, it follows that, where z(m l ) is a standard Gaussian  Proposition 5 1  Utility mutual information is lower bounded by the contrastive NCE-binary objective, Proof  We suppress the reliance on h since this is directly passed through 
By definition of mutual information, we have, Our network model learns π R + (y|m) from rolled-out trajectories, R + , using our policy  The prior of our network state, π R − (y), can be modeled from rolling out a random trajectory, R−  Unfortunately, it is intractable to model π R + (y|m) and π R − (y) directly during iterative learning, but we can sample y + ∼ π R + (y|m) and y − ∼ π R − (y) directly from our network during training 
It has been shown that log p(y|m) provides a bound on mutual information , with the expectation over l p(m l , y l )  However, we need a tractable understanding of the information Y   In the information bottleneck, Y represents the desired outcome  In our setup, y is coordination information that helps create the desired output, such as any action a −  
This implies, y =⇒ a −   Since the transition is known, it follows that a − =⇒ s − f , a random future state  Thus, we have, π This is similar to the proof for lemma A 5, but requires assumptions on messages m from the emergent language  We note that when m is random, the case defaults to lemma A 5  Thus, we assume we have at least input-oriented information in m given sufficiently satisfying equation 2  Given a sufficient emergent language, it follows that y =⇒ a + , where a + is an intention action based on m 
Similarly, since the transition is known, a + =⇒ s + f , a desired goal state along the trajectory  Thus, we have, π R + (y|m) = p(s = s + f |y, m)  Recall the following (as shown in ), which we have adapted to our communication objective, Proposition A 3 (rewards → probabilities)  The Q-function for the goal-conditioned reward function r g (s t , m t ) = (1 − γ)p(s = s g |y t ) is equivalent to the probability of state s g under the discounted state occupancy measure:
and Lemma A 4  The critic function that optimizes equation 8 is a Q-function for the goal-conditioned reward function up to a multiplicative constant 1 The critic function f (s, m, s f ) = y enc(s f ) represents the similarity between the encoding y = enc(s, m) and the encoding of the future rollout s f  
Given lemmas A 5 A 6 A 8 and proposition A 7, it follows that equation 8 is the NCE-binary (InfoMAX ) objective, Î(M j , Y i ) = log σ(f (s, m, s + f )) + log 1 − σ(f (s, m, s − f )) which lower bounds the mutual information, I(M j , Y i ) ≥ Î(M j , Y i )  The critic function is unbounded, so we constrain it to [0, 1] with the sigmoid function, σ( * ) 
We suppress the reliance on h since this is directly passed through  By definition of mutual information, we have, Our network model learns π R + (y|m) from rolled-out trajectories, R + , using our policy  The prior of our network state, π R − (y), can be modeled from rolling out a random trajectory, R− 
Unfortunately, it is intractable to model π R + (y|m) and π R − (y) directly during iterative learning, but we can sample y + ∼ π R + (y|m) and y − ∼ π R − (y) directly from our network during training  It has been shown that log p(y|m) provides a bound on mutual information , with the expectation over l p(m l , y l ) 
However, we need a tractable understanding of the information Y   Lemma A 5  π R − (y) = p(s = s − f |y)  In the information bottleneck, Y represents the desired outcome  In our setup, y is coordination information that helps create the desired output, such as any action a −   This implies, y =⇒ a −   Since the transition is known, it follows that a − =⇒ s − f , a random future state 
Thus, we have, π R − (y) = p(s = s − f |y)  Lemma A 6  π R + (y|m) = p(s = s + f |y, m)  This is similar to the proof for lemma A 5, but requires assumptions on messages m from the emergent language  We note that when m is random, the case defaults to lemma A 5  Thus, we assume we have at least input-oriented information in m given sufficiently satisfying equation 2  Given a sufficient emergent language, it follows that y =⇒ a + , where a + is an intention action based on m 
Similarly, since the transition is known, a + =⇒ s + f , a desired goal state along the trajectory  Thus, we have, π R + (y|m) = p(s = s + f |y, m)  Recall the following (as shown in ), which we have adapted to our communication objective, Proposition A 7 (rewards → probabilities)  The Q-function for the goal-conditioned reward function r g (s t , m t ) = (1 − γ)p(s = s g |y t ) is equivalent to the probability of state s g under the discounted state occupancy measure:
and Lemma A 8  The critic function that optimizes equation 8 is a Q-function for the goal-conditioned reward function up to a multiplicative constant 1 p(s f ) : exp(f * (s, m, s f ) = 1 p(s f ) Q π s f (s, m)  The critic function f (s, m, s f ) = y enc(s f ) represents the similarity between the encoding y = enc(s, m) and the encoding of the future rollout s f  
Given lemmas A 5 A 6 A 8 and proposition A 7, it follows that equation 8 is the NCE-binary (InfoMAX ) objective, which lower bounds the mutual information, I(M j , Y i ) ≥ Î(M j , Y i )  The critic function is unbounded, so we constrain it to [0, 1] with the sigmoid function, σ( * ) 



ground_truth: An unsupervised method based on the information bottleneck and contrastive learning.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: 对于PD3.0协议，FS312BH支持的最高诱骗电压是多少？

Read the following text and answer briefly.

Context 4:  Voltage pulses ranging between 4-6 V are applied by the tip of a combined STM/AFM system, which result in cleavage of one C-H bond at each of the pentagonal apices of 6, thereby leading to the generation of 5 (Fig  ) 
In the main text, we focus on the generation and characterization of 5 on insulating surfaces  Generation and characterization of 5 on coinage metal surfaces is shown in Supplementary Fig    )  Blue and orange colors represent spin up and spin down densities, respectively  c, Probability density of the SOMOs of 5OS (isovalue: 0 001 e -Å -3 ) 
d, DFT-calculated bond lengths of 5OS  e, Constant-height I(V) spectra acquired on a species of 5 assigned as 5OS, along with the corresponding dI/dV(V) spectra  Open feedback parameters: V = -2 V, I = 0 17 pA (negative bias side) and V = 2 V, I = 0 17 pA (positive bias side)  Acquisition position of the spectra is shown in Supplementary Fig    f, Scheme of many-body transitions associated to the measured ionic resonances of 5OS 
Also shown are STM images of assigned 5OS at biases where the corresponding transitions become accessible  Scanning parameters: I = 0 3 pA (V = -1 2 V and -1 5 V) and 0 2 pA (V = 1 3 V and 1 6 V)  g, Laplace-filtered AFM image of assigned 5OS  STM set point: V = 0 2 V, I = 0 5 pA on bilayer NaCl, Δz = -0 3
Å  The tip-height offset Δz for each panel is provided with respect to the STM setpoint, and positive (negative) values of Δz denote tip approach (retraction) from the STM setpoint  f and g show the same molecule at the same adsorption site, which is next to a trilayer NaCl island  The bright and dark features in the trilayer NaCl island in g correspond to Cl -and Na + ions, respectively 
Scale bars: 10 Å (f) and 5 Å (g)  To experimentally explore the electronic structure of 5, we used bilayer NaCl films on coinage metal surfaces to electronically decouple the molecule from the metal surfaces  Before presenting the experimental findings, we summarize the results of our theoretical calculations performed on 5 in the neutral charge state (denoted as 5 0 ) 
We start by performing DFT calculations on 5 0 in the gas phase  Geometry optimization performed at the spin-unrestricted UB3LYP/6-31G level of theory leads to one local minimum, 5OS, the geometry of which corresponds to the open-shell resonance structure of 5 (Fig  , the label OS denotes open-shell) 
The triplet electronic configuration of 5OS is the lowest-energy state, with the openshell singlet configuration 90 meV higher in energy  Geometry optimization performed at the restricted closed-shell RB3LYP/6-31G level reveals two local minima, 5para and 5ortho, the geometries of which (Fig  ) exhibit bond length alternations in line with the presence of a para-or an ortho-QDM moiety, respectively, in the as-indacene core of the closed-shell resonance structures of 5 (Fig  )  
Relative to 5OS in the triplet configuration, 5para and 5ortho are 0 40 and 0 43 eV higher in energy, respectively  Additional DFT results are shown in Supplementary Fig    To gain more accurate insights into the theoretical electronic structure of 5, we performed multireference perturbation theory calculations (Supplementary Fig  ) based on quasi-degenerate second-order n-electron valence state perturbation theory (QD-NEVPT2) 
In so far as the order of the ground and excited states are concerned, the results of QD-NEVPT2 calculations qualitatively match with DFT calculations  For 5OS, the triplet configuration remains the lowest-energy state, with the open-shell singlet configuration 60 meV higher in energy  The energy differences between the open-and closed-shell states are substantially reduced in QD-NEVPT2 calculations, with 5para and 5ortho only 0 11 and 0 21 eV higher in energy, respectively, compared to 5OS in the triplet configuration 
We also performed nucleus-independent chemical shift calculations to probe local aromaticity of 5 in the openand closed-shell states  While 5OS in the triplet configuration exhibits local aromaticity at the terminal benzenoid rings, 5OS in the open-shell singlet configuration, 5para and 5ortho all display antiaromaticity (Supplementary Fig  ) 
The choice of the insulating surface determines the charge state of 5: while 5 adopts neutral charge state on the high work function bilayer NaCl/Au(111) surface (irrespective of its openor closed-shell state, Supplementary Fig  ), 5 exhibits charge bistability between 5 0 and the anionic state 5 -1 on the lower work function bilayer NaCl/Ag(111) and Cu(111) surfaces (Supplementary Figs  ) 
In the main text, we focus on the characterization of 5 on bilayer NaCl/Au(111)  Characterization of charge bistable 5 is reported in Supplementary Figs    We first describe experiments on 5 on bilayer NaCl/Au(111), where 5 exhibits a geometry corresponding to the calculated 5OS geometry, and an open-shell electronic configuration 
We compare the experimental data on this species to calculations on 5OS with a triplet configuration, as theory predicts a triplet ground state for 5OS  For 5OS, the calculated frontier orbitals correspond to the SOMOs ψ1 and ψ2 (Fig  ), whose spin up levels are occupied and the spin down levels are empty 
Figure shows the DFT-calculated bond lengths of 5OS, where the two salient features, namely, the small difference in the bond lengths within each ring and the notably longer bond lengths in the pentagonal rings, agree with the open-shell resonance structure of 5 (Fig  )  Figure shows an AFM image of 5 adsorbed on bilayer NaCl/Au(111) that we assign as 5OS, where the bond-order differences qualitatively correspond to the calculated 5OS geometry (discussed and compared to the closed-shell state below) 
Differential conductance spectra (dI/dV(V), where I and V denote the tunneling current and bias voltage, respectively) acquired on assigned 5OS exhibit two peaks centered at -1 5 V and 1 6 V (Fig  ), which we assign to the positive and negative ion resonances (PIR and NIR), respectively  Figure shows the corresponding STM images acquired at the onset (V = -1 2
V/1 3 V) and the peak (V = -1 5 V/1 6 V) of the ionic resonances  To draw a correspondence between the STM images and the molecular orbital densities, we consider tunneling events as many-body electronic transitions between different charge states of 5OS (Fig  )  Within this framework, the PIR corresponds to transitions between 5 0 and the cationic state 5  
At the onset of the PIR at -1 2 V, an electron can only be detached from the SOMO ψ1 and the corresponding STM image at -1 2 V shows the orbital density of ψ1  Increasing the bias to the peak of the PIR at -1 5 V, it becomes possible to also empty the SOMO ψ2, such that the corresponding STM image shows the superposition of ψ1 and ψ2, that is, |ψ1| 2 + |ψ2| 2 (ref 
)  Similarly, the NIR corresponds to transitions between 5 0 and 5 -1   At the NIR onset of 1 3 V, only electron attachment to ψ2 is energetically possible  At 1 6 V, electron attachment to ψ1 also becomes possible, and the corresponding STM image shows the superposition of ψ1 and ψ2  The observation of the orbital densities of SOMOs, and not the hybridized HOMO and LUMO, proves the open-shell ground state of assigned 5OS 
Measurements of the monoradical species with a doublet ground state are shown in Supplementary Fig    Unexpectedly, another species of 5 was also experimentally observed that exhibited a closedshell ground state  In contrast to 5OS, where the frontier orbitals correspond to the SOMOs ψ1 and ψ2, DFT calculations predict orbitals of different shapes and symmetries for 5para and 5ortho, denoted as α and β and shown in Fig   
For 5ortho, α and β correspond to HOMO and LUMO, respectively  The orbitals are inverted in energy and occupation for 5para, where β is the HOMO and α is the LUMO  Fig  shows an AFM image of 5 that we assign as 5para  We experimentally infer its closed-shell state first by using qualitative bond order discrimination by AFM 
In high-resolution AFM imaging, chemical bonds with higher bond order are imaged brighter (that is, with higher frequency shift Δf) due to stronger repulsive forces, and they appear shorter   In Fig  , we label seven bonds whose bond orders show significant qualitative differences in the calculated 5ortho, 5para (Fig  ) and 5OS (Fig  ) geometries 
In 5para, the bonds b and d exhibit a higher bond order than a and c, respectively  This pattern is reversed for 5ortho, while the bond orders of the bonds a-d are all similar and small for 5OS  Furthermore, in 5para bond f exhibits a higher bond order than e, while in 5ortho and 5OS bonds e and f exhibit similar bond order (because they belong to Clar sextets) 
Finally, the bond labeled g shows a higher bond order in 5para than in 5ortho and 5OS  The AFM image of assigned 5para shown in Fig  indicates higher bond orders of the bonds b, d and f compared to a, c and e, respectively  In addition, the bond g appears almost point-like and with enhanced Δf contrast compared to its neighboring bonds, indicative of a high bond order (see Supplementary Fig  for height-dependent measurements) 
These observations concur with the calculated 5para geometry (Fig  )  Importantly, all these distinguishing bond-order differences are distinctly different in the AFM image of 5OS shown in Fig  , which is consistent with the calculated 5OS geometry (Fig  )  In the AFM images of 5OS (Fig  and Supplementary Fig  ), the bonds a-d at the pentagon apices appear with similar contrast and apparent bond length 
The bonds e and f at one of the terminal benzenoid rings also exhibit similar contrast and apparent bond length, while the central bond g appears longer compared to assigned 5para  Further compelling evidence for the closed-shell state of assigned 5para is obtained by STM and STS  dI/dV(V) spectra acquired on an assigned 5para species exhibit two peaks centered at -1 4 V (PIR) and 1 6 V (NIR) (Fig  ) 
STM images acquired at these biases (Fig  ) show the orbital densities of β (-1 4 V) and α (1 6 V)  First, the observation of α and β as the frontier orbitals of this species, and not the SOMOs, strongly indicates its closed-shell state  Second, consistent with AFM measurements that indicate good correspondence to the calculated 5para geometry, we observe β as the HOMO and α as the LUMO 
For 5ortho, α should be observed as the HOMO and β as the LUMO  We did not observe molecules with the signatures of 5ortho in our experiments  We observed molecules in open-(5OS, Fig  ) and closed-shell (5para, Fig  ) states in similar occurrence after their generation from 6 on the surface  We could also switch individual molecules between open-and closed-shell states as shown in Fig  and Supplementary Fig   
To this end, a change in the adsorption site of a molecule was induced by STM imaging at ionic resonances, which often resulted in movement of the molecule  The example presented in Fig  shows a molecule that was switched from 5para to 5OS and back to 5para  The switching is not directed, that is, we cannot choose which of the two species will be formed when changing the adsorption site, and we observed 5OS and 5para in approximately equal yields upon changing the adsorption site 
The molecule in Fig  is adsorbed on top of a defect that stabilizes its adsorption geometry on bilayer NaCl  At defect-free adsorption sites on bilayer NaCl, that is, without a third layer NaCl island or atomic defects in the vicinity of the molecule, 5 could be stably imaged neither by AFM nor by STM at ionic resonances (Supplementary Fig  ) 
Without changing the adsorption site, the state of 5 (open-or closedshell) never changed, including the experiments on bilayer NaCl/Ag(111) and Cu(111), on which the charge state of 5 could be switched (Supplementary Figs  )  Also on these lower work function surfaces, both open-and closed-shell species were observed for 5 0 and both showed charge bistability between 5 0 (5OS or 5para) and 5 -1 (Supplementary Figs  ) 
The geometrical structure of 5 -1 probed by AFM, and its electronic structure probed by STM imaging at the NIR (corresponding to transitions between 5 -1 and the dianionic state 5 -2 ), are identical within the measurement accuracy for the charged species of both 5OS and 5para  When cycling the charge state of 5 between 5 0 and 5 -1 several times, we always observed the same state (5OS or 5para) when returning to 5 0 , provided the molecule did not move during the charging/discharging process 
Based on our experimental observations we conclude that indeno[1,2-a]fluorene (5), the last unknown indenofluorene isomer, can be stabilized in and switched between an open-shell (5OS) and a closed-shell (5para) state on NaCl  For the former, both DFT and QD-NEVPT2 calculations predict a triplet electronic configuration 
Therefore, 5 can be considered to exhibit the spin-crossover effect, involving magnetic switching between high-spin (5OS) and low-spin (5para) states, coupled with a reversible structural transformation  So far, the spin-crossover effect has mainly only been observed in transition-metal-based coordination compounds with a near-octahedral geometry  
The observation that the switching between open-and closedshell states is related to changes in the adsorption site but is not achieved by charge-state cycling alone, indicates that the NaCl surface and local defects facilitate different electronic configurations of 5 depending on the adsorption site 
Gas-phase QD-NEVPT2 calculations predict that 5OS is the ground state, and the closed-shell 5para and 5ortho states are 0 11 and 0 21 eV higher in energy  The experiments, showing bidirectional switching between 5OS and 5para, indicate that a change in the adsorption site can induce sufficient change in the geometry of 5 (leading to a corresponding change in the ground state electronic configuration) and thus induce switching 
Switching between open-and closed-shell states in 5 does not require the breaking or formation of covalent bonds , but a change of adsorption site on NaCl where the molecule is physisorbed  Our results should have implications for single-molecule devices, capitalizing on the altered electronic and chemical properties of a system in π-diradical open-shell and closed-shell states such as frontier orbital and singlet-triplet gaps, and chemical reactivity 
For possible future applications as a single-molecule switch, it might be possible to also switch between open-and closed-shell states by changing the local electric field, such as by using chargeable adsorbates   Scanning probe microscopy measurements and sample preparation  STM and AFM measurements were performed in a home-built system operating at base pressures below 1×10 -10 mbar and a base temperature of 5 K  Bias voltages are provided with respect to the sample 
All STM, AFM and spectroscopy measurements were performed with carbon monoxide (CO) functionalized tips  AFM measurements were performed in non-contact mode with a qPlus sensor   The sensor was operated in frequency modulation mode with a constant oscillation amplitude of 0 5 Å  STM measurements were performed in constantcurrent mode, AFM measurements were performed in constant-height mode with V = 0 V, and I(V) and Δf(V) spectra were acquired in constant-height mode 
Positive (negative) values of the tip-height offset Δz represent tip approach (retraction) from the STM setpoint  All dI/dV(V) spectra are obtained by numerical differentiation of the corresponding I(V) spectra  STM and AFM images, and spectroscopy curves, were post-processed using Gaussian low-pass filters 
Au(111), Ag(111) and Cu(111) surfaces were cleaned by iterative cycles of sputtering with Ne + ions and annealing up to 800 K  NaCl was thermally evaporated on Au(111), Ag(111) and Cu(111) surfaces held at 323 K, 303 K and 283 K, respectively  This protocol results in the growth of predominantly bilayer (100)-terminated islands, with a minority of trilayer islands 
Sub-monolayer coverage of 6 on surfaces was obtained by flashing an oxidized silicon wafer containing the precursor molecules in front of the cold sample in the microscope  CO molecules for tip functionalization were dosed from the gas phase on the cold sample  Density functional theory calculations  DFT was employed using the PSI4 program package  
All molecules with different charge (neutral and anionic) and electronic (open-and closed-shell) states were independently investigated in the gas phase  The B3LYP exchangecorrelation functional with 6-31G basis set was employed for structural relaxation and singlepoint energy calculations  The convergence criteria were set to 10 −4 eV Å −1 for the total forces and 10 −6 eV for the total energies 
Multireference calculations  Multireference calculations were performed on the DFToptimized geometries using the QD-NEVPT2 level of theory , with three singlet roots and one triplet root included in the state-averaged calculation  A (10,10) active space (that is, 10 electrons in 10 orbitals) was used along with the def2-TZVP basis set  
Increasing either the active space size or expanding the basis set resulted in changes of about 50 meV for relative energies of the singlet and triplet states  These calculations were performed using the ORCA program package   Nucleus-independent chemical shift (NICS) calculations  Isotropic nucleus-independent chemical shift values were evaluated at the centre of each ring using the B3LYP exchangecorrelation functional with def2-TZVP basis set using the Gaussian 16 software package  
Starting materials (reagent grade) were purchased from TCI and Sigma-Aldrich and used without further purification  Reactions were carried out in flame-dried glassware and under an inert atmosphere of purified Ar using Schlenk techniques  Thin-layer chromatography (TLC) was performed on Silica Gel 60 F-254 plates (Merck) 
Column chromatography was performed on silica gel (40-60 µm)  Nuclear magnetic resonance (NMR) spectra were recorded on a Bruker Varian Mercury 300 or Bruker Varian Inova 500 spectrometers  Mass spectrometry (MS) data were recorded in a Bruker Micro-TOF spectrometer  The synthesis of compound 6 was developed following the two-step synthetic route shown in Supplementary Fig  , which is based on the preparation of methylene-bridge polyarenes by means of Pd-catalyzed activation of benzylic C-H bonds  
Supplementary Figure | Synthetic route to obtain compound 6  The complex Pd2(dba)3 (20 mg, 0 02 mmol) was added over a deoxygenated mixture of 1,3-dibromo-2,4-dimethylbenzene (9, 100 mg, 0 38 mmol), boronic acid 10 (178 mg, 1 14 mmol), K2CO3 (314 mg, 2 28 mmol) and XPhos (35 mg, 0 08 mmol) in toluene (1:1, 10 mL), and the resulting mixture was heated at 90 °C for 2 h 
After cooling to room temperature, the solvents were evaporated under reduced pressure  The reaction crude was purified by column chromatography (SiO2; hexane:CH2Cl2 9:1) affording 11 (94 mg, 76%) as a colorless oil  The complex Pd(OAc)2 (7 mg, 0 03 mmol) was added over a deoxygenated mixture of terphenyl 11 (90 mg, 0 27 mmol), K2CO3 (114 mg, 0 83 mmol) and ligand L (26 mg, 0 06 mmol) in NMP (2 mL) 
The resulting mixture was heated at 160 °C for 4 h  After cooling to room temperature, H2O (30 mL) was added, and the mixture was extracted with EtOAc (3x15 mL)  The combined organic extracts were dried over anhydrous Na2SO4, filtered, and evaporated under reduced pressure  The reaction crude was purified by column chromatography (SiO2; hexane:CH2Cl2 9:1) affording compound 6 (8 mg, 11%) as a white solid  in AFM imaging due to their reduced adsorption height compared to the rest of the carbon atoms 
We attribute this observation to the significantly different lattice parameter of Cu(111) (2 57 Å) compared to Au(111) and Ag(111) (2 95 Å and 2 94 Å, respectively) , such that the apical carbon atoms of the pentagonal rings of 5 adsorb on the on-top atomic sites on Au(111) and Ag(111), but not on Cu(111) 
Our speculation is based on a previous study of polymers of 1 on Au(111) by Di Giovannantonio et al  , where both tilted and planar individual units of 1 were observed depending on whether the apical carbon atoms of the pentagonal rings in 1 adsorbed on the on-top or hollow sites of the surface, respectively 
Given the strong molecule-metal interaction, we found no electronic state signatures of 5 on all three metal surfaces  STM set point for AFM images: V = 0  e, Frontier orbital spectrum of 5 -1   In the anionic state, ψ2 becomes doubly occupied and ψ1 is the SOMO  Filled and empty circles denote occupied and empty orbitals, respectively 
For each panel, zero of the energy axis has been aligned to the respective highest-energy occupied orbital 



ground_truth: 48V.