InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma
Authors: Xiaoxuan Hou
Jiayi Yuan
Joel Z Leibo
Natasha Jaques
Venue: ICLR 2025 Poster (2025)
Keywords: multi-agent reinforcement learning
climate change
ai for climate
Primary area: datasets and benchmarks
TL;DR: We introduced, InvestESG, a novel multi-agent reinforcement learning (MARL) benchmark designed to study the impact of Environmental, Social, and Governance (ESG) disclosure mandates on corporate climate investments.

Abstract: **InvestESG** is a novel multi-agent reinforcement learning (MARL) benchmark designed to study the impact of Environmental, Social, and Governance (ESG) disclosure mandates on corporate climate investments. The benchmark models an intertemporal social dilemma where companies balance short-term profit losses from climate mitigation efforts and long-term benefits from reducing climate risk, while ESG-conscious investors attempt to influence corporate behavior through their investment decisions. Companies allocate capital across mitigation, greenwashing, and resilience, with varying strategies influencing climate outcomes and investor preferences. We are releasing open-source versions of InvestESG in both PyTorch and JAX, which enable scalable and hardware-accelerated simulations for investigating competing incentives in mitigate climate change. Our experiments show that without ESG-conscious investors with sufficient capital, corporate mitigation efforts remain limited under the disclosure mandate. However, when a critical mass of investors prioritizes ESG, corporate cooperation increases, which in turn reduces climate risks and enhances long-term financial stability. Additionally, providing more information about global climate risks encourages companies to invest more in mitigation, even without investor involvement. Our findings align with empirical research using real-world data, highlighting MARL's potential to inform policy by providing insights into large-scale socio-economic challenges through efficient testing of alternative policy and market designs.

Published as a conference paper at ICLR 2025
INVESTESG: A MULTI-AGENT REINFORCEMENT LEARNING BENCHMARK FOR STUDYING CLIMATE
INVESTMENT AS A SOCIAL DILEMMA
Xiaoxuan Hou" Jiayi Yuan" Joel Leibo", Natasha Jaques?.3
Foster School of Business, University of Washington
Paul G. Allen School of Computer Science and Engineering, University of Washington 'Google Deepmind
xxhou@uw edu, jiayiy9lcs wttpees.wwhiingtoneeeeee jal8google.com
nje njecs. washington. edu
ABSTRACT
InvestESG is novel multi-agent reinforcement learning (MARL) benchmark designed to study the impact of Environmental, Social, and Govermance (ESG) disclosure mandates corporate climate investments. The benchmark models intertemporal social dilemma where companies balance short-term profit losses from climate mitigation efforts and long-term benefits from reducing climate risk, while ESG-conseious investors attempt to influence corporate behavior through their investment decisions. Companies allocate capital across mitigation, green- washing, and resilience, with varying strategies influencing climate outcomes and investor preferences. We are releasing open-source versions of InvestESG in both PyTorch and JAX[ which enable scalable and aerdware-accelraate simulations for investigating competing incentives in mitigate climate change. Our exper- iments show that without ESG-conscious investors with sufficient capital, cor- porate mitigation efforts remain limited under the disclosure mandate. However, when critical mass of investors prioritizes ESG, corporate cooperation increases, which in turn reduces climate risks and enhances long-term financial stability. Additionally, providing more information about global climate risks encourages companies to invest more mitigation, even without investor involvement. Our findings align with empirical research using real-world data, highlighting MARL's potential to inform policy by providing insights into large-scale socio-cconomic challenges through efficient testing of alternative policy and market designs.
INTRODUCTION
Climate change poses persistent threat to global stability, with droughts, storms, fires, and flooding becoming more intense and frequent (Christopher B Field & Dahe 2012), leading disruption of the natural ecosystem and significant impacts the global economy. Addressing climate change requires coordinated efforts across multiple sectors, particularly from large corporations, which are reportedly responsible for over 70% of global industrial greenhouse gas emissions (Griffin & Heede 2017]. While adaptation adaptation--preparnng for the inevitable consequences of climate change tends to be party-specific and often driven by financial incentives, mitigation -rrruciion-rruuiinn reducing
an intertemporal social dilemma (Leibo et al. 2017 Hughes et al. 2018), where the benefits of re- duced emissions shared globally yet the costs borne locally OOlson Jr 1971 Dahlman 1979 Buchanan & Stubblebine/ 2006). As corporations are inherently self-interested, they are unlikely to reduce emissions voluntarily without external incentives or regulations.
Numerous policies have been proposed to address this challenge. Among these, mandatory Environ- mental, Social, and Governance (ESG) disclosures have recently been hotly debated the United States. The Securities and Exchange Commission's 's (SEC) ESG proposal, which would require pub- licly traded companies disclose climate-related risks, mitigation strategies, and greenhouse gas
Github repo: http:////////tthh..oo//aaaaaaaaa Invest

---

Published as a conference paper at ICLR 2025
Mandate ESG disclosure
decision based on corporate profit margins ESG
ESG Score
Government/Puble Sectors
Invest in emission mit mitiaaiion
Invest in
Invest greenwashing
Investors
Corporations
The InvestESG Environment. Corporations choose how much to invest in mitigating emissions, which affects their ESG Score. Climate-conscious investors can see ESG Scores when deciding how much to invest in each company. However, companies can engage in greenwashing inexpensively and falsely improve ESG scores without actually mitigating climate change. InvestESG is social dilemma, where selfish, profit- motivated corporations will not invest mitigation without further incentives, leading to increased climate risks and decreased global wealth.
Figure I:
emissions from their operations, has attracted over 15,000 comments, making one of the most contentious proposals in the SEC's history (SEC) 2024b| CNBC 2024). This has resulted in an indefinite delay in enactment the policy to allow for further discussion (SEC) 2024a). The U.S. is not alone in facing such pushback; similar delays are unfolding in the European Union and Ko- rea (Bloomberg News] 2024; Korea Economic Daily 2023). This highlights the need for thorough research to effectively inform the design and implementation of these policies.
Traditional economics and policy research relies on either empirical analysis which which does not en- able testing possible new policies (Doshi et al. 2013 Li & Wul [2200] Krueger al. [ or theoretical economics models, which are often limited to scenarios with only two agents (e.g.. Friedman et al. (202I), or single-period games (e.g., Pástor et (02I). In contrast, Multi-Agent Reinforcement Learning (MARL) enables simulating complex interactions between multiple agents over extended time periods, under diverse hypothesized policy settings. Leveraging MARL to ad- dress large-scale scale socio- economic questions is growing field (Hertz et al.] (023). Prior work has demonstrated the potential of MARL design effective taxation schemes that enhance both equality and productivity (Zheng et al. 2021), highlighting its relevance for tackling social challenges.
We propose using multi-agent reinforcement learning (MARL) to explore the impact of the ESG dis- closure policy. We introduce InvestESG, an open-source MARL benchmark, to examine how profit- driven corporations balance short-term profits with long-term climate investments and whether ESG- informed investor choices influence corporate behavior. The simulation involves two agent types: companies and investors. Companies allocate funds to mitigation, greenwashing, and resilience, while investors choose investment portfolios based on their preferences for financial returns versus ESG benefits. This creates an intertemporal social dilemma (Hughes et al.) 2018), where both agent classes must weigh immediate and local costs against long-term and global gains. Using Schelling diagrams, we demonstrate that in a fully profit-driven environment, social dilemma arises, but with sufficiently ESG-conscious investors with enough capital, climate mitigation becomes optimal for some or all corporations. However, if companies are able to greemwash to cheaply increase ESG scores without genuine mitigation, the environment once again becomes a social dilemma.
Our experiments with state-of the- MARL implementation yield findings that align with real- world empirical evidence and provide novel insights. First, a sufficient number of highly ESG- conseious investors are needed to incentivize corporate mitigation efforts. In such cases, climate- focused companies emerge and attract most climate-conscious investments, while others prioritize profit maximization. When only some investors are climate-foussed the market bifurcates, with mit- igating companies aligning with conscious investors. Additionally, sharing climate risk information helps companies to increase mitigation efforts, even when investors are not present.
More broadly, we demonstrate the potential of using a MARL framework to inform policy debates in the field of climate change. Importantly, our results compatible broadly with the existing theoretical and empirical literature that examines this question, which shows its validity in predict- ing directional behaviors of rational decision makers. Assessing the effectiveness of policy is
2

---

Published as a conference paper at ICLR 2025
inherently challenging, due to the fact that policy experiments are often prohibitively expensive and impractical to conduct, and even when they are feasible, it can be extremely time-conumiing Given work provides a vector for studying this prob-
the urgency of addressing climate change, lem, creating a simulated environment where a broad range of regulations can be explored and tested efficiently provide novel insights into the problem.
Our aim is not to claim that our model fully represents real-world complexity. It remains a "first- principles" model; however, it captures aspects that are challenging to integrate into a single frame- work using traditional economic approach. We present InvestESG as a challenge for the machine learning community: for researchers interested in developing better cooperative agents, or improved MARL algorithms, InvestESG represents a benchmark that could actually inform policymaking.
RELATED WORK
Creating a benchmark environment that analyzes the interplay between corporations and investors and their impacts on climate change mitigation requires various domain knowledge and connects multiple streams of studies. We closely examined three streams of literature.
Conventional economic methods are limited by either generalizability or tractability. Existing ESG disclosure research in economics, business, and public policy rely on either empirical data (Doshi et al. 2013; Li & Wu| 2020 Krueger et al. (02T) or simplified theoretical models (Polinsky & Shavell|[201 Kalkanci & Plambeck [2200] Cho et al., 2019 Pástor et al. 2021 Friedman et al., 2021]. Empirical analyses, while grounded in weal-world data, struggle with generalizability and testing counterfactual policies. Theoretical models provide formalized equilibria and explore coun- terfactuals but are limited by tractability, modeling either multiple agents in single-period games et al. 202T) or only two agents over limited time periods (e.g.. FFiedman et al. 2021). By proposing a MARL framework, our method overcomes these limitations by enabling the simu- lation of complex. multi-agent systems over extended time horizons under diverse policy settings, which allows for emergent behaviors and better captures complex socio-economic dynamics among diverse agents (Hertz et al. 2023).
Current MARL benchmarks and social dilemma environments were not designed to model specific policy problems. Various MARL benchmarks have been created study multi-agent co-
ordination and cooperation; however, they are often limited to simplistic particle simulations (Lowe)
al. 2017 or vidcogames (Agapiou al. 2023] Carroll et al. 2020 Samvelyan et al. 2019 Bettini it al. 2024b a). which have little direct real-world implication. Sequential social dilemmas (SSD)
(Leibo et al. 2017) are spatially and temporally extended multi-agent environments in which the payoff to an individual agent for defecting is higher, but if all agents defect the payoff is lower. SSDs go beyond traditional game-theoretic environments like Prisoner's Dilemma because the com- plexity of the solving the SSD depends on not only addressing the misalignment between individual and collective rationality, but doing so when the negative consequences short-sighted actions may take long time to manifest. Prior research has examined methods to promote cooperation in SSDs by incorporating inequity aversion in agents (Hughes et al. 22018), rewarding agents for influencing others' actions (Jaques et al. 2018), enabling agents provide incentives to others (Yang et al., 2020a), and controlling for behavioral diversity (Bettini et al. 2024a).. Many of these studies are in- spired by factors that drive human cooperation in social dilemmas. However, much of this research has been conducted in environments with no direct real-world implications (e.g. [eibo et al. 2017]] 2021. In contrast, our environment directly addresses the problem of climate change, and is de- signed with critical trade-offs around a specific policy question. We hope to encourage researchers interested in addressing social dilemmas focus on an environment with potential beneficial social impact on the problem of climate change (Bisaro & Hinkel| 2016).
RL benchmarks can be effective in focusing AI research on climate change issues. Learning to Run a Power Network (L2RPN) is single-agent RL. benchmark focused on improving power grid efficiency (Marot et al. 202T). This work has spawned multiple competitions, and it is cur- rently hosted by the Electric Power Research Institute (EPRI) in conjunction with several other energy companies, government agencies, and universities L2RPN continues to foster innovative and meaningful collaboration across institutions, showing the potential impact of this type of sim-
httpp::/ httpss//www.eprt com/12rpn
3

---

Published as a conference paper at ICLR 2025
plified, simulated RL benchmark. However, L2RPN is focused single-agent RL, whereas explore a multi-agent, multi-party social dilemma. The only other MARL climate benchmark we aware of was proposed by Zhang et al. (2022), and is focused studying the dynamics of in- ternational climate negotiations, by incorporating an integrated assessment model simulating global climate and economic systems. In contrast, our environment, InvestESG, was designed with focus on a more targeted policy question regarding ESG disclosure mandates, making our approach more directly applicable an ongoing and highly debated policy issue.
PROBLEM SETTING
Corporations face two key trade-off's in climate action. The first is between short-term costs and long-term benefits, as climate efforts often require upfront investments with returns realized over time. The second is between self-interest and collective benefit, as emission reductions benefit all, but the costs are borne by those who act, which incentivizes free-riding and discourages collective action. These trade-offs motivate us to conceptualize corporate climate actions as an intertemporal social dilemma (Leibo et al.) 2017 Bisaro & Hinkel, 2016), which by definition, involves a conflict between immediate self-interest and longer-term collective interests.
We introduce InvestESG, a MARL benchmark environment designed to evaluate the impact of ESG disclosure mandates on corporate climate investments. The environment is grounded well- established finance and economics literature that explores similar questions, providing a conceptual framework for studying corporate and investor dynamics in response to ESG policies. Pastor et al. (2021) and Pedersen et al. (2021) proposed analytical models of single-period equilibria, where companies either choose or are assigned fixed levels of "greenness," and investors optimize portfo- lios to balance financial returns with preferences for green assets. We follow a similar setting where companies choose mitigation effort levels every period and receive a single metric ESG score that reflects their greenness. Investors derive utility from the profits generated by their portfolios and, based on their level of LESG--onncciounnee with high ESG scores. We extend these frameworks by (1) modeling the evolution of firm-investor strategies over the long run, which poses significant analytical and numerical challenges (Pakes) McGuiree 200T), (2) incorporating climate evolution, where risks worsen over time but can be mitigated by firm efforts, and (3) introducing greenwashing (Lyon & Maxwell 2011) where com- panies cheaply inflate ESG scores without genuine mitigation, which raises issues of information asymmetry and principal-agent problems (Grossman & Hart 1992).
may also receive positive utility by investing in firms
We aim to address the following research questions: (1) Do ESG-conscious investors incentivize companies undertake mitigation efforts, and how does the level of ESG-consciousness influence these outcomes? (2) Does the presence of heterogeneous investors lead to bifurcation in agent strate- gies, where only some companies engage in mitigation attract ESG-conscious investors, while others and their investors opt to free-ride? (3) Are companies likely to engage in greenwashing? (4) What measures can enhance the effectiveness of ESG disclosure mandates?
THE INVESTESG ENVIRONMENT
In this section, we provide a brief overview of the environment. The detailed mathematical for- mulation is presented in Appendix 9 The simulation starts in 2021 and runs through 2120, with each period corresponding one year. The environment includes two main components: (1) an evolving climate and economic system, and (2) two types of agents: M company agents C, for i {I..... } and N investor agents I, for j E [1.....N).
Climate and Economic Dynamics. The environment is initialized with a climate risk parameter, Po. representing the probability of at least one adverse climate event occurring. The value of Po is benchmarked against Masson-Delmottee et al. (202T), reflecting the likelihood of extreme heat, heavy precipitation, and drought events. In the absence of mitigation efforts, climate risk increases over time, aligning with the IPCC's 4 C warming scenario. Figure 2al depicts how increased cli- mate risks and adverse climate events increase over time in a scenario where companies are solely profit-motivated. Company agents mitigate the growth of climate risk by investing in emissions mirigation. Climate events are modeled as Bernoulii processes, with their probabilities determined by the climate risk of the corresponding year. Multiple events occur within single year, illus-
4

---

Published as a conference paper at ICLR 2025
trated by the red dashed lines in Figure 2a In addition to the evolving climate risks, the environment incorporates a baseline economic growth rate 7, set to 10% by default, aligned with the historical average annual return of the S&P 500 over the past century (Damodaran 2024a). Company agents' capital levels K grow rate ? each year, barring climate events. If adverse event occurs, company agents lose a portion of their total capital according to their respective climate resilience parameter L economic losses drive a company's 's remaining capital into negative territory, the company is declared bankrupt.
Action Space. Each company agent C; in period selects ac- tions from a continuous vector u (4f, "g.g. ) where uC,,, represents the share of capital allocated to mitigation, ui to greenwashing, and 1 to building climate resilience.
Overall Metrics Over Time
Cursdative Es0 invesmment
Climaee Ria
Clmate mobossty
Each investor agent first selects the set of companies it will invest in. Specifically, agent Zj in period t selects an action from binary vector of length M, a, corresponding to the M company agents.
Investmeit
(a..1--... (,M))
Modeling ESG Disclosure. With the ESG disclosure mandate in place, each company agent receives an updated ESG score p Q in period t, calculated Q ",,m + Buk, .g* based on mitigation and greenwash spending, where 3 > indicates
(a) Environment dynamics over the course of single 100 year episode.
Severe Climate Events Over 120-barl
that greenwashing is cheaper than genuine mitigation in terms of building an ESG-friendly image.
Awrage Sever fuems
The environment simulates partially observable Markov game M defined over contin- uous, multi-dimensional state space. The system state at pe- riod t is characterized by the climate risk parameter P., each company agent's state vector, consistent of its capital level, its ESG, and its climate resilience. Each investor agent's state represented by their investment portfolio and cash levels. All company and investor agents share a common observation space, with the state of all companies and investors. Exten- sions that incorporate additional observable information, like climate risk, are explored in Section 5
State and Observation Space.
(b) Average number of severe climate event occurrences over 10-ycar period for the scenario part (a).
Figure 2:
Status quo scenario where all agents are only profit-motivated. In (a), mitigation spending (blue curve) is minimal, leading climate risk (yel- low curve) to increase over time. Ad- verse weather event occurrences are shown dotted lines; red lines indi- cate multiple adverse events a sin- gle year. (b) plots the average number severe climate events over the episode in (a), showing how increasing climate risk leads to more frequent extreme weather events.
State Transition. At the beginning of period f, investors col- lect their investment holdings from period f- 1 and redistribute their capital according to a If an investor opts not to invest, all capital remains as cash. Companies then make climate- related spending, after which system climate risk and compa- nies' individual resilience are updated. Meanwhile, compa- nies receive updated ESG scores, and the occurrence of climate events are simulated. Afterwards, companies' profit margins, Pt are computed as a function of their climate-related spend- ing c. default economic growth 7, and losses due to climate events, L.C and the number of climate events X₁. Finally, company capitals are updated, and so are investor holdings and cash based on the profit margins of their portfolios.
Rewards.
The single-perion reward for company C, solely based on its profit, reflecting the assumption that companies are profit-driven. The reward for in- vestor Z, is the summation of two components: (1) the portfolio profit margin, and (2) the weighted average ESG score of the investor's portfolio multiplied by the investor's ESG preference, a
Social Outcome Metrics. We evaluate agent performance based on two key social outcome met- rics: the final climate risk level, Puuo, and the total market wealth at the end of the period, W W100- Additionally, we keep track of other social outcome metrics such as total number of severe climate events and number of companies that go bankrupt.
5

---

Published as a conference paper at ICLR 2025
0000
colvoar Flydl
Peyal Comportla Rayer
607
Delecte
Cray Road
Payal
1.00
Cuppeitar Papor
Compinato Neyal
Hn cooceting
Sarae Goppaaiin
Coppene sowei
of
0/3 Conscious Investors
(h) 2/3 Conscious Investors
(c) 3/3 Conscious Investors
(d) 3/3 Conscious &
(e) 3/3 Conscious & Greenwashing
Resilience Spending
Figure 3: Schelling diagrams demonstrating that the environment constitutes social dilemma. The graphs compare payolI's between cooperation (mitigation, blue lines) and defection (no mitigation. red lines) for focal company, given varying number of other cooperating companies. Yellow lines represent the average payoff across all companies when the focal company defects. Subfigure (a) illustrates the selfish scenario, where all three investors consistently prioritize financial returns (a = 0, for j = 1, 3). Here, defection always yiclds higher payoffs for the focal company than cooperation, leading all companies defect. However, widespread defection results in lower overall profits, as the average payoff (yellow) increases with greater cooperation, demonstrating the environment constitutes a social dilemma.. Subfigure (b) and (c) correspond to two and three infinitely ESG-consee CSo-conscious investors (a = 20), respectively. In (b). cooperation yields higher payoff's than defection for the focal company when few others cooperate. In (c), cooperation outperforms defection in all cases. Therefore, (b-c) demonstrate how investor behavior can transform the environment, eliminating the social dilemma by aligning corporate incentives with mitigation. Subfigures (d) and (c) build on (c) with three ESG-conscious investors. Subfigure (d) introduces resilience spending, while (c) adds greenwashing. The latter reintroduces a social dilemma, where corporations again avoid mitigation.
4.1 THE TRADE-OFF FOR THE AGENTS FRAMES CLIMATE CHANGE AS SOCIAL DILEMMA
Companies' trade-off between short-term private costs and long-term collective benefits create a social dilemma within the enviromment. To illustrate this, we use empirical Schelling diagrams (Hughes et al., 2018), which plot the payoff of following either a cooperative or defecting policy, depending on the number of other cooperating company agents in a 5-company, 3-investor scenario.
In our case, cooperation represents altruistic investment in mitigating emissions. We simulate a co- operative policy as one which consistently invests 0.5% of capital in mitigation and 0 in greenwash- ing or resilience building. In Figure 3a to 3c a defector policy takes zero action in all of mitigation, greenwashing, and resilience. As shown in Figure Ba when investors are solely profit-driven, the payoff for following defector policy is always higher than the cooperator payoff. However, if all agents fail to cooperate, overall payoff's will be extremely low, showing that the environment repre- sents social dilemma. Figures 3b) and [c illustrate how the payoffss for cooperation and defection evolve as the proportion of ESG-conscious investors with positive ESG preferences increases. No- tably, Figure 3bb suggests the possibility of a bifurcated equilibrium when investor preferences are mixed. Specifically, climate-friendly companies may attract ESG-conscious investors, while other companies free ride and attract profit-motivate investors. Figure 3c shows that when all investors prioritize ESG-friendly firms, cooperation dominates defection in all cases. Therefore, with enough ESG-conscious investors the environment is no longer social dilemma, and companies are actually greedily motivated to invest in mitigation to attract investors.
Although Figure 3c appears promising, the option to greenwash and spend on resilience complicates the tradc-offs companies face. In Figure 3el we define the defection policy as investing 0.5% of capital annually in resilience. This makes defection slightly more attractive compared to Figure [3]] as resilience spending results in higher capital gains than taking no action. Figure 3d presents an alternative scenario where defecting com- panies invest 0.3% annually in greenwashing" In this case, cooperation becomes significantly more challenging. because companies can attract ESG-conscious investors without actually mitigating their emissions. The addition of greenwashing turns the environmental back into social dilemma.
Greenwashing and Climate Resilience Investment.
"Since company actions are continuous, we selected 0.5% a benchmark for illustrating the social dilemma structure in the Schelling diagram. This aligns with real examples of cooperative corporations, such as Patag- onia, where ~0.3% of its valuation is allocated annually to climate initiatives. The diagram's 's shape remains consistent across other tested values.
'For the Schelling diagram, we set the greenwashing coefficient 3 = 2. As result, 0.3% spending on greenwashing leads to higher ESG score than 0.5% spent on mitigation.

---

Published as a conference paper at ICLR 2025
Checle
Figure 4: Ending values for all metrics averaged over the last 100 episodes; error bars show std. eIr. over 3 random seeds. We compare the status quo scenario with solely profit-driven investors (investors with ESG consciousness level of 0), both with and without the ESG disclosure mandate, scenarios involving three ESG-conscious investors with ESG consciousness level of a = 0.5, a = 1, and a = 10. These results indicate that merely disclosing ESG scores is insufficient to resolve the social dilemma if investors are not interested in investing in climate- friendly companies. However as investors' level of ESG consciousness increases, the ESG mandate results in consistent improvements in mitigation, climate risk, and market wealth.
4.2 MGENI-AGENT REINFORCEMENT LEARNING BASELINES
We are motivated to simulate companies and investors as independent, selfishly motivated agents that specialize in maximizing their own expected reward. We employ the state-of-the-art Independent PPO (IPPO) algorithm (De Witt et al., 2020; Yu et al. 2022) so each agent has its own policy parameters, and agents do not share parameters among themselves. InvestESG is implemented in both PyTorch (Paszke et al. 2017) and JAX (Bradbury et al., 2018), capable of simulating the investment behaviors in massively parallel sense by vectorization on both CPUs and GPUs. By default, 5 companies are modeled with an initial capital of $10 trillion each, alongside 3 investors, each starting with $16 trillion. This creates total beginning market wealth of $98 trillion, which is comparable to the global stock market cap (WFE Statistics 2023). We conduct the experiments in 5-company, 3-investor environment by default unless specified otherwise. All the following results are averaged based on 3 runs of different random seeds, with the standard error plotted in the shaded region Fl. Following economic literature that argues more than 10 agents are often needed to observe meaningful group size effects (Arifovic et al. 2023), we provide additional results in Appendix [T] from settings with 10 companies and 10 investors, as well as 25 companics and 25 investors, which are consistent with our findings with the 5-company, 3-investor setting?
5
MAIN RESULTS
Level of investors' ESG-consciousness increases mitigation efforts. To evaluate the effectiveness
of ESG disclosure mandates in incentivizing emissions mitigation by companies, we conduct three experiments: (1) Status Quo: All agents are profit-motivated, and no ESG scores are released; (2)
Status Quo with Mandate: ESG scores are disclosed, but investors remain profit-driven (a = 0): (3) Mandate with ESG- CCG-Consssous Investors: ESG scores are disclosed, and investors are ESG-conscious
(a > 0). In this first set of experiments, we study the default setting in which both greenwashing and resilience spending are disabled.
The final system climate risks are shown in Figure 4 The values for Status Quo with Mandate and Status Quo are similar, indicating that mandatory ESG disclosure alone does not significantly incentivize mitigation when investors prioritize profits. However, we see that mitigation spending increases consistently corresponding to each increase ESGG-cnnsciousess level (ar 0.5, 1, 10), leading to corresponding improvements climate risks and market wealth, with the best values
*In simulating climate events as described in Equation 2. we use a fixed random seed across training episodes. This reduces stochasticity while ensuring event occurrence varies with cpisode-specific climate risks. This design choice is based on two considerations. First, unlike MARL agents lcarning from scratch, real- world companies possess baseline understanding of trade-offs and can reasonably anticipate future risks; more predictable training environment mirrors this insight. Second, our study aims not to develop agents that solve climate change under all stochastic scenarios, but rather to efficiently train agents that grasp the strategic trade-offs faced by companies and investors. Limiting stochasticity makes learning and convergence easier without compromising the study's 's structural conclusions. This design is also consistent with classic versions of climate-ecen econoeiccoocccccccc simulation models like DICE and RICE, where the link between economic development and CO2 mass and the damage functions of global warming are largely deterministic (Nordhaus] 1992 Nordhaus Yang 1996; Nordhaus (017). The implementation also includes the option to not use a fixed random seed. "hh 25-by-25 agent setting aligns with Zhang et al. (2022), which employs 27 agents in a MARI. frame-
work to address climate change issues.
7

---

Published as a conference paper at ICLR 2025
Inwestor & ESG coroc ouunnand
Invester L: comccosanenrii
Inwestor 22 coc
22
106
(a) Investment matrix the latest episode for investors with different level of ESG con- sciousness: 0, 10, 10.
(b) Investors with different level of ESG consciousness make different investment decisions.
Figure 5: Investigating the effects the level of ESG consciousness in the case of companies and investors, where investor O is profit driven (a 0), and investors 1 and 2 are deeply climate-conscious (af a²² = 10). In (a) and (b). Company 0 (purple) learns be the leading mitigator. The figures plot the investors focus investing in
investment distribution for cach investor, showing that more the more conseciou companies, mirroring the market bifurcation results of the Schelling diagrams.
Total Mitigation Greenwoch Amount
Total Mitgation Greemaash Ansount
140
130
100
200
100
0.5
100
50k
100
200
(a) Mitigation and greenwash- ing amount for greenwashing coefficient 2, 10, 20.
(b) Mitigation and greenwash- ing amount, zoomed in for early episodes.
(c) Climate risk for various
greenwashing coefficient, com- pared with status quo and status quo with esg=10.
Figure 6: Mitigation, greenwashing, and climate risk for greenwashing coefficient = 2, 10, 20, with initial exploration episodes zoomed in in (b). It shows that when both companies and investors are IPPO agents, regardless of greenwashing cost, companies initially explore greenwashing and mitigation equally but quickly abandon greenwashing and invest in mitigation to attract ESG-conscious investors. In these experiments, the final climate risk is similar to the value when greenwashing is not enabled, suggesting greenwashing does not hinder mitigation investment.
achieved for a 10. In this setting, a few leading companies attract the majority ESG-conscious investments, forming positive feedback loop where better market performance draws further in- vestments (see Appendix Figure and TOd] for a demonstration for such pattern in the compa- nies, 10 investors case). These findings align with research showing that ESG disclosure mandates encourage Siimate-friendly efforts driven by societal and stakeholder pressures (Cormier & Magnan 1999 Gamerschlag et al. 2011 Fama & French [007]] Friedman & Heinle 20e6 Chen al. 2018 Ioannou & Serafeim 2019), ,, and that fund managers are willing to sacrifice financial returns for ESG
benefits (Krueger et al. 20
Heterogeneous investor preferences lead to bifurcation in agent strategies. Research shows that investors have varying preferences for ESG efforts and respond in different ways (Amel-Zadeh & Serafeim (018).. In the scenario plotted in Figure we initialized three investors with different lev- els of ESG-conssiosssssss, with Investor representing the solely profit-sceking investor (a 0), and Investor I and 2 representing highly ESG-conscious investors (az a²a 10), which corre- sponds the scenario presented in the Schelling diagram in Figure 3b Figure ba and 6c highhighh divergence in both investor and company behavior. Profit-driven Investor O distributes investments firms,
evenly across companies, while ESG-focused investors (1 and 2) favor dimate-conscionn such as Company O, which prioritizes mitigation. Such divergence extends to company strategies: Company learns to attract more ESG investment by focusing on mitigation, while others prioritize financial returns at the expense of some investor interest.
The possibility of greenwashing does not significantly undermine mitigation efforts. Building on research that examines the existence and extent of greenwashing (Wu et al. 2020 Marquis al.
2016 Siano et al. 2017). and the concern that the ESG disclosure policy can backfire with green- explore whether companies adjust their strategies when greenwashing is permitted. Based on the Schelling diagram in Figure 33 and Figure we predict that investors would be misled by greenwashing, prompting companies prioritize greenwashing over mitigation
washing Hale 202
8

---

Published as a conference paper at ICLR 2025
Marterrlete
Her Merkiirie
Clinuta Bax
Total
sdbo
6000
6000
23
30001
3000
24
290
2004
2000
9650
2dbo
9:50
(a) More information for investors and companies
(b) More information for companies (no investors)
Figure 7: Effect providing more information about climate risk to both investors and companies in the default 5-company--iivesstor case (a), or companies only in a Accompay---investte case (b). These results show that simply providing more information about climate risk to companies can help them coordinate to increase mitigation efforts.
due to its lower cost, leading to waste of resources. To verify greenwashing can attract climate- conscious investors in our simulation, we hard-code companies to follow either greenwashing or mitigation strategy. Figure P shows how in this scenario the greenwashing company earns a higher ESG score and succeeds in attracting climate-conscious investors. However, this trend is not ob- served when simulating agents with IPPO. Figure plots results under varying greenwashing costs, represented by coefficient 3 in Equation 3 where larger indicates cheaper greenwashing. All investors have an ESG-consciousnes. level of = 1. The results show that regardless of green- washing cost, companies initially explore greenwashing and mitigation equally but quickly drop greenwashing as the main strategy. This likely occurs because, in the early episodes, investors have not yet linked ESG scores to their investment strategies, so do not respond to greenwashing ef- forts with increased investment. Instead, greenwashing presents an immediate cost for companies, which does not pay off with reduced climate risk, so they quickly learn to avoid it. In contrast, even without immediate investor rewards, companies may recognize the long-term benefits of mit- igation (which leads to higher collective returns), and be incentivized to continue those efforts at first, even if they later learn to defect. This mirrors reality, where if investors are slow to adjust their strategies, companies may abandon greenwashing early but persist in low levels of mitigation for either its long-term climate benefits or the foresight of regulations and societal pressure that will eventually nudge them towards sustainable operation. This is consistent with empirical litera- ture that despite the emergence of some greenwashing, ESG disclosure mandates overall encourage mitigation (Fiechter et al. 2022).
Providing additional information about climate risk increases mitigation efforts. In this sce- nario, we test whether providing additional climate-related information to agents can affect mitiga- tion behavior. Therefore, we provide the climate event probability and climate event occurrences as additional information in the observation space for both companies and investors. In the default 5 compaan--3-nnvettee scenario, having additional climate-related information reduces the ending system climate risks, as shown in Figure 7a Figure 7b companies are provided with the same additional climate-related information. Here climate infor- mation leads to a large reduction in climate risk, with the final risk (=s 0.7) comparable the risk achieved in the a = 1 scenario in Figure 4 This result suggests that dlimate-related information helps both companies and investors make better environmental decisions. Even in the absence of ESG-focused investors, educating companies about the overall system can significantly improve outcomes. This aligns with literature showing that raising awareness encourages positive corporate responses (Delmas & Toffel 2008 Bowen 2000).
tests a scenario with no investors, in which
Total
Climate Bick
Market Total Moalth
2000
35000
33000
0.7
5300
0.
0.5
Sploode
70k
422
225
1001
20%
43%
30c
50k
1001
(a) Mitigation resilience spending
(b) Climate risk
(c) Market total wealth
Figure& 8: Effect of allowing resilience spending. When resilience is allowed, companies invest significantly more in resilience than in mitigation, resulting in ending episode higher market wealth, and lower climate risk.
Effects of resilience spending, In this case, we allow companies to invest in resilience spending to improve their own robustness climate events, without affecting the global climate risk. Although

---

Published as a conference paper at ICLR 2025
investors continue to respond to companies' ESG scores, resilience spending is excluded from the calculation of ESG scores. As shown in Figure g when resilience allowed, companies invest significantly more in resilience than in mitigation. By investing in resilience, companies are more resilient to the climate event and therefore are able to maintain more capital to invest in actual mitigation, compared to the case when resilience is not allowed, reflected inf8c] Therefore mitigation spending when resilience is allowed is actually slightly higher compared to when it is not an option, resulting in comparable final climate risk in Figure 8b This suggests that companies are financially incentivized to prioritize resilience investments, which can actually enable greater commitment to climate mitigation efforts.
Additional experiments. In addition to the experiments presented above, we also explore settings where (1) company agents' actions are initialized with data from real-world companies, (2) agents' decisions are locked in for five years to simulate capital flexibility challenges, (3) companies' climate resilience parameter E are random and vary across events and companies, and (4) the conditions for companies going bankrupt are made more strict. These experiments yield directional conclusions consistent with those presented in the main text. Detailed results can be found in Appendix []
DISCUSSION AND CONCLUSION
In this paper, we introduce InvestESG, MARL environment simulating long-term interactions between companies and investors under varying ESG disclosure policies. For policymakers, In- vestESG shows that mandatory ESG disclosure, paired with highly ESG-conscious investors, can drive corporate mitigation efforts. Additionally, greenwashing poses lesser challenge than antic- ipated, and providing high-quality system-wide information effectively motivates action from both corporations and investors. For economies and policy researchers, InvestESG introduces MARL as a promising tool to complement traditional empirical and theoretical methods, allowing scalable policy testing in a simulated environment. Our model predicts agent behaviors consistent with em- pirical evidence and uncovers novel insights. For the machine learning community, InvestESG presents a novel multi-agent benchmark, fostering the development of RL algorithms that tackle complex social dilemmas, competition, and long-term strateg, and pushing forward AI applications in real-world, high-impact domains. We encourage the machine learning community to develop algorithmic innovations for InvestESG that can inspire practical actions to address climate change.
Limitations. We acknowledge that InvestESG simplifies various aspects of real-world climate evo- lution, financial markets, and regulatory frameworks. These simplifications were intentional for two reasons: (1) rather than fully simulating the global market and climate complexities, which arguably infeasible for single model, we aim to develop "ffrst-principles" model that offers di- rectional insights for policy design; (2) we designed the benchmark to be simple enough for the ML community to iterate with reasonable computational resources (e.g., single GPU). By lowering computational barriers, we hope to encourage broader participation from the ML. community.
At this stage, InvestESG effectively captures the core incentive structures necessary to evaluate the policy in question. We anticipate future improvements to include more complexity, allowing researchers and policymakers select the appropriate level of granularity for their needs.
Future Work. Our goal is to spark discussions and evolve InvestESG into a robust tool for policy design, with input from experts in machine learning, climate change, and economics. We envision three key areas for future work: (1) Richer environment. For instance, we plan to incorporate employees and customers, who may also be attracted to more climate-friendly companies. We also envision capturing complex market and environmental dynamics between industries (e.g., include insurance companies (Brogi al. 2022 ), and enhancing investor decision- making to move beyond binary choices to better reflect real-world complexities. (2) Enhanced policy design. We propose testing policies that strengthen ESG disclosure mandates, such as guidelines on Scope 1 and emis- sions, which, while costly, may ease the evaluation of companies' efforts. We also suggest exploring ways to redesign the simple ESG score to better inform investors and motivate corporate actions. (3) Richer agent dynamics. For instance, we plan to allow investors to learn ESGGconsciousness level through interaction with the environment, which can change overtime. Given the challenges of passing new regulations, we aim to explore if companies and investors can self-regulat through coordination. For instance, we suggest testing if collective funds or peer monitoring can enhance mitigation or lead to collusion.
10

---

Published as a conference paper at ICLR 2025
7 ETHICS STATEMENT
This research is directly aimed at improving human well-being by addressing the existential threat posed by climate change, one of the most pressing issues of our time. Our work seeks to develop fully reproducible, open-source multi-agent reinforcement learning (MARL) benchmark, enabling the highest standard of transparency and scientific rigor. The benchmark designed be freely accessible to all researchers, and performant enough to run efficiently commonly available GPU resources, fostering inclusivity. By simulating real-world social dilemmas between corporations and investors, aim to spark novel solutions for reducing emissions and mitigating climate change risks. No human subjects are involved this research.
8 REPRODUCIBILITY STATEMENT
To promote reproducibility, we provide our source code and corresponding configuration files used for running the experiments in open source. The details about installation and sample code for running the experiment are also included at https: https://github. com/ jiayii Invest BSG
REFERENCES
John P. Agapiou, Alexander Sasha Vezhnevets, Edgar A. Duéeeee-uzumannn Guzmán, Jayd Matyas, Yiran Mao, Peter Sunehag. Ruphael Köster, Udari Madhushani, Kavya Kopparapu, Ramona Comanescu, DJ Strouse, Michael B. Johanson, Sukhdeep Singh, Julia Haas, Igor Mordatch, Dean Mobbs, and Joel Z. Leibo. Melting pot 2.0, 2023. UtL.tttps://arxx wttp:/awwwv..raaaaaaaeooooeoooee
wat.vvorr///aaa /ab// /21p:/22111cc 01221.13746
Amir Amel-Zadeh and George Serafeim. Why and how investors use esg information: Evidence from global survey. Financial analysts journal, 74(3):87-103, 2018.
Jasmina Arifovic, Cars Hommes, Anita Kopányi-Peuker, and Isabelle Salle. Ten isn't large! group size and coordination in large-scale experiment. American Economic Journal: Microeconomics, 15(1):580-617, 2023.
Matteo Bettini, Ryan Kortvelesy, and Amanda Prorok. Controlling behavioral diversity in multi- agent reinforcement learning, arXiv preprint arXiv:2405.1503 2024a.
Matteo Bettini, Amanda Prorok, and Vincent Moens. Benchmarl: Benchmarking multi-agent rein- forcement learning. Journal of Machine Learning Research, 25(217):1-10, 2024b.
Alexander Bisaro and Jochen Hinkel. Governance of social dilemmas in climate change adaptation. Nature Climate Change, ((4):354-359) 2016.
Bloomberg News. EU to Delay ESG Reporting Rule for Some Sectors by Two Years, 2024. URL
www..bll.. http:///ww.....oeee....../////ea
sg www.reppottiggguge ruwe- some- ouw.ee---o----eee---ttttt------- oue=truel
Frances Bowen. Environmental visibility: a trigger of green organizational response? Business
stralegy and the environment, 9(2)-22-107, 2000.
James Bradbury, Roy Frostig. Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Manderman-Milhe and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http:
//githube com cottb//cc/aa/m-mc///mm
Marina Brogi, Antonella Cappiello, Valentina Lagasio, and Fabrizio Santoboni. Determinants of
insurance companies' environmental, social, and governance awareness. Corporate Social Re- sponsibility and Environmental Management, 2955)133771 2022.
James M Buchanan and Wm Craig Stubblebine. Externality. In Inframarginal Contributions to Development Economics, pp. 55-73. World Scientific, 2006.
11

---

Published as a conference paper at ICLR 2025
Barbara Buchner. Annual finance for climate action surpasses usd 1 trillion, but far from levels needed to avoid devastating future losses, 2023,
basses-usd-1-tril1
fating-future--lsss loses/ Accessed 27-11-2024.
Micah Carrol,, Rohin Shah, Mark K. Ho, Thomas L. Griffiths, Sanjit A. Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordination, 2020. URL
https://arxix: org/ abs 1910. 05789
Yi-Chun Chen, Mingyi Hung, and Yongxiang Wang. The effect of mandatory esr disclosure on firm profitabilitty and social externalities: Evidence from china. Journal of accounting and economics, 65(1):169-190, 2018.
Soo-Haeng Cho, Xin Fang, Sridhar Tayur, and Ying Xu. Combating child labor: Incentives and in- formation disclosure in global supply chains. Manufacturing & Service Operations Management, 21(3):692-711 2019.
Thomas F Stocker Christopher B Field, Vicente Barros and Qin Dahe. Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation: Special Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, 2012.
CNBC. The sec votes this week on controversial climate change rule: Here's what's at stake, 2024.
http:ttts:///ww http:///www..ooooooo///////////s ontroversial- -limamt--mateeeeeeeeeeeeee-eee
Denis Cormier and Michel Magnan. Corporate environmental disclosure strategies: determinants,
costs and benefits. Journal of Accounting, Auditing & Finance, 14(4))429-411, 1999.
Carl Dahlman. The problem of externality. The journal of law and economics, 22(1):141-162, 1979.
Aswath Damodaran. Historical returns on stocks, bonds, and bills, 2024a. https://ppp S. sterr.. nyu. edu/ aatiodardaa http://owoo...aagaaaaaaaaaaaaaaa New York University Stern School of Business.
Aswath Damodaran. Capital expenditures by sector, 2024b. URL http:::/ppsaaae sttp::///.aeessee........ stern. Inyu.eda//adaaodaa//ee adamaaaz//odadeeee Bateiddddeeee___eeeeeeeeeee htwpeeeeagg//tttafaleeeeee/eeeee html Accessed 27-11-
2024.
Sebastião Vieira de Freitas Netto, Marcos Felipe Falcão Sobral, Ana Regina Bezerra Ribeiro, and Gleibson Robert da Luz Soares. Concepts and forms of greenwashing: A systematic review. Environmental Sciences Europe, 32:1-12, 2020.
Christian Schroeder De Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020.
Magali A Delmas and Michael W Toffel. Organizational responses environmental demands:
Opening the black box. Strategic management journal,
2008.
Anil R Doshi, Glen WS Dowell, and Michael W Toffel. How firms respond to mandatory informa- tion disclosure. Strategic Management Journal, 34(10): 121100):12099111 1231, 2013.
Ishan P. Durugkar, Clemens Rosenbaum, Stefan Dembach, and Sridhar Mahadevan. Deep reinforce-
ment learning with macro-actions, 2016.
04615
Javier El-Hage. Fixing esg: Are mandatory esg disclosures the solution to misleading esg ratings? Fordham J. Corp. Fin. L., 26:359, 2021.
European Investment Bank. What drives firms' investment in climate action? evidence from the 2022-2023 eib investment survey. Technical report, European Investment Bank, 2023. URL http:://ww..ebb.oro////ii//
/tt.ooonn//222111-/aaatt wwwhhat-driie--ee firn--irssiii- iveattent-in-e imate Accessed: 2024-11-26.
12

---

Published as a conference paper at ICLR 2025
Eugene F Fama and Kenneth R French. Disagreement, tastes, and asset prices. Journal of financial economics, 83(3):667-683 6896677-699988 2007.
J Doyne Farmer, Cameron Hepburn, Penny Mealy, and Alexander Teytelboym. A third wave in the economics of climate change. Environmental and Resource Economics, 62:329-357, 2015.
Peter Fiechter, Jörg-Markus Hitz, and Nico Lehmann. Real effects of widespread esr reporting mandate: Evidence from the european union's ' csr directive. Journal of Accounting Research, 60 19:1499-1549, 2022.
Henry Friedman and Mirko S Heinle. Taste, information, and asset prices: Implications for the valuation of esr. Review of Accounting Studies, 21:740-767, 2016.
Henry L Friedman, Mirko Stanislav Heinle, and Irina Luneva. A theoretical framework for esg reporting to investors. Available at SSRN 3932689, 2021.
Ramin Gamerschlag, Klaus Möller, and Frank Verbeeten. Determinants of voluntary csr disclosure:
empirical evidence from germany. Review of managerial science, 5:233-262. 2011.
David Gardiner and Associates. Nearly half of fortune 500 companies engaged in major climate initiatives. https: es oow7://671.eeecceeaaaaaarreee000 S. 2.$20Fortume320500 2023, Accessed: 2024-11-26.
Paul Griffin and CR Heede. The carbon majors database.
CDP carbon majors report 2017, 14,
2017.
Sanford J Grossman and Oliver D Hart. An analysis of the principal-agent problem. In Foundations of Insurance Economics: Readings in Economics and Finance, pp. 302-340. Springer, 1992.
Uri Hertz, Raphael Koster, Marco Janssen, and Joel Leibo. Beyond the matrix: Experimental approaches to studying oocial-coooogcal systems. 2023.
Edward Hughes, Joel Z Leibo, Matthew Phillips, Karl Tuyls, Edgar Dueñezz-Gzzman, Antonio Garefa Castañeda, lain Dunning, Tina Zhu, Kevin McKee, Raphael Koster, al. Inequity aver- sion improves cooperation in intertemporal social dilemmas. Advances in neural information processing systems, 31, 2018.
Ioannis Ioannou and George Serafeim. Corporate sustainability: A strategy? Harvard Business School Accounting & Management Unit Working Paper, (19-065), 2019.
Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Çaglar Gülçchre, Pedro A. Ortega, DJ Strouse, Joel Z. Leibo, and Nando de Freitas. Intrinsic social motivation via causal influ- ence multi-agent RL. CoRR, abs/1810.08641) 2018. URL http: http://arxxv..oor///b/// 10.48647
Basak Kalkanci and Erica L Plambeck. Managing supplier social and environmental impacts with voluntary versus mandatory disclosure to investors. Management Science, 66(8):3311-3322 2020.
Korea Economic Daily. Korea to enhance esg disclosures part of corporate sustainability efforts, 2023. URL https://p/// Accessed: 2024-08-22.
localllbbl..... com httpaa/caacaaae//eeee///////////
Philipp Krueger, Zacharias Sautner, Dragon Yongjun Tang, and Rui Zhong. The effects of mandatory esg disclosure around the world. Journal of Accounting Research, 2021.
Joel 7, Leibo, Vinicius Zambaldi, Mare Lanctot, Janusz Marecki, and Thore Graepel. Multi-agent reinforcement learning in sequential social dilemmas. arXiv preprint 1002v:7702.00000 2017.
Joel Z Leibo, Edgar A Dueñez-Guzman, Alexander Vezhnevets, John P Agapiou, Peter Sunehag, Raphael Koster, Jayd Matyas, Charlie Beattie, Igor Mordatch, and Thore Graepel. Scalable eval- uation of multi-agent reinforcement learning with melting pot. International conference on machine learning, pp. 6187-6199. PMLR, 2021.
13

---

Published as a conference paper at ICLR 2025
Jun Li and Di Wu. Do corporate social responsibility engagements lead to real environmental, social, and governance impact? Management Science, 66(6):2564-2555 2020.
Ryan Lowe. Yi I Wu, Aviv Tamar, Jean Harb, OpenAl Pieter Abbeel, and Igor Mordatch. Multi- agent actor-critic for mixed hoperative-compeittvy environments. Advances in neural informa- tion processing systems, 30, 2017.
Thomas P Lyon and John W Maxwell. Greenwash: Corporate environmental disclosure under threat of audit. Journal of economics & management strategy, 20(1):3-41) 2011.
Antoine Marot, Benjamin Donnot, Gabriel Dulac-Amold, Adrian Kelly, Aidan O'Sullivan, Jan Viebahn, Mariette Awad, Isabelle Guyon, Patrick Panciatici, and Camilo Romero. Learning to run a power network challenge: a retrospective analysis. CoRR, abs/2103.03104, 2021. URL
https://axxi:: org/ abs 220.. 03104
Christopher Marquis, Michael W Toffel, and Yanhua Zhou. Scrutiny, norms, and selective disclo- sure: A global study of greenwashing. Organization Science, 37(2):483-500 2016.
P. Zhai, A. Pirani, S.L. Connors. C. Péan, S. Berger, N. Caud, Y. Chen, V. L. Goldfarb, M.I. Gomis, M. Huang, K. Leitzell, E. Lonnoy, J.B.R. Matthews, T.K. Maycock, T. Waterfield. O. Yelekçi, R. Yu, and B. Zhou (eds.). Climate Change 2021: The Physical Science Basis. Contribution of Working Group to the Sixth Assessment Report of the Intergovernmentall Panel on Climate Change. Intergovermental Panel on Climate Change, 2021. In Press.
MSCI Inc. Msci esg ratings, 2024. URL /httpss:// www. msci.... cww caan/sustainatle inves t cing/essg-ratinggs Accessed: 2024-09-19.
William D Nordhaus. An optimal transition path for controlling greenhouse gases. Science, 258 (5086):1315-1319) 1992.
William D Nordhaus. Revisiting the social cost of carbon. Proceedings of the National Academy of Sciences, 114(7):1518-155 1523,, 2017.
William D Nordhaus and Zili Yang. A regional dynamic general-equilibrium model of alternative climate-change strategies. The American Economic Review, pp. 741-765, 1996.
Mancur Olson Jr. The Logic of Collective Action: Public Goods and the Theory of Groups, with a new preface and appendix, volume 124. harvard university press, 1971.
Ariel Pakes and Paul McGuire. Stochastic algorithms, symmetric markov perfect equilibrium, and the 'curse' of dimensionality. Econometrica, 6265) 2001.
Climate Impact Partners. Fortune global 500 climate commitments, 2024. URL httph::tpp//yy commi
elimateimpact cww htt.a//aaao...aaaaaaaaaaaaaaaaaa
tments/ Accessed: 2024-11-26.
L'uboš Pástor, Robert F Stambaugh, and Lucian A Taylor. Sustainable investing in equilibrium. Journal of financial economics, 142(2)550-571, 2021.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPN- W, 2017.
A Mitchell Polinsky and Steven Shavell. Mandatory versus voluntary disclosure of product risks. The Journal of Law, Economics, & Organization, 28(2):360-377 2012.
H.-O. Pörtner, D.C. Roberts, M. Tignor, E.S. Poloczanska, K. Mintenbeck, A. Alegría, M. Craig, S. Langsdorf, S. Löschke, V. Möller, A. Okem, and B. Rama (eds.). Climate Change 2022: Im- pacts. Adoptation, and Vulnerability, Contribution Report of the Intergovermmental Panel on Climate Change. Cambridge University Press, Cam bridge, UK and New York, NY, USA, 2022. doi: 10.1017/97810099332 0093258844.
Working Group Il to the Sixth Assessment
14

---

Published as a conference paper at ICLR 2025
Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dor- mann. Stable-baselines:: Reliable reinforcement learning implementations. Journal of Machine Learning Research, (2(268)118,, 2021. hRLphttp:// jmlr wwtpppppppppp//........
4.
Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung. Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. arXiv preprint arXiv:1902.04043, 2019.
SEC. The enhancement and standardization of climate-related disclosures for investors; delay of
effective dates, 2024a. URL httpa//ww.....aaa.////////////aa
jeenallethve--deelaa--------eeee Acceess. 2024-08-22.
Accessed:
SEC. Sec adopts rules enhance and standardize climate-related disclosures for investors, Mar
2024b. /et2bu/ssssts:///ww....e...
Yin Shi and Xiaoni Li. An overview of bankruptey prediction models for corporate firms: A
systematic literature review. Intangible Capital, 15(2):114-277, 2019. ISSN 1697-9818. doi:
10.3926/ic.1354 URLhttps! www. intangiblecapitaal. htrp://eeapppaaaa..ooooeeeeee wwwp/rrre.e...oo..o.oo..o....e.e httppinpwp..pp..pppppepppppppp php/ic/a
Eticle/view/1354
P.R. Shukla, J. Skea, R. Slade, A. Al Khourdajie, R. van Diemen, D. McCollum, M. Pathak, S. Some, P. Vyas, R. Fradera, M. Belkacemi, A. Hasija, G. Lisboa, S. Luz, and Malley (eds.). Climate Change 2022: Miligation of Climate Change. Contribution of Working Group III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press. Cambridge, UK and New York, NY, USA, 2022. doi: 01.1017/978100915599
Alfonso Siano, Agostino Vollero, Francesca Conte, and Sara Amabile. "more than words": Expand- ing the taxonomy of greenwashing after the volkswagen scandal. Journal of business research, 71:27-37,2017
S&P Global. S&p global esg scores, 2024. URL https: httpp::/www.mrrrpppccc....e spglobal comm http:///datttttttsss/pooolaaaeee scc-seoess---ssss-------------e- (1701)s-(111) Accessed: 2024-09-19.
WFE Statistics. Market Capitalisation Q3 2023. World Federation of Exchanges, 2023, URL
https://focus. world wwwpsa/wwwcc.nnn..ooooooo
capitalisatic har.://g.....r...at.ieee//rrttte htpps//aaarrttttpppalaaaaaa.aaaa
n-a3-2023 Accessed: 2024-09-27.
Yue Wu, Kaifu Zhang, and Jinhong Xie. Bad greenwashing, good greenwashing: Corporate social responsibility and information transparency. Management Science, 66(7):3095-3122, 2020.
Jiachen Yang, Ang Li, Mehrdad Farajtabar, Peter Sunehag, Edward Hughes, and Hongyuan Zha. Learning incentivize other learning agents. CoRR, abs/2006.06051, 2020a. URL https:
/
Zhi Yang, Thi Thu Huong Nguyen, Hoang Nam Nguyen, Thi Thuy Nga Nguyen, and Thi Thanh Cao. Greenwashing behaviours: Causes, taxonomy and consequences based on a systematic literature review. Journal of business economics and management, 1486-177755 2020b.
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:2461 1-24624, 2022.
Tianyu Zhang, Andrew Williams, Soham Phade, Sunil Srinivasa, Yang Zhang. Prateek Gupta, Yoshua Bengio, and Stephan Zheng. Ai for global climate cooperation: modeling global climate negotiations, agreements, and long-term cooperation in rice-n. arXiv preprint arXiv:2208.0777 07004, 2022.
Stephan Zheng, Alexander Trott, Sunil Srinivasa, David C. Parkes, and Richard Socher. The AI economist: Optimal economic policy design via two-level deep reinforcement learning, 2021.
URL https:////// hatpps:////xiiii/ wawpiviiii.. org/ abs//b/s..... 02755
15

---

Published as a conference paper at ICLR 2025
In the Appendix, will cover the following details of our work.
Technical details of the InvestESG environment Appendix Math formulation of the environment.
Implementation details Appendix forcement learning experiments.
10
Implementation details about multi-agent rein-
Additional experiments Appendix 11 Details about additional experiments, including the results with more agents in 11.1 the impacts of resilience spending on the stakeholders behavior and mitigation level in ??, the effects of seeding companies with real-world date
in [1.2] the effects of lock-in period for agent decisions in [1.3] the effects of uncertain
climate event damage in 11.4 and the effects of bankruptcy mechanism in 11.5
TECHNICAL DETAILS OF THE INVESTESG ENVIRONMENT
The simulation starts 2021 and runs through 2120, with each period corresponding one year. The environment includes two main components: (1) an evolving climate and economic system, and (2) two types of agents: M company agents C, for {I.....M} M) and N investor agents Iy for j E NN.
Climate and Economic Dynamics. The environment is characterized by three climate risk param- eters: extreme heat probability (P.)), heavy precipitation probability (Pi", ). and drought probability (P:)) in year t. Initial climate risks are set P 0.28, Pe 0.13, and P 0.17 following the IPCC estimate
et al., (02I). resulting in an overall climate risk of Po 0.48, probability at least one adverse climate event a year. Without mitigation efforts, these risks increase linearly over time, reaching the IPCC's 4 CC scenario scenario by 2100, which corresponds
to the 80th period in our 100-period simulation. By that point, we observe P = 0.94, P = 0.27,
and P 0.41 (or an overall climate risk of P = 0.97). We then extrapolate this trend through to period 100. Figure [a depicts how increased climate risks and adverse climate events increase over time scenario where companies are solely profit-motivated. Company agents can mitigate the growth of climate risk by investing in emissions reduction. The change in climate risk PŁ for event e {h, p, d} in year t is governed by the function:
Net
P.
+ Po. for (h,p,d).
X,U,,
where Um is the cumulative mitigation spending from all agents by period t. If U.. 0. risks increase linearly to reach Po by 2100 as explained earlier. When Uwm >> the growth rate of climate risk decreases. The parameters Xe are calibrated based on Shukla et al. (2022). which estimates that an annual mitigation investment of $2.3 trillion is required to achieve IPCC's 1.5 C scenario. The model fits A so that such investment levels would yield 1.5 'C scenario climate risks by 2100. Climate events are modeled as independent Bernoulli processes, allowing for multiple events within year (red dashed lines in Figure 2ap. Let X2,,, Xt.p Xt,e E {0, 1} represent the occurrence of each climate event, and X, denote the total number of events in period i, determined by Equation
X, Xt, Xt.p + Xt,e: where Xt,e (emoouli(P?). for ph,
(2)
In addition to the evolving climate risks, the environment incorporates baseline economic growth rate Y, set to 10% by default, aligned with the historical average annual return of the S&P 500 over the past century (Damodaran 2024a). Company agents' capital levels K.: grow at rate each year, barring climate events. If an adverse event occurs, company agents lose a portion of their total capital according to their respective climate resilience parameter L,C If economic losses drive company's 's remaining capital into negative territory, the company is declared bankrupt.
Company Action Space. Each company agent C; in period selects actions from continuous tor C: (uC, u.c uC: where uf; represents the share of capital allocated to mitigation, uf: to greenwashing, and C, to building climate resilience. The action space for each company is de- fined as continuous 3-dimensional unit cube, u [0. 1]3. If the of the three ratios exceeds
16

---

Published as a conference paper at ICLR 2025
in any period, the company agent is deemed to be overspending, resulting in bankruptcy. Mitigation spending directly reduces the system-wit wide climate risk as explained above. Greenwashing involves deceptive marketing or accounting tactics that allow companies to appear filmaee-fiennlle at a low cost without providing any real benefits to society (de Freitas Netto et al. 2020; Yang et al. 2020b)- Resilience spending enhances the company's climate resilience by lowering its vulnerability L,; but does not reduce emissions and therefore does not mitigate system-wide climate risk. In the default setting, we disable greenwashing and resilience spending to focus on testing companies' mitigation efforts. These options are later enabled examine their effects, as will become clear in Section 5
Investor Action Space. Investor agents first select the companies they want invest in. Specifically, agent I in period t selects an action from a binary vector of length M, a. = (at)).... a a(,s). corresponding to the M company agents. Each entry of = indicates that investor I, invests in company C, in period I, and J - otherwise. The investor's action space at time t is thus A = {0, 1I Once the choices are made, investors capitals are distributed equally among these chosen companies, as will be detailed later in Equation
Modeling ESG Diselosure. With the ESG disclosure mandate in place, each company agent re- ceives an updated ESG score Qi: in period t, calculated as
Qft = f + BuL..
where 3 > 1 indicates that greenwashing is cheaper than genuine mitigation in terms of building an ESG-friendly image. This mirrors simple ESG ratings provided by some agencies, which typically range from 1 to 100 (S&P Global 2024) or use letter-based ratings (MSCI Inc. 2024)..
State and Observation Space. The environment simulates a partially observable Markov game M defined over a continuous, multi-dimensional state space. The system state at period / is character- ized by the three climate risk parameters P. (P.h. PP, P! each company agent's state vector s: (K. of I, where K is the capital level, Q is the ESG score, and L, is the climate resilience. Each investor agent's state is represented by their investment portfolio and cash levels S ( ... H M C,ar:(")) C²²),, where H represents investor Ij's 's holdings in company C,, and C.7, is the investor's cash level. The full system state at period f is thus S. = (Pr. (S.: M (S ) All company and investor agents share common observation space, denoted as O,. In the default setting, O = (15 M ) Extensions that incorporate additional observable informa- tion, like climate risk, are explored in Section
State Transition. The environment's 's state transition T proceeds as follows. At the beginning of period f, investors collect their investment holdings from period t - and redistribute their capital according to a?" Denote ||a.1 = Em Iz during period t and let K.- = EM H c represent the total capital of investor I, at the start of period f. Companies reach an interim capital level after returning old investments, Σn 112,2 K or 0 if the and receiving new ones, with the investment from investor I, calculated as a |a.1 investor opts out of investing, shown in Equation
4 as the number of companies investor Z₁ invests in
N
N
kC
(+1.interim = =KC
for i = 1,...,,A M
(4)
Companies then make climate-related spending using the interim capital, as described in Equations 6 Here, Ut. represents the cumulative mitigation spending by all company agents up to period f, while US denotes the cumulative resilience spending by company Ct.
Umm Um-
KC:
Kitlintermm
(5)
U.C = U-1,r + ucj kC: for i = 1, 1,..., M
(6)
17

---

Published as a conference paper at ICLR 2025
While Ut.m is then plugged into Equation I where system climate risks are updated. Equation states that a company's 's climate resilience, L scales with the proportion of cumulative resilience investment relative to its capital, and that increasing resilience becomes progressively more chal- lenging due diminishing returns (Pörtner et al. (022).
L Li' exp ( (-1fe, U + u1 KC
(intttir ), for i I...., M.
KC: iinteiterim
At the same time, the occurrence of climate events are simulated according to Equation 2 and companies receive updated ESG scores based on Equation[]] Afterwards, companies' profit margins, o are computed using Equation 8 factoring in climate-related spendings uf, e u default economic growth 7, and losses due to climate events, which are influenced by companies' climate vulnerability L and the number of climate events X1 as defined in Equation 2
p. = (1- -1- - -
-ao-- (1))) (1-x111))))) 1, for i I....,,M. M.
(8)
Company capitals K investor holdings H H4,,, and investor cash positions Ch are updated
according to Equations to 11 Company capitals k are updated according to Equation by scaling the interim capital levels by profit margin.
k = (1 + Pi (f iinterim for I,...,M. M.
(9)
Equation 10 adjusts investor holdings based company profit margins in their portfolios.
K
1221 o Pf))
for i = -..... M.j. = I.
(10)
||ee|
If an investor chooses not invest, all capital remains as cash, as shown in Equation 11
cHi So. if
[allll +
for j = 1....,, N.
(11)
K.P,
if [arll Zz =
Rewards. The single-period reward for company C, is solely based on its profit margin, given by re k4 k iittiite for i l,.... M, reflecting the assumption that companies are profit-
driven. The reward for investor I, is r - a E. Hii,, Qfi for j I...,,
K. The first component is the portfolio return ratio, and the second component represents the weighted average ESG score of the investor's portfolio adjusted by the investor's ESG preference, az,,
EM, K
Social Outcome Metrics. We evaluate agent performance based on two key social outcome metries: the final climate risk level, P100- defined as P10o = - (1 Pioo) PP ( - Pd Pl and the -M
total market wealth at the end of the period, W1o0. defined as Wi00 = Zim1 K 100 + 2i=1 K I
100
10
IMPLEMENTATION DETAILS
10.1 INDEPENDENT-PPO
To test how self-interest agents learn to respond incentives in the environment, we employ a state- of-the-art MARL algorithm based on Independent PPO. Each agent has its own policy parameters, and agents do not share parameters among themselves. This is because we are interested in simulat- ing companies and investors as independent, selfishly motivated agents that specialize in maximizing their own expected reward. The policy model is simple Multi-Layer Perceptron (MLP) network, with input as the capital, resilience and margin of each company, along with the investments and capital of each investor. Additional information can added the observation as 4.2 All com- pany and investor agents share common observation space. To implement Independent-PPO with different roles, we built upon the Stable Bascline repository (Raffin et al. 2021). Each policy model is an MLP network with two layers of size 256 and 128 and with tanh activation layers, and update each policy after 5 episodes during the training. See Table [ for more details.
In the default setting, we assign equal amount of initial capitals to companies and investors, which is a rough representation of the current market. For each experimental scenario, we run the learning algorithms for 30k episodes, over 3 trials with different random seeds.
18

---

Published as a conference paper at ICLR 2025
ML.P layers
Activation layers
PPO n_steps
PPO learning rate
PPO entropy
Gradient Clipping
coefficient
256, 128
tanh
500
3 10-5
0.01
0.2
Table 1: IPPO policy training parameters. The rest of the parameters are the default as described in
Stable Baselines 3 (Raffin et al. 2021
3099 Greenwash, Mitigation, ESG Score by Company
Investment Matrix
0200
Company_0 0
Company
LO64
3274.77
170724
3003-8
Company_2
0.30
Company 3
Company_ 4
Investor ID
Velue
0.000
2003
0025
Greenwash
Mitiiaatonn
ESG Score
Company
(a) ESG-conssious investors can be distracted by greenwashing.
Figure 9: ESG-conscious investors be distracted by greenwashing. and heavily invest company that greenwashes. Here we examine scenario where Company hard-coded to invest in real mitigation while Company only invests in greenwashing. Investors have ESG consciousness level of 0, 1, 10.
11 ADDITIONAL EXPERIMENTS
11.1 SCALE UP THE NUMBER AGENTS
10b show the experiment results when the number of company and investor agents are scaled up 25-by-25. The increased number of agents reveal the same directional story as the main results shown in Figure 4 where highly DSG-conscious investors motivate mitigation efforts from companies, resulting in lower ending climate risk and higher total market wealth. Figure T0c Lod zoom into a single episode of the 10-company, 10-investor case, with investor ESG-consciousness set 0 and 10, respectively. In the ideal case where all investors are highly ESG-conscious, leading mitigating company attracts the majority of the investment, which is consistent with the pattern shown Figure 6a
Figure 10a
11.2 SEEDING COMPANIES WITH REAL- WORLD DATA
To ground the agents in real-world data, we seeded company agents' actions with real-world cor- porate behaviors. According to Gardiner Associates (2023): European Investment Bank (2023);
Partners (2024), approximately 50% of large companies are currently investing in climate mitiga-
tion. Globally, about 1% of GDP allocated climate finance annually (Buchner) (023). Since
GDP can be roughly viewed as the counterpart of a company' 's sales, and based on data from an NYU database [Damodaran] 2024b), which estimates the average sales-to-capital ratio across sectors to be between 0.8 and 1.28, approximate sales and capital to be of similar magnitudes. Consequently, we seeded 50% of companies to invest between 0.5% and 1% of their total capital into mitigation, re- flecting weal-world investment levels. As shown in Figures Tla Tlb and Tlc. when seeded with real data, the total corporate mitigation efforts and resulting climate risk levels eventually align closely with the baseline scenario.
19

---

Published as a conference paper at ICLR 2025
Climate Risk
Market Total Wealth
0.6
3000
2500
2000
1500
1000
Episode
Episode
10k
20k
30k
40k
10k
20k
30k
40k
25 companies + 25 investors w/ ESG consclouaness = 10
25 companies 25 investors w/ ESG consciousness 10
25 companies 25 investors ESG consciousness
25 companies 25 investors w/o ESG consciousness
(a) Ending climate risk of 25 companies and 25 investors,
(b) Ending total market wealth 25 companics and 25 investors.
104
(c) Company capital over time, company ESG score over time, and investment matrix of 10 compa- nies and 10 investors with ESG consciousness 0 in one episode.
Caloaen spnat ouar Ine
Compayes fran oar
Marb
Lat aoweiii
(d) Company capital over time, company ESG score over time, and investment matrix of 10 compa- nies and 10 investors with ESG consciousness 10 in one episode.
Figure 10: (a)(b) shows the final climate risk and market total wealth for the case of 25 companics and 25 investors. Similar to the default Soompany-and---nveettt case, when investors are highly conscious, the final climate risk would be decreased. (c)(d) shows the ending episode company capitals over time, company ESG score over time, and investment matrix environments with 10 companies. When investors are ESG conscious, the investments are more concentrated on the mitigating company compared to the case when the investors are not ESG conscious. Consequently, the capitals of mitigating companies are much larger compared mitiiiiiga companies in the case when the investors are ESG conscious, while the capitals of mitigating companies are comparable to non non- --tiiiaiiiinnggg companies when the investors are not ESG conscious.
11.3 LOCK-IN INVESTMENTS
In reality, the decision-making processes of both companies and investors can be less flexible than modeled, where companies and investors update their strategies annually. To reflect the capital inflexibility, we implemented a 5-year lock-in period for agent decisions. This approach allows the climate to evolve more rapidly than agents can respond. Despite this constraint, the results remain consistent with the directional findings presented in the main text, illustrated in Figures IId Tle and ITi suggesting that agents would achieve similar results using a macro-action reinforcement learning approach (Durugkar al.] However, the learning curves for the locked-in investment
20

---

Published as a conference paper at ICLR 2025
Total Mitigation Amount
Climate Risk
Market Total Wealth
200
150
3000
100
0.8
2000
0.7
50
0.6
Episodo
Episoda
10k
20k
30k
40k
0
10kk
20k
30K
43k
0
10k.
20k
30k
40k
Seoded wi rodl seo (013
Status ou manddtte
Seadiddion rea--ooric
Statusnqao mundess
Sredeelog red
Statie monderte
(a) Mitigation spending
(b) Climate risk
(c) Market total wealth
Total Mittigation Amount
Climate Risk
Market Total
250
4000
0.9
30000
100
1000
Episode
Episode
Episode
20kr
2001
0
20k
4.0K
60K
80N
1024
0
20k
40%
60k
80k
1001
Locked Inseentment
Statu quaree monddine
Leckedde Invendrment
Stalts manndaae
(d) Mitigation spending
(c) Climate risk
(f) Market total wealth
Figure 11: (a)-(c Effect of seeding with real-world data. (d)-(f) Effect of 5-year lock-in period for agent cisions. All comparisons are made against the default case with an ESG disclosure mandate, in which investors have zero ESG-consciousness, and company actions are restricted to mitigation efforts only.
cases exhibit greater volatility and slower convergence compared to the default case, indicating that capital inflexibility indeed increases the challenges of addressing climate change.
11.4 UNCERTAIN CLIMATE EVENT DAMAGE
Given the significant uncertainty surrounding the economic damages of climate change (Farmer al. 2015), conducted additional experiments where companies' resilience parameters, L representing economic losses from extreme climate events, were modeled as Gaussian random vari- ables I, ~ N(H, ar) clipped within the range [0,1], varying across both events and companies. This randomness allows for extreme climate damages, including the potential bankruptcy of some companies, which could incentivize risk-averse strategies where companies engage in greater mit- igation efforts. Conversely, the randomness complicates the ability of company agents to learn the long-term benefits of mitigation, potentially encouraging short-sighted behaviors or greenwashing.
Figures 12a to 12c illustrate the effects of uncertain climate event damage compared the default mandate case, where no investors are ESG-conscious, and company actions are restricted to mitiga- tion only. Both scenarios result in similar levels of total mitigation and climate risk. However, agents in the uncertain damage scenario require more time to leam the optimal mitigation level due to the increased uncertainty the environment. Despite the similarities, the uncertain damage scenario re- sults in significantly lower total market wealth due to potentially some high-tail losses in early years. When considering the wealth discrepancy, companies the uncertain damage scenario ultimately adopt more aggressive mitigation strategy relative to their capital. This behavior suggests that the high uncertainty may have prompted risk-averse strategies.
Uncertain climate event damage also incentivizes the companies to focus more on short-term benefits by attracting investments, compared to the case with fixed damage when the companies first explore greenwashing strategy and then settle down doing little greenwashing. As shown in 12d to 12g] when threatened with uncertain climate event damage, companies increase their greenwashing amount to attract immediate investments from ESG-consciouns investors.
21

---

Published as a conference paper at ICLR 2025
Mitigation Amount
Climate Risk
Market Total Wealth
200
4000
150
0.9
3000
100
1000
Episode
Episode
10k
40k
10k
20%
30k
40k
0
30k
20k
30k
40k
no dimott
of mate
mandene
W uncersen cámura demagss
-Settn mundon
(a) Mitigation spending
(b) Climate risk
(c) Market total wealth
Hitigrrion Amoust
Clinate RSA
Market Total Woolth
ToGal Greenwach wwrnsccchhmmmmnt
0.6
(d) Mitigation spending
(c) Climate risk
Market total wealth
(g) Greenwash spending
Wingrtion Amoont
Market Total Meatts
5002
155
2002
1002
Epoode
124
25
35c
40c
(h) Mitigation spending
(i) Climate risk
(i) Market total wealth (k) Number of bankrupt companies
Figure 12: (a)-(c) Effect of uncertain climate event damage. (d)-(g) Effect of uncertain climate event dam- age on greenwash spending. (h)-(k) Effect of more strick bankruptcy mechanism and its combination with uncertain climate event damage.
301
(a) ESG consciousness = 0.5
(b) ESG consciousness =
sata 11 201
(c) ESG consciousness 10
(d) Ending climate risk increases as the number of sharcholders increases.
Figure 13: (a)-(c) shows the ending episode investment matrix for investors with different level of ESG consciousness. (d) shows ending climate risk vs. number companies under the setting where no investors are ESG-conscious. When the number of companies increases, each company has less capital, and therefore companies have to spend greater portion of their capital to see the same mitigation effect.
22

---

Published as a conference paper at ICLR 2025
11.5 MORE STRICT BANKRUPTCY MECHANISM
Additionally, to make our model more enriched, we implemented a more strict bankruptcy mecha- nism for company agents: if company agent has margin worse than negative 10% for 3 consec- utive years, a red flag signaling potential financial distress, is deemed as bankrupt (Shi Li 2019). As shown in the comparison between orange and pink lines 12h 12i and [2j the new bankruptcy mechanism does not cause significant difference in the level of mitigation from the status quo with mandate case. Given that the market growth rate is high, companies are not severely threatened by bankruptcy despite that the new bankruptcy mechanism is in place, and therefore refrain from investing in climate mitigation.
However, when the stricter bankruptey mechanism is combined with uncertain climate event dam- age, as depicted by the green lines in Figures 12h and [2K] a significant increase in mitigation and more bankrupteies damage and the bankruptcy mechanism creates an immediate risk of bankruptcy for company agents, thereby strongly incentivizing their mitigation efforts.
observed. This indicates that the combination of uncertain climate event
12 SUPPLEMENTAL FIGURES
Figure 13 shows few supplemental plots which may be of interest. Figure 13a investor investments concentrate in mitigating companies as they become increasingly more ESG- conscious. In Figure [3d examine the impact of varying the number of companies under the status quo scenario with 5 companies and investors who have zero ESG-consciousness, while maintaining an initial wealth distribution of 50-50 between companies and investors. In the absence of ESG-conscious investors, the ending system climate risk increases as the number of companies grows. This likely because higher number of companies results in each company having less capital, requiring them to allocate larger proportion of their resources to achieve the same mitiga- tion effect.
13c reveal how
23


Reviews:
[{"review_id": "IzZNWXqhZ0", "reviewer": "Reviewer_9btN", "summary": "The paper concentrates on developing a multi-agent reinforcement learning framework (which they name InvestESG), to be used to student individual and collective outcomes from company investment and climate risk. It is an overall well written paper on a timely and relevant problem, for which we clearly need to better understand how to drive investors decisions to align individual and collective objectives. The paper is completely application-driven, in the sense that the authors do not develop new methodology, or a new solution approach. They mainly focus on describing how the framework should look like for the purpose of the application. They then generate a lot of simulation results to study various aspects of the problem (e.g., greenwashing, different levels of ESG consciousness, etc.)", "questions": "I think some of the key points to consider are:\n- questioning assumptions, e.g. rationality of the agents, why they would behave as if employing RL, etc.\n- convincing us that simulating a system with a limited number of agents provides us with insights that are relevant for systems for very large number of agents\nI clearly recognise that such issues may be more generally valid for the case of MARL environments and broader than for the case of this paper only. However, here, in view of the importance of the application, I find these issues particularly relevant.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The main strength of the paper is the topic is focuses on, and the idea to bring some momentum towards the development of a general platform to simulate a multi-agent system with focus on climate risk and company behaviour. Another strength (but which may also be seen as a weakness - see below) is that the framework is simple, in the sense that it is easily interpretable and flexible enough to interact with e.g. policy makers. The authors also aim to produce some relevant results, which may be seen as of value by policy-makers.", "weaknesses": "As mentioned by authors in a last part of the paper, maybe the main weakness is the simplicity of the framework, which may prevent a broad audience from accepting that it realistically model a real-world situation, and that it may be ring some relevant insights to be used as input to policy-making. In my opinion, it feels like an oversimplified and stylised approach where, depending on a few assumptions and a few modelling changes, we could get the model to do completely different things. Therefore, I believe that quite more work is necessary for such a paper, starting with the importance of the underlying assumptions, assessing the impact of modelling choices, sensitivity analyses, etc. I am not critical of the fact the authors are engaging in such developments - I am saying instead that I feel more work is necessary before sharing this work/paper with the world.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jNXsIDMMSY", "reviewer": "Reviewer_mQaB", "summary": "This paper presents InvestESG, a novel multi-agent reinforcement learning (MARL) benchmark designed to simulate and analyse the impact of varying Environmental, Social, and Governance (ESG) disclosure policies through the social dilemma paradigm. InvestESG  uses two types of agents: companies and investors. Companies allocate capital across mitigation, greenwashing, and resilience, with varying strategies influencing climate outcomes and investor preferences. The findings are consistent with empirical research using real-world data. They capture the positive impact of companies using information about global climate risks to determine their level of investment in mitigation, even without investor involvement. The paper is beautifully written and rigorous.", "questions": "I think the paper could be extended in several possible directions, as indicated in the Weaknesses section, for a more significant contribution. Another possible direction would be to implement and assess the impact of other PPO policies than IPPO on the overall behaviour and insights.\n\nPotential relevant papers are suggested below.\n\nBisaro, Alexander, and Jochen Hinkel. \"Governance of social dilemmas in climate change adaptation.\" Nature Climate Change 6, no. 4 (2016): 354-359.\n\nBettini, Matteo, Amanda Prorok, and Vincent Moens. \"Benchmarl: Benchmarking multi-agent reinforcement learning.\" Journal of Machine Learning Research 25, no. 217 (2024): 1-10.\n\nBettini, Matteo, Ryan Kortvelesy, and Amanda Prorok. \"Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning.\" arXiv preprint arXiv:2405.15054 (2024).\n\nBrogi, Marina, Antonella Cappiello, Valentina Lagasio, and Fabrizio Santoboni. \"Determinants of insurance companies' environmental, social, and governance awareness.\" Corporate Social Responsibility and Environmental Management 29, no. 5 (2022): 1357-1369.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper represents a novel contribution in a highly relevant, high-impact domain, at the intersection between climate change and MARL. It is beautifully written and self-contained, with rigorous specifications of the InvesESG environment. The implementation details and code are provided, and overall the paper makes a good case for a MARL benchmark for studying climate investment through the social dilemma paradigm, via two agent types: companies and investors. InvestESG is designed to simulate and analyse the impact of varying Environmental, Social, and Governance (ESG) disclosure policies on corporate climate investments. In InvestESG, companies allocate capital across mitigation, greenwashing, and resilience, with varying strategies influencing climate outcomes and investor preferences. The findings are consistent with empirical research using real-world data. The results capture the positive impact of companies using information about global climate risks to determine their level of investment in mitigation, even without investor involvement.", "weaknesses": "The main weakness of the paper consists in its simplifying assumptions, in terms of the types of agents, and the considered scenarios, analysis and discussions. These limitations are acknowledged in the paper. Due to these reasons, I believe that, in the current format, the paper makes an insufficient contribution for a top conference like ICLR.\n\nFor a more significant contribution, this work could be extended in one or more possible directions: extend the agent types (possibly consider insurance companies/market?), add more complex agent behavior, learn parameters and behaviors from real data, include more social outcome metrics (in addition to the final climate risk level and the final total market wealth, at the end of the simulation period) and/or include additional features, such as agent bankruptcy, and a dynamic number of agents.\n\nAssuming the agent-types remain just companies and investors, increasing the number of companies and investors, and learning their behavior from real world data, may be a sufficient extension for a more significant contribution.\n\nIn the longer term, the initial vision of InvestESG would benefit from a more diverse agent space, for a more realistic climate-change problem specification (however, this is not essential for a significant contribution).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z22aJDXMbQ", "reviewer": "Reviewer_fPY9", "summary": "The paper presents InvestESG, a MARL environment that studies the impact of ESG disclosures on company and investor agents. The benchmark is meant to simulate companies' investment decisions into climate mitigation, green washing and resilience spending as a social dilemma.\n\nSpecifically, the contributions are:\n\n* InvestESG, a climate-economic environment in which investors fund companies, which make decisions about how much to invest in climate-related spending over 100 years starting in 2020.\n  * Climate risks grow linearly in the absence of any mitigation\n  * Companies decide how much to spend on mitigation, greenwashing and resilience\n  * Investors decide which companies to invest in based on their preferences, which trade off between profits and climate efforts documented by ESG disclosures.\n  * As the simulation proceeds, companies make profits which they return to investors, while climate risks grow, resulting in a higher probability of extreme events.\n  * Agents are modelled using IPPO\n\n* A set of experiments shedding light on agent behaviour in InvestESG\n  * With no ESG disclosures, purely profit-driven decisions result in suboptimal collective outcomes\n  * The impact of ESG disclosures depending on how many and how much investors care about ESG reports when choosing which companies to invest in\n  * Whether companies leverage greenwashing when it is allowed in InvestESG\n  * Whether visibility of the climate-related risk probabilities impacts agent behaviour\n\n* Conclusions for policymakers and researchers\n  * Mandatory EST disclosure paired with ESG-conscious investors can drive corporate mitigation efforts.\n  * Knowledge of climate risks motivates investors and companies\n  * Agent behaviour is consistent with empirical evidence\n  * InvestESG is an example of using MARL to tackle complex social dilemmas in real-world, high-impact domains", "questions": "- What does a sensitivity analysis of the Beta parameter do to the results?\n\n- How would more longer capital investment timelines (e.g. min 5 year lock-in) impact the trained agents?\n\n- Do observations include past climate events?\n\n- How did you calibrate the 0.5% investment of capital into mitigation?\n\n- Why do you think that agents are so insensitive to the value of Beta as shown in figure 6 b)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "# Originality\n\n* **Novel MARL application to ESG disclosures**: Even though Zhang et al. explore MARL in the policy space, as far as I know, this is the only MARL simulator that looks at ESG disclosure impact in this scenario.\n\n* **Novel problem formulation**: the authors cleanly describe the relationship between companies and investors with a two agent type system, as well as an ESG disclosure component.\n\n# Quality \n\n* **Relevant problem setup**: key decisions are captured by the problem setup. The ESG disclosure abstraction is simple and elegant. The reward structure effectively represents a social dilemma.\n\n* **Extensive experimental results**: the authors go through many scenarios with InvestESG to analyze different outcomes.\n\n# Clarity\n\n* The paper is **well structured**, and makes for a smooth read with little to no cognitive breaks.\n\n* The work is **well situated** within the literature on MARL simulators, and they contrast well with similar work.\n\n* The design and implementation of InvestESG is **clearly laid out**. \n\n* The work makes judicious use of **relevant visualizations**, such as Schelling diagrams.\n\n# Significance\n\n* The analysis is **timely and relevant** given the current discussions around ESG disclosures.\n\n* The conclusions around the preferences of investors for climate-active companies is impactful.\n\n* The use of MARL to study social dilemmas is an important subject of study.", "weaknesses": "# Soundness\n\n* **The economic agents are not grounded in the economics literature**. This leads to issues such as capital being perfectly flexible across time steps. In traditional economics models, investments in capital last and they are not flexible. Here, there seems to be an implied assumption of perfectly flexible capital, which is unrealistic. Starting with an existing model of economic agents (with a citation), highlighting its limitations for InvestESG and then explaining how you extend to agent to accommodate for these limitations would be a much more compelling presentation. \n\n* **Investor decisions are binary**, as opposed to continuous across all companies. Making investor decisions floats, i.e. a vector whose sum is capped at one, would allow for proportional investments across different companies. This is essential for investor diversification, which would also enable interesting extensions like regional damages to companies (i.e. climate events could affect subsets of agents either chosen at random or chosen somehow).\n\n* Figure 7 b) is highly confusing. It looks like **with climate information, risk is *maximized* and market wealth is *minimized***. I'm not sure what exactly is going on in this plot, but it doesn't fit with the storyline of the paper. That is, it certainly does not look like more information improves decision making in this plot, if anything the effects of more information are catastrophic for both climate risk and market wealth.\n\n* Figure 2b could be improved by showing the average number of events at each year across many episodes, as opposed to a single episode. \n\n* The **number of agents is limited**. Granted, it is more than 2. However, it would be interesting to scale it up to more and see what types of behaviour emerge. There are group size effects that can emerge at scale in economics, e.g. see https://www.aeaweb.org/articles?id=10.1257/mic.20200290. This shows in section 9.2 of the paper in the appendix, but given the implications of such a result, it would be very important to expand upon these results.\n\n# Presentation\n\n* The paper is well structured, but the plots are a pain to read. The labels and ticks are too small, and the axes are not annotated. \n\n* If you use a pdf format for your images instead of png, you can avoid the graininess when zooming in, which is necessary because of the label sizes.\n\n* The results section could benefit from additional structure. It would be less dense and easier to read if you highlighted which of your results you consider the main results, and which you consider additional.\n\n* I found the description of schelling diagrams fairly unclear, it took me a minute to get it.\n\n* It should be ICLR 2025, not 2024. Please make sure the template you used is up to date.\n\n* Inconsistent use of \"MARL\" and \"multi-agent RL\"\n\n- Might benefit for a problem setting section, where you introduce important concepts like bifurcated equilibria\n\n- Typo: 3.2 \"self-interest\" -> \"self-interested\"\n# Contribution\n\n* The importance of the contributions are weakened by what Figure 9 d) is suggesting, since there are many companies in the world. It seems to me that, without addressing the concerns raised by this result, your conclusions for policymakers do not hold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)