# Determining the power of investors over Frontier AI Labs is strategically important to reduce x-risk

*Produced as part of the* [*ML Alignment & Theory Scholars Program*](https://www.matsprogram.org)* \- Summer 2024 Cohort*

This post presents the basic story behind my current research direction at MATS. I'm looking for feedback on whether the basic premises hold up, and which research questions should I prioritize.

The role of investors in fueling the AGI race is unclear
========================================================

Most Frontier AI Labs acknowledge the extreme risks of AGI development, invest in safety, and seem to care at least somewhat about making sure their systems don't destroy the world.

However, Frontier Labs currently appear to be in a race to build AGI, which most fear will lead Labs to make risky bets and not take adequate safety measures to avoid catastrophic outcomes, in order to have a chance of winning the race. A common explanation is that Labs are for-profit companies, which prevents them from putting safety first, because investors would force them to put profit first and continue to rapidly develop capabilities even though executives know the risks.

But do investors actually have so much influence over companies that they can force them to build dangerous models against their best judgement?

There are strong economic arguments for why companies that prioritize profit are rewarded in the long run, both through selection effects (profit-seeking companies are less likely to go bankrupt) and through capital allocation (non-profit-seeking companies don't receive investment or don't even get started). However, these mechanisms only work on average over long periods of time, and in the short term investors regularly fail to keep the companies they invest in from making stupid decisions and going bankrupt (e.g. FTX).

So I think it is unclear how much power investors have over Frontier Labs' overall strategy for building AGI, or over specific short-term decisions like whether to train a particular dangerous model. I expect their effective power to vary by company, by type of decision and by timescale. Their power will depend on specific corporate structures, executives, boards, internal cultures, ownership types and applicable corporate laws.

Identifying the power of investors over Labs can inform AI governance strategy
==============================================================================

I expect the details of investors' influence over Labs' decisions to be relevant to several strategic considerations:

*   Whether widespread investor awareness of the extreme dangers of AGI development would change the behavior of Labs
*   Whether regulations that change the financial incentives of Labs would succeed in changing their behavior.
*   Whether Labs would avoid actions that would harm the rest of their investors' portfolios (e.g. avoiding automation of certain industries, avoiding large-scale catastrophes).
*   Whether executives could take actions that would disempower investors, empower themselves, or take any kind of large-scale unilateral action enabled by AGI (e.g. pivotal acts, implementing their own ideology in the AGI, taking military or political control, distributing profits through a UBI).
*   And probably many others

Research direction: Investing how investors influence Frontier AI Labs
======================================================================

I expect that there are many open questions in this area, and that studying them could help governance researchers make better decisions and policy proposals.

Here is a list of questions that seem important to investigate:

*   What are the different ways in which investors have formal control over Frontier Labs? Over which timescales does this control take place?
*   How can we quantify the amount of influence each stakeholder has over the Labs' decisions?
*   How does the amount of investor influence over the development of AGI change depending on timelines lengths?
*   How would the bursting of the AI bubble or other macroeconomic events change the power of investors?
*   How do different Labs compare in the amount of influence investors have over their decisions?
*   How does the fiduciary duty of companies to investors work? What is the duration of a typical breach of fiduciary duty case? In what case could Labs investors win? Would it be likely to work?
*   What are the different ways in which investors have informal power over labs? How does this compare with their formal power?

All feedback is welcome! Feel free to comment on whether the basic premises hold up, whether this is an impactful research direction, what research questions I should prioritize, or how this work should be published.