trentmkelly/LessWrong-43k huggingface dataset:
Using game theory to elect a centrist in the 2024 US Presidential Election
by Ebenezer Dukakis — 2024-04-05T00:46:22.949Z

Crossposted from the EA Forum

TL;DR

A nonpartisan group like No Labels could privately offer US congresspeople this deal: If enough congresspeople pledge to the deal, they all agree to switch their Presidential endorsement to a compromise candidate. If not enough pledge, then pledging still gets them some other benefit, such as a campaign donation or endorsement. Such a scheme could generate a lot of utility.

Executive Summary

Many Americans are unsatisfied with the way their democracy is working, and deeply concerned with one or both of the major candidates for the 2024 presidential election. Furthermore, previous EA Forum discussion has identified electoral reform as a possible top cause area. It may be time to explore alternatives to the primary-election system used by US political parties to select presidential nominees since the late 1960s. In this post I propose a dominant assurance contract mechanism for coordinating endorsements around an alternative centrist candidate. The proposed contract works as follows: If a political big shot (congressperson, pundit, etc.) signs the contract, and certain thresholds in the contract are reached (in terms of the number of contract signatures / candidate poll numbers / etc. by a particular date), then signers agree to switch their endorsement to a compromise candidate. If those thresholds are not reached, then signers should still get some sort of bonus, perhaps in the form of a campaign donation, endorsement, etc. This bonus ensures that signing the contract looks attractive in all scenarios, which makes it more likely that the target threshold will be reached. If this scheme works as described, it could provide a foundation for long-lasting electoral reform in the United States.

Background information

Why expect a centrist candidate to do well in America's 2024 presidential election?

With all the sound and fury around American elections, you might expect most Americans to have a strong party preference. In fact, 43% of Americans call themselves independent. "Independent" is easily the nation's most popular affiliation, comfortably ahead of either Republican or Democrat — source.

Furthermore, the Electoral College amplifies the voting power of undecided voters in "swing states". Common sense suggests these voters are likely to prefer a centrist candidate.

What is negative partisanship?

Although many Americans d...
New report: A review of the empirical evidence for existential risk from AI via misaligned power-seeking
by Harlan — 2024-04-04T23:41:26.439Z
Link: https://blog.aiimpacts.org/p/new-report-a-review-of-the-empirical

Visiting researcher Rose Hadshar recently published

a review of some evidence for existential risk from AI, focused on empirical evidence for misalignment and power seeking

. (Previously from this project: a blogpost outlining some of the

key claims that are often made about AI risk

, a series of

interviews

of AI researchers, and a

database

of empirical evidence for misalignment and power seeking.)

In this report, Rose looks into evidence for:

Misalignment,[1] where AI systems develop goals which are misaligned with human goals; and
Power-seeking,[2] where misaligned AI systems seek power to achieve their goals.

Rose found the current state of this evidence for existential risk from misaligned power-seeking to be concerning but inconclusive:

There is empirical evidence of AI systems developing misaligned goals (via specification gaming[3] and via goal misgeneralization[4]), including in deployment (via specification gaming), but it's not clear to Rose whether these problems will scale far enough to pose an existential risk.

Rose considers the conceptual arguments for power-seeking behavior from AI systems to be strong, but notes that she could not find any clear examples of power-seeking AI so far.

With these considerations, Rose thinks that it’s hard to be very confident either that misaligned power-seeking poses a large existential risk, or that it poses no existential risk. She finds this uncertainty to be concerning, given the severity of the potential risks in question. Rose also expressed that it would be good to have more reviews of evidence, including evidence for other claims about AI risks[5] and evidence against AI risks.[6]

^

“An AI is misaligned whenever it chooses behaviors based on a reward function that is different from the true welfare of relevant humans.” (

Hadfield-Menell & Hadfield

, 2019)

^

Rose follows (Carlsmith, 2022) and defines power-seeking as “active efforts by an AI system to gain and maintain power in ways that designers didn’t intend, arising from problems with that system’s objectives."

^

"Specification gaming is a behaviour that satisfies the literal specification of an objective without achieving the intended outcome." (

Krakovna et al.

, 2020).

^

"Goal misgeneralization is a specific form of robustness failure for learning algorithms in which the learned program competently pursues an undesired goal that leads to

...
Quick evidence review of bulking & cutting
by jp — 2024-04-04T21:43:48.534Z

Epistemic status: fairly fast non-comprehensive literature review by a non-expert
Content warning: I advise against reading this if you believe you have an eating disorder

My ideal body aesthetic would be to have defined muscles and low body fat. Maybe this is also true of you. Maybe you’ve heard of cycling between seasons of building muscle (bulking) and losing fat (cutting) as a way to achieve that aesthetic. Should you?

Theory

It is easier to add muscle when on a caloric surplus[1] (more on that citation later). The theory is that it’s so much easier to build muscle this way, that you should spend several months intentionally eating more calories than you need, and accept that you’ll probably gain some body fat as well. But not to worry, you can then spend some months eating fewer calories than you burn in order to reduce your body fat percentage. Chasing these two hares separately putatively leads to a better result along both dimensions than a steady state.

In addition, there’s a natural advantage to this approach from the dynamic where you care more about the low body fat percentage during the summer, where your muscle definition is more apparent.

Quick notes:

You definitely want to be doing resistance training (weightlifting) the whole time. Because that’s the primary way to build muscle mass, and to avoid losing muscle when you’re cutting.
I’ll assume that you are moderately physically fit — you’re not brand new to weightlifting, but you’re also not particularly close to body-builder levels of jacked.
It’s definitely possible to cut body fat below healthy levels. When I’m talking about low numbers I’m aiming for I’m thinking of something like 10%-ish, though I haven’t really looked into this.

Evidence from bodybuilders

One piece of evidence is that most (maybe ~all) body builders do this.[2] Body builders have a similar goal to us — I wouldn’t go as far as they go, but building muscle definition is the name of their game, and they certainly achieve it. I would downweight this evidence on the basis of several factors that lead me to believe that the superiority of this technique might fail to generalize:

Bodybuilders have a competition season where they want to have a very low body fat percentage, often pushing down below sustainable levels. (I’ve seen numbers around 4%.)
The very low levels of body fat targeted by body builders is sufficient

...
LLMs for Alignment Research: a safety priority?
by abramdemski — 2024-04-04T20:03:22.484Z

A recent short story by Gabriel Mukobi illustrates a near-term scenario where things go bad because new developments in LLMs allow LLMs to accelerate capabilities research without a correspondingly large acceleration in safety research.

This scenario is disturbingly close to the situation we already find ourselves in. Asking the best LLMs for help with programming vs technical alignment research feels very different (at least to me). LLMs might generate junk code, but you can keep pointing out the problems with the code, and the code will eventually work. This can be faster than doing it myself, in cases where I don't know a language or library well; the LLMs are moderately familiar with everything.

When I try to talk to LLMs about technical AI safety work, however, I just get garbage.

I think a useful safety precaution for frontier AI models would be to make them more useful for safety research than capabilities research. This extends beyond applying AI technology to accelerate safety research within top AI labs; models available to the general public (such as GPT-N, Claude-N) should also accelerate safety more than capabilities.

What is wrong with current models?

My experience is mostly with Claude, and mostly with versions of Claude before the current (Claude 3).[1] I'm going to complain about Claude here; but everything else I've tried seemed worse. In particular, I found GPT4 to be worse than Claude2 for my purposes.

As I mentioned in the introduction, I've been comparing how these models feel helpful for programming to how useless they feel for technical AI safety. Specifically, technical AI safety of the mathematical-philosophy flavor that I usually think about. This is not, of course, a perfect experiment to compare capability-research-boosting to safety-research-boosting. However, the tasks feel comparable in the following sense: programming involves translating natural-language descriptions into formal specifications; mathematical philosophy also involves translating natural-language descriptions into formal specifications. From this perspective, the main difference is what sort of formal language is being targeted (IE, programming languages vs axiomatic models).

I don't have systematic experiments to report; just a general feeling that Claude's programming is useful, but Claude's philosophy is not.[2] It is not obvious, to me, why this is. I've...
On Leif Wenar's Absurdly Unconvincing Critique Of Effective Altruism

by Bentham's Bulldog — 2024-04-04T19:01:00.332Z

Crossposted here, on my blog.

Leif Wenar recently

published a critique

of effective altruism that seems to be getting a lot of hype. I don’t know why. There were a few different arguments in the piece: some terrible and others even worse. Yet more strangely, he doesn’t object much to EA as a whole—he just points to random downsides of EA and is snarky. If I accepted every claim in his piece, I’d come away with the belief that some EA charities are bad in a bunch of random ways, but believe nothing that imperils my core belief in the goodness of the effective altruism movement or, indeed, in the charities that Wenar critiques.

I’m not going to quote Wenar’s entire article, as it’s quite long and mostly irrelevant. It contains, at various points, bizarre evidence-free speculation about the motivations of effective altruists. He writes, for instance, “Ord, it seemed, wanted to be the hero—the hero by being smart—just as I had. Behind his glazed eyes, the hero is thinking, “They’re trying to stop me.””

I’m sure this is rooted in Ord’s poor relationship with his mother!

At another point, he mistakes MacAskill’s statement that there’s been a lot of aid in poor countries and that things have gotten better for the claim that aid is responsible for the entirety of the improvement. These strange status games about credit and reward and heroism demonstrate a surprising moral shallowness, caring more about whether people take credit for doing things than what is done. He says, for instance, after quoting MacAskill saying it’s possible to save a life for a few thousand dollars:

But let’s picture that person you’ve supposedly rescued from death in MacAskill’s account—say it’s a young Malawian boy. Do you really deserve all the credit for “saving his life”? Didn’t the people who first developed the bed nets also “make a difference” in preventing his malaria?

Well, as a philosopher, Wenar should know that two things can both cause something else. If there’s a 9-judge panel evaluating an issue, and one side wins on a 5-4, each judge caused the victory, in the relevant, counterfactual sense—had they not acted, the victory wouldn’t have occurred. MacAskill wasn't talking about apportioning blame or brownie points—just describing one’s opportunity to do enormous amounts of good. Would Wenar object to the claim that it would be important to vote if you knew your candidate would be bet...
Run evals on base models too!
by orthonormal — 2024-04-04T18:43:25.468Z

(Creating more visibility for a comment thread with Rohin Shah.)

Currently, DeepMind's capabilities evals are run on the post-RL*F (RLHF/RLAIF) models and not on the base models. This worries me because RL*F will train a base model to stop displaying capabilities, but this isn't a guarantee that it trains the model out of having the capabilities.

Consider by analogy using RLHF on a chess-playing AI, where the trainers reward it for putting up a good fight and making the trainer work hard to win, but punish it for ever beating the trainer. There are two things to point out about this example:

Running a simple eval on the post-RLHF model would reveal a much lower ELO than if you ran it on the base model, because it would generally find a way to lose. (In this example, you can imagine the red team qualitatively noticing the issue, but the example is an artificially simple one!)
The post-RLHF model still has much of its chess knowledge latently available, in order to put up a good fight across the full range of human ability. Possibly it's even superhuman at chess—I know I'd have to be better than you at chess in order to optimize well for an entertaining game for you. But that won't show up in its ELO.

So it seems to me like running evals on the base model as well as the post-RL*F model is an extremely sensible precaution against (1), and I'd love to be reassured either that this is unnecessary for some really obvious and ironclad reason, or that someone is already working on this.

And I don't have any good suggestion on (2), the idea that RL*F could reinforce a capability while also concealing it.
Let's Fund: Impact of our $1M crowdfunded grant to the Center for Clean Energy Innovation
by Hauke Hillebrandt — 2024-04-04T16:28:32.371Z
Link: https://lets-fund.org/clean-energy/eval/

Let's Fund researches pressing problems, like climate change, and then crowdfunds for nonprofits working on effective policy solutions. One such policy is clean energy innovation (e.g. via more grant funding for scientists to invent better solar panels).

Making clean energy cheaper has many benefits because it reduces:

Emissions
Energy poverty
Air pollution (which kills millions a year)
Revenue for autocratic petrostates[1]
Extreme climate risks (since if countries agreements to reduce emissions like Paris were to break down, cheaper clean energy hedges against this[2])

Since 2019, we’ve crowdfunded $1M for the Center for Clean Energy Innovation (CCEI) at ITIF, a non-profit think tank in DC. One example of our grantees work is researching the effects of higher and smarter clean energy R&D spending and communicating the results to policy-makers. Our research showed that this is the most effective climate policy[3] and was featured on Vox (which Bill Gates retweeted![4]).

As a result, ~2000 donors crowdfunded a $1M+ for CCEI to do more think tank work (e.g. do research, talk to policy-makers, etc.).

Here I show how with our grant, CCEI might have e.g. shifted >$100M from less effective clean energy deployment (e.g. subsidies) to more neglected and effective clean energy R&D. The donations might avert a ton CO₂ for less than $0.10.

That a leading think tank can cause such shifts becomes plausible, if we look at the pivotal ('hingey') timeline of a political climate so favorable that climate budgets went up by an unprecedented scale:

2020: Big Government Dems win the presidency, house and a razor-thin margin senate majority. Then a CCEI researcher gets a job advising Biden's climate envoy, John Kerry, who had endorsed and blurbed CCEI's Energizing America report and which has been called 'a very influential report', and advice for Biden on how to reform the energy innovation system.[5]

2021: COVID leads to a massive stimulus that includes ~$42B for clean energy RD&D, doubling the yearly budget- an ~$10B increase: [6]

US DoE RD&D budget in 2021 $ - Source: Energizing Innovation in FY 2023: Interactive Dataviz | CCEI

This US leadership led 16 countries to pledge ~$100B for the Clean Energy Technologies Demonstration Challenge recently.

These increases were politically tractable thanks to tens of thousands of clima...
The Buckling World Hypothesis - Visualising Vulnerable Worlds
by Rosco-Hunter — 2024-04-04T15:51:59.151Z

Motivation.

Mark Zuckerberg’s notorious motto, “move fast and break things'' [1], reflects a mindset shared by many of the most powerful entrepreneurs in Silicon Valley. This mindset rests on the assumption that the benefits of discovering advanced technologies will ultimately outweigh any disruptions (i.e., broken things) created along the way. However, Nick Bostrom’s vulnerable world hypothesis (VWH) [2] presents a sobering alternative. It supposes that there exist certain advanced technologies whose discovery would break society to the point of devastation. The purpose of this article isn’t to argue that we live in a vulnerable world, but rather to examine and visualise what such a world might look like. The intention is to provide a useful tool for policymakers governing advanced technologies, regardless of whether the VWH is proven true. In order to clarify the dynamics of a vulnerable world, we'll first explore a simpler phenomenon, that of a buckling ruler.

Introducing Buckling.

Imagine yourself holding a standard plastic ruler horizontally between your hands and applying some level of force to either end. For small forces, the straight ruler appears resilient (A). In this low-force regime, a small perturbation to the ruler’s curvature is quickly resolved as the ruler snaps back into shape. However, there is a critical threshold at which the force becomes sufficient to provoke a dramatic change. At this point, the ruler succumbs to instability and buckles either downwards (B) or upwards (C). This illustrates how buckling can flip formerly stable states (a straight ruler) into instability while establishing new stable states (a curved ruler) that were previously unstable. Mathematicians would depict this in the diagram below [3].

Buckling as a model of the VWH.

The diagram presented below adapts our model of the buckling ruler to visualise a vulnerable world, replacing “force” with “technological progress'' and “curvature” with “state power.” In this article, we employ a straightforward characterisation of “state power” as the capacity of the state to prevent and mitigate threats to its security. To qualify what may otherwise be an overly abstract diagram, we label three regions along the y-axis. In particular, we define regions of absolute tyranny, where a collapse of the existing order appears implausible, and absolute anarch...
Can AI Transform the Electorate into a Citizen’s Assembly?
by Rosco-Hunter — 2024-04-04T15:45:31.075Z

Introduction.
When at their best, democracies are able to transform diverse beliefs into effective real-world policies. This ideal is achievable when citizens are well-informed, engaged, and open [1]. However, these favourable democratic conditions are increasingly undermined by the rise of misinformation [2] and polarisation [3], fuelled by the influence of AI. In this article, we explore how AI may yet solve the very problems it’s created and, in doing so, modernise our democracies to meet the demands of the 21st century. Before we explore AI’s potential contributions, let’s first outline and critique an existing democratic structure that will be relevant to our proposal.

Citizens Assemblies: A Democratic Success Story.
A Citizens Assembly involves a small, randomly selected group of citizens tasked with formulating policy recommendations on specific issues. Following expert briefings and active group debate, these recommendations are distilled into policy proposals, which are submitted to parliament. Members of parliament then use their expertise to scrutinise these proposals before potentially enacting them as policy.

Citizens assemblies engage everyday citizens in the political process beyond an infrequent and potentially disengaged act of voting. This fosters a deeper comprehension of legislation, bolsters government transparency, and cultivates a sense of democratic self-efficacy [4]. Moreover, the compromise required for a randomly selected group of individuals to reach a consensus provides an intuitive antidote to political polarisation. However, these advantages are inherently limited to the small group of citizens that partake in the assembly. Can we not extend a citizen’s assembly to encompass the whole electorate instead of just a small subset?

The Challenge of Scale.
There are two critical factors that impede our ability to extend citizen assemblies:

Information Overload: Humans are wired for small-group discussions; therefore, we’re unable to digest the contributions of the millions of interlocutors that inhabit our large-scale societies.
Insufficient Expertise: Experts lack the bandwidth to directly guide each member of the electorate to write informed and effective policies.

In the following section, we will explore how AI can address these challenges and augment assembly via a digital platform for the entire electorate. But what would su...
AI Discrimination Requirements: A Regulatory Review
by Deric Cheng — 2024-04-04T15:43:58.008Z

This article is the fifth in a series of ~10 posts comprising a 2024 State of the AI Regulatory Landscape Review, conducted by the

Governance Recommendations Research Program

at

Convergence Analysis

. Each post will cover a specific domain of AI governance (e.g.

incident reporting

,

safety evals

,

model registries

, etc.). We’ll provide an overview of existing regulations, focusing on the US, EU, and China as the leading governmental bodies currently developing AI legislation. Additionally, we’ll discuss the relevant context behind each domain and conduct a short analysis.

This series is intended to be a primer for policymakers, researchers, and individuals seeking to develop a high-level overview of the current AI governance space. We’ll publish individual posts on our website and release a comprehensive report at the end of this series.

What are discrimination requirements for AI? Why do they matter?

Discrimination requirements for AI are rules and guidelines aimed at preventing AIsystems from perpetuating or amplifying societal biases and unfairly disadvantaging certain groups of people based on protected characteristics like race, gender, age, religion, disability status, or sexual orientation. As AI increasingly powers high-stakes decision making in areas like hiring, lending, healthcare, criminal justice, and public benefits, these systems are likely to adversely impact certain subsets of the population without algorithmic bias management.

For example, an algorithm designed to identify strong resumes for a job application is

likely to predict correlations between the sex of a candidate and the quality of their resume

, reflecting existing societal biases (and therefore perpetuating them). As a result, certain classes of individuals may be adversely impacted by an algorithm that contains inherently discriminatory word associations.

Other

examples for algorithmic discrimination include

:

Biases in the type of online ads presented to website users
Biases in the error rates of facial recognition technology by race and gender
Biases in algorithms designed to predict risk in criminal justice

The usage of discriminatory factors such as sex, ethnicity, or age has been expressly prohibited by longstanding anti-discriminatory legislation around the globe, such as

Title VII of the US Civil Right Act of 1964

, the

U.N.

...