trentmkelly/LessWrong-43k huggingface dataset:
GPT-7: The Tale of the Big Computer (An Experimental Story)
by Justin Bullock — 2023-07-10T20:22:26.918Z

In the not-too-distant future, a remarkable transformation took place. The world had seen the rise and fall of many technologies, but none as impactful as the data processing machines. These machines, born from the marriage of silicon and code, were not just tools; they were partners in our quest for knowledge and prosperity. And they were surprisingly good at winning trivia nights, which was both amusing and slightly unsettling.

GPT-7, the first of these machines to truly change the world, was a marvel of its time. It was like a digital Sherlock Holmes, with an insatiable appetite for data and an uncanny ability to generate human-like text. It was the brainchild of a group of dedicated scientists and engineers, who poured their collective knowledge and resources into its creation. And, like any proud parent, they were slightly terrified of what they had created. After all, it's not every day you give birth to a superintelligent machine.

GPT-7's problem-solving capabilities were unparalleled. It could sift through vast amounts of data, find patterns that eluded even the most skilled human analysts, and propose solutions that were both innovative and effective. One of its most notable achievements was in the field of healthcare. It analyzed countless medical records, research papers, and clinical trials, and developed new treatment protocols that significantly improved patient outcomes. Diseases that were once considered incurable were now manageable, and in some cases, even curable. It was like having a digital House M.D., minus the snarky comments and the cane.

There was a time when the world was on the brink of a major energy crisis. Traditional sources of energy were depleting rapidly, and renewable energy technologies were not yet efficient enough to meet the global demand. It was a problem that had stumped the best human minds for decades. But for GPT-7, it was just another puzzle to solve.

The machine analyzed countless research papers, patents, and simulations, and proposed a solution that was as ingenious as it was unexpected. It involved a combination of solar power, energy storage, and a radical new approach to energy distribution. The solution was implemented, and within a few years, the energy crisis was averted. It was a triumph of machine intelligence, a testament to the power of data and algorithms. And it was a little eerie, like watching a chess...
Cost-effectiveness of professional field-building programs for AI safety research
by Dan H — 2023-07-10T18:28:36.677Z

Summary

This post explores the cost-effectiveness of AI safety field-building programs aimed at ML professionals, specifically the Trojan Detection Challenge (a prize), NeurIPS ML Safety Social, and NeurIPS ML Safety Workshop.

We estimate the benefit of these programs in ‘Quality-Adjusted Research Years’, using cost-effectiveness models built for the

Center for AI Safety

(introduction post here, full code

here

).

We intend for these models to support — not determine — strategic decisions. We do not believe, for instance, that programs which a model rates as lower cost-effectiveness are necessarily not worthwhile as part of a portfolio of programs.

The models’ tentative results, summarized below, suggest that field-building programs for professionals compare favorably to ‘baseline’ programs — directly funding a talented research scientist or PhD student working on trojans research for 1 year or 5 years respectively. Further, the cost-effectiveness of these programs can be significantly improved with straightforward modifications — such as focusing a hypothetical prize on a more ‘relevant’ research avenue, or running a hypothetical workshop with a much smaller budget.

Program	Cost (USD)	Benefit (counterfactual expected QARYs)	Cost-effectiveness (QARYs per $1M)
Trojan Detection Challenge	65,000	26	390
NeurIPS ML Safety Social	5200	150	29,000
NeurIPS ML Safety Workshop	110,000	360	3300
Hypothetical: Power Aversion Prize	50,000	490	9900
Hypothetical: Cheaper Workshop	35,000	250	7000
Baseline: Scientist Trojans	500,000	84	170
Baseline: PhD Trojans	250,000	8.7	35

For readers who are after high-level takeaways, including which factors are driving these results, skip ahead to the

cost-effectiveness in context

section. For those keen on understanding the model and results in more detail, read on as we:

Give important disclaimers. (Read more.)
Direct you to background information about this project. (Read more.)
Walk through the model. (Read more.)
Contrast these programs with one another, and with funding researchers directly. (Read more.)
Consider the scalability and robustness properties of the model. (Read more.)

Disclaimer

This analysis is a starting point for discussion, not a final verdict. The most critical reasons for this are that:

These models are reductionist. Even if we have avoided other pitfalls associated with cost-effectiveness an

...
Cost-effectiveness of student programs for AI safety research
by Dan H — 2023-07-10T18:28:18.073Z

Summary

This post explores the cost-effectiveness of field-building programs for students, specifically the Atlas Fellowship (a rationality program, with some AI safety programming), MLSS (an ML safety course for undergraduates), a top-tier university student group, and undergraduate research stipends.

We estimate the benefit of these programs in ‘Quality-Adjusted Research Years’, using cost-effectiveness models built for the

Center for AI Safety

(introduction post here, full code

here

). Since our framework focuses on benefits for technical AI safety research exclusively, we will not account for other benefits of programs with broader objectives, such as the Atlas Fellowship.

We intend for these models to support — not determine — strategic decisions. We do not believe, for instance, that programs which a model rates as lower cost-effectiveness are necessarily not worthwhile as part of a portfolio of programs.

The models’ tentative results, summarized below, suggest that student groups and undergraduate research stipends are considerably more cost-effective than Atlas and MLSS. (With many important caveats and uncertainties, discussed in the post.) Additionally, student groups and undergraduate research stipends compare favorably to ‘baseline’ programs — directly funding a talented research scientist or PhD student working on trojans research for 1 year or 5 years respectively.

Program	Cost (USD)	Benefit (counterfactual expected QARYs)	Cost-effectiveness (QARYs per $1M)
Atlas	9,000,000	43	4.7
MLSS	330,000	6.4	19
Student Group	350,000	50	140
Undergraduate Stipends	50,000	17	340
Baseline: Scientist Trojans	500,000	84	170
Baseline: PhD Trojans	250,000	8.7	35

For readers who are after high-level takeaways, including which factors are driving these results, skip ahead to the

cost-effectiveness in context

section. For those keen on understanding the model and results in more detail, read on as we:

Give important disclaimers. (Read more.)
Direct you to background information about this project. (Read more.)
Walk through the model. (Read more.)
Contrast these programs with one another, and with funding researchers directly. (Read more.)
Test the robustness of the model. (Read more.)

Disclaimers

This analysis is a starting point for discussion, not a final verdict. The most critical re...
Modeling the impact of AI safety field-building programs
by Dan H — 2023-07-10T18:27:24.807Z

Summary

AI safety field-building is a large space, with many possible programs. The Center for AI Safety (CAIS) has commissioned cost-effectiveness models to support decisions regarding how to prioritize between these programs. Dan provided high-level guidance early on in the process as well as a small amount of sanity checks later on.

This analysis is a starting point for discussion, not a final verdict. Numbers should not be taken out of context and are not necessarily endorsed fully by CAIS. Rather, we believe that these models:

Provide a framework for modeling the effectiveness of field-building programs.
Make explicit underlying assumptions which then can be the focal point of further discussions.
Provide one perspective (and by no means the only perspective) on the cost-effectiveness of different programs.

These models have so far assisted our evaluations of field-building programs for students and professionals, detailed here and here respectively.

This post:

Provides motivations for using explicit models. (Read more.)
Details the metric used in the model to evaluate field-building programs. (Read more.)
Guides you on adopting or adapting the models for your own work. (Read more.)
Teases tentative results, and analyzes an important crux: one’s preference for current versus future research. (Read more.)

Motivations for explicit models

We expect that many readers of this post will already be on-board with the practice of using explicit models to guide decision-making — historically, cost-effectiveness analyses have been core to effective altruism. At the same time, however, we expect that few do this in practice, in part because evaluating the cost-effectiveness of AI safety programs might be more challenging than evaluating global development or animal welfare programs. This section justifies our belief that using explicit models for AI safety is worthwhile in practice.

We see two core benefits:

Enhanced strategic decision-making, for CAIS as well as other organizations.
Improved communication between organizations.

When making strategic decisions, we are often interested in questions such as:

What assumptions does our view on the impact of program X depend on?
Which information is most important for us to learn about program X?
At what scale should we operate program X?
How should we make allocation decisions between programs?

The first benefit — enhanced strategic d...
I think Michael Bailey's dismissal of my autogynephilia questions for Scott Alexander and Aella makes very little sense
by tailcalled — 2023-07-10T17:39:26.325Z

I am autogynephilic, and there's been a lot of autogynephilia talk lately. One subject that sometimes comes up, but hasn't been discussed much on LessWrong, is how common autogynephilia is in cis women.

Two datasets that are sometimes used for this question are Scott Alexander's and Aella's. Part of how they happened to be made is that I reached out to Scott and Aella, suggesting experimental questions for assessing autogynephilia, in ways that might function in cis women too and not just cis men[1]:

Picture a very beautiful woman. How sexually arousing would you find it to imagine being her?
Do you find the thought of masturbating alone as a woman to be erotic?

This is somewhat different from usual autogynephilia measures, which contain questions such as:

Did you ever feel sexually aroused when putting on females' underwear or clothing?
Have you ever become sexually aroused while picturing your nude female breasts?

The reason I didn't include these latter questions is because it seems likely to me that they will be interpreted differently for males and females (e.g. if males do not have female anatomy, then they cannot be aroused by it literally, so instead they get aroused by imagining some other female anatomy that they don't actually have), and because these usual questions seem very bad if taken literally ("ever" and "while" rather than "how frequently" and "by" seem like there could be a lot of ways to get affirmative answers while not actually being autogynephilic - though the low rate of endorsement among women suggests to me that they are not taking it literally?).

Anyway, Michael Bailey (activist researcher for autogynephilia ideology) responds in Aporia Magazine, saying "It's important not to confound y'know being seen as a woman or having a female body from having sex with a partner while having a female body or the prospect of going out on a date while wearing sexy clothes".

I don't think this makes any sense. "Do you find the thought of masturbating alone as a woman to be erotic?" explicitly states that one doesn't have a partner around. "Picture a very beautiful woman. How sexually arousing would you find it to imagine being her?" doesn't suggest a partner any more than "Did you ever feel sexually aroused when putting on females' underwear or clothing?" or "Have you ever become sexually aroused while picturing your nude female breasts?" does. This seem...
Incentives from a causal perspective
by tom4everitt — 2023-07-10T17:16:28.373Z

Post 4 of

Towards Causal Foundations of Safe AGI

,

preceded by Post 1: Introduction, Post 2: Causality, and Post 3: Agency.

By Tom Everitt, James Fox, Ryan Carey, Matt MacDermott, Sebastian Benthall, and Jon Richens, representing the Causal Incentives Working Group. Thanks also to Toby Shevlane and Aliya Ahmad.

“Show me the incentive, and I’ll show you the outcome” – Charlie Munger

Predicting behaviour is an important question when designing and deploying agentic AI systems. Incentives capture some key forces that shape agent behaviour,[1] which don’t require us to fully understand the internal workings of a system.

This post shows how a causal model of an agent and its environment can reveal what the agent wants to know and what it wants to control, as well as how it will respond to commands and influence its environment. A

complementary result

shows that some incentives can only be inferred from a causal model, so a causal model of the agent’s environment is strictly necessary for a full incentive analysis.

Value of information

What information would an agent like to learn? Consider, for example, Mr Jones deciding whether to water his lawn, based on the weather report, and whether the newspaper arrived in the morning. Knowing the weather means that he can water more when it will be sunny than when it will be raining, which saves water and improves the greenness of the grass. The weather forecast therefore has

information

value

for the sprinkler decision, and so does the weather itself, but the newspaper arrival does not.

We can quantify how useful observing the weather is for Mr Jones, by comparing his expected utility in a world in which he does observe the weather, to a world in which he doesn’t. (This measure only makes sense if we can assume that Mr Jones adapts appropriately to the different worlds, i.e. he needs to be

agentic

in this sense.)

The causal structure of the environment reveals which variables provide useful information. In particular, the

d-separation criterion

captures whether information can flow between variables in a causal graph when a subset of variables are observed. In single-decision graphs, value of information is possible when there is an information-carrying path from a variable to the agent’s utility node, when conditioning on the decision node and its parents (i.e. the “observed...
Is the Endowment Effect Due to Incomparability?
by Kevin Dorst — 2023-07-10T16:26:06.508Z
Link: https://kevindorst.substack.com/p/is-the-endowment-effect-due-to-incomparability

Thanksgiving break, early 2000s. A hectic time in the Dorst household. Grandparents are in town. The oversized puppy threatens books, lamps, and (he thinks) squirrels. There are meals to make, grandparents to impress, children to manage.

But most important—at least, according to said children—a decision must be made:

PlayStation 2 or GameCube?

We’d read the reviews, perused the games, and written pro/con lists. Though torn, we had a tentative verdict: PlayStation. It was cooler, faster, more “mature”.

Then, while casually strolling through my parents’ closet (as one does, during the holidays), I noticed a box. A GameCube box. Uh oh. It wasn’t too late. There was time yet to orchestrate an exchange—to get a cooler, faster, more mature console.

We didn’t. After all (we quickly concluded), the GameCube had better games, nicer controllers, and was more fun than the PlayStation, anyways. Thank goodness they got the GameCube!

This is an instance of what economists call the endowment effect: owning an item makes people value it more.

It’s one of the key findings that motivated the rise of behavioral economics, for it’s difficult to explain using standard economic models. Those models assume complete, stable preferences between options—meaning that if you prefer a PlayStation to a GameCube, then owning the latter shouldn’t change that.

There’s been a spirited debate about the causes of the endowment effect, but many proposals on the table assume that it’s in some way irrational.

My proposal: The endowment effect might be due to the fact that—unlike standard economic models—most real-world decisions are rife with incomparability: neither option is better than the other, but nor are they precisely equally good.

This idea is independently motivated, and offers a simple explanation of some of the main empirical evidence.

The Empirical Evidence

There are two main type of experiments supporting the endowment effect.

First, the valuation paradigm. Half of our subjects are randomly assigned as “buyers”, the rest as “sellers”. An item—say, a university-branded coffee mug—is presented. Buyers are simply shown it, while sellers are given it.

Buyers are asked how much money they’d be willing to pay (WTP) to buy the mug. Sellers are asked how much money they’d be willing to accept (WTA) to sell the mug.

The result? Buyers are willing to pay less—way less—than sellers are willing to accep...
Frontier AI Regulation
by Zach Stein-Perlman — 2023-07-10T14:30:06.366Z
Link: https://arxiv.org/pdf/2307.03718.pdf

This paper is about (1) "government intervention" to protect "against the risks from frontier AI models" and (2) some particular proposed safety standards. It's by Markus Anderljung, Joslyn Barnhart (Google DeepMind), Jade Leung (OpenAI governance lead), Anton Korinek, Cullen O'Keefe (OpenAI), Jess Whittlestone, and 18 others.

Abstract

Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term “frontier AI” models — highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model’s capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.

Executive Summary

The capabilities of today’s foundation models highlight both the promise and risks of rapid advances in AI. These models have demonstrated significant potential to benefit people in a wide range of

...
Why is it so hard to change people's minds? Well, imagine if it wasn't...
by Celarix — 2023-07-10T13:55:38.239Z

Epistemic status: Tying together the great works of others into something less great

I think the mind's defenses against change in its beliefs are a form of anti-parasitism.

Society commonly bemoans the difficulty in changing people's minds. We wish we could change the minds of our friends and family about all sorts of issues: vaccines, policy issues, religious beliefs or lack thereof, and on and on.

We struggle to convince ourselves of things, too. Diet, exercise, sleep, laziness or workaholism. We make the same New Year's Resolutions, year in and year out, only to drop them after a week or two, just like every year past.

When we try to change someone's mind, even our own, we do so in a remarkably useless way. If we're not flinging angry insults on Facebook, at best we're running down the same old list of reasons to believe X that they've already heard, processed, and created counterexamples for years ago. When we try to change our own minds, we try pumping ourselves up, insulting ourselves, or just declaring Today to be The Day that Things Change™.

I've been in more arguments and debates than I care to count. They usually don't lead anywhere: I say my side, they say theirs, and we both walk away even more convinced of our respective correctness. I've been the recipient of much well-meaning advice over the years, too, but my mind is excellent at making up counterexamples or not being swayed by the words (most of which are unchanged since I first heard them). And, of course, I've told myself many things about how to live and what to do. Sometimes I even believe my mind has finally integrated them.

In some sick meta joke, our belief that mere arguments and debates and advice can change minds is itself resistant to change. We get caught in the same old arguments that go exactly the same way. We use the same old talking points, believing subconsciously that this time, they'll surely work. Our debates become mere reflex, yet we never question our methods when they invariably fail again.

The mind's defenses are powerful. If the part of the brain that is able to integrate new information is damaged, we see a module of the brain completely take over and spin up arbitrary stories defending pre-existing beliefs.

Even in healthy people, our beliefs color our perception. Any ambiguity is resolved, automatically, subconsciously, in favor of what we already know. The misanthrope...