trentmkelly/LessWrong-43k huggingface dataset:
If AI is in a bubble and the bubble bursts, what would you do?
by Remmelt — 2024-08-19T10:56:03.948Z

"The AI bubble is reaching a tipping point", says Sequoia Capital.

AI companies paid billions of dollars for top engineers, data centers, etc. Meanwhile, companies are running out of 'free' data to scrape online and facing lawsuits for the data they did scrape. Finally, the novelty of chatbots and image generators is wearing off for users, and fierce competition is leading to some product commoditisation.
No major AI lab is making a profit yet (while downstream GPU providers do profit). That's not to say they won't make money eventually from automation.

It looks somewhat like the run-up of the Dotcom bubble. Companies then too were awash in investments (propped up by low interest rates), but most lacked a viable business strategy. Once the bubble burst, non-viable internet companies got filtered out.

Yet today, companies like Google and Microsoft use the internet to dominate the US economy. Their core businesses became cash cows, now allowing CEOs to throw money at AI as long as a vote-adjusted majority of stakeholders buys the growth story. That marks one difference with the Dotcom bubble. Anyway, here's the scenario:

How would your plans change if we saw an industry-wide crash?
Let's say there is a brief window where:

Investments drop massively (eg. because the s-curve of innovation did flatten for generative AI, and further development cycles were needed to automate at a profit).
The public turns sour on generative AI (eg. because the fun factor wore off, and harms like disinformation, job insecurity, and pollution came to the surface).
Politicians are no longer interested in hearing the stories of AI tech CEOs and their lobbyists (eg. because political campaigns are not getting backed by the AI crowd).

Let's say it's the one big crash before major AI labs can break even for their parent companies (eg. because mass-manufacturing lowered hardware costs, real-time surveillance resolved the data bottleneck, and multi-domain-navigating robotics resolved inefficient learning).

Would you attempt any actions you would not otherwise have attempted?
Thinking About Propensity Evaluations
by Maxime Riché — 2024-08-19T09:23:55.091Z

Warning: This post was written at the start of 2024 as part of the AISC project "Evaluating Alignment Evaluations". We are not especially satisfied with the quality reached, but since we are not planning to work on it anymore, we are releasing it as a Work-In-Progress document.

Introduction

In this post, we cover:

What are propensity evaluations?
How do they differ from capability evaluations?
Which propensities are currently evaluated?
A few approaches to propensity evaluation

Propensity evaluations (

What's up with "Responsible Scaling Policies"? — LessWrong

,

A Causal Framework for AI Regulation and Auditing

—

Apollo Research

,

We need a Science of Evals — Apollo Research

) evaluate which kind of behavior a model prioritizes over other kinds. Groups of behaviors need to be specified for each propensity evaluation. A common denominator to all propensity evaluations is that the clusters of behaviors being compared are not created using only capability-related features. Otherwise, the evaluations would be called capability evaluations, not propensity evaluations. Propensity evaluations are characterized by the use of at least some non-capability-related features to create the clusters of behaviors that are studied.

Propensity evaluations include evaluations of alignment in actions, and those will likely be needed when dangerous capability levels are reached, which could be as soon as this year. For more detail, see the section:

Motivation

.

We will mostly focus on a special case of propensity evaluations: propensity evaluations with low capability-dependence. These can be visualized as evaluations of priorities between clusters of behaviors that are constructed without almost no capability-related features. For more detail, see the section:

Defining propensity evaluations

.

Takeaways

Propensity evaluations evaluate which kind of behaviors a model prioritizes in terms of action, and NOT in terms of motivation.
Most evaluations used at the moment are either capability evaluations or

propensity evaluations

with high capability-dependence. We argue that this can cause misrepresentations or misunderstandings when studying scaling trends of propensities.

Propensity evaluations

(with low capability-dependence) differ from capability evaluations in the following ways:

The nature of the behaviors evaluated.
The infor

...
A Taxonomy Of AI System Evaluations
by Maxime Riché — 2024-08-19T09:07:45.224Z

Warning: This post was written at the start of 2024 as part of the AISC project "Evaluating Alignment Evaluations". We are not especially satisfied with the quality reached, but since we are not planning to work on it anymore, we are releasing it as a Work-In-Progress document.

Introduction

TLDR

We assemble and extend existing taxonomies of AI system evaluations into a more exhaustive but still Work-In-Progress taxonomy of AI system evaluations.
The taxonomy aims to bring more clarity about the characteristics of AI system evaluations. Not only about what they evaluate, but also how the evaluation is done and how the results can be used.
You are welcome to contribute to improving the taxonomy. If you spot misleading or incorrect content, please inform us, and we will correct it or at least add warnings.

Introduction to evaluations

For an introduction to model evaluations, see

A starter guide for evals — LessWrong

. For clarification about what evaluations are relative to characterizations, see also

What is the difference between Evaluation, Characterization, Experiments, and Observations

.

Motivation

AI system evaluations (which include model evaluations and evaluation of

scaffoldings

), are used by AI researchers to further capability or safety research.
They are used by AI labs to assess the capability, usefulness, and risks of their models.
They are also used by governance organizations for evaluating current risks and to be used as triggers in conditional policies (

ref

,

ref

).

Clarifying and understanding the characteristics of AI system evaluations is important to better use and design them.

Audience

The taxonomy is written for people who already know a minimum about model evaluations and AI systems. For example, you already read

A starter guide for evals

, and you already worked with or analyzed evaluations for a few days or more. For audience members with less knowledge, you may expect to not understand all the content.

The goal of the taxonomy

Our taxonomy focuses on features of AI system evaluations.

AI system evaluations: Evaluate properties of AI systems (e.g., capabilities, propensities, beliefs, latency)

It does NOT try to include the features relevant for the following evaluations:

Control evaluations

: Evaluate the level of control the overseer has over an AI system.

Understanding-based evaluations

: Eval

...
Beware the science fiction bias in predictions of the future
by nsokolsky — 2024-08-19T05:32:47.372Z
Link: https://nsokolsky.substack.com/p/beware-the-science-fiction-bias-in

We want to know the future, so we like reading predictions. These predictions can significantly vary in how long their time horizon is:

Seconds to Minutes: poker players, high-frequency traders
Minutes to Hours: day traders, in-game sports betting
Days to Weeks: weather forecasts, outcomes of sports tournaments
Months to Years: hedge funds, election outcomes, venture capitalists
Decades to Centuries: technological trends, climate change projections
Centuries to Millennia: rise and fall of countries and civilizations
Millions to Billions of Years: life cycle of stars, ultimate fate of the Solar System and our universe

There’s three elements in how predictions are evaluated by society:

Accuracy: easy for short time horizons, hard for long ones. Poker players will quickly lose their money if their judgement is poor. Weather models will go out of favor if they consistently predict sun when it rains. But a VC fund can continue operating for a decade or two despite making glaring mistakes.
Political bias: if you like nuclear power you’ll be inclined to believe predictions promising a fusion power revolution. If you dislike Trump you’ll be skeptical of a model projecting him as the winner of the next election.
Quality of storytelling: a prediction that follows the patterns of good fiction is more interesting to read and would thus be more popular, just like Star Wars is always going to be more popular than a book on space engineering. I would call this the science fiction bias.

We all like a good story

Imagine the year is 2011 and you’re writing a movie about a future global pandemic. You hire a scientist known for helping eradicate smallpox (Larry Brilliant) as your consultant and overall try to make your story somewhat plausible. You title the movie ‘Contagion’ and release it with the following plot elements:

The virus has a fatality rate of 25-30% in every age group.
Infection develops extremely rapidly, with people dying within 48 hours after being infected.
It infects a large number of people in every country within weeks after the first human infection.
Society comes close to a full collapse, people are scared to leave their homes.
The US enforces a very harsh quarantine in every city.
And, of course, patient zero is Gwyneth Paltrow eating at a fancy restaurant.

Nine years later a global pandemic happens in real life but (fortunately) the plotline is much more mundane:

The fatali

...
Interdictor Ship
by lsusr — 2024-08-19T04:59:18.487Z

The standard operating procedure against a primitive forest-dwelling enemy is to retreat to orbit and then bombard the planet until there isn't a forest anymore. However, the only reason the Galactic Empire was in the Alpha Centauri A system in the first place was because of the fragile mineral resources underneath that forest. Dropping tungsten rods at hypersonic speeds would risk destroying the only thing of value on Pandora.

Alien aborigines armed with bows and arrows damaged Imperial legitimacy across the galaxy. It was like losing a battle to Ewoks.

The Emperor's solution had been to hire an Ewok.

Mitth'raw'nuruodo was a dwarf by Na'vi standards, but the blue alien stood a head above most humans. Originally hired as a translator, the Imperials on Pandora quickly noticed that the only patrols who came back alive were those that followed Mitth'raw'nuruodo's advice. Pretty soon, the moon was pacified and Mitth'raw'nuruodo was its de facto king.

Nobody liked the idea of an alien being in control of such a strategically-valuable moon. To get rid of him, they promoted Mitth'raw'nuruodo to Admiral. In space, many parsecs away from Pandora, the humans under Mitth'raw'nuruodo's command couldn't pronounce "Mitth'raw'nuruodo".

That was fine, thought Mitth'raw'nuruodo. Everyone just called him "Thrawn".

Amateurs talk strategy. Professionals talk logistics.

The Imperial Navy Defense Acquisitions Board (INDAB) originally met on Coruscant, but was moved to the Life Star for security reasons. Idle chitchat usually preceded the important negotiations.

"What I don't get is why we call it the 'Life Star'," said Chief Bast, "This thing blows up planets. Shouldn't it be called the 'Death Star'?"

"Do you want us to look like the bad guys?" said General Tagge, "The Department of Defense isn't called the 'Department of War'. The Department of Justice isn't called the 'Department of Incarceration'. The Department of Education isn't called the 'Department of Child Indoctrination'. Calling this megastructure the 'Life Star' buys us legitimacy for the low, low price of zero Galactic Credits."

"But won't people call us out on our Bantha fodder when we call things the opposite of what they really are?" said Chief Bast.

"Humans don't. Aliens sometimes make a fuss about it," General Tagge said, "No offense, Admiral."

"None taken," said Thrawn.

"Speaking of which, I've read your recent report,...
Why you should be using a retinoid
by GeneSmith — 2024-08-19T03:07:41.722Z

If you want the 60 second version of this post that just tells you what to do, click here to skip to the summary.

There is a cheap, widely available, extremely effective treatment for skin aging that has been around for decades and almost no one outside of dermatologists and beauty bloggers seems to know about it.

It's called a retinoid. I first learned about their existence a few months ago after looking in the mirror one day and noticing I was starting to get permanent wrinkles around my mouth. Naturally, I wondered if there was anything I could do to fix them.

An ex of mine was a skincare addict and had perhaps the nicest skin of anyone I have ever met. I texted her to ask for advice and she recommended I use a retinoid.

Since I didn't know what those were or how they worked, I watched a YouTube video. Thus began my 3 month journey down the rabbit hole of skin care product reviews and progress videos. In this post I'll summarize what I've learned.

What are retinoids?

Retinoids are a family of medications derived from vitamin A. In the same way that Ozempic was originally developed as an anti-diabetes drug and later turned out to have a broader set of benefits, retinoids were originally developed to treat acne but turned out to do far more than clear up breakouts.

These effects can be summed up as "improving almost everything about skin". If we had medications that worked as well for other organs as retinoids work for skin, people would probably live well into their hundreds. It's actually kind of remarkable just how well retinoids work.

Exactly HOW retinoids work is a little difficult to describe because they seem to do so many different things. Here's a brief list:

Retinoids increase collagen production
They decrease degradation of collagen within the skin
They protect the extracellular matrix by reducing the activity of metalloproteinases
They thicken the epidermis, which tends to thin as we age
They increase the formation of blood veseels, which makes the skin's color look nicer and speeds wound healing
They increase the levels of fibronectin and tropoelastin, which makes for firmer, bouncier skin

These things just sound kind of vague and boring until you start to look at people who have used retinoids for a long time. Here's a screenshot of "Melissa55" on YouTube, a woman in her late 60s that has been using Retin-A (the first available retinoid) for 28 years.

This wo

...
Liability regimes for AI
by Ege Erdil — 2024-08-19T01:25:01.006Z

For many products, we face a choice of who to hold liable for harms that would not have occurred if not for the existence of the product. For instance, if a person uses a gun in a school shooting that kills a dozen people, there are many legal persons who in principle could be held liable for the harm:

The shooter themselves, for obvious reasons.

The shop that sold the shooter the weapon.

The company that designs and manufactures the weapon.

Which one of these is the best? I'll offer a brief and elementary economic analysis of how this decision should be made in this post.

The important concepts from economic theory to understand here are Coasean bargaining and the problem of the judgment-proof defendant.

Coasean bargaining

Let's start with Coaesean bargaining: in short, this idea says that regardless of who the legal system decides to hold liable for a harm, the involved parties can, under certain conditions, slice the harm arbitrarily among themselves by contracting and reach an economically efficient outcome. Under these conditions and assuming no transaction costs, it doesn't matter who the government decides to hold liable for a harm; it's the market that will ultimately decide how the liability burden is divided up.

For instance, if we decide to hold shops liable for selling guns to people who go on to use the guns in acts of violence, the shops could demand that prospective buyers purchase insurance on the risk of them committing a criminal act. The insurance companies could then analyze who is more or less likely to engage in such an act of violence and adjust premiums accordingly, or even refuse to offer it altogether to e.g. people with previous criminal records, which would make guns less accessible overall (because there's a background risk of anyone committing a violent act using a gun) and also differentially less accessible to those seen as more likely to become violent criminals. In other words, we don't lose the ability to deter individuals by deciding to impose the liability on other actors in the chain, because they can simply find ways of passing on the cost.

The judgment-proof defendant

However, what if we imagine imposing the liability on individuals instead? We might naively think that there's nothing wrong, because anyone who used a gun in a violent act would be required to pay compensation to the victims which in principle could be s...
Something Is Lost When AI Makes Art
by utilistrutil — 2024-08-18T22:53:46.951Z

That's right: a neartermist take! Cower before its sublime wrath!

The Resurrection of the Author

Here, hold my bee.

— Someone with beauty in their eye

There are many theories of aesthetics that seek to explain the value of art and the nature of beauty. On some of these theories, the artist's role is subordinate to the viewer's. Aesthetic value is located in a private experience between the viewer and the artistic object. The beauty of an intricate painting is not too different from the beauty of a sunset: the painter is irrelevant.

If you think of art in these terms, you might be excited by the prospect of a proliferation of cheap and beautiful AI-generated artwork. Even if you don't think art is especially enjoyable, you might still hold a consequentialist view on which art plays a historical role in capturing the zeitgeist of the time, or a didactic role in holding up a mirror to society. Arguably, AI-generated art could carry out both functions better than human-generated art.

I am receptive to all this! But there is an argument against AI-generated art that I worry is overshadowed by artists' vocal concerns about job loss. Is the argument important enough to outweigh the benefits of AI-generated art? Uh, I don't know, probably not, you decide.[1]

The argument is this: The most significant way that art has impacted my life is by fostering a feeling of connection to artists. The art is just a conduit by which I learn about an artist's experiences: then I relate to the artist and feel more understood and less alone in my own experiences. Art is uniquely suited to this purpose, in part because it allows the artist to obfuscate their message from unsympathetic audiences.

James Baldwin said it best:

You think your pain and your heartbreak are unprecedented in the history of the world, but then you read. It was Dostoevsky and Dickens who taught me that the things that tormented me most were the very things that connected me with all the people who were alive, or who ever had been alive. Only if we face these open wounds in ourselves can we understand them in other people. An artist is a sort of emotional or spiritual historian. His role is to make you realize the doom and glory of knowing who you are and what you are. He has to tell, because nobody else in the world can tell, what it is like to be alive.[2]

AI-generated art lacks this property because the artist did not ...
Scaling Laws and Likely Limits to AI
by Davidmanheim — 2024-08-18T17:19:46.597Z

A number of arguments about AI capabilities have deep roots in different expectations about how scaling can work, and historical analogues are often raised. This is true on both sides of the debate, with

AI Snake Oil noting the end of scaling for processors and aircraft speeds

, an Open Philanthropy researcher with

a draft report looking at trends and biological anchors

, work for

Metaculus on compute scaling trends

, and

Epoch AI

tracking scaling trends and expected limits. This leads to questions in the other direction as well;

AI Impacts looks at discontinuities in progress

, considering the question of whether scaling might jump, rather than proceed at “only” exponential pace.

One very basic issue is how to choose the metrics in question, and how the metrics relate to the actual dynamics in question - a topic which itself is a more general interest of mine. The question of what metric continues to improve over time is a critical but difficult one - and is deeply related to whether we expect progress from AI in general.

We can see that performance on a given test is a bad metric, for (at least) three reasons. First, there is a built-in limit of perfection on the test, and previous tests of AI have been discarded once ML obviously has beaten them. The Turing test has been abandoned, perhaps because one obvious way to tell the difference between a current LLM system and a human is that a human won’t have near-universal expertise and knowledge. Similarly, almost

all new LLMs outperform humans on knowledge-based tests like SQuAD

. Second, many benchmarks have leaked into training datasets for many modern LLMs, so that many models can answer multiple choice questions from these datasets correctly by picking A,B, C, or D even when the text of the options are not provided. Similarly,

many models perform substantially worse

on test sets with similar-difficulty but distinct questions. If this wasn’t sufficient evidence, their success rate on questions which are nonsensical or impossible far exceeds chance. This brings us to the third and final point; large test sets often contain a proportion of questions which are unreasonable or garbage - as shown for

MMLU

, and

CoQA

.

But whatever it is that LLMs are improving at doing over time, they are clearly imperfect, and there is room to improve. Still, on one hand, obviously, some limits exist. Physics dictates the sp...
What is "True Love"?
by johnswentworth — 2024-08-18T16:05:47.358Z

Meta: I recently entered the dating market, so naturally I have lots of random thoughts on the subject which you all get to suffer through for a while. Your usual diet of dry math and agency theory will resume shortly.

Obviously the phrase “true love” has been so thoroughly overdone in so much fiction as to lose all substantive meaning. That’s what happens when we leave important conceptual work to would-be poets. We’re here to reclaim the term, because there’s a useful concept which is very naturally described by the words “true” and “love”.

You know

that thing

where, when you’re smitten by someone, they seem more awesome than they really are? Your brain plays up all the great things about them, and plays down all the bad things, and makes up stories about how great they are in other ways too? And then you get even more smitten by them? All that perceived-wonderfulness makes your attraction a steady state? That’s part of normal being-in-love.

… and there’s something “false” about it. Like, in some sense, you’re in love with an imaginary person, not the real person in front of you. You’re in love with this construct in your head whose merits are greater and shortcomings more minor than the real person who triggered the cascade in your heart.

But what if you can see the target of your affection with clear eyes and level head, without the pleasant tint of limerance skewing your perception, and still feel a similar level of love? What if they are actually that good a fit to you, not just in your head but in real life? Well, the obvious name for that would be “true love”: love which is not built on a map-territory mismatch, but rather on perception of your loved one as they really are.

And that does actually seem like a pretty good fit for at least some of the poetry on the subject: loving your partner as they truly are, flaws and all, blah blah blah.

Alas, “false” love can still feel like “true” love from the inside as it’s happening. To tell it’s happening, you’d need to either be really good at keeping a level head, rely on feedback from other people you trust, or just wait until the honeymoon stage passes and find out in hindsight.