trentmkelly/LessWrong-43k huggingface dataset:
Ninety-five theses on AI
by hamandcheese — 2024-05-16T17:51:03.638Z

Originally posted to SecondBest.ca ; Zvi responds here.

I. Oversight of AGI labs is prudent

It is in the U.S. national interest to closely monitor frontier model capabilities.
You can be ambivalent about the usefulness of most forms of AI regulation and still favor oversight of the frontier labs.
As a temporary measure, using compute thresholds to pick out the AGI labs for safety-testing and disclosures is as light-touch and well-targeted as it gets.
The dogma that we should only regulate technologies based on “use” or “risk” may sound more market-friendly, but often results in a far broader regulatory scope than technology-specific approaches (see: the EU AI Act).
Training compute is an imperfect but robust proxy for model capability, and has the immense virtue of simplicity.
The use of the Defense Production Act to require disclosures from frontier labs is appropriate given the unique affordances available to the Department of Defense, and the bona fide national security risks associated with sufficiently advanced forms of AI.
You can question the nearness of AGI / superintelligence / other “dual use” capabilities and still see the invocation of the DPA as prudent for the option value it provides under conditions of fundamental uncertainty.
Requiring safety testing and disclosures for the outputs of $100 million-plus training runs is not an example of regulatory capture nor a meaningful barrier to entry relative to the cost of compute.

II. Most proposed “AI regulations” are ill-conceived or premature

There is a substantial premium on discretion and autonomy in government policymaking whenever events are fast moving and uncertain, as with AI.
It is unwise to craft comprehensive statutory regulation at a technological inflection point, as the basic ontology of what is being regulated is in flux.
The optimal policy response to AI likely combines targeted regulation with comprehensive deregulation across most sectors.
Regulations codify rules, standards and processes fit for a particular mode of production and industry structure, and are liable to obsolesce in periods of rapid technological change.
The benefits of deregulation come less from static efficiency gains than from the greater capacity of markets and governments to adapt to innovation.
The main regulatory barriers to the commercial adoption of AI are within legacy laws and regulations, mostly not prospective AI-spec

...
GPT-4o My and Google I/O Day
by Zvi — 2024-05-16T17:50:03.040Z

At least twice the speed! At most half the price!

That’s right, it’s GPT-4o My.

Some people’s expectations for the OpenAI announcement this week were very high.

Spencer Schiff: Next week will likely be remembered as one of the most significant weeks in human history.

We fell far short of that, but it was still plenty cool.

Essentially no one’s expectations for Google’s I/O day were very high.

Then Google, in way that was not in terms of its presentation especially exciting or easy to parse, announced a new version of basically everything AI.

That plausibly includes, effectively, most of what OpenAI was showing off. It also includes broader integrations and distribution.

It is hard to tell who has the real deal, and who does not, until we see the various models at full power in the wild.

I will start with and spend the bulk of this post on OpenAI’s announcement, because they made it so much easier, and because ‘twice as fast, half the price, available right now’ is a big freaking deal we can touch in a way that the rest mostly isn’t.

But it is not clear to me, at all, who we will see as having won this week.

So what have we got?

The GPT-4o Announcement

OpenAI: GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, and image and generates any combination of text, audio, and image outputs.

It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window) in a conversation.

It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.

…

With GPT-4o, we trained a single new model end-to-end across text, vision, and audio, meaning that all inputs and outputs are processed by the same neural network. Because GPT-4o is our first model combining all of these modalities, we are still just scratching the surface of exploring what the model can do and its limitations.

They are if anything underselling the speedup factor. This is a super important advance in practical terms. In other languages, they cut the number of tokens quite a lot, so the speed and cost advancemen...
AI #64: Feel the Mundane Utility
by Zvi — 2024-05-16T15:20:02.956Z

It’s happening. The race is on.

Google and OpenAI both premiered the early versions of their fully multimodal, eventually fully integrated AI agents. Soon your phone experience will get more and more tightly integrated with AI. You will talk to your phone, or your computer, and it will talk back, and it will do all the things. It will hear your tone of voice and understand your facial expressions. It will remember the contents of your inbox and all of your quirky preferences.

It will plausibly be a version of Her, from the hit movie ‘Are we sure about building this Her thing, seems questionable?’

OpenAI won this round of hype going away, because it premiered, and for some modalities released, the new GPT-4o. GPT-4o is tearing up the Arena, and in many ways is clearly giving the people what they want. If nothing else, it is half the price of GPT-4-Turbo, and it is lightning fast including fast web searches, which together have me (at least for now) switching back to ChatGPT as my default, after giving Gemini Advanced (or Pro 1.5) and Claude Opus their times in the sun, although Gemini still has the long context use case locked up.

I will be covering all that in another post, which will be out soon once I finish getting it properly organized.

This post covers some of the other things that happened this past week.

Due to the need to triage for now and ensure everything gets its proper attention, it does drop a number of important developments.

I did write the post about OpenAI’s model spec. I am holding it somewhat for final editing and to update it for GPT-4o, but mostly to give it space so anyone, especially at OpenAI, will have the time to read it.

Jan Lieke and Ilya Sutskever have left OpenAI, with Jan Lieke saying only ‘I resigned.’ That is a terrible sign, and part of a highly worrisome pattern. I will be writing a post about that for next week.

Chuck Schumer’s group issued its report on AI. That requires close attention.

Dwarkesh Patel has a new podcast episode with OpenAI Cofounder John Schulman. Self-recommending, only partially obsolete, again requires proper attention.

For now, here is all the other stuff.

For example, did you know that Big Tech is spending a lot of money in an attempt to avoid being regulated, far more than others are spending? Are you surprised?

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Munda

...
AISN #35: Lobbying on AI Regulation Plus, New Models from OpenAI and Google, and Legal Regimes for Training on Copyrighted Data
by Dan H — 2024-05-16T14:29:21.683Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-35-lobbying?r=7oh0&triedRedirect=true

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

OpenAI and Google Announce New Multimodal Models

In the current paradigm of AI development, there are long delays between the release of successive models. Progress is largely driven by increases in computing power, and training models with more computing power requires building large new data centers.

More than a year after the release of GPT-4, OpenAI has yet to release GPT-4.5 or GPT-5, which would presumably be trained on 10x or 100x more compute than GPT-4, respectively. These models might be released over the next year or two, and could represent large spikes in AI capabilities.

But OpenAI did announce a new model last week, called GPT-4o. The “o” stands for “omni,” referring to the fact that the model can use text, images, videos, and audio as inputs or outputs. This new model modestly outperforms OpenAI’s previous models on standard benchmarks of conversational skill and coding ability. More importantly, it suggests a potential change in how people interact with AI systems, moving from text-based chatbots to live verbal discussions.

OpenAI employees talking with GPT-4o in a live demo of the new model.

Google DeepMind demoed a similar model, called Project Astra. It can watch videos and discuss them in real-time. This model is intended to be part of a path towards building AI agents that can act autonomously in the world. Google also announced improvements to their Gemini series of closed source models, and Gemma series of open source models.

One interesting note for those interested in AI policy is that these models could potentially be deemed illegal in the European Union. The EU AI Act prohibits:

the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons.

Users can ask multimodal AI systems like GPT-4o and Project Astra to look at a person’s face and assess whether they’re happy, sad, angry, or surprised. Does this mean that these models will be illegal...
FMT: a great opportunity for (soon-to-be) parents
by EternallyBlissful — 2024-05-16T13:24:27.975Z

Executive summary

Fecal Microbiota Transplant (FMT) is a procedure that involves transferring the stool of healthy people to the guts of unhealthy people. The bacteria in the healthy person’s stool helps to rebalance the unhealthy person’s dysbiotic (imbalanced) gut microbiome, making their microbiome healthier, disease-resistant, and more youthful. Think of FMTs as a kind of super probiotic to optimize your gut health!

Since the microbiome affects almost all aspects of human health, functioning, and development, FMTs are a promising treatment for a huge variety of health conditions, including multiple sclerosis, ALS, neurodegenerative diseases like Alzheimer's, autism, chronic fatigue syndrome, long Covid, and many more. FMTs from young donors might even have rejuvenating effects (Forget young blood - you want young poop!...).

FMTs can easily and safely be done at home without a doctor with fairly little effort - both for the donor and recipient. All the donor needs to do is put the stools in a vacuum zip bag, remove the oxygen, store them in a freezer, and have them picked up at your doorstep by a dry-ice delivery service to the recipient. See chapter "How to FMT" for more details.

FMT treatment could help a huge amount of people, but it’s severely bottlenecked by a lack of proper donors. Donors must not only be free of various diseases and pathogens, but also exceptionally healthy and with excellent microbiomes — something that's alarmingly rare these days. See the chapter "FMT donor criteria" below. You know you have a great microbiome if you fulfill most of the criteria.

For reasons explained below, babies have a much higher chance than adults of being good FMT donors. Babies can be FMT donors from a few months old (since the procedure just involves taking their stool from their diaper), and usually the younger the donor, the better. This is especially true if the parents take extra steps to ensure that the baby has a great microbiome.

More specifically, becoming parents can ensure that there’s a high chance that the young baby will be a great FMT donor by taking the following steps during pregnancy and after birth:

6-12+ months before birth, the pregnant mother starts taking care of her microbiome (probiotics, lifestyle, diet, etc.) to ensure it's as good as possible at the beginning of the child's life and later on.

This is

...
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
by Gunnar_Zarncke — 2024-05-16T13:09:39.265Z
Link: https://arxiv.org/abs/2405.06624

Authors: David "davidad" Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, Joshua Tenenbaum

Abstract:

Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches.
The Dunning-Kruger of disproving Dunning-Kruger
by kromem — 2024-05-16T10:11:33.108Z

In an online discussion elsewhere today someone linked this article which in turn linked the paper Gignac & Zajenkowski, The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data (PDF) (ironically hosted on @gwern's site).

And I just don't understand what they were thinking.

Let's look at their methodology real quick in section 2.2 (emphasis added):

2.2.1. Subjectively assessed intelligence
Participants assessed their own intelligence on a scale ranging from 1 to 25 (see Zajenkowski, Stolarski, Maciantowicz, Malesza, & Witowska, 2016). Five groups of five columns were labelled as very low, low, average, high or very high, respectively (see Fig. S1). Participants' SAIQ was indexed with the marked column counting from the first to the left; thus, the scores ranged from 1 to 25. Prior to providing a response to the scale, the following instruction was presented: “People differ with respect to their intelligence and can have a low, average or high level. Using the following scale, please indicate where you can be placed compared to other people. Please mark an X in the appropriate box corresponding to your level of intelligence.” In order to place the 25-point scale SAIQ scores onto a scale more comparable to a conventional IQ score (i.e., M = 100; SD = 15), we transformed the scores such that values of 1, 2, 3, 4, 5… 21, 22, 23, 24, 25 were recoded to 40, 45, 50, 55, 60… 140, 145, 150, 155, 160. As the transformation was entirely linear, the results derived from the raw scale SAI scores and the recoded scale SAI scores were the same.

Any alarm bells yet? Let's look at how they measured actual results:

2.2.2. Objectively assessed intelligence
Participants completed the Advanced Progressive Matrices (APM; Raven, Court, & Raven, 1994). The APM is a non-verbal intelligence test which consists of items that include a matrix of figural patterns with a missing piece. The goal is to discover the rules that govern the matrix and to apply them to the response options. The APM is considered to be less affected by culture and/or education (Raven et al., 1994). It is known as good, but not perfect, indicator of general intellectual functioning (Carroll, 1993; Gignac, 2015). We used the age-based norms published in Raven et al. (1994, p. 55) to convert the raw APM scores into percentile scor

...
A case for fairness-enforcing irrational behavior
by cousin_it — 2024-05-16T09:41:30.660Z

There's a long-standing and possibly unsolvable puzzle about how AIs should behave in game-theoretic situations with each other. The simplest example is the Ultimatum Game, where player A proposes how a dollar should be split between A and B, and B either accepts or rejects. In case of rejection both A and B get nothing. There are many Nash equilibria, one for each possible split, making the game indeterminate.

You can put all kinds of complexities on top of the game, like making A and B computer programs that can make conclusions about each other, but the essential nature of indeterminacy remains: the players have to pick a point on the Pareto frontier, their interests being directly opposed, making it a tug-of-war. The game is so simple that any complicated analysis seems almost hopeless.

However, when people play this game in reality, it seems that they bring in other considerations, not just choose what's best for them. The person being offered 20% of the pot will often reject. The reason for such behavior seems to come from a notion of fairness.

This points the way to how AIs could solve the puzzle as well. Imagine you're an AI forced to play some complicated ultimatum-type game with another AI. Then you could ignore the strategic picture of the game entirely, and focus only on what outcome seems "fair", in the sense that you and the other player get about equal amount of whuffies (however understood). And if the other player offers you an unfair deal, you could "flip the table" and make them get nothing, even at cost to you. As long as the "flip the table" option is available to you, this seems a viable approach.

Maybe this is a very simple idea, but it flips my understanding of game-theoretic situations on its head. Until today I thought that the game matrix, what actions are available to players, was the important part. And things like each player's utility scaling were merely afterthoughts. But under the "fairness" view, the figure and the ground invert. Now we care only about comparing the players' utilities, making sure everyone gets roughly equal amount of whuffies. The particular strategic details of each game matter less: as long as each player has access to a "flip the table" strategy, and is willing to use that strategy irrationally when the outcome seems unfair, that's enough.

Of course this can fail if the two players have incompatible views on ...
Against "argument from overhang risk"
by RobertM — 2024-05-16T04:44:00.318Z

Epistemic status: I wrote this in August 2023, got some feedback I didn't manage to incorporate very well, and then never published it. There's been less discussion of overhang risk recently but I don't see any reason to keep sitting on it. Still broadly endorsed, though there's a mention of a "recent" hardware shortage which might be a bit dated.

I think arguments about the risks of overhangs are often unclear about what type of argument is being made. Various types arguments that I've seen include:

Pausing is net-harmful in expectation because it would cause an overhang, which [insert further argument here]
Pausing is less helpful than [naive estimate of helpfulness] because it would cause an overhang, which [insert further argument here]
We shouldn't spend effort attempting to coordinate or enforce a pause, because it will cause [harmful PR effects]
We shouldn't spend effort attempting to coordinate or enforce a pause, because that effort can be directed toward [different goal], which is more helpful (conditional on proponent's implicit model of the world)

I think it's pretty difficult to reason about e.g. (4) without having good object-level models of the actual trade-offs involved for each proposed intervention, and so have been thinking about (1) and (2) recently. (I haven't spent much time thinking about (3); while I've seen some arguments in that direction, most have been either obvious or trivially wrong, rather than wrong in a non-obvious way.)

I think (1) generally factors into two claims:

Most alignment progress will happen from studying closer-to-superhuman models
The "rebound" from overhangs after a pause will happen so much faster than the continuous progress that would've happened without a pause, that you end up behind on net in terms of alignment progress at the same level of capabilities

I want put aside the first claim and argue that the second is not obvious.

Heuristic

Arguments that overhangs are so bad that they outweigh the effects of pausing or slowing down are basically arguing that a second-order effect is more salient than the first-order effect. This is sometimes true, but before you've screened this consideration off by examining the object-level, I think your prior should be against.

Progress Begets Progress

I think an argument of the following form is not crazy: "All else equal, you might rather smoothly ...
How can I make the most of Less Online/Camp/Manifest?
by ErioirE — 2024-05-16T02:19:28.516Z

I spent several weeks psyching myself into buying tickets to what is essentially my first 'vacation' in my adult life (I'm 27 and I typically dislike traveling and enjoy my job more than average).
I'm optimistic it will be enjoyable but social events are not something I'm particularly adept at navigating. I'm relatively new to LW but I've ravenously consumed a large portion of the curated library of content.
In the past few years I've made some significant updates in my lifestyle and worldview and I'm concerned that I'll feel out of my depth among those who are more 'in their element'.
My objectives are:

Learn new things and update existing knowledge
Meet and connect with people with common interests, skills and personalities
Avoid making a total fool of myself (partial is inevitable).
Also related, are there going to be any social events/activities aimed at singles so I can try some awkward attempts at meeting/flirting and getting rejected in an efficient manner?