trentmkelly/LessWrong-43k huggingface dataset:
2024 in AI predictions
by jessicata — 2025-01-01T20:29:49.132Z

Follow-up to: 2023 in AI predictions.

Here I collect some AI predictions made in 2024. It's not very systematic, it's a convenience sample mostly from browsing Twitter/X. I prefer including predictions that are more specific/testable. I'm planning to make these posts yearly, checking in on predictions whose date has expired. Feel free to add more references to predictions made in 2024 to the comments. (Thanks especially @tsarnick and @AISafetyMemes for posting about a lot of these.)

Predictions about 2024

I'll review predictions from previous posts that are about 2024.

the gears to ascension: "Hard problem of alignment is going to hit us like a train in 3 to 12 months at the same time some specific capabilities breakthroughs people have been working on for the entire history of ML finally start working now that they have a weak AGI to apply to, and suddenly critch's stuff becomes super duper important to understand." (conceded as false by author)

John Pressman: "6-12 month prediction (80%): The alignment problem as the core of AI X-Risk will become a historical artifact as it's largely solved or on track to being solved in the eyes of most parties and arguments increasingly become about competition and misuse. Few switch sides." (conceded as false by author)

Predictions made in 2024

December 2024

Gary Marcus:

Prediction: By end of 2024 we will see

7-10 GPT-4 level models

No massive advance (no GPT-5, or disappointing GPT-5)

Price wars

Very little moat for anyone

No robust solution to hallucinations

Modest lasting corporate adoption

Modest profits, split 7-10 ways

(since 2024 has already ended, this can be evaluated to some degree; I would say he's approximately correct regarding non-agent models, but o1 and o3 are big advances ("massive" is about right), and constitute more moat for OpenAI. He rates himself as 7/7.)

September 2025

teortaxesTex: "We can have effectively o3 level models fitting into 256 Gb VRAM by Q3 2025, running at >40 t/s. Basically it’s a matter of Liang and co. having the compute and the political will to train and upload r3 on Huggingface."

October 2025

Jack Gallagher: "calling it now - there's enough different promising candidates rn that I bet by this time next year we mostly don't use Adam anymore."

December 2025

Elon Musk: "AI will probably be smarter than any single human next year. By 2029, AI is probably smarter than all human...
Approaches to Group Singing
by jefftk — 2025-01-01T12:50:01.877Z

Singing together in groups can be a great feeling, building a sense of
togetherness and shared purpose. While widespread literacy means
getting everyone singing the same words isn't too hard, how do you
get everyone on the same melody? This has often been a problem for
our

secular solstices, but is also one
many groups have handled in a range of ways. Here are the options I
know about:

Use broader cultural knowledge:

Choose songs that are already well-known. A random group of
people in the US will have maybe a few hundred songs they could get
through well with no prep, that people learned from hearing them over
and over. Some are children's songs (Old MacDonald, Ba Ba Black
Sheep), others are well known older pop songs (Hey Jude, YMCA),
holiday songs (Jingle Bells, Rudolph), folk songs (This Land Is Your
Land, Amazing Grace), movie songs (A Spoonful of Sugar, Over the
Rainbow), etc.

Write new words to well-known songs. At our gatherings we've
sung songs adapting the music from The
Mary Ellen Carter, The
Twelve Days of Christmas, Why
Does the Sun Shine, Sinner
Man, etc.

Use written music. Many churches traditionally took this
approach, some using shaped notes to be
easier to learn, and it can expand to sight-reading four-part harmony
for a very full sound. This does require more advance work: it's not
enough to have a song leader and accompanist, you also need to find,
buy, or draft an arrangement in appropriate notation. This also only
works within a culture of singing from written music, or if your group
is a big enough deal in participants lives (ex: weekly gatherings)
that many will learn to read music specifically for your events.

Develop internal group knowledge:

Build up your own songs. If you keep doing the same songs with
the same people, after 2-5 repetitions the group will know them. No
one knows Brighter
than Today outside the secular solstice context, but since we do
it every year (and some of our attendees have heard it at events
elsewhere) it goes well. This works a lot better with groups that
meet more often: weekly is great; yearly is hard.

Send out recordings in advance. If people listen to recordings
in advance they can show up with the melody learned, ready to sing
together. Many people will only need to listen once or twice before
they can join in with others singing as a group. This also requires

...
Alienable (not Inalienable) Right to Buy
by FlorianH — 2025-01-01T12:19:03.691Z

We have overly simplistic principles of market organization that don't square with human reality, giving too much freedom to indulge our short-term impulses, and not enough tools to help our disciplined, long-term selves say no to it. Suffering from over-consumption in ways our long-term self could unlikely agree with is too much of a norm to be considered the acceptable exception to the rule. So we require a rethinking of "markets" and "freedom" for a shift on a societal level - rather than lay the burden on individuals' willpower alone. The right to buy should become alienable: We require societal structures & laws that support our long-term self to restrict the future short-term self. I must be able to impose today that tomorrow I'll be unable to get a chocolate.

Status of post: Exploration with many questions remaining open (technical, psychological, economical, legal) but a core direction I'm convinced should be explored in more detail.

The Issue: Ain't of steel

Markets let us buy anything society can produce, anytime.[1] At first glance, that sounds great.

The crux? Society fails to empower us to undo this if ever we want to. It doesn't enable the individual to prevent herself from consuming anything anytime in any quantity.

Seeing the consequences, e.g. >40% of adults obese in the US, some 14% diabetic, the conclusion seems clear: The unconstrained consumption possibilities are highly destructive.[2] Very, very difficult to justify to let happen if there are any means to help limit the problem. The issue is by far not niche. We systematically don't resist to things we want ourselves to resist to, and in terms of life quality it risks to negate a significant junk of the gains we have from the economic progress we've made in recent centuries.

The Short-Term vs. Long-Term Self

We're not a coherent self. We're - a bit stylized - a short-term & long-term self.

Short-termie is the cannot-resist-to-temptation self that becomes the fat sick American if let loose, or the needlessly-alcoholic, or the gambling addict, the smoker struggling to quit, the TV- or Youtube-Junkie, or the hourly-procrastination-newsmedia-scroller like I am (the latter arguably being a bit less sad and/or easier to prevent if dearly wanted).

Long-term self is the one who genuinely cares about the future well-being, is better at resisting temptations, and is genera...
AGI is what generates evolutionarily fit and novel information
by onur — 2025-01-01T09:22:55.841Z
Link: https://solmaz.io/agi-informational-fitness

I had this idea while taking a shower and felt that I had to share it. It most likely has flaws, so I would appreciate any feedback at info@solmaz.io. My hunch is that it could be a stepping stone towards something more fundamental.

As the world heads towards Artificial General Intelligence—AGI—people rush to define what it is. Marcus Hutter historically described it as

AI which is able to match or exceed human intelligence in a wide class of environments

(...)

hypothetical agent that can perform virtually all intellectual tasks as well as a typical human could

(see his most recently published book)

whereas OpenAI historically described it as

a highly autonomous system that outperforms humans at most economically valuable work

and more recently, according to a The Information report

an AI system that can generate at least $100 billion in profits for OpenAI

which apparently could be the threshold at which Microsoft loses access to OpenAI models, according to the legal agreement between OpenAI and Microsoft.

Acknowledging all of this and other possible definitions, I want to introduce a definition of AGI that relates to information theory and biology, which I think could make sense:

An AGI is an autonomous system that can generate out-of-distribution (i.e. novel) information, that can survive and spread in the broader environment, at a rate higher than a human can generate.

Here, "survival" can be thought of as mimetic survival, where an idea or invention keeps getting replicated or referenced instead of being deleted or forgotten. Some pieces of information, like blog posts auto-generated for SEO purposes, can quickly vanish, are ephemeral and so recently have started being called "AI slop". Others, such as scientific theories, math proofs, books such as Euclid's Elements, and so on, can persist across millennia because societies find them worth copying, citing, or archiving. They are Lindy.

In that way, it is possible to paraphrase the above definition as "an autonomous system that can generate novel and Lindy information at a rate higher than a human can do".

Like Hutter's definition, the concept of environment is crucial for this definition. Viruses thrive in biological systems because cells and organisms replicate them. Digital viruses exploit computers. Euclid's Elements thrives in a math-loving environment. In every case, the information's persisten...
The OODA Loop -- Observe, Orient, Decide, Act
by Davis_Kingsley — 2025-01-01T08:00:27.979Z

United States Air Force Colonel John Boyd was a fighter pilot and military strategist who developed several important strategic theories. While serving as a jet fighter instructor, he was nicknamed "Forty-Second Boyd" because he had a standing offer that he could go up in his plane and defeat any opponent in a simulated dogfight in forty seconds or less -- and do it starting from an extremely unfavorable position! If he failed, he would owe the challenger $40 -- but purportedly he never failed! Further, he was not only able to accomplish this feat against trainees, but also against various visiting pilots who challenged him.[1]
Boyd's concepts have been credited as instrumental to the United States's dramatic victory in the Persian Gulf War ("Operation Desert Storm"), and his insights into aircraft design have been highly influential as well. Boyd was a very unconventional thinker in some ways and did not always get along well within the system, but was nevertheless able to achieve very impressive results.

While some of Boyd's concepts, like the energy-maneuverability theory, are very specific to air combat, he also had ideas that could be applied much more broadly. Perhaps the most well known of these is his concept of the decision cycle, sometimes known as the OODA loop.

It's worth noting that I would not consider myself some superlative expert in this area. I have done substantial reading on the topic, gone over some of Boyd's old briefing slides, listened to some recorded material from Boyd's presentations, applied some of these concepts in my own life, and taught an experimental class at several CFAR workshops covering these principles; however, I am not a military strategist or aircraft designer, nor have I directly practiced this in high-level business strategy or an equivalent area. Take what I have to say here with a grain of salt!

What are OODA Loops?

Basic Definition

OODA is an acronym that stands for Observe, Orient, Decide, Act; the idea is that in order to make a decision, one goes through these steps.

In the Observe step, one gathers information about the situation around them. In Boyd's original context of fighter aircraft operations, we can imagine a pilot looking out the canopy, checking instruments, listening to radio communications, etc.[2]

In the Orient step, one takes the information gathered in the Observe step, integrates it with ...
Comment on "Death and the Gorgon"
by Zack_M_Davis — 2025-01-01T05:47:30.730Z

(some plot spoilers)

There's something distinctly uncomfortable about reading Greg Egan in the 2020s. Besides telling gripping tales with insightful commentary on the true nature of mind and existence, Egan stories written in the 1990s and set in the twenty-first century excelled at speculative worldbuilding, imagining what technological wonders might exist in the decades to come and how Society might adapt to them.

In contrast, "Death and the Gorgon", published in the January/February 2024 issue of Asimov's, feels like it's set twenty minutes into the future. The technologies on display are an AI assistant for police officers (capable of performing research tasks and carrying on conversation) and real-time synthetic avatars (good enough to pass as a video call with a real person). When these kinds of products showed up in "'90s Egan"—I think of Worth's "pharm" custom drug dispenser in Distress (1995) or Maria's "mask" for screening spam calls in Permutation City (1994)—it was part of the background setting of a more technologically advanced world than our own.

Reading "Gorgon" in 2024, not only do the depicted capabilities seem less out of reach (our language model assistants and deepfakes aren't quite there yet, but don't seem too far off), but their literary function has changed: much of the moral of "Gorgon" seems to be to chide people in the real world who are overly impressed by ChatGPT. Reality and Greg Egan are starting to meet in the middle.

Our story features Beth, a standard-issue Greg Egan protagonist[1] as a small-town Colorado sheriff investigating the suspicious destruction of a cryonics vault in an old mine: a naturally occurring cave-in seems unlikely, but it's not clear who would have the motive to thaw (murder?) a hundred frozen heads.

Graciously tolerating the antics of her deputy, who is obsessed with the department's trial version of (what is essentially) ChatGPT-for-law-enforcement, Beth proceeds to interview the next of kin, searching for a motive. She discovers that many of the cryopreserved heads were beneficiaries of a lottery for terminally ill patients in which the prize was free cyronic suspension. The lottery is run by OG—"Optimized Giving"—a charitable group concerned with risks affecting the future of humanity. As the investigation unfolds, Beth and a colleague at the FBI begin to suspect that the lottery is a front for a creative...
Fireplace and Candle Smoke
by jefftk — 2025-01-01T01:50:01.408Z

We celebrated New Year's Eve at my dad's, including a fire in the
fireplace. I was curious how much the wood smoke went up the chimney
vs collecting in the room, and decided to take some measurements. I
used the

M2000
that I got when investigating whether a ceiling fan could be

repurposed as an
air purifier.

Here's what I found:

I started the meter running at 4:30pm, and we started the fire at
about 5:30pm. I didn't write down the specific time because I thought
it would be evident from the chart [1] but actually I can't see it at
all.

Then at 6:45pm we lit Hanukkah candles, and the smoke from the matches
being blown out had a very sharp effect. Particulate levels stayed
high for the rest of the time, with both the fireplace and candles,
which I attribute to the candles.

[1] Several years ago I remember reading Sam Harris' blog post The
Fireplace Delusion, which argues that while we consider wood fires
to be wholesome they're actually worse than smoking. And argues that
this feeling of "wait, but wood fires are good!" is useful for
understanding what religious folks are thinking when presented with
atheism. Several years later his post had gotten jumbled in my head
into saying that fireplace fires cause bad air quality in your own
home, and so when I ran this experiment I was expecting to see quite
high levels. On rereading, however, he spends a lot of time talking
about externalities: the wood smoke that goes up my chimney goes, in
part, into many other people's houses, causing a small bit of harm in
each. So no conflict there.
Merry Sciencemas: A Rat Solstice Retrospective
by leebriskCyrano — 2025-01-01T01:08:36.433Z
Link: https://leebriskcyrano.com/pseudomantic/

I TAKE this blog very seriously.

My half-dozen readers are counting on me for accurate, unbiased takes on Bay Area culture—a genuine read on the pulse of the collective consciousness. So when a friend invited me to the 2024 Secular Winter Solstice festival, I knew I had to deliver some serious boots-on-the-ground reporting of the event.

I wouldn't consider myself a "rationalist" by any means, despite my profoundly autistic qualities. Nor do I sympathize with the "post-rationalist" navel-gazing endemic to the theater kid slash hippie types. No, it feels we've taken a specific moment in 2019 and churned it into a gray paste. None of it appeals to me. If I were to aspire to a label I would be a romantic—I want to feel things—but it's hard to wring poetry out of a life fixed to a monitor, so like most boring people I turn to chemical enhancement.
Riffing on Machines of Loving Grace
by an1lam — 2025-01-01T01:06:45.122Z
Link: https://an1lam.substack.com/p/riffing-on-machines-of-loving-grace

I wrote a post thinking through what sorts of impacts "geniuses in a datacenter" might have on biology in the near-ish term.

Disclaimer: it's not focused on alignment, even though I recognize alignment is very important, and tacitly assumes no intelligence explosion. It's obviously fine for readers to have the response that these caveats make it pointless to even think about, but I'd prefer not to end up spending tons of space in the comments debating whether it should have focused on that.

Here's the intro to the post as a teaser:

Introduction

Dario Amodei's essay,

Machines of Loving Grace

, has been living rent free in my head since he published it two months ago. In it, he lays out a positive vision for how superhuman AI systems could accelerate biological progress. Niko McCarty also wrote a response essay,

Levers for Biological Progress

, and Adam Marblestone wrote a

tweet thread

(which is begging to be expanded into an essay). These inspired me to write up my own thoughts riffing on the implications of the sort of biological acceleration Dario describes.

Sitting down to write, I felt overwhelmed by the number of threads there were to pull on, so rather than going for comprehensiveness, I decided to “riff” on a few implications I find particularly compelling:

Molecular design is ripe for acceleration
AIs would be superhuman experiment planners
Automation could finally penetrate into early stage exploratory research
AIs will like modular therapeutics even more than I do (high bar)
AIs’ discoveries will surprise, and likely upset (some of) us
new chinese stealth aircraft
by bhauth — 2025-01-01T00:19:10.644Z
Link: https://bhauth.com/blog/military/chinese%20planes.html

Recently, 2 Chinese military aircraft were seen flying for the first time. Some people wanted to read about my thoughts on them. In this post, I'll be referring to them as "Diamond" and "Dart" based on their shapes. Speculative designations being used elsewhere are:

Diamond = Chengdu J-36

Dart = Shenyang J-XS

some articles

Instead of embedding photos here, I'll just link to some articles with pictures:

business-standard

aerosociety

yahoo

what the photos show

aircraft size

Diamond seems to be ~22m long, with a central weapon bay long enough for the PL-17 or YJ-83 (6.4m), and 2 smaller bays long enough for the PL-15 (4m). It could probably carry glide bombs too. Its wing area is quite large for a fighter aircraft. The planform is similar to a F-16XL, and scaling that up to 22m length would be ~50 tons MTOW.

Dart is smaller, and its bays seem big enough for the PL-15 but not the PL-17. So, it's meant to operate closer to its targets, but the PL-15 is still bigger and longer-range than current US air-to-air missiles.

aerodynamics

Diamond has thin delta wings. Sweep is ~50°, quite high. It looks designed to go Mach 2 in a straight line at high altitude.

Dart has higher aspect ratio wings. It should have better turning at subsonic speeds, but probably has less range than Diamond at supersonic speeds and a lower max altitude. It should have significantly shorter takeoff distance than Diamond.

control surfaces

Both aircraft have no vertical stabilizer. Normally, those are important for preventing uncontrolled yaw to keep the aircraft pointed forwards.

Diamond has a lot of separate ailerons in the back, which could control yaw by increasing drag on 1 side. That's how the B-2 did things. Diamond also has thrust vectoring, as indicated by things including space between the exhaust nozzles; I suspect that's meant to be the main way Diamond controls yaw.

Dart has fewer ailerons, but has some funky protrusions on the wingtips - I wonder if those are exhaust nozzles for bleed air from the engines for yaw control. If the wingtip things aren't for controlling yaw, then Dart definitely needs thrust vectoring, but it seems designed for lower cost than Diamond and thrust vectoring does increase cost.

stealth

I haven't done simulations or anything, but Diamond seems about as stealthy vs aircraft radar as the F-22, and more stealthy from above or vs low-frequency radar.

The advant...
The Roots of Progress 2024 in review
by jasoncrawford — 2025-01-01T00:02:06.441Z
Link: https://newsletter.rootsofprogress.org/p/2024-in-review

2024 was a big year for me, and an even bigger year for the Roots of Progress Institute (RPI). For one, we became the Roots of Progress Institute (with a nice new logo and website). Here’s what the org and I were up to this year. (My annual “highlights from what I read this year” are towards the end, if you’re looking for that.)

The Progress Conference

Progress Conference 2024, hosted by RPI together with several great co-presenters, was the highlight of my year, and I think some other people’s too. We’ve already covered it in previous writeups, but in case you’re just tuning in: well over 200 people attended (with hundreds on the waitlist); dozens of great speakers, including Tyler Cowen, Patrick Collison, and Steven Pinker; and over 30+ participant-led “unconference” sessions on a variety of topics from healthcare to medieval Chinese technology. Several people told us it was the best conference they had ever attended, full stop. (!) See the writeups from Scott Alexander, Noah Smith, Packy McCormick, or Bryan Walsh (Vox), to pick a few.

Most of the talks are now online, and most of the rest will be up soon.

The RPI Fellowship

In 2024 we also ran the second cohort of the Roots of Progress Fellowship. Two dozen talented writers completed the program, publishing dozens of essays and almost doubling their audiences. I was thrilled with the talent we attracted to the program this year and excited to see where they’re going to go. See our recent writeup of the program.

My writing

In 2024 I published 17 essays (including this one) totaling over 37,000 words. That’s about half of last year, which decline I attribute in part to being involved in the programs mentioned above, and to doing fundraising. Also, about half of those essays, and well over half the words, were for my book-in-progress, The Techno-Humanist Manifesto, and that is some of the hardest writing I’ve done.

Highlights:

Longest post (4,400 words): The Life Well-Lived, part 2, from Chapter 4 of The Techno-Humanist Manifesto
Most liked on Substack: Announcing The Techno-Humanist Manifesto
Most commented on Substack: What is progress?
Most upvoted on Hacker News: Why you, personally, should want a larger human population
Most upvoted on LessWrong: Biological risk from the mirror world

My audience

In 2024:

My email subscribers (via Substack) grew 82% to almost 33k
Followers on the social network formerly known as Twitter

...