trentmkelly/LessWrong-43k huggingface dataset:
Zuckerberg’s Dystopian AI Vision
by Zvi — 2025-05-06T13:50:01.545Z

You think it’s bad now? Oh, you have no idea. In his talks with Ben Thompson and Dwarkesh Patel, Zuckerberg lays out his vision for our AI future.

I thank him for his candor. I’m still kind of boggled that he said all of it out loud.

We will start with the situation now. How are things going on Facebook in the AI era?

Oh, right.

Sakib: Again, it happened again. Opened Facebook and I saw this. I looked at the comments and they’re just unsuspecting boomers congratulating the fake AI gen couple

Deepfates: You think those are real boomers in the comments?

This continues to be 100% Zuckerberg’s fault, and 100% an intentional decision.

The algorithm knows full well what kind of post this is. It still floods people with them, especially if you click even once. If they wanted to stop it, they easily could.

There’s also the rather insane and deeply embarrassing AI bot accounts they have tried out on Facebook and Instagram.

Compared to his vision of the future? You aint seen nothing yet.

Zuckerberg Tells it to Thompson

Ben Thompson interviewed Mark Zuckerberg, centering on business models.

It was like if you took a left wing caricature of why Zuckerberg is evil, combined it with a left wing caricature about why AI is evil, and then fused them into their final form. Except it’s coming directly from Zuckerberg, as explicit text, on purpose.

It’s understandable that many leave such interviews and related stories saying this:

Ewan Morrison: Big tech atomises you, isolates you, makes you lonely and depressed – then it rents you an AI friend, and AI therapist, an AI lover.

Big tech are parasites who pretend they are here to help you.

When asked what he wants to use AI for, Zuckerberg’s primary answer is advertising, in particular an ‘ultimate black box’ where you ask for a business outcome and the AI does what it takes to make that outcome happen. I leave all the ‘do not want’ and ‘misalignment maximalist goal out of what you are literally calling a black box, film at 11 if you need to watch it again’ and ‘general dystopian nightmare’ details as an exercise to the reader. He anticipates that advertising will then grow from the current 1%-2% of GDP to something more, and Thompson is ‘there with’ him, ‘everyone should embrace the black box.’

His number two use is ‘growing engagement on the customer surfaces and recommendations.’ As in, advertisi...
Will protein design tools solve the snake antivenom shortage?
by Abhishaike Mahajan — 2025-05-06T13:11:07.725Z
Link: https://www.owlposting.com/p/will-protein-design-tools-solve-the

Another note: Just yesterday, the same day this article was released, the New York Times put this out: Universal Antivenom May Grow Out of Man Who Let Snakes Bite Him 200 Times. I was scooped! Somewhat. I added an addendum section discussing this paper at the bottom.

Introduction

There has been a fair bit of discussion over this recent ‘creating binders against snake venom protein’ paper from the Baker Lab that came out earlier this year, including this article from Derek Lowe.

For a quick recap of the paper: the authors use RFDiffusion (a computational tool for generating proteins from scratch) to design proteins that bind to neurotoxic protein found in snake venom, preventing it from interacting with the body. They offer structural characterization results to show binding between their created protein binder and the protein in question (three-finger toxins), and in-vivo results in mice demonstrating that their created protein confers protection against the designed protein as proof.

It’s excellent work, and potentially one of the first times that RFDiffusion has been used in such a straightforward way for a clearly useful problem (though others disagree with me on this being the first time). But, there’s a few obvious questions here: why computationally design binders against venoms if we already have antivenoms available on the market? What further work remains to be done to create binders against any arbitrary venom? And when can we expect that?

I asked these questions, fell down a surprisingly rabbit hole, and decided to compile the results together to release as an article. This is the result.

Let’s get started!

The dismal state of antivenom production

Vaccines are the prototypical ‘clearly good for humanity, but terrible profit incentives’ drug. You spend billions on R&D and clinical trials, give someone a shot once, and then they’ll never need the drug again (or need it once every decade). As such, you end up with these insane cases of promising vaccines being left by the wayside for decades due to economic reasons (see the malaria vaccine debacle), with millions of lives being lost in the process.

In many ways, antivenom production mirrors the problem of vaccines, but on a smaller scale.

But you needn’t take my word for it! As it happens, the Works in Progress magazine has already published a lengthy essay on how horrifically bad the state of antivenom ma...
Utah Court Case Over State Law Regarding "Personhood" for Nonhuman Intelligences
by Stephen Martin — 2025-05-06T12:54:40.344Z

Under US law, the concept of being a "person" is separate from that of being a "human". Certain non-human entities like corporations have been granted limited forms of "personhood" under the law, as famously litigated in the Hobby Lobby case. This status gives them rights and duties such as for example the ability to sue or be sued. The concept of legal personhood also comes up often in legal debates around abortion law, such as the decision in Roe v. Wade;

The Constitution does not define ‘person’ in so many words. Section 1 of the Fourteenth Amendment contains three references to ‘person.’ ... But in nearly all these instances, the use of the word is such that it has application only postnatally. None indicates, with any assurance, that it has any possible pre-natal application

The application of legal personhood to sufficiently advanced and autonomous digital intelligences is getting attention in both legal scholarship[1][2] and legislative action. One development of particular importance has come recently out of Utah, where the state legislature passed a bill stating:

Notwithstanding any other provision of law, a governmental entity may not grant legal personhood to, nor recognize legal personhood in:

(1)artificial intelligence;
(2)an inanimate object;
(3)a body of water;
(4)land;
(5)real property;
(6)atmospheric gases;
(7)an astronomical object;
(8)weather;
(9)a plant;
(10)a nonhuman animal; or
(11)any other member of a taxonomic domain that is not a human being.

This bill both limits the actions an autonomous digital intelligence could take, and what legal protections they would enjoy. Actions such as forming a corporation, filing a lawsuit, and protections such as the right to counsel or a speedy trial may be threatened (on a state level) by this bill. As such this bill has implications for both alignment and model welfare discussions.

Since being passed, the law has been challenged in state court by the Non-Human Rights Project (NHrP), an organization primarily focused on animal welfare, who filed a lawsuit challenging H.B. 249 as unconstitutional;

In effect, H.B. 249 prohibits the Utah judiciary and all Utah legislative assemblies (now and in the future) from granting any legal right to, or recognizing any legal right in, any entity that is not a member of the species Homo sapiens

Consequently, H.B. 249 violates the Utah Constitution, specifically Art. V, § 1 (S

...
Global Risks Weekly Roundup #18/2025: US tariff shortages, military policing, Gaza famine.
by NunoSempere — 2025-05-06T10:39:28.293Z
Link: https://blog.sentinel-team.org/p/global-risks-weekly-roundup-182025

Executive summary

Shortages of some goods expected in the US starting later this month, because of tariffs.
The power outage on the Iberian Peninsula was the worst in Europe since 2003, and Europe may have been close to a more widespread blackout.
Trump wants to involve the military in policing within the US.
Negotiations between the US and Iran over its nuclear weapons program have stalled.
Microsoft will host xAI’s Grok model in its cloud service, in a move to reduce Microsoft’s dependency on OpenAI.
Gaza is running out of food, and the UN warns of famine.
India-Pakistan tensions remain high.

Geopolitics

United States

US economy

A recommendation for those in the US: Because imports are slowing, especially from China, shortages of some imported goods are expected to become noticeable in the coming weeks and to become substantial around August. If tariffs continue, some of these shortages may become pronounced; prices will increase dramatically for other goods. We recommend that those in the US consider buying imported goods before shortages and large price increases arrive.

Forecasters also discussed the possibility of shortages caused merely by self-fulfilling prophecy. For instance, the majority of toilet paper for US consumption is produced domestically, and so, one might naïvely think that as shelves start to empty of other products, toilet paper inventories will not be affected. However, shelves might empty of some products simply because people expect them to, even of products that are not substantially affected by tariffs.

A store in Spain, Mercadona, sold out of toilet paper during the recent power outage, via Virginia López-Hidalgo

Ocean shipping volumes arriving in the US are expected to start falling off this week. Volumes arriving at the Port of Los Angeles from Asia are expected to be 35% lower this week, and total volumes arriving at New York ports are expected to be 40% lower. Subsequent weeks are expected to see 60% lower volumes than normal, as companies have halted or reduced imports. These reductions in arriving port volumes will be felt in stores over the coming weeks and months.

We also wonder whether tariffs might affect the US food supply to a limited extent. The US imports about 15% of its food, by value, and a lot of these exports aren’t luxuries. Food prices fell by 0.1% in March, compared to February, but we expect to see food prices increase ov...
OpenAI's Jig May Be Up
by Vale — 2025-05-06T08:51:21.607Z

It wasn't long ago that OpenAI was the undisputed leader in the AI race. Despite this, they appear to be struggling to remain competitive. I believe they've hit a tipping point and must either take drastic action to avoid being overtaken by rapidly advancing competitors and the structural advantages of big tech or start planning their out.

There is no doubt that OpenAI had an edge when they released ChatGPT. It was a hit. Apparently it surprised even them, and it left their competition floundering. Google tried to get people to care about Bard, Meta started cramming Meta AI into everything they could, Microsoft jumped on the OpenAI bandwagon, and everyone else dropped everything to join the fray. Despite this, my assessment is that OpenAI are no longer frontrunners in their domain.

Frontrunners No Longer

OpenAI are being carried by the wave of success they built and riding it out. I found myself wondering, "If OpenAI launched today, with the models they currently have, would they be cared about as much as they are now?" There is no doubt in my mind that they would certainly be cared about, but I'm not so sure they'd have such the recognition or status they have now.

The walls around OpenAI's moat are crumbling. They're certainly doing impressive work, and they are most definitely still competitive for now, but the lead they once had has disappeared, and at their current trajectory they'll be falling behind quickly. Many of OpenAI's researchers have moved on to roles at other companies, such as Anthropic.

Publicly available information and the long gap since significant architectural updates suggest that progress on GPT-5 may have encountered significant challenges or shifted priorities. The lack of a major public leap in core model capabilities gives the impression of slower progress compared to the rapid advancements seen earlier, or potentially a focus on areas like multimodal integration or safety that take more time. OpenAI has managed some decent improvements, but nothing to push them significantly ahead, and the competition has caught up and begun encroaching.

They've found themselves in a Half-Life 3 situation. It has gained a mythos beyond reality, and Sam Altman only continues to build it up further by throwing hype at hyped. No matter what GPT-5 is like, it won't meet the legendary expectations that it has garnered. At least Valve hasn't continued to hyp...
My Reasons for Using Anki
by Parker Conley — 2025-05-06T07:01:01.045Z
Link: https://parconley.com/my-reasons-for-using-anki/

Introduction

In some circles, having an Anki habit seems to hold similar weight to clichés like "you should meditate", "you should eat healthy", or "you should work out". There's a sense that "doing Anki is good", despite most people in the circles not actually using memory systems.

I've been using my memory system, Anki, daily for two or more years now. Here are the high-level reasons I use memory systems. I don't think memory systems are a cure-all; on occasion, I doubt their value. However, Anki provides enough benefit for me to spend 1h/day reviewing flashcards. This blog post explains my reasons for spending >100 hours using Anki this past college semester. This blog post will provide insight for both people with a memory system practice and those who are considering one.

~my anki heatmap~

My reasons for using Anki

Learn things quickly and effectively

Above all, my use of Anki doesn't fit into neat learning projects. The most meaningful and interesting Anki cards have come from spontaneous cards guided by my natural curiosity and

learn drive

.

Having an Anki habit is like being able to drive. Reviewing cards every day isn’t always useful. However, sometimes you really need to learn something. Just like sometimes you need to drive somewhere. Just like knowing how to drive is useful when you need to drive somewhere, having an Anki habit is useful when you need to learn something.

Examples of meaningful flashcards that don’t fit into formal learning projects include: months of the year; basic React concepts; an important foreign person's name; when daylight saving time is; meaningful personal memories; the basics of equity compensation;

a map of world history

.

I think the largest marginal benefit of Anki comes from being able to learn random things quickly and remember them for as long as I like. However, to enable this benefit of Anki for learning things quickly, I need to have a consistent time spent reviewing. My more formal reasons for using Anki, discussed below, support the existence of my daily review time. They are also intrinsically useful in their own ways.

Study for classes

As of writing this, I'm a junior studying computer science. The instrumental use case I have for Anki is as follows:

Having a high GPA on my resume signals to employers that I'm conscientious and, at one point, I knew something about computer science.
In order to get a high GPA, I need

...
It's 'Well, actually...' all the way down
by benwr — 2025-05-06T05:44:06.843Z
Link: https://www.benwr.net/2025/05/05/well-actually-all-the-way-down.html

Some people (the “Boubas”) don’t like “chemicals” in their food. But other people (the “Kikis”) are like, “uh, everything is chemicals, what do you even mean?”

The Boubas are using the word “chemical” differently than the Kikis, and the way they’re using it is simultaneously more specific and less precise than the way the Kikis use it. I think most Kikis implicitly know this, but their identities are typically tied up in being the kind of person who “knows what ‘chemical’ means”, and… you’ve gotta use that kind of thing whenever you can, I guess?

There is no single privileged universally-correct answer to the question “what does ‘chemical’ mean?”, because the Boubas exist and are using the word differently than Kikis, and in an internally-consistent (though vague) way.

The Kikis are, generally speaking, much more knowledgeable about the details of chemistry. They might hang out around chemists, whose intellectual ancestors invented the precise definition of the word “chemical”. Or they (we) might just be the kind of person to bank on their knowledge of things-like-chemistry. It’s easy to assume that the Kikis must be right that “avoiding chemicals” is silly, since they seem to know so much more.

But the Boubas are pointing to a real thing that Kikis seem weirdly inclined to assume away: The modern world does intentional large-scale chemical engineering, and this is meaningfully different in its scale and quickly-shifting nature than nearly anything that came before 1661. You can’t necessarily use the same analogical and cultural reasoning you would have used in 1660 to tell you what’s safe and what isn’t, in a world where everything you touch was chemically engineered.

Which is not to say that the Boubas are right to avoid all (Bouba-defined) chemicals in their food! But it is to say that reversed stupidity is not intelligence: You should be less confident in your understanding of the long-term consequences of eating Red Dye Number 40 than in your understanding of the long-term consequences of eating non-GMO wheat. Pretending otherwise is silly. At the same time, risks do not inherently outweigh benefits! Red Dye Number 40 is very pretty! I’d guess that food preservatives have saved or enabled millions of lives! I haven’t looked it up; I’m going to live with the risk that I’m wrong, and so are you.
Five Hinge‑Questions That Decide Whether AGI Is Five Years Away or Twenty
by charlieoneill — 2025-05-06T02:48:23.215Z

For people who care about falsifiable stakes rather than vibes

TL;DR

All timeline arguments ultimately turn on five quantitative pivots. Pick optimistic answers to three of them and your median forecast collapses into the 2026–2029 range; pick pessimistic answers to any two and you drift past 2040. The pivots (I think) are:

Which empirical curve matters (hardware spend, algorithmic efficiency, or revenue)
Whether software‑only recursive self‑improvement (RSI) can accelerate capabilities faster than hardware can be installed.
How sharply compute translates into economic value once broad “agentic” reliability is reached.
Whether automating half of essential tasks ignites runaway growth or whether Baumol’s law keeps aggregate productivity anchored until all bottlenecks fall
How much alignment fear, regulation, and supply‑chain friction slow scale‑up

The rest of this post traces how the canonical short‑timeline narrative AI 2027 and the long‑timeline essays by Ege Erdil and Zhendong Zheng + Arjun Ramani diverge on each hinge, and proposes concrete bets that will force regular public updates.

Shared premises

Six doublings in frontier training compute between GPT‑2 (2019) and GPT‑4 (2023)
GPT‑4‑level systems demonstrably replace some cognitive tasks
Alignment is non‑trivial; nobody claims a free deployment lunch

Agreement in the forecasting/timelines community ends at the tempo question.

Hinge #1: Which curve do we extrapolate?

The first divide concerns what exactly we should project into the future. Short‑timeline advocates emphasise frontier‑training compute and algorithmic efficiency, or even just the general amalgamation of all benchmarks as "intelligence extrapolation". They point to six straight doublings in effective training FLOP between GPT‑2 and GPT‑4, and they cite scaling‑law papers showing a 1.6x yearly reduction in compute required to reach any fixed loss. This is the engine behind the claim in AI 2027, that “CapEx grows one hundred‑fold in four years.” Long‑timeline authors reply that the best public proxy for capital actually deployed (NVIDIA datacentre revenue, which I think is a flawed metric for other reasons) stopped growing exponentially after the ChatGPT launch. Either you extrapolate benchmark/efficiency performance and hope that eventually gets you to automation, or you only believe if the work itself is actually a...
Nonprofit to retain control of OpenAI
by Archimedes — 2025-05-05T23:41:28.812Z
Link: https://openai.com/index/evolving-our-structure/

The OpenAI Board has an updated plan for evolving OpenAI’s structure.

OpenAI was founded as a nonprofit, and is today overseen and controlled by that nonprofit. Going forward, it will continue to be overseen and controlled by that nonprofit.

Our for-profit LLC, which has been under the nonprofit since 2019, will transition to a Public Benefit Corporation (PBC)–a purpose-driven company structure that has to consider the interests of both shareholders and the mission.
Unexpected Conscious Entities
by Gunnar_Zarncke — 2025-05-05T22:14:57.829Z

Epistemic status: This is an overly confident, unpolished draft that needs more research, but a type of research I'm not good at, I guess. I rather post it now and see if it resonates.

There could be many entities around us that are conscious without us noticing. This is because we don't have a clear, testable theory of consciousness. How would know if the Pando forest is conscious[1]? Consciousness is still a muddled concept and has many competing theories that cover multiple layers of abstraction and involve multiple aspects or attributes entities must have to be considered conscious. Here, I am interested in some aspects of consciousness common in many theories of consciousness[2] and want to illustrate how some unusual entities may actually possess them.

Perception: The ability to detect, interpret, and respond to stimuli from the environment.
Perceptual Processing and Attention: The filtering and focusing mechanism that determines which stimuli are prioritised for further processing.
Stable Awareness Patterns: Sustained cognitive focus and recognition of consistent patterns within perceived information.[3]
Self-Perception: The capacity to reflect on and recognise oneself as a distinct entity with unique characteristics.
Response to Events: The enactment of actions or changes in behavior in reaction to perceived events or stimuli.
(Episodic) Memory Formation: The process of encoding, storing, and retrieving personal experiences and events.
Intentionality and Goals: The conscious direction of thoughts and actions toward achieving desired outcomes.
Learning and Adaptation: The ability to incorporate new information and experiences to modify behaviors and understanding.
Communication: The exchange of information using verbal, written, or non-verbal methods to convey meaning.
Expressing Emotions: The articulation or display of feelings through various expressive means.
Theory of Mind: The capability to attribute mental states to oneself and others, understanding that others have perspectives and intentions distinct from one's own.

It turns out that many aspects of consciousness can be found in entities that are very unlike human beings.

This post was originally inspired by a discussion wheter LLMs could be conscious. I see the behavioral analogs in LLM outputs, but had trouble reconciling the rsulting claims with some elements that I intuitively felt nece...