trentmkelly/LessWrong-43k huggingface dataset:
Thinking Insect Suffering Is The Biggest Deal In The World Is Surprisingly Intuitive
by Bentham's Bulldog — 2025-05-19T15:30:36.626Z

I think insect suffering is the worst thing in the world by far. I know it sounds weird! But please hear me out—ideally with an open mind!

Many views seem intuitive but only because of bias. To southern slave-owners, the permissibility of slavery was intuitive. But this was only as a result of bias and a profound failure of empathy. They only considered things from their own perspectives, never thinking about what it would be like to be a tormented and subjugated slave. Had they done so, they’d have correctly judged slavery to be the abomination it was.

I think the same thing is true about insect suffering. People have the strong intuition that it doesn’t matter at all. But when one really reflects, this turns out to be for unreliable and superficial reasons. Insect suffering is genuinely important, and we only neglect it because of bias and irrationality. I think there are four considerations which when considered collectively make vivid the significance of the

10^18 insects

presently suffering.

(Note: when I say insect, I’m really talking about plausibly-conscious arthropods—which is a class of very numerous organisms including crabs, shrimp, spiders, and other things that aren’t technically buts—but repeatedly using scientific terms like “arthropod” is lame, so I’m just going to talk about insects.)

Consider first: you shouldn’t directly trust your intuition that insect suffering doesn’t matter. You’re not an insect, you have no natural empathy towards insects, there’s social incentive not to care about insect welfare, and caring about them is inconvenient. Just like you shouldn’t trust the intuitions of white slave owners who have no empathy towards slaves, you shouldn’t trust your own direct intuitions about insects. If you feel no empathy towards a creature for superficial reasons, relating to its size and the way it looks, you’re obviously not in a position to reliably judge its worth.

(Note to stupid people: I am not making a comparison between black people and insects. I am making a broader point about untrustworthy intuitions.)

Second: insects plausibly can suffer a great deal. The

most detailed report

ever compiled on the subject estimated that they suffer at least 1% as intensely as we do, and

on average around 10%

. That could, of course, be an underestimate, but it could also be a dramatic overestimate.

If a creature can suffer, to decide how much its ...
Thoughts on "Antiqua et nova" (Catholic Church's AI statement)
by jchan — 2025-05-19T13:30:32.712Z

On January 28, 2025 (during the pontificate of Pope Francis) the Catholic Church put out its position paper on AI: Antiqua et nova: Note on the Relationship Between Artificial Intelligence and Human Intelligence. Here are my thoughts, which I wrote back in February, but not much has changed since then, and the new pope has continued citing AI as of particular concern to the Catholic Church.

"With all your modern science, are you any closer to understanding the mystery of how a robot walks or talks?" "Yes, you idiot. The circuit diagram is right here on the inside of your case!" (Futurama)

Four characteristics of humanity

Section III is the real meat of the document. It lists four essential characteristics, and argues that AI falls short of the fullness of humanity in each respect. These are: rationality, truth-seeking, embodiment, and relationality.

I will address embodiment and relationality first, since these two things are connected, and they are in my opinion the most interesting ones. By comparison, I understand less of what (if anything) the document has to say about rationality and truth-seeking, so I'll address those briefly afterward.

Embodiment and relationality

Lately I've been on a contrarian kick advocating these stances in opposition to the "modal rationalist" views of disembodiment and individuality. However, if I take off the contrarian hat, I suppose my actual position is something like:

The general view of rationalists (including myself in the past) is too far biased towards the extreme of disembodiment and individuality because of certain philosophical premises which are usually left unexamined, and which are not empirically-based. The correct position is probably somewhere more in the embodiment/relationality direction on the spectrum.

As a purely descriptive matter, embodiment and relationality seem to more accurately explain the way people actually behave, as opposed to modal rationalism. Understanding this is important for living in society. It also poses a conundrum for attempts to define a "utility function (CEV) of humanity" - if most humans are not utilitarians, how can there be such a thing?

What are embodiment/disembodiment and relationality/individuality? It is helpful to define these positions ostensively by giving examples of the views on the extremes of these two spectra.

Disembodiment:

What matters (for morality and pe

...
FLF Fellowship on AI for Human Reasoning: $25-50k, 12 weeks
by Oliver Sourbut — 2025-05-19T13:25:46.353Z
Link: https://www.flf.org/fellowship

The Future of Life Foundation is launching a fellowship on AI for Human Reasoning, which we think could be decisive in averting existential risk.

Fellowship on AI for Human Reasoning

Apply by June 9th | $25k–$50k stipend | 12 weeks, from July 14 - October 3

Join us in working out how to build a future which robustly empowers humans and improves decision-making.

FLF’s incubator fellowship on AI for human reasoning will help talented researchers and builders start working on AI tools for coordination and epistemics. Participants will scope out and work on pilot projects in this area, with discussion and guidance from experts working in related fields. FLF will provide fellows with a $25k–$50k stipend, the opportunity to work in a shared office in the SF Bay Area, and other support.

In some cases we would be excited to provide support beyond the end of the fellowship period, or help you in launching a new organization.

We’re looking for researchers and builders who want to work on tools to help people make sense of complex situations and coordinate to face challenges - especially the many hazards we face in navigating the future of AI.

Whether you expect gradual or sudden AI takeoff, and whether you're afraid of gradual or acute catastrophes, it really matters how well-informed, clear-headed, and free from coordination failures we are navigating into and through AI transitions. Just the occasion for human reasoning uplift!

Technology shapes the world we live in today. The technologies we develop now — especially AI-powered technologies — will shape the world of tomorrow.

We are concerned that humanity may fumble this ball. High stakes and rapid, dynamic, changes mean that leaders and other decision-makers may be disoriented, misunderstand the situation, or fail to coordinate on necessary actions — and steer the world into gradual or acute catastrophe.

The right technology could help. The rise of modern AI systems unlocks prospects for big improvements here; AI tools might help everyone track the state of play, make decisions they stand behind, and act in sync with others. This could defuse race dynamics, prevent narrowly-interested cliques from exploiting less coordinated groups, and avoid catastrophic errors of judgement.

We believe:

The world is radically underinvested in these beneficial applications;

Many people have not yet had the space to take these prospec

...
[Paper] Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods
by markov — 2025-05-19T10:38:22.570Z

"If you cannot measure it, you cannot improve it." - Lord Kelvin

The science of AI safety evaluations is still nascent, but it is making progress! We know much more today than we did two years ago.

We tried to make this knowledge accessible by writing a literature review and systematizing all the knowledge.

We wanted to go beyond a central source that just summarized collected knowledge, so we put a lot of effort into distillation, and disentangling concepts that are often presented in a messy way. We created original visualizations and gathered others from many different sources to accompany these explanations.

E.g. Disentangling truth, honesty, hallucination, deception and scheming.

The review provides a taxonomy of AI safety evaluations along three dimensions:

What properties do we measure? (dangerous capabilities, propensities, and control)
How do we measure them? (Behavioral and internal techniques)
How do we integrate evaluations into broader frameworks? (Model Organisms , Responsible Scaling Policies, ...)

We also bring up some limitations that safety evaluations might face including things like: "sandbagging" (strategic underperformance on tests), organizational "safetywashing" (misrepresenting capability improvements as safety advancements), or other more fundamental inherent challenges like proving absence rather than presence of capabilities.

The text is available in many different places:

Arxiv
Google Docs (Open to comments)
AI Safety Atlas

This paper is intended to be part of a larger body of work called the AI Safety Atlas. We think of it as chapter 5 in a comprehensive collection of literature reviews collectively forming a textbook for safety.
A widely shared AI productivity paper was retracted, is possibly fraudulent
by titotal — 2025-05-19T10:18:24.505Z

Confidence notes: I am a physicist working on computational material science, so I have some familiarity with the field, but don't know much about R&D firms or economics. Some of the links in this article were gathered from a post at pivot-to-ai.com and the BS detector.

The paper "Artificial Intelligence, Scientific Discovery, and Product Innovation" was published as an Arxiv preprint last December, roughly 5 months ago, and was submitted to a top economics journal.

The paper claimed to show the effect of an experiment at a large R&D company. It claimed the productivity of a thousand material scientists was tracked before and after the introduction of an machine learning material generation tool. The headline results was that the AI caused a 44% increase in materials discovery at the firm, with a productivity increase of 81% for top-decile scientists.

This research was breathlessly reported on in the atlantic, the Wall street journal, and in the news section of Nature. Nobel economics prize winner Darren Acemoglu promoted the research and was acknowledged for his support in the paper. The pre-print article was shared widely, and has been cited dozens of times already in the academic literature. I have seen it cited several times on this forum, most notably by @80000_Hours in their case for AGI by 2030. I myself looked at the paper and took it's findings at face value, something I am now kicking myself for[1].

At some point someone else with some computational materials science expertise noticed serious issues with the paper, and took their concerns to Acemoglu, who in turn took it up with MIT directly. And then everything started to fall apart. Both Acemoglu and MIT have publicly withdrawn support from the paper and urged that it be retracted. In a press release, MIT stated:

"Earlier this year, the COD conducted a confidential internal review based upon allegations it received regarding certain aspects of this paper. While student privacy laws and MIT policy prohibit the disclosure of the outcome of this review, we are writing to inform you that MIT has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper. Based upon this finding, we also believe that the inclusion of this paper in arXiv may violate a

...
The Fragility of Naive Dynamism
by Davidmanheim — 2025-05-19T07:51:11.932Z

Movement alone isn’t progress, and there are dangers in ignoring dimensions and directions of acceleration.

Note: This is part of my Substack on Exploring Cooperation. I’m delaying my next planned post to respond to a recent post by Helen Toner. (But just in case anyone is worried, I’ll definitely return to the theory of language and what it has to do with AI misalignment - and if that sounds interesting, feel free to subscribe.)

Progress, Acceleration, and the Fragility of Civilizational Defaults

In debates about progress, societies seem trapped between two crude narratives. On one side, all progress is framed as inherently heroic—proof of human ingenuity, resilience, and ambition. On the other, any concerns about technologies, much less calls for regulation or opposition, is cast as luddism and bureaucratic overreach fighting the engines of prosperity. But the simplistic counter-narrative that technology leads to ruin, while slow and natural changes were what created an imaged beautiful past is also deeply confused.

We need to reject the simplistic choice between the supposed fragile dynamism leading to ruin and the imagined pastoral beauty of only allowing slow changes.

It is true, as many in progress studies claim, that building things, deploying infrastructure, improving logistics, and expanding capacity is an almost unalloyed good, too often suppressed by governance choke points. Additionally, but differently, many argue that the solution is pushing the frontiers of AI, synthetic biology, or other civilization-scale technologies to accelerate progress.

But once the two statements are made, it becomes clear that conflating material progress and technological acceleration fails to distinguish between building things, on one hand, and pushing intentionally disruptive technologies on the other. And that means both acceleration and luddism are not just wrong, but category errors, each lumping two very different things together. But too many technologists celebrate acceleration as an unmitigated virtue, even though the governance systems which make progress beneficial were built for a slower, more predictable world, and default to clumsy attempts at control.

This essay tries to explain how the frames are confused, what a better mental model looks like, then, finally, in light or the risks from future artificial intelligence, gestures towards potential solutions tha...
Dreams of Ideas
by Joseph Miller — 2025-05-19T02:15:58.790Z

CGP Grey describes a phenomenon I think of as 'dreams of ideas' that I find useful as a tool to know when to stop working on a project.

You can be working on something and you are thinking about what it could be, but what is hard to know is if that thought in your head is more like when your brain dreams.
There is a phenomenon that I always think of as like, you wake up and you think you had a great idea in your dream, but what you really had was the idea of a great idea. You didn't have a great idea, but you had all the emotional resonance to it being a good idea in your dream. You're like, "Oh, why can't I think of the details?". It's like, because there weren't any details there at all...
Sometimes there just isn't the better version of the thing. Like, that's not real. You can't get the details right because those details don't exist. You're having some kind of different response to the thing that you're working on, which is, "This thing isn't good. I can imagine a better version of it." But can you really? Because, like, the details are what matter.

I sometimes get an idea for a good story and feel inspired to try writing some fiction. Then I sit down to write and realize that I don't really know what my idea was beyond a vague image of one tiny slice of a story.

Sometimes I think of awesome research ideas and feel excited to start a project. Then I sit down to start coding and realize that I can't implement it because my idea wasn't really complete or coherent. I had a vibe of the sort of the thing I'd like to have done, but not an actual plan for what to do.

When to cull ideas

This sort of sounds like the opposite advice to Butterfly Ideas but it's actually not in contradiction. When an idea is first born, it should be nurtured and protected from excessive rigor. But it's not healthy for it to remain in that state for too long. You need to set a 5 minute timer and think hard about the thing to see if you can find the details to make it work. Later in the lifecycle, once it's had time to develop, if an idea hasn't matured, then it's time to start asking if this is a real idea or just a mirage.

Examples

I think it's particularly useful to think about this very late in the lifecycle of an idea. Once dozens of people have thought about something for months or years, then there should be very clear details in place, or very good excuses for why the idea isn't concr...
What We Talk About When We Talk About Objective Functions
by Waddington — 2025-05-18T22:48:17.512Z

tl; dr: We can better understand common objective functions (reward, prediction, fitness, control) as all being related to a singular, overarching objective.

Reward? Prediction? Fitness?

In their 2021 paper Reward is enough, DeepMind researchers argue that "intelligence, and its associated abilities, can be understood as subserving the maximization of reward."

This is a response not just to the idea that Attention Is All You Need, but also to predictive processing, a theoretical framework in neuroscience where prediction-error minimization is the star of the show, rather than reward maximization.

"The whole function of the brain is summed up in: error correction." So wrote W. Ross Ashby, the British psychiatrist and cyberneticist, some half a century ago. Computational neuroscience has come a very long way since then. There is now increasing reason to believe that Ashby's (admittedly somewhat vague) statement is correct, and that it captures something crucial about the way that spending metabolic money to build complex brains pays dividends in the search for adaptive success. In particular, one of the brain's key tricks, it now seems, is to implement dumb processes that correct a certain kind or error: error in the multi-layered prediction of input.

―Andy Clark, Whatever next? Predictive brains, situated agents, and the future of cognitive science (2013)

This battle between Reward and Prediction is an old one. The behaviorists were Team Reward. The cyberneticists were Team Prediction. Reward maximization is closely linked to the idea of fitness maximization and Darwinian evolution. Error minimization is closely linked to the overarching notion of control.

Could we possibly fit everything under the same umbrella?

Selectionism

Let's take a step back.

In 2023, a peculiar paper was published in PNAS: On the roles of function and selection in evolving systems.

We identity universal concepts of selection—static persistence, dynamic persistence, and novelty generation—that underpin function and drive systems to evolve through the exchange of information between the environment and the system. Accordingly, we propose a "law of increasing functional information": The functional information of a system will increase (i.e., the system will evolve) if many different configurations of the system undergo selection for one or more functions.

The universe started out in its lowes...
Letter to The Honourable Evan Solomon
by Annapurna — 2025-05-18T16:34:52.232Z
Link: https://jorgevelez.substack.com/p/canada-ai-development

With Canada's new Government and Cabinet in place, we must act quickly and decisively in the global AI race. To ensure Canada leads and that AI development benefits all Canadians, urgently securing resources and fostering partnerships is critical. I've sent the following letter to our new Minister of Artificial Intelligence and Digital Innovation today, hoping to spark wider discussion and encourage action.

Dear Minister Solomon,

Congratulations on your election as Member of Parliament and your new role as Minister of Artificial Intelligence and Digital Innovation.

I am writing to urge your ministry to immediately engage with major international players in the AI infrastructure space. This matter is urgent, as significant, finite resources are currently being deployed. By engaging now, Canada can position itself at the forefront to deploy and develop these resources, increasing its chances to gain a pivotal role in the global AI infrastructure build-up.

Canada is exceptionally well-positioned to become a regional hub for AI development and infrastructure. We possess a unique combination of critical assets:

Proximity and Relationship with the United States: Despite recent geopolitical developments, Canada is a natural ally to the United States. All major private enterprises engaged in this field have a physical presence in Canada, and considerable cross-border investments already exist. The United States government is determined to lead in this domain, and Canada possesses all that is necessary to be its most productive ally.
Abundant and Diverse Energy Resources: Canada can offer the vast and reliable energy supply crucial for AI data centres and operations.
Strategic Assets: Canada offers ample land, freshwater, and access to the financial capital necessary for large-scale infrastructure projects.
World-Class Human Capital: We have a deep pool of talent in AI research and development, as well as the skilled workforce required to construct and manage state-of-the-art facilities.
Strong Innovation Ecosystem: Canada was the first country to launch a national AI strategy and continues to foster a supportive environment for AI innovation and investment.

The global race to build AI capabilities is accelerating. This week, the United States signed two landmark agreements with the United Arab Emirates and Kingdom of Saudi Arabia (KSA), positioning these Middle Eastern allies...