trentmkelly/LessWrong-43k huggingface dataset:
The boat
by RomanS — 2024-11-22T12:56:45.050Z

A short science fiction story about our ancestors and the ethical responsibility we have towards them.

Old Ana's legs fought against her now, but still she led her granddaughter up the mountain path. The girl's torch made shadows dance on the rocks. The river sang in the distance – the same river that had shown Ana her great making many moons ago.

"Look, Kaya. When I was young like you, I watched the fallen trees in water. They floated, always. One day I saw a big log and thought – what if we made it hollow, like a nest?"

Kaya knew this tale well – how her grandmother made the first boat. How the people of their tribe laughed and said her head was touched by bad spirits. How she worked through many moons, scraping out the old tree's heart with bone tools. How their mouths opened wide when they saw that in this hollow tree, they could ride the river.

"But listen, little one," – Ana stopped to draw breath. – "One day our people will make boats that float up there," she pointed to the night sky. "They will go between the bright spots that shine in the dark, will cross the sky river."

Kaya looked at her grandmother with big eyes: "But how?"

"I do not know," – Ana smiled. – "But our people are clever. They will find the way. And maybe..." – she touched her chest where her heart beat – "maybe one day they will be so clever they can bring back to life those who have long gone. They'll read our traces as I read animal tracks in the snow. With the traces, they'll track down our spirits among the sand grains."

At last, they came to the cave. Inside, the torch showed the walls of their ancestors – hundreds of hands marked in earth-red.

"Now it's your turn, Kaya," – Ana took out a small pot of red ochre. – "Place your hand next to mine. Let those who come here many, many moons from now see us together, so they don't forget about bringing us back."

The awakening was gentle. Ana opened her eyes to an unfamiliar cave of smooth blue walls. Next to her, Kaya was already sitting up, exactly as she remembered her.

"Welcome back," said a spirit, the voice like many voices together. "It will take a while to iterate over all possible human minds. Luckily, you two left some data to narrow down the search".

Ana looked at her own hands - they were strong again, not gnarled by age. She could feel every part of her body singing with life.

"Your boat," the voice continued, "we found it...
Which things were you surprised to learn are metaphors?
by Gordon Seidoh Worley — 2024-11-22T03:46:02.845Z

I love this question, but I've enjoyed noticing answers to its opposite more. What are some things you thought weren't metaphors but were surprised to learn actually are metaphors.

A classic example is that, at least in English, time is often described using distance metaphors. For example, we talk about things taking a "long" or a "short" amount of time, about events that happened in the "distant past" or will happen in the "near future", and how two events can be said to happen "close" or "far apart" from one another in time.
LLM chatbots have ~half of the kinds of "consciousness" that humans believe in. Humans should avoid going crazy about that.
by Andrew_Critch — 2024-11-22T03:26:11.681Z

Preceded by: "Consciousness as a conflationary alliance term for intrinsically valued internal experiences"

tl;dr: Chatbots are probably "conscious" in a variety of important ways. We humans should probably be nice to each other about the moral disagreements and confusions we're about to uncover in our concept of "consciousness".

Epistemic status: I'm pretty sure my conclusions here are correct, but also there's a good chance this post won't convince you of them if you're not on board with my preceding post.

Executive Summary:

I'm pretty sure Turing Prize laureate Geoffrey Hinton is correct that LLM chatbots are "sentient" and/or "conscious" (source: Twitter video), I think for at least 8 of the 17 notions of "consciousness" that I previously elicited from people through my methodical-but-informal study of the term (as well as the peculiar definition of consciousness that Hinton himself favors). If I'm right about this, many humans will probably soon form steadfast opinions that LLM chatbots are "conscious" and/or moral patients, and in many cases, the human's opinion will be based on a valid realization that a chatbot truly is exhibiting this-or-that referent of "consciousness" that the human morally values. On a positive note, these realizations could help humanity to become more appropriately compassionate toward non-human minds, including animals. But on a potentially negative note, these realizations could also erode the (conflationary) alliance that humans have sometimes maintained upon the ambiguous assertion that only humans are "conscious" or can be known to be "conscious".
In particular, there is a possibility that humans could engage in destructive conflicts over the meaning of "consciousness" in AI systems, or over the intrinsic moral value of AI systems, or both. Such conflicts will often be unnecessary, especially in cases where we can obviate or dissolve the conflated term "consciousness" by simply acknowledging in good faith that we disagree about which internal mental process are of moral significance. To acknowledge this disagreement in good faith will mean to do so with an intention to peacefully negotiate with each other to bring about protections for diverse cognitive phenomena that are ideally inclusive of biological humans, rather than with a bad faith intention to wage war over the disagree...
Reading RFK Jr so that you don’t have to
by braces — 2024-11-22T00:59:19.583Z

Some people in my orbit suggested reading Robert F. Kennedy Jr’s book The Real Anthony Fauci.

Here’s my story of wading through a few pages of the book and trying to understand the basis of the claims. My takeaway: there's a lot of sloppiness here, and several times the book’s claim isn’t supported by the citation.

Often, though, the citation is fine. The arguments are carried by shoddy science and massive leaps in causal attribution. For a lot of the anti-vaccine and anti-lockdown takes, there’s at least some study by some scientists showing really bad effects. And the scientists are real—it’s just a small sample or poor design.

Sorting things out honestly is hard, especially with bad faith actors using sloppy language to describe highly selected but serious-sounding findings. My notes on the text follow.

Lockdowns

First, the Introduction has a section on the harms from lockdowns (p.11). I gravitated toward it because it had a lot of numbers. I’m open to the claims that lockdowns were net negative, with severe and diverse harms.

Here’s the first paragraph:

As Dr. Fauci’s policies took hold globally, 300 million humans fell into dire poverty, food insecurity, and starvation. “Globally, the impact of lockdowns on health programs, food production, and supply chains plunged millions of people into severe hunger and malnutrition,” said Alex Gutentag in Tablet Magazine. According to the Associated Press (AP), during 2020, 10,000 children died each month due to virus-linked hunger from global lockdowns. In addition, 500,000 children per month experienced wasting and stunting from malnutrition—up 6.7 million from last year’s total of 47 million—which can “permanently damage children physically and mentally, transforming individual tragedies into a generational catastrophe.” In 2020, disruptions to health and nutrition services killed 228,000 children in South Asia. Deferred medical treatments for cancers, kidney failure, and diabetes killed hundreds of thousands of people and created epidemics of cardiovascular disease and undiagnosed cancer. Unemployment shock is expected to cause 890,000 additional deaths over the next 15 years.

We’ll go sentence-by-sentence.

As Dr. Fauci’s policies took hold globally, 300 million humans fell into dire poverty, food insecurity, and starvation.

I couldn’t find the source for this. When I search “300 million” and...
Don't want Goodhart? — Specify the damn variables
by  — 2024-11-21T22:45:26.538Z

In everyday life, if something looks good to a human, then it is probably actually good (i.e. that human would still think it’s good if they had more complete information and understanding). Obviously there are plenty of exceptions to this, but it works most of the time in day-to-day dealings. But if we start optimizing really hard to make things look good, then Goodhart’s Law kicks in. We end up with instagram food - an elaborate milkshake or salad or burger, visually arranged like a bouquet of flowers, but impractical to eat and kinda mediocre-tasting.

Why Agent Foundations? An Overly Abstract Explanation

I expect that the main problem with Goodhart's law is that if you strive for an indicator to accurately reflect the state of the world, once the indicator becomes decoupled from the state of the world, it stops reflecting the changes in the world. This is how I interpret the term 'good,' which I dislike. People want a thermometer to accurately reflect the patterns they called temperature to better predict the future — if the thermometer doesn't reflect the temperature, future predictions suffer.

Now I return to the burger example — suppose a neural network operator starts optimizing certain parameters to make a burger picture increase the café's profit. Suppose there are several initially optimizable parameters — the recognizability of the burger's image, the anticipated 'sense of pleasure' upon viewing, the presence of necessary ingredients, a non-irritating background, clear visibility of the image, and others. If we are solving the task of 'increasing sales from a picture,' we are not solving the problem of feeding the hungry; we are solving a narrower task — which means that optimizing the taste of the burger may not be needed for this task. For example, if we optimize for reducing the time spent on a task, we can neglect the efforts to fix one of the variables.

In this example, the task was not to create the most appealing burger and at the same time maximize the taste and convenience of consumption. That would be a different function.

If you indeed were solving a narrower task — that is, only creating the most sense of pleasure-inducing picture with maximization of other parameters — and then looked back, puzzled as to why the hungry weren't fed by this procedure, bringing Goodhart's law into the discussion is madness; it stress...
Don't want Goodhart? — Specify the variables more
by YanLyutnev — 2024-11-21T22:43:48.362Z

In everyday life, if something looks good to a human, then it is probably actually good (i.e. that human would still think it’s good if they had more complete information and understanding). Obviously there are plenty of exceptions to this, but it works most of the time in day-to-day dealings. But if we start optimizing really hard to make things look good, then Goodhart’s Law kicks in. We end up with instagram food - an elaborate milkshake or salad or burger, visually arranged like a bouquet of flowers, but impractical to eat and kinda mediocre-tasting.

Why Agent Foundations? An Overly Abstract Explanation

I expect that the main problem with Goodhart's law is that if you strive for an indicator to accurately reflect the state of the world, once the indicator becomes decoupled from the state of the world, it stops reflecting the changes in the world. This is how I interpret the term 'good,' which I dislike. People want a thermometer to accurately reflect the patterns they called temperature to better predict the future — if the thermometer doesn't reflect the temperature, future predictions suffer.

Now I return to the burger example — suppose a neural network operator starts optimizing certain parameters to make a burger picture increase the café's profit. Suppose there are several initially optimizable parameters — the recognizability of the burger's image, the anticipated 'sense of pleasure' upon viewing, the presence of necessary ingredients, a non-irritating background, clear visibility of the image, and others. If we are solving the task of 'increasing sales from a picture,' we are not solving the problem of feeding the hungry; we are solving a narrower task — which means that optimizing the taste of the burger may not be needed for this task. For example, if we optimize for reducing the time spent on a task, we can neglect the efforts to fix one of the variables.

In this example, the task was not to create the most appealing burger and at the same time maximize the taste and convenience of consumption. That would be a different function.

If you indeed were solving a narrower task — that is, only creating the most sense of pleasure-inducing picture with maximization of other parameters — and then looked back, puzzled as to why the hungry weren't fed by this procedure, bringing Goodhart's law into the discussion is madness; it stresses me out. The varia...
Aligning AI Safety Projects with a Republican Administration
by Deric Cheng — 2024-11-21T22:12:27.502Z

We’ve previously written a piece on

soft nationalization

, discussing the growing importance of US national security on the control and regulation of powerful AI models.

The upcoming shift from Biden’s administration to a Republican one is likely to necessitate some changes in strategy and framing for AI safety initiatives.

US national security interests, closely followed by US economic interests, are likely to dominate any regulation of AI systems in the next four years. A Trump administration is likely to prioritize projecting an image of American innovation, power, and speed regarding AI technologies.

Republicans have indicated that they support initiatives such as pro-business deregulation, military capabilities, and American chip manufacturing. Despite some evidence that Republican policymakers are beginning to raise

concerns about existential risks

, the incoming administration will likely be hesitant to pass legislation that meaningfully restricts industry interests in pursuit of AI safety.

National security interests that a Trump administration will likely pursue include:

Increasing controls around Chinese & foreign access to AI models and chips
A progressive increase in public / private partnerships developing AI technologies for US military and intelligence goals
International cooperation in the context of promoting American interests and avoiding multilateral agreements that restrict US AI corporations

Economic interests that a Trump administration will likely pursue include:

Policies encouraging rapid innovation and AI capabilities development
Broad deregulation for AI development – or at the minimum, a reluctance to engage in AI safety governance that restricts AI labs
Corporation-friendly policies and increasing amounts of regulatory capture

Despite this focus, we think AI safety initiatives are still very much on the table in the upcoming administration. There is still significant overlap between national security interests and AI safety initiatives. For example, strategies to increase monitoring of the semiconductor supply chain could improve both US national security outcomes (by reducing chip smuggling to China) and AI safety outcomes (by capacity building for the USG).

As a result, we think that AI safety projects should prioritize “leaning into” the common ground between US national interes...
Entropic strategy in Two Truths and a Lie
by dkl9 — 2024-11-21T22:03:28.986Z
Link: https://dkl9.net/essays/entropic_2t1l.html

The classic social game "two truths and a lie" asks each player, at their turn, to say three facts as the title says, which other players, listening, then seek to tease apart. It gets boring if listeners can easily and reliably tell which "fact" is false. To make it more exciting, the natural strategy would be to pick two absurd true facts and one dull fake fact.

I picked my diet by poring over a spreadsheet
I've been to Mexico twice
I butter my bread on both sides

With this strategy used at a moderate level, listeners mostly guess the more absurd true fact as the probable lie. At an extreme, listeners correctly recognise the lie as the most plausible-sounding one. We can do better.

Some people (me), by confusing people, derive fun; some witnesses (my friends), by watching, derive collateral fun. Disagreement begets excitement, which, except when violent, likewise adds to fun. Information-theoretic entropy roughly measures confusion and disagreement. The optimal "two truths and a lie" strategy is the most fun strategy is the one making the greatest entropy from the listeners' guesses is that which makes each "fact" seem equally probable as the lie.

I can bench-press my body mass
I eschew my phone for a full day per week
I got personally featured on a minor news show

This balance becomes numerically apparent if listeners declare their guesses. Even if the game focuses wholly on the speakers, the entropy will become cognitively apparent as listeners notice a similar truth-propensity between all three statements.
The Three Warnings of the Zentradi
by Trevor Hill-Hand — 2024-11-21T20:28:45.567Z

This is mostly some ramblings and background notes for a fanfiction, and should not be taken seriously as a real-world argument, except insofar as I would hope it could become good enough to be a real-world argument if I were smart enough and worked on it enough and got the right feedback. I would love to hear criticism on any or all of it, and your ideas on where or how else the story of Macross/Robotech has interesting ideas to explore.

Beyond the Machine's Eye: Power, Choice, and the Crisis of Human Agency

Imagine teaching a computer to play chess. You give it clear rules about what makes a "good" move - capturing pieces, controlling the center, protecting the king. The computer gets incredibly good at following these rules.

But here's the thing: it can never ask whether chess is worth playing.

This might seem like a silly example, but it points to something crucial about the challenges we face as machine intelligence becomes increasingly powerful. Systems optimized for specific goals - whether winning chess games or maximizing "engagement" - can't step outside their programming to question whether those goals are worthwhile.

To understand these challenges better, let's look at a story about space warriors called the Zentradi from the anime "Macross" (also known as, in a sense, "Robotech"), and how they optimized themselves into extinction.

Part I: How to Optimize Your Civilization Away

Imagine you're part of an advanced spacefaring civilization called the Protoculture. You face genuine existential threats - hostile aliens, cosmic disasters, internal conflicts. You decide you need a military force to survive.

The reasonable decision: Create an elite warrior force, the Zentradi, genetically engineered for combat effectiveness. Give them their own ships and resources so they can operate independently, without endangering civilian lives.

Seems sensible. What could go wrong?

Your warrior force is effective but has problems:

Personal relationships affect combat decisions

Cultural activities distract from training

Individual preferences create coordination issues

Emotional bonds make warriors hesitate

The reasonable decision: Start limiting these "inefficiencies." Restrict relationships. Standardize routines. Optimize for pure military effectiveness.

Still seems rational. You're just removing obvious problems.

Your warriors are now more effective, but you notice:

U

...
Which things were you surprised to learn are not metaphors?
by Eric Neyman — 2024-11-21T18:56:18.025Z

People with aphantasia typically think that when someone says to "picture X in your mind", they're being entirely metaphorical. If you don't have a mind's eye, that's a super reasonable thing to think, but it turns out that you'd be wrong!

In that spirit, I recently discovered that many expressions about "feelings in your body" are not metaphorical. Sometimes, people literally feel a lump in their throat when they feel sad, or literally feel like their head is hot ("hot-headed") when they're angry.

It seems pretty likely to me that there are other non-metaphors that I currently think are metaphors, and likewise for other people here. So: what are some things that you thought were metaphors, that you later discovered were not metaphors?
Epistemic status: poetry (and other poems)
by Richard_Ngo — 2024-11-21T18:13:17.194Z
Link: https://www.narrativeark.xyz/p/epistemic-status-poetry-and-other

Epistemic status: poetry

Epistemic status: I think this is right, but I’d like people to read it carefully anyway.
Epistemic status: mainstream, normal, totally boring science. If you disagree with any of it, take that up with the Science Czar.
Epistemic status: the sort of post that shouldn’t need an epistemic status tag because it’s so obviously satire.

Epistemic status: I’ve spent around 100 hours thinking about this argument, and now feel like I have a solid understanding of it.
Epistemic status: satisfied.
Epistemic status: a little speculative, a little liberated. A little alive in its own way.

Epistemic status: I spent several weeks in a monastery in Wisconsin with my thoughts as my only companions. Between meditations, I ruminated obsessively on a single idea. The fruits of my cognitive labors are laid out below.
Epistemic status: this post would’ve been a peer-reviewed paper if I had any intellectual peers.
Epistemic status: maximal. I am the epistemic alpha at the top of the epistemic status hierarchy. I am the territory that everyone else is trying to map.

Epistemic status: what is an episteme anyway? Why state a static status? Am I compressing my mind onto a single frozen dimension simply to relieve you from the burden of having to evaluate my claims yourself?
Epistemic status: the mental state of first realizing that you’re allowed to be wrong after all, that it’s not the end of the world, not even if someone much smarter than you gives an argument you can’t refute that literally uses the phrase “literally the end of the world”. Please update accordingly.
Epistemic status: games.

Epistemic status: the content of this post is so true that it has satiated my desire for truth. It’s so true that my prediction error has gone negative. It feels so fucking good.
Epistemic status: divine revelation. There's nothing you could say that would make me doubt these ideas. The voices of the gods have tattooed them into my mind, and I am utterly transformed.
Epistemic status: I have laid my soul on the page in front of you. You could not tear this ontology away from me without tearing me apart. It is the great oak tree at the center of the garden of myself, whose roots hold together the soil of my identity.

I’m pretty confident that this stuff makes sense, but who really knows?

For Boltzmann

The mayfly parts of me that spent their last
Splinter of consciousness writing this word—...