trentmkelly/LessWrong-43k huggingface dataset:
Reality is Fractal-Shaped
by silentbob — 2024-12-17T13:52:16.946Z

Tl;dr: Much of common sense and personal insights are just first order approximations, broadly generalizing complex situations to a single supposed truth. But often the truth value of a claim depends on many hidden factors. In this post I argue that it’s important to maintain a general awareness of the imprecise nature of seemingly binary statements, and to recognize when it’s appropriate to dig deeper and figure out these hidden variables and how they interact. Often we should replace “Is X true?”-type questions with “When is X true?”, which has a better shot at leading to more interesting and accurate insights.

Many human disagreements come down to something like the following:

Alan: Did you know, Earth's surface is water?
Betty: What, that is not true! We are standing on Earth right now, and it is clearly not wet.
Alan: Well this is just anecdotal evidence. Haven't you read the science that most of Earth's surface is covered by water?
Betty: Of course I know this science. But surely you don't believe it is water all the way down? Do you know the science that beneath all this water, there is actually a lot of rock at the ground? So Earth's surface is solid after all.
...

Why is this argument so silly? Alice and Bob argue about "the Earth's surface" as a single thing, such that claims about it, e.g. what material it consists of, can be answered in a binary fashion. But whether or not that is true or false is not a well-defined question that one can just answer. It depends not only on your exact definitions but also, crucially, on many implicit, even hidden factors. In this case, your exact location on the planet, as well as e.g. the time of year.

Of course nobody would have that exact argument, because in this context the futility of it becomes instantly obvious. Yet, many other arguments, as well as individual misconceptions, are of a very similar nature, where a complex question that depends on many hidden factors is falsely treated as a binary one.

Note that even probabilistic or grayscale thinking are often not sufficient to resolve this, because they still treat the question as one that essentially has one answer. Replacing the binary with a spectrum or probability is often an improvement, but it doesn’t address the inherent hidden complexity of most statements.

Example 1: People Can’t Estimate Calories

I once had a conversation with a ...
Trying to translate when people talk past each other
by Kaj_Sotala — 2024-12-17T09:40:02.640Z

Sometimes two people are talking past each other, and I try to help them understand each other (with varying degrees of success).

It’s as if they are looking at the same object, but from different angles. Mostly they see the same thing – most of the words have shared meanings. But some key words and assumptions have a different meaning to them.

Often, I find that one person (call them A) has a perspective that’s easier for me to understand. It comes naturally. But B’s perspective is initially harder. So if I want to translate from B to A, I first need to understand B.

I remember a time when I sat listening to two people having a conversation, both getting increasingly agitated and repeating the same points without making progress. Four of us were playing a cooperative board game together. The situation was something like…

(I don’t remember the exact details anymore, and communicating the exact details would require explaining game mechanics that aren’t important in this context, so I’ll give a partially-fictional version that tries to have the same rough shape as the original situation. See this comment for something that tries to offer a more accurate summary.)

We had been making plans about our next move. Person A had promised that they would make a particular play. When the time came, they noticed that there was a better play they could make instead, so they did that. Person B became upset. The conversation went something like:

A: I’ll make this play.
B: What? That’s not what we agreed on.
A: That doesn’t matter – look, this play is better because it has these consequences.
B: You can’t just say that it doesn’t matter, you promised to make a different play.
A: But this play would have a better outcome in terms of what we all want.
B: Yes but you promised to play differently, you can’t just ignore that. Our previous agreement matters.
A: Okay if you don’t want me to play like this, I can still play the way that we originally discussed, too.
B: That’s not the point, you can play the way you intended now.
A: ??? So… It is okay if I make this new move?
B: Yes but my point is that you promised to do the move that we previously discussed.
A: … but that doesn’t matter since the new move is better?
B: It matters! Kaj was counting on you to make the old move, and he needs to be able to count on you when doing plans!
A: But Kaj can just do this other thing instead now, and that’s ev...
What is "wireheading"?
by Vishakha — 2024-12-17T07:49:50.957Z
Link: https://aisafety.info/questions/NKRA/What-is-%22wireheading%22

This is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback.

The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.

Wireheading happens when an agent's reward mechanism is stimulated directly, instead of through the agent’s goals being achieved in the world.[1]

One way of visualizing this is if we imagine a counter in the agent’s head which determines how satisfied it is with its situation. We say that the agent is wireheaded if something reaches inside its head to directly increase the counter. Now it very satisfied with its situation, even though its “goals” have not been achieved.

The term comes from experiments in which rats with electrodes implanted into their brains could activate their pleasure centers at the press of a button. Some of the rats repeatedly pressed the pleasure button until they died of hunger or thirst.

AI safety researchers sometimes worry that an AI may wirehead itself by accessing its reward function directly and setting its reward to its maximum value.[2] This could be benign if it caused the AI to simply stop doing anything, but it could be problematic if it caused a powerful AI to take actions to ensure that we don’t stop it from wireheading.

There’s another way in which wireheading occasionally comes up. Some thought experiments in which a powerful AI is given a relatively simple goal, like "make humans happy", conclude that this would lead the AI to “wirehead” humans — e.g., by pumping us full of heroin, or by finding other ways to make us feel good while leaving out a lot of what makes life worth living.[3]

Both of these problems would be categorized as outer misalignment.

Further reading:

Advanced artificial agents intervene in the provision of reward
Model-based RL, Desires, Brains, Wireheading
A definition of wireheading

^

Another (overlapping) term for an agent interfering with its reward is reward tampering.

^

Some, including proponents of shard theory, argue that this is unlikely to happen.

^

At least according to most people — some pure hedonists would argue only pleasure and pain matter.
1 What If We Rebuild Motivation with the Fermi ESTIMATion?
by P. João — 2024-12-17T07:46:40.547Z

From a General Vision of a Method to a Foundation for Systematic Self-Improvement

In today's information-saturated world, the quest for optimizing cognitive performance is more critical than ever. While we intuitively recognize moments when we're at our best—deeply motivated, sharply focused—capturing and reproducing these mental states remains elusive.

Enter the States Metric (SM), a novel framework inspired by the logic of Fermi estimation. Like Fermi's famous method for breaking down big questions into manageable parts, SM offers a way to quantify and systematically optimize cognitive states such as motivation, focus, and creativity.

The Problem: Cognitive Overload

The flood of information we face daily often leads to suboptimal decisions and wasted mental energy. How do we avoid this? Many of us ask vague questions like, “Was I productive today?”—but this rarely provides actionable answers.

Instead, the States Metric (SM) transforms this ambiguity into measurable progress by asking:

How meaningful was your progress toward your Moments of Upper Motivation?
How effectively did you balance your biological, emotional, social, and intellectual resources?

By breaking these questions into manageable components, SM empowers us to systematically evaluate, adjust, and optimize our cognitive processes.

Core Objectives and Principles

Our adaptive system for evidence-based personal evaluation rests on three core principles:

Maximize potential universe states
Account for individual differences
Enable systematic validation

This framework organizes:

Goals
Routines
Tasks
Scientific Personal Journals

The States Metric Framework

The States Metric (SM) evaluates possible cognitive states to identify the point of maximum entropy.

Just as Enrico Fermi broke down complex questions into manageable calculations, SM transforms subjective concepts into practical data. This enables the construction of a personal system that calibrates goals, designs effective routines, and maximizes adaptive potential.

For the SM we can sue Moment of Upper Motivation (MUM)—the point where your cognitive resources align to produce maximum efficiency and satisfaction.

Fermi Estimation Meets Cognitive States

The logic behind SM mirrors the famous Fermi estimation technique, which breaks complex questions into smaller, solvable parts. Here's how the two align:

1. Hierarchical Decomposi...
Where do you put your ideas?
by CstineSublime — 2024-12-17T07:26:06.685Z

I am currently looking for a system which will help me execute some of my massive backlog of ideas. By “ideas” I include my hundreds and hundreds of story outlines for films with a handful of finished screenplays, but also things like: alternative income streams, or day jobs, or skills or abilities I’d like to learn/get (coding, traditional animation, dance the Tango, conversational Italian), as well as a host of other projects.
Before I get to the determining how to better pick which ideas I should pursue (Update: see my investigation of my idea choosing decision making model here), I was wondering if there was any more I could do to optimize my current idea recording method. Some of this overlaps with the GTD concept of the "Someday" bucket. But what I don't like about that is that I'd very much like to ensure I review and act upon some of these ideas.
So how do most of you record your ideas? Where do you put them? Where do you keep them, not so in the dark as to never seen sunlight again?
I'm from a film background, so my knowledge of idea-capture is influenced by anecdotes of everyone from Vladimir Nabokov's index cards which in the early gestational stage he describes as "...including the accumulation of seemingly haphazard notes, the secret arrowheads of research", Joan Rivers or Bob Hope's archives of decade's of accumulated jokes organized by subject, or the unfiltered and uncensored NO-NO sessions of Robert Clampett at the birth of a cartoon. To Jerry Lewis typing out between shows the screenplay the Bell-Boy, which to be fair is more of a anthology of isolated jokes than a continuous narrative?
But what about when you have an idea for an app? For research? For an algorithm? Or yes, career moves and similar choices? Where do you put that idea? How do you ensure you don't lose it so that you can maximize the chances of doing something with it?
Elevating Air Purifiers
by jefftk — 2024-12-17T01:40:05.401Z

I'm pretty interested in indoor air quality, primarily for

reducing
disease transmission, but also for reducing the effect of

small particles from cooking. Air
purifiers work well, but if you do leave them on the ground they're in
the way and vulnerable to children and pets.

Elevating them can

help a lot, and I decided
to have another go at this.

I had two purifiers to work with, the same two I compared yesterday:

Coway AP-1512,
240 CADR @ $170

Airfanta
3Pro, 413 CADR @ $150

For the AP-1512 I added a shelf above head height:

I was careful to route the cord where it couldn't pull the purifier
down on someone's head.

The 3Pro was light enough that I could use
the bottom as a shelf. I attached shelf brackets with countersunk m4
machine screws, left over from my folding portable laptop monitor:

And mounted it to the side of a kitchen cabinet:

It's not flush with the cabinet: there's a 1.5" gap. I'm not sure how
much this decreases the efficacy of the filter on that side, but
probably not too much?

I'm pretty happy with both of these!

Comment via: mastodon, bluesky
A dataset of questions on decision-theoretic reasoning in Newcomb-like problems
by Caspar Oesterheld — 2024-12-16T22:42:03.763Z
Link: https://arxiv.org/abs/2411.10588

I’ve spent a lot of the last few years working on issues related to acausal cooperation. With LLMs being clearly dominant over recent years, I’ve now led a team to make a benchmark to figure out how good LLMs are at decision theory and whether and when they lean more CDT or EDT. We hope to expand this dataset in the future, including by incorporating questions that try to measure the updatelessness dimension. Hopefully, this dataset will be useful for future interventions aimed at improving acausal interactions.

Abstract:

We introduce a dataset of natural-language questions in the decision theory of so-called Newcomb-like problems. Newcomb-like problems include, for instance, decision problems in which an agent interacts with a similar other agent, and thus has to reason about the fact that the other agent will likely reason in similar ways. Evaluating LLM reasoning about Newcomb-like problems is important because interactions between foundation-model-based agents will often be Newcomb-like. Some ways of reasoning about Newcomb-like problems may allow for greater cooperation between models.

Our dataset contains both capabilities questions (i.e., questions with a unique, uncontroversially correct answer) and attitude questions (i.e., questions about which decision theorists would disagree). We use our dataset for an investigation of decision-theoretical capabilities and expressed attitudes and their interplay in existing models (different models by OpenAI, Anthropic, Meta, GDM, Reka, etc.), as well as models under simple prompt-based interventions. We find, among other things, that attitudes vary significantly between existing models; that high capabilities are associated with attitudes more favorable toward so-called evidential decision theory; and that attitudes are consistent across different types of questions.

Twitter thread:

How do LLMs reason about playing games against copies of themselves? 🪞We made the first LLM decision theory benchmark to find out. 🧵

Decision theory tackles questions of rational choice, especially in interactions with copies or simulations of yourself. Rare for humans but potentially very important for language models!

Our team, which includes academic decision theory researchers, spent hundreds of hours hand-generating 400+ multiple-choice questions to test how well LLMs reason about two key decision theories: causal and evidentia

...
A practical guide to tiling the universe with hedonium
by Vittu Perkele — 2024-12-16T21:25:44.431Z
Link: https://perkeleperusing.substack.com/p/practical-considerations-for-a-hedonium

This is a blog post I recently made about what tiling the universe with hedonium (which I believe is morally entailed by a belief in hedonic utilitarianism) would actually involve in practice. It's easy to just say "make as much hedonium as possible," but what is hedonium, anyways, and how would you actually go about making it? I hope you find it interesting, just follow the link to read the full post if you're interested in what the prerequisites for converting the universe into hedonium actually are.
AI Safety Seed Funding Network - Join as a Donor or Investor
by Alexandra Bos — 2024-12-16T19:30:43.812Z

Summary

You can now

express interest

in joining

Catalyze Impact

's seed funding networks for AI safety organizations. We offer a non-profit Seed Funding Circle for individuals and organizations with $15k+ annual donation capacity and an Investor Network for VCs and angel investors with $20k+ annual investment capacity. Joining does not create any obligation to invest or donate. For more information, please visit

this page

.

Express your Interest

With the AI safety field still being remarkably small relative to the challenge it addresses, new organizations can have an outsized impact on humanity's future. At Catalyze, an incubator for AI safety organizations, we aim to facilitate this. We are currently open to expressions of interest to join either or both of our funding networks.

Seed Funding Circle

This network focuses on providing early-stage funding to exceptional non-profit founding teams which aim for large contributions to the AI safety field. A relatively small amount of capital (of around $50k-300k) from our network can serve as a catalyst, enabling the launch of new ambitious AI safety organizations.

Circle members can either personally make funding decisions, or defer to independent regrantors who are experts in the AI safety field. Each round features 3-6 organization proposals, primarily incubated through Catalyze Impact's selective incubation program. The circle is a good fit for individuals & organization representatives who have the capacity to donate $15k or more per year and who are excited to support high-leverage interventions while embracing a hits-based, risk-tolerant approach. Joining the circle does not create an obligation to donate a minimum amount in any proposal round.

Investor Network

Through this network, we facilitate early-stage investments in exceptional founding teams which aim for a strong impact on AI safety. These companies play into emerging large-scale market demands, with the AI Assurance Technology market - which enables the safe use and governance of AI -

projected to grow to $276 billion by 2030

.

This network is a good fit for venture capitalists and angel investors with the capacity to invest $20k or more per year and who care about furthering the AI safety field. Start-ups eligible to apply have typically not received investment before and are primarily incubated through our ...
Is this a better way to do matchmaking?
by Chris Lakin — 2024-12-16T19:06:14.574Z

When friends ask me to matchmake them, they usually list traits they want: "Kind, curious, growth-minded..." (Sometimes they even write date-me docs!) But I find it really hard to think of the right people from these descriptions.

Here's what I'm exploring, and suspect works better: "You know [person]? Do you know anyone kinda like them?" (Where "person" could be their ex, or that close friend they wish they were attracted to.)

Matching by description is hard. When someone lists desired traits, I need to:

Understand what they mean by each trait
Think of people I know
Check if they match
Somehow weigh everything together

But when I meet their person, I instantly get it - their vibe, how they talk, their humor, how they move through the world.

Plus, there's usually a gap between what people think they want and what works for them. Their trait list comes from trying to rationalize past attractions and guess future ones. But seeing who they've actually had chemistry with shows me what actually works.

An example also captures subtle compatibility factors they might not even be aware of. Maybe they say they want someone "growth-minded," but what they really click with is a specific style of playful intellectual banter that's hard to describe but easy to spot.

I've recently started trying to match people this way, with varying results. Would love to hear if others have successfully done this before!

I just wish I knew how to make this scalable (like, how do you do this on the internet?) or work even when you don't know the example person that well. If you have ideas, let me know!
I read every major AI lab’s safety plan so you don’t have to
by sarahhw — 2024-12-16T18:51:38.499Z
Link: https://longerramblings.substack.com/p/i-read-every-major-ai-labs-safety

I recently completed the AI Safety Fundamentals governance course. For my project, which won runner-up in the Technical Governance Explainer category, I summarised the safety frameworks published by OpenAI, Anthropic and Deepmind, and offered some of my high-level thoughts. Posting it here in case it can be of any use to people!

Note that I wrote this to be (hopefully) accessible to a reader without prior knowledge of AI safety, and it is not totally comprehensive – but can hopefully be a useful for access point for anyone who wants to know more about what these frameworks actually say.

A handful of tech companies are competing to build advanced, general-purpose AI systems that radically outsmart all of humanity. Each acknowledges that this will be a highly – perhaps

existentially

– dangerous undertaking. How do they plan to mitigate these risks?

Three industry leaders have released safety frameworks outlining how they intend to avoid catastrophic outcomes. They are OpenAI’s

Preparedness Framework

, Anthropic’s

Responsible Scaling Policy

and Google DeepMind’s

Frontier Safety Framework

.

Despite having been an avid follower of AI safety issues for almost two years now, and having heard plenty about these safety frameworks and how promising (or disappointing) others believe them to be, I had never actually read them in full. I decided to do that – and to create a simple summary that might be useful for others.

I tried to write this assuming no prior knowledge. It is aimed at a reader who has heard that AI companies are doing something dangerous, and would like to know how they plan to address that. In the first section, I give a high-level summary of what each framework actually says. In the second, I offer some of my own opinions.

Note I haven’t covered every aspect of the three frameworks here. I’ve focused on risk thresholds, capability evaluations and mitigations. There are some other sections, which mainly cover each lab’s governance and transparency policies. I also want to throw in the obvious disclaimer that I have not been comprehensive here and have probably missed some nuances despite my best efforts to capture all the important bits!

What are they?

First, let’s take a look at how each lab defines their safety framework, and what they claim it will achieve.

Antrophic’s Responsible Scaling Policy is the most concretely defined of the three:

“a public commi...