trentmkelly/LessWrong-43k huggingface dataset:
Is there a term for 'the mistake of making a decision based on averages when you could cherry picked instead'?
by freedomandutility — 2021-05-25T19:18:01.604Z

I see lots of situations where let's say, Mike is aware that either method A and method B that can be used to carry out a task.

Observational data shows that on average, system A outperforms system B, so seeing this, Mike decides to use system A.

But, the best ever result on the task was achieved with system B, and the conditions under which that was achieved could be easily replicated.

So really, Mike would be better off using system B and replicating those exact conditions - Mike could have cherry picked and recreated the best scenario, but made a decision based on averages instead.

Is there a term for this? And if not, what should the term be?
Decoupling deliberation from competition
by paulfchristiano — 2021-05-25T18:50:03.879Z

I view intent alignment as one step towards a broader goal of decoupling deliberation from competition.

Deliberation. Thinking about what we want, learning about the world, talking and learning from each other, resolving our disagreements, figuring out better methodologies for making further progress…
Competition. Making money and racing to build infrastructure, managing political campaigns and maneuvering within the political system, running ads to persuade people, fighting wars…

Competition pushes us to become the kind of people and communities who can win a fight, to delegate to whichever kind of AI is available first, and to adopt whatever ideologies are most memetically fit.

Deliberation pushes us to become the kind of people and communities who we want to be, to delegate only when we trust an AIs judgment more than our own, and to adopt views that we really believe.

I think it’s likely that competition is going to accelerate and become more complex over the next 100 years, especially as AI systems begin to replace humans and compete on our behalf. I’m afraid that this may derail human deliberation and lead us to a place we don’t want to go.

Decoupling

I would like humans and humanity to have the time, space, and safety to grow and change in whatever way we decide — individually and collectively — that we want to.

You could try to achieve this by “pausing” competition. Alice and Bob could agree to stop fighting while they try to figure out what they want and work out their disagreements. But that’s a tall order — it requires halting not only military conflict, but any economic development that could put someone at an advantage later on. I don’t want to dismiss this kind of ambitious goal (related post), but I think it’s uncertain and long-term enough that you probably want a stop-gap solution.

An alternative approach is to “decouple” competition from deliberation. Alice and Bob keep competing, but they try to make sure that deliberation happens independently and the result isn’t affected by competition. (“Pausing” is the special case of decoupling where deliberation finishes before competition starts.)

In a world without AI, decoupling is possible to a limited extent. Alice and Bob can spend time competing while planning to deliberate later after the dust has settled(or have their descendants deliberate). But it’s inevitable that Alice and Bob...
Knowledge is not just map/territory resemblance
by Alex Flint — 2021-05-25T17:58:08.565Z

Financial status: This is independent research. I welcome financial support to make further posts like this possible.

Epistemic status: This is in-progress thinking.

This post is part of a sequence on the accumulation of knowledge. Our goal is to articulate what it means for knowledge to accumulate within a physical system.

The challenge is this: given a closed physical system, if I point to a region and tell you that "knowledge is accumulating" in this region, how would you test my claim? What are the physical characteristics of the accumulation of knowledge? We do not take some agent as the fundamental starting point but instead take a mechanistic physical system as the starting point, and look for a definition of knowledge accumulation in terms of physical patterns.

In this post I explore directly looking within the territory for a physical map that resembles the territory.

Example: Shipping container

In the shipping container example from the previous post, if we found within some region of interest a physical map with markings that clearly corresponded to the arrangement of items in the shipping container, then we could say, yes, knowledge exists within this region. If we revisited this region over several time steps and observed the markings on the map coming into closer and closer correspondence to the configuration of the overall shipping container then we could say that knowledge is accumulating there.

Example: Sailing ship

Suppose we examined a sailing ship from a few hundred years ago and asked whether knowledge was accumulating on that sailing ship. If we found physical sheets of paper with drawings of nearby coastlines then we could say, yes, there is a process happening on this ship that is resulting in the accumulation of knowledge.

Resemblance

But what exactly does it mean for one thing to resemble another? The original lesswrong 1.0 had the following header at the top of each page, pointing at a certain concept of map/territory resemblance:

Suppose we converted both the "map" and "territory" image to grayscale and then looked at a 2D histogram of pixel intensities with pixels from the "map" on the X axis and pixels from the "territory" on the Y axis.

We would find that the "map" and "territory" pixel intensities are predictive of one another, in the sense that knowing the "map" pixel intensity gives us some information about the corresponding "territo...
Abstraction Talk
by johnswentworth — 2021-05-25T16:45:15.996Z

I gave a talk a couple weeks on abstraction. It moves very fast, but it covers more material in one place than any individual post I've written, and includes a few things which haven't showed up in posts yet (especially near the end).

Slides are here. The slides are hard to see sometimes in the video; I recommend keeping them open to the side of the video.
Starting a Rationalist Meetup during Lockdown
by Felix Karg — 2021-05-25T15:43:47.211Z

Authors: fkarg (my blog), crtltab, wilm

Backstory and Intentions

In October 2020, wilm moved to Karlsruhe and was disappointed by the lack of a local LessWrong community. In order to change this, he reached out to fkarg and together they decide to try to establish a LW Meetup Group in Karlsruhe.

Despite the ongoing pandemic, which made in-person gatherings impossible, the Meetup Group has been going great! In this post we want to share what we have learned about starting and running a local meetup group during lockdown.

By sharing our experience we want to encourage others (you, YES YOU) to also initiate a meetup group. Lockdown is no excuse, and online might just work out!

In the following we will explain how we got started, how we organize our meetings in terms of content and structure and provide information about what did and did not work well.

Obligatory disclaimer: What worked for us might not work for you, and things that didn't work for us might work very well for you.

Why create a Meetup

Our reason to create a meetup was simple: to bring together aspiring rationalists from our area, to learn together, to share progress, to help each other, and to socialize. It also helps to get to know new people and to learn about how other people think.

Overview

The first step in creating the meetup group was writing a post on LessWrong to announce the meetup. Since the initial video-call, we have been meeting online almost every week. With a regular influx of new members, our group has grown to about a dozen regular attendees.

Similarly, the structure and content of our meetings has evolved over time. Our weekly schedule looks something like this currently:

19:00 – 19:10 People slowly coming in and welcoming each other

19:10 – 19:20 Introductions (if someone new is present)

19:20 – 20:00 Topic 1: Hammertime Sequence

Topic 2

[Topic 3]

Scheduling Topics for next Week

Feedback

[End of Meetup]

Open Socializing

The first few meetups were considerably shorter, but recently the open socializing went until around 21:30 to 22:00 o'clock.

(If you want to start your own meetup, it is advisable to adjust the structure and content to the interests and preferences of the people attending.)

From our experience, coming up with interesting and worthwhile content is nothing to be worried about as (so far) we always had more ideas for discussion topics or other activities than we manag...
Oracles, Informers, and Controllers
by ozziegooen — 2021-05-25T14:16:22.378Z

Epistemic Status: I think the axes are straightforward enough. I'm not particularly sure about the names, happy to hear other sorts of suggestions.

Scholarly Status: Push/pull systems have been discussed before. I'm sure there's other naming for this sort of thing in the engineering/design literature, but it probably hasn't been optimised as well to fit in with the terminology of Superintelligence. This was another fairly quickly written post.

This article builds on ideas from this one, but that one is not a prerequisite.

In

Superintelligence

, Nick Bostrom proposed the terminology of oracles, genies, and sovereigns. These metaphors are clear and have achieved some popularity. I recommend

this post

by Katja Grace for an overview and some further thinking.

This terminology might be incomplete. I think that oracles have a close parallel to genies, but there's no information-only parallel to sovereigns. Here, I propose two options.

Oracles

To reiterate from Superintelligence, an oracle is a system that answers questions. We have similar systems now, though they are weaker than what Bostrom considered. When you enter search queries into Google or many other search programs, you are using them as oracles.

Pure functions

don't modify the external world and are used to in-essence answer particular queries. Some consultants are asked to solve particular questions or problems. These all represent types of weak oracles.

Informers

In comparison to an oracle, an informer is a system that actively pushes select information to recipients. AI assistants are starting to veer into this territory. They suggest particular actions without being prompted.

Some sorts of prompts given by Siri include:

"Dinner with Josh looks important! Turn on Do Not Disturb until the end of this event!" -

link

"Call in to Portfolio work Session. Event at 10:00 AM found in Calendar" -

link

"Tell Mum Happy Birthday!" Birthday found in Contacts" -

link

A relevant suggestion that

surprised one user

.

Technical systems also have many sorts of push messaging systems. These would also count under the informer category, though they are more boring. Modern recommender systems (like Youtube and Facebook recommendations) are arguably examples of informers.

Controllers

Lastly, we have the controller. The controller is the extreme form of the informer. At this point, prompts are available for nearly all of hu...
We should probably buy ADA?
by mako yass — 2021-05-24T23:58:05.395Z

Edit 2022: It seems like better systems have come along, and a crypto winter has set in, and Taleb argues that BTC will eventually go to zero and I'm inclined to agree, platforms will keep improving, it may be a long time before any of them are good enough to stick as a standard, applications will migrate, most platform tokens will eventually be worthless, if there's ever a unit of exchange in crypto we do not yet know how it will hold its value.
So I'd no longer advocate buying much of anything until we're really pretty sure that no further improvements to the technology can be made, and then still be extremely wary of volatility, don't expect platform tokens to hold value far beyond the actual usefulness of the platform.

Epistemic Status: Buying, but, the following does not constitute investment advice. There are some big open questions and some intuitions I can't start to explain (see the bottom).

Purpose: I am writing this post because I want the sorts of people who take the alignment problem seriously to have more money, but you will still need to do some research of your own. What I have here is more of a series of argument outlines than a complete analysis. My purpose here isn't to accelerate adoption of Cardano, really. I don't think Cardano needs my help.

ADA is the currency of Cardano. Cardano is (soon will be) a smart contract platform comparable to Ethereum.

I think the main load-bearing part of the argument here is that it looks like Cardano will end up have a year or two with the lead to itself, and whoever takes that lead, now, I don't think they would lose it again for a very very long time, because after Proof of Stake, Scaling, and Updatability, there don't seem to be any major holes left to fill, that's it, the biggest limitations of smart contracts will have been solved, developers will largely not be tempted away.

But I'm not completely sure. We need more clarity on this.

Legitimacy

Present here in abundance are the qualities upon which credit grounds its meaning, responsibility, reliability. For the following reasons, many people will rightly want to believe in Cardano:

The project is focusing suitably hard on solving the real human problems that blockchains can actually help with: building the missing infrastructures for banking/identity/loans/credit in the developing world

They've already secured a deal with the Ethiopian government to integrat

...
Books on Infant and Early Childhood Development?
by moderock — 2021-05-24T23:44:25.829Z

Recently, a friend of mine recommended "BabySparks", an app providing a set of development milestones and targeted growth activities for babies and toddlers.

This feels like the sort of thing I should be able to pull from a few books, rather than a subscription-based app. I'm also not particularly interested in Yet Another App, nor am I particularly keen on handing my infant's personal data to Yet Another Social Media company.

I'm especially interested in understanding the varying stages of development, and in activities to perform with aforementioned infant during their first year.

So: Any non-app resources that LessWrongers would care to recommend?
Questions are tools to help answerers optimize utility
by ozziegooen — 2021-05-24T19:30:30.270Z

Epistemic & Scholarly Status: Fairly quickly written. I’m sure there’s better writing out there on the topic somewhere, but I haven’t found it so far. I have some confidence in the main point, but the terminology around it makes it difficult to be concrete.

TLDR
The very asking of a question presupposes multiple assumptions that break down when the answerer is capable enough. Questions stop making sense once a questioner has sufficient trust in the answerer. After some threshold, the answerer will instead be trusted to directly reach out whenever is appropriate. I think this insight can help draw light on a few dilemmas.

I've been doing some reflection on what it means to answer a question well.

Questions often are poorly specified or chosen. A keen answerer should not only give an answer, but often provide a better question. But how far can this go? If the answerer could be more useful by ignoring the question altogether, should they? Perhaps there is some fundamental reason why we should desire answerers to act as

oracles

instead of more general information feeders.

My impression is that situations where we have incredibly intelligent agents doing nothing but answer questions are artificial and contrived. Below I attempt to clarify this.

Let's define some terminology:

Asker: The agent asking the question.

Answerer: The agent answering the question. It could be the same as the asker, but probably later in time. Agent here just means “entity”, not agent vs. tool agent.

Asked question: The original question that the asker asks.

Enlightened question: The question that the asker should have asked, if they were to have had more information and insight. This obviously changes depending on exactly how much more information and insight they have.

Ideal answer: The best attempt to directly answer a question. This could either be the asked question or an enlightened question. Answer quality is evaluated for how well it answers the question, not how well it helps the asker.

Ideal response: The best response the answerer could provide to the asker. This is not the same as the ideal answer. Response quality is evaluated for how it helps the answer, not how well it answers the question.

Utility: A representation of one's preferences. Utility function, not utilitarianism.

Examples

Question: What's the best way to arrive at my dentist appointment today?

The answer to the stated ...
A.D&D.Sci May 2021 Evaluation and Ruleset
by abstractapplic — 2021-05-24T16:25:13.704Z

This is a followup to the D&D.Sci post I made last week; if you haven’t already read it, you should do so now so you know what I'm talking about here.

Ruleset

Generation code is now up here; this section goes through the important points.

Species

The carcasses obtained by the stranger are 41% Mild Boar, 31% Jungle Mammoths, 5% Jewel Beetles, and 23% Dragons (5% Green, 2% Gray, 8% Blue, 8% Red).

Days Since Death

The days since each creature’s death is modelled by rolling two d10s and taking the lowest result.

(For the rest of this post, let “[DSD]” stand in for “Days Since Death”.)

Butchery

Mild Boar

The revenue from a boar is found by rolling a d4, a d8, a d12, and a d20, then summing every result greater than [DSD].

EV is graphed below.

Jungle Mammoths

The revenue from a Jungle Mammoth carcass is given by 20+10d4-3*[DSD]

EV is graphed below.

Dragons

Valuable components of Dragons are Scales, Tongue, Heart and Spleen.

A Dragon has 5d8 Scales; these are worth 1sp apiece, unless it’s Red, in which case they’re worth 2sp apiece (this is the only way colour is relevant).

A Tongue is worth 10sp, so long as the corpse is less than three days dead. A Heart is worth 30sp until day five, and a Spleen is worth 5sp until day seven. Once these thresholds are passed, the organs lose their magic and become worthless.

EV is graphed below.

Jewel Beetles

The revenue from a Jewel Beetle is found by starting with a value of 1 and rolling a d6 repeatedly. On a 1, stop rolling and take the value as your revenue; on a 6, double the value and keep rolling; on any other roll, add your result to the value and keep rolling.

EV is infinite. This is cold comfort to those who won the one on auction, which happened to be worth only 18sp.

Universes

I got 7 entries. These were randomly assigned between two universes, and the universe with three human players got an NPC to even the odds.

The NPC

To hedge my bets against an unexpected level of cooperation – if all or most of the human players made low bids, I didn’t want the NPC to ruin their fun – I knew I had to make NPC behaviour derived from player behaviour, while not granting it any unfair advantage. Therefore, for each lot, the NPC bid is randomly selected from the seven player-submitted bids.

(Three of you become aware that one of your compatriots appears to be a flickering, translucent tangle of selves. The auctioneer, noticing your concern, reassures you...
Problems facing a correspondence theory of knowledge
by Alex Flint — 2021-05-24T16:02:37.859Z

Financial status: This is independent research. I welcome financial support to make further posts like this possible.

Epistemic status: These ideas are still being developed.

Outline

I am interested in recognizing entities that might exert significant power over the future.

My current hypothesis is that knowledge of one’s environment is a prerequisite to power over one’s environment.

I would therefore like a good definition of what it means for an entity to accumulate knowledge over time.

However, I have not found a good definition for the accumulation of knowledge. In this sequence I describe the definitions I’ve tried and the counterexamples that I’ve come up against.

Introduction

The entities that currently exert greatest influence over the future of our planet — humans — seem to do so in part by acquiring an understanding of their environment, then using that understanding to select actions that are likely to achieve a goal. Humans accumulate knowledge in this way as individuals, and are also able to share this understanding with others, which has led to the accumulation of cultural knowledge over time. This has allowed humankind to exert significant influence over the future.

More generally, life forms on this planet are distinguished from non-life in part by the accumulation of genetic knowledge over time. This knowledge is accumulated in such a way that the organisms it gives rise to have a capacity for goal-directed action that is optimized for features of the environment that have been discovered by the process of natural selection and are encoded into the genome.

Even though genetic knowledge accumulates over many lifetimes and cognitive knowledge accumulates during a single lifetime, for our present purposes there is no particular need to distinguish "outer knowledge accumulation" from "inner knowledge accumulation" as we do when distinguishing outer optimization from inner optimization in machine learning. Instead, there are simply processes in the world that accumulate knowledge and which we recognize by the capacity this knowledge conveys for effective goal-directed action. Examples of such processes are natural selection, culture, and cognition.

In AI alignment, we seek to build machines that have a capacity for effective goal-directed action, and that use that capacity in a way that is beneficial to all life. We would particularly like...