trentmkelly/LessWrong-43k huggingface dataset:
Why you are psychologically screwed up
by R Bruns — 2022-02-18T17:07:00.758Z

Linkpost for https://allegedwisdom.blogspot.com/2022/02/why-you-are-psychologically-screwed-up.html

Epistemic status: deliberately written in simplified tale/myth mode. I could go off on so many tangents on the actual complexity and nuance of everything, but I wanted to keep it tight. I encourage you to 'unpack' the points by comparing them to things you know, and/or framing what I say in different ways, to see if the main theme/message makes any sense to you.

1: Monkeys Become Ants

In the beginning there were monkeys. Individuals had their own models of physical reality, and desired things, and took actions based on their desires and reality models. This often involved fighting the other monkeys to get something. Life was solitary, poor, nasty, brutish, and short.

And then something happened. Some monkeys became more cooperative and social. They formed packs, and the packs were more successful than the individuals. The monkeys in packs coordinated their actions and learned from each other. Homo sapiens crushed its competition and filled the world.

Making the pack work required a lot of psychological engineering. People had to be programmed to replace their own desires and world models with the shared desires and models of the pack, at least when it was necessary for the pack to cooperate and do something. People who were unable or unwilling to do this were selected against.

Over time, cultures and religions and ideologies learned new tricks to enable cooperation at larger scales. They became more aggressive about replacing the desires and world-models of people with those of the group. People were acculturated to deeply internalize the group's desires and have faith in the group's description of reality, such that going against either was somewhere between morally abhorrent and unthinkable.

The groups that were capable of generating cooperation/brainwashing at scale crushed the ones that had less ability to do this. Eventually giant hiveminds formed, after figuring out how to brainwash millions of people into useful cooperation. They came to rule the world, and people who were unable or unwilling to be assimilated into these mega-hiveminds were selected against.

In short, monkeys were psychologically manipulated into sometimes acting like ants. It didn't matter how traumatizing this process was, because the hive minds that did it won the fight against those w...
Simulation arguments
by Joe Carlsmith — 2022-02-18T10:45:55.541Z

“If Nature herself proves artificial, where will you go to seek wildness? Where is the real out-of-doors?”

– C.S. Lewis

(Cross-posted from Hands and Cities. Content warning: weird)

Summary:

This post examines “simulation arguments” – i.e., arguments that we should assign significant probability to living in a computer simulation.
I distinguish between two types. The first type starts with our “standard story” about the world, and tries to argue that on this story, unless something prevents the creation of suitably many sims suitably like us, we’re probably sims. I think this version fails: on our standard story, none of the sims will make our observations. For example, they won’t observe the specific set of books on my bookshelf.
This sort of objection feels cheatsy – and ultimately, I think, it doesn’t work on the best version of the argument. But I use a series of cases to bring out its similarity to less cheatsy-feeling cases: for example, cases in which it appears that all the sims that will ever get made are squid-people with tentacle arms. Simulation arguments, I suggest, should say the same thing about all these cases.
But the “indifference principles” that classic simulation arguments rely on get confused about this. In particular, they end up having to hand-wave about which things we count as “knowing,” and/or about which “reference classes” are “admissible.” This confusion makes it seem like classic arguments require an unprincipled kind of “selective skepticism” – one that risks undermining itself.
I think we do better to focus on a second type of simulation argument, which relies centrally on a claim about what sorts of priors it’s reasonable to have over “structurally similar” worlds that contain your observations. In particular, instead of saying “on your standard story, you’re probably a sim,” this version says: “if that’s your story, you should put lots of weight on these alternative, structurally similar stories, too – on priors, such stories are similarly likely, and they contain your observations, too. But on those stories, you’re a sim.”
I think this second type of argument works better – and that it brings more to the table than a standard skeptical “what if.” Indeed, I think that classic formulations of the simulation argument are implicitly leaning on something in the vicinity. But bringing out the underlying assum

...
Monks of Magnitude
by Duncan Sabien (Inactive) — 2022-02-18T07:48:58.764Z

Sometimes I encounter a concept and it immediately embeds itself in my culture, such that it feels like I always knew it. This is a short description of such a concept; it was a near-perfect match for a way I already thought about things, and has since become a useful handle that I find myself explaining to others (so that I can subsequently reference it) roughly once a month.

There is a certain story, which I will not name here in order to reduce the spoiler-y nature of the following description.

In that story, there is a monastery, and the monastery is divided into tiers, or levels.

(I may not be precisely representing the story, here, but rather how the thing in the story ended up being recorded by my brain.)

Some monks are 1-day monks. They come out from their seclusion once a day, and mingle with the regular people, and share their insights, and make new observations, and then retreat back to their private spaces to muse and meditate.

Some monks are 10-day monks. They are much like the 1-day monks, except they come out only every 10 days.

The 10-day monks tend to think longer thoughts, and wrestle with subtler or more complex problems, than the 1-day monks. This is treated by the culture of the monastery as natural and correct. Of course the 10-day monks address a different set of problems; if 10-day monks and 1-day monks were good for the same purposes, the two different Orders wouldn't need to exist.

There are also 100-day monks, who come out only a few times per year.

There are also 1,000-day monks, who come out only every few years.

There are also 10,000-day monks, who come out only every thirty years or so.

There are also 100,000-day monks (the structure of this society has led them to be better at solving problems overall, which has allowed for some advances in longevity tech).

"Come out" may be a bit of a misnomer; in fact, it is the case that the most valuable insights of a given Order tend to be fully comprehensible only to monks of one, mmmmaaaaybe two Orders below them. So the 100,000-day monks, when they report in, mostly speak only to the 10,000-day monks, who are responsible for distilling and transferring relevant insights to the 1,000-day monks, who are responsible for distilling and transferring relevant insights to the 100-day monks, etc.

(It's also the case, as some readers have pointed out in the comments belo...
Implications of automated ontology identification
by Alex Flint — 2022-02-18T03:30:53.795Z

Financial status: supported by individual donors and a grant from LTFF.

Epistemic status: early-stage technical work.

This write-up benefited from conversations with John Wentworth.

Outline

This write-up is a response to ARC’s request for feedback on ontology identification, described in the ELK technical report.

We suppose that a solution to ELK is found, and explore the technical implications of that.

In order to do this we operationalize "automated ontology identification" in terms of a safety guarantee and a generalization guarantee.

For some choices of safety guarantee and generalization guarantee we show that ontology identification can be iterated, leading to a fixed point that has strange properties.

We explore properties of this fixed point informally, with a view towards a possible future impossibility result.

We speculate that a range of safety and generalization guarantees would give rise to the same basic iteration scheme.

In an appendix we confirm that impossibility of automated ontology identification would not imply impossibility of interpretability in general or statistical learning in general.

Introduction

In this write-up we consider the implications of a solution to the ontology identification problem described in the ELK technical report. We proceed in three steps. First, we define ontology identification as a method for finding a reporter, given a predictor and a labeled dataset, subject to a certain generalization guarantee and a certain safety guarantee. Second, we show that, due to the generalization and safety guarantee, ontology identification can be iterated to construct a powerful oracle using only a finite narrow dataset. We find no formal inconsistency here, though the result seems counter-intuitive to us. Third, we explore the powers of the oracle by asking whether it could solve unreasonably difficult problems in value learning.

The crux of our framework is an operationalization of automated ontology identification. We define an "automated ontology identifier" as meeting two formal requirements:

(Safety) Given an error-free training set, an automated ontology identifier must find a reporter that never answers "YES" when the true answer is "NO" (though the converse is permissible). This mirrors the emphasis on worst-case performance in the ELK report. We say that a reporter meeting this requirement is ‘conserva

...
A scheme for sampling durable goods first-hand before making a purchase
by riceissa — 2022-02-17T23:36:35.665Z

A problem when trying to purchase some classes of products (like pillows, computer mice, and sleep masks) is that it is difficult to sample lots of them first-hand before making a purchase. This forces one to rely on product reviews or to settle for one's limited experience (such as just buying the same kind that a friend has that seems good enough, or trying the few available samples at a brick and mortar store).

Here is a naive scheme to allow a group of people to sample a bunch of kinds of a single product before making a purchase:

Suppose there are n contender products.
Find n people.
Person 1 buys all n products and collects payment equal to the average cost of the products from each of the other people.
Person 1 tries all products, then ships all of the products to person 2.
From here the process repeats: person 2 tries all the products, then ships them all to person 3, etc.
At the end, each person buys their favorite product. (In this naive scheme, the products used during the sampling process are thrown away.)

At the end of the scheme, each of the people will have their favorite product, for an average cost equal to twice the average cost of the products, plus shipping costs. (There's some subtlety here where, e.g. if better products tend to be more expensive, then you would expect people to pay more than the average.)

The above scheme can be improved in simple ways, such as not throwing away the products used during the sampling process, and instead letting people keep them (and then somehow distributing the savings). The more diverse people's tastes are, the higher the savings. The contender products can also be chosen in "smart" ways such as by reading product reviews online (not adding such a preliminary research step would be silly, since it would just be emulating what brick and mortar stores already do). If more than n people are found, the overhead can also be reduced (with N people, n products, and an average cost p of products, the expected cost will be p+np/N, plus shipping).

Some things I like about this scheme:

It allows people to have first-hand experience of all of the products. This has several benefits:

eliminates reliance on product reviews
respects people's idiosyncratic preferences (in fact, the scheme encourages idiosyncrasy because the more people prefer different pro

...
Is the competition/cooperation between symbolic AI and statistical AI (ML) about historical approach to research / engineering, or is it more fundamentally about what intelligent agents "are"?
by Edward Hammond — 2022-02-17T23:11:29.817Z

I have found that comprehensive overviews of artificial intelligence (Wikipedia, SEP article, Norvig and Russel's AI: A Modern Approach) make reference to symbolic AI and statistical AI in their historical context of the former preceding the latter, their corresponding limitations etc. But I have found it really difficult to dissect this from the question of whether the divide / cooperation between these paradigms are about the implementation of engineering of intelligent agents, or if they are getting at something more fundamental about the space of possible minds (I use this term to be as broad as possible considering anything we would label as a mind, regardless of ontogeny, architecture, physical components etc)?

I have given a list of questions below, but some of them are mutually exclusive, i.e. some answers to one question make other questions irrelevant. The fact that I have a list of questions is a demonstration of the fact I find it difficult to find what the boundaries of the discussion are supposed to be. Basically, I haven't been able to find anything that begins to answer the title question. And so I wouldn't expect any comment to answer each of my subquestions one by one, but to treat them as an expression of my confusion to maybe try an point me in some good directions. Immense thanks in advance, this has been one of those questions strangling me for a while now.

While trying to concern oneself as little as possible with the implementation or engineering of minds, what is the relationship between symbolic AI, connectionism, and the design space of minds?

When we talk about approaches to AI “failing”, is this in terms of practicality / our own limitations? I.e. without GPUs, in some sense “deep learning fails”. And by analogy, symbolic AI’s “failure” isn’t indicative of the actual structure of the space of possible minds.
Or is it more meaningful. I.e. the “failure of symbolic AI in favor of statistical methods” is because ‘symbolic AI’ simply doesn’t map onto the design space of minds.

Are symbolic AI and machine learning merely approaches to design an intelligent system? I.e. there are regions in the design space of minds that are identifiable as ‘symbolic’ and others as ‘connectionist/ML’.
Do all minds need symbolic components and connectionist components? And if so, what about the human brain? The neural network / artificial neural n

...
The Kindness Project
by soth02 — 2022-02-17T20:24:38.732Z

AI Safety is a group project for us all. We need everyone to participate - the ESFPs to the INTJs!

Capturing the essence and subtleties of core values needs input across a broad span of humanity.

Assumption 1 - large language models will be the basis of AGI.

Assumption 2 - One way to add the abstraction of a value like "kindness is good" into the model is to add a large corpus of written material on Kindness during training (or retraining).

The Kindness Project is a website with a prompt, like a college essay. Users add their stories to the open collection based on the prompt: "Tell a story about how you impacted or were impacted by someone being kind". This prompt is translated for all languages to maximize input.

The end goal is that there is a large and detailed system of nodes in the model around the abstraction of Kindness that represents our experiences.

There would be sister projects based around other values like Wisdom, Integrity, Compassion, etc.

The project incentivizes participation through contests, random drawings, partner projects with schools, etc.

Submissions are filtered for plagiarism, duplicates, etc.

Documents are auto-linked back to reddit for inclusion in language model document scrapers.
Interlude: On Realness
by LoganStrohl — 2022-02-17T19:02:16.551Z

I have a final line of thought on “the territory” that’s a lot less tidy than what you’ve read so far. Everything else in this sequence is a distillation, an attempt to communicate large principles that I’ve brought into sharp focus over time. In this interlude, I’ll be doing something different.

Here, I’ll be sharing a thread of investigation that’s still very much open for me. I’m not entirely sure what it’s about, but I think it has something to do with what it feels like to lower the map.

If you’re so inclined, I invite you to approach this as a fellow naturalist/investigator/curious kitten. I invite you to puzzle with me over how you might tell when you’re heavily map-focused, and when you’re not. What does it feel like to lower the map, to have it knocked forcefully from your hands, or to suddenly encounter something that’s not mostly made of your own perceptions and interpretations? And how does that sort of thing happen?

It is very interesting to me that it’s possible for some things to feel especially real.

One morning in college, I was half-sleeping through a poetry lecture in a dim classroom, when I looked out the window and saw a tree branch covered in spider silk. The silk shimmered in the sunlight. Strands of it hung from the branch and wafted in the breeze, and as they moved, the sunlight seemed to drip down the strands in waves.

It caught me. I don’t remember anything about the lecture, but I’ll never forget the silk. Seeing it felt a lot like waking up from a dream. That tree seemed more real than anything in the classroom, and more real than anything else in my life from the previous month.

I’ve puzzled over this perception of “realness” off and on for years. I’ve collected a lot of reference experiences that are rich in this property, and I’ve tried to compare them to each other.

Here are a few quick examples:

A person said, “I’ll be real with you,” and then began to say things that did in fact seem more “real”. My brain says they “stopped lying”, but I think it’s more complicated than that.
Reciting the sentence “I have seen the smoke rising from the pipes of lonely men in shirtsleeves leaning out of windows” (and also much of the rest of “The Love Song Of J. Alfred Prufrock”)
I was out for a walk in the early morning when I saw a patch of moss covered in dew.
The main character Max in the Dave Eggers novel The Wild Things feels more rea

...
Covid 2/17/22: Coasting Downwards
by Zvi — 2022-02-17T15:20:01.294Z

As cases and hospitalizations continue to drop, restrictions are slowly being lifted and life is returning to its new normal. Masks are still sticking around in many places where they no longer make physical sense, and will doubtless continue to do this for some time, with the most important one being schools. Still, the tide has turned, and they are in less such places every day.

The convoy continues to stick around in Ottawa, although the blockades at the border crossings have been cleared. Canada invoked the Emergencies Act and is doing some things that it seems important to know about. I covered the situation yesterday in its own post and I ask you to confine all convoy-related comments there.

Technical Note: Ever since their attack on Scott Alexander, I have been avoiding linking to The New York Times. I will continue to follow this policy on links directly to the paper itself but will no longer fully avoid secondary links such as to Twitter posts that in turn link to NYT. They’re still on the enemies list but no need to go too crazy. That’s especially true given that we learned this week that the doxing using hacked information is considered acceptable standard procedure for those who anger power, and I want my rules to be consistent.

Executive Summary

Cases continue their decline as do hospitalizations and deaths.
Vaccinations for young kids indefinitely postponed for no good reason.
Convoy seems important, might want to check that out.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 900k cases (-36%) and 14,700 deaths (-13%).

Results: 855k cases (-40%) and 15,194 deaths (-11%).

Prediction for next week: 555k cases (-35%) and 11,400 deaths (-25%).

Deaths

Deaths have variable lags so we continue not to see as much decline as we might hope for, but we see hospitalization numbers dropping steadily each day and next week we will be three weeks out from a dramatic case drop and four weeks from the peak, so we should see a larger decline. For cases, I presume behaviors will adjust so I expect a somewhat smaller decline in cases than before, but weather is increasingly on our side as well and I don’t think there is that big a behavioral adjustment in many places, and the Maine and Maryland thing is weird, so I still expect good results. We have had much lower case counts at various points last year.

...
Twitter thread on postrationalists
by Eli Tyre — 2022-02-17T09:02:54.806Z

I wrote the following as a thread on twitter. Kaj Sotala asked me to share it on LessWrong as well, so I have reproduced it here with minimal editing.
(If you are interested in my twitter content, but don't use twitter, I recommend reading it on threadreader. The user experience of reading long twitter threads on twitter is pretty frustrating, and threadreader is somewhat better. Also, I think engaging with twitter is net-bad for most people, and I feel morally obliged to make sure my content is hosted elsewhere, so that I don't add to the incentive for people to do things that are bad for them.)

Here's my theory of the cluster of people that call themselves post-rationalists.
Long ago, @ESYudkowsky wrote the sequences. The sequences had a lot of explicit content, but they also had a "vibe."

The vibe was something like "intelligence dominates everything; the way to power and winning is thinking really well. If you think well enough, you will be awesome."
Also, "arrogance and snark about how insane everyone / the world is."
Also, "ambitious, heroic, glorious."

People read the sequences, and they got whatever they got out of them, but a LOT of what people picked up on was the "vibe".

For instance, one major thing that I got from the sequences is that "rationalists should win". That I should be able to get what I want by being clever, and that I shouldn't often be in situations where I'm not getting what I want. And I should be indignant about them.
[Note: when I say "indignant" here, I don't mean an attitude of "this is unfair. Now I will mope." It was more like, "I'm better than this. I will now rise above."]

In 2015, this became a sort of mantra for me.
In places where other people would be satisfied with the lemons that life gave them, _I_ should win.

The actual post where it says that rationalists should win is about Newcobm's problem: it's a [reasonably] abstract point about the philosophy of rational choice.

I mostly wasn't, in this instance, making use of the explicitly stated content. It's not like I was using some _specific_ reasoning technique that is more effective than baseline, and that's why I would win.
I was adopting an attitude that I picked up on.

(Compare this to the case where a person reads the sequences and starts doing explicit Bayesian calculations, [edit: regularly], and expects to do better

...