trentmkelly/LessWrong-43k huggingface dataset:
What Decision Theory is Implied By Predictive Processing?
by johnswentworth — 2020-09-28T17:20:51.946Z

At a fairly abstract/stylized level, predictive processing models human cognition and behavior as always minimizing predictive error. Sometimes, the environment is "fixed" and our internal models are updated to match it - e.g. when I see my untied shoelace, my internal model updates to include an untied shoelace. Other times, our internal model is "fixed", and we act on the environment to make it better match the model - e.g. "wanting food" is internally implemented as a strong expectation that I'm going to eat soon, which in turn makes me seek out food in order to make that expectation true. Rather than having a utility function that values food or anything like that, the decision theory implied by predictive processing just has a model in which we obtain food, and we try to make the model match reality.

Abstracting out the key idea: we pack all of the complicated stuff into our world-model, hardcode some things into our world-model which we want to be true, then generally try to make the model match reality.

While making the model match reality, there will be knobs we can turn both "in the model" (i.e. updates) and "in reality" (i.e. actions); there's no hard separation between the two. There will be things in both map and reality which we can change, and there will be things in both map and reality which we can't change. It's all treated the same. At first glance, that looks potentially quite useful for embedded agency.

(My own interest in this was piqued partly because a predictive-processing-like decision theory seems likely to produce abstraction boundaries which look like Cartesian boundaries. As in that post, it seems like some of the intuitive arguments we make around decision theories would naturally drop out of a predictive-processing-like decision theory.)

What problems does such a decision theory run into? What sort of things can we hardcode into our world-model without breaking it altogether? What things must be treated as "fixed" when making the model match reality? Does such an approach have any "invariant" implications, i.e. implications independent of which model we're trying to match? What further requirements are there on the target model in order for a predictive-processing-style agent to have "good" behavior, in the ways characterized by other decision theories?

This is intended to be an open-ended research question, but off-the-cuff thought...
What are examples of Rationalist fable-like stories?
by Mati_Roy — 2020-09-28T16:52:13.500Z

I will post some of the collection I've acquired. I have ideas for more. And would be interested in seeing more.

If there's enough interest, and with the permission of the authors, I would like to make a book of Rationalist fables / parables (but with a more generic name). Maybe it could be adapted for children. Thoughts?

Let me know if you're interested in helping me with this project, including but not limited to: finding stories, selecting stories, adapting them, reaching out to authors, writing more stories, writing their morales (?), making a book cover, (self?) publishing it, marketing it, choosing a title, donating money to contract external help, etc.
Macro-Procrastination
by Neel Nanda — 2020-09-28T16:07:48.670Z
Link: https://www.neelnanda.io/blog/32-macro-procrastination

The Problem

As readers of this blog will know, I massively procrastinate. And this is a pretty big problem in my life. But it’s a problem I’m aware of, that mostly resolves itself. I have deadlines, accountability mechanisms, and the latent, creeping guilt of something I know I’m putting off. And all of my solutions centre around noticing this creeping guilt and converting into action. And generally, the thing will get done! There are costs - it’s often done late, or to a lower standard. There’s often a lot of wasted motion, time spent on mindless busywork with opportunity costs for what I actually care about. But the costs are bounded, and I can force myself to get things done in the end.

But, when I actually think about it, this system is weird. There’s a major disconnect here. I get over procrastination because I feel guilty about it. But I care about procrastination as an issue because on some level I want to do these tasks. These tasks are important to my long-term goals and procrastination is bad because it holds me back from my goals. I am allocating my time badly, and this is sad. The guilt is a useful incidental property of the fact that there are deadlines, but the fact that it’s a problem is nothing to do with the deadlines. Procrastination is fundamentally a problem of prioritisation, and bad allocation of time.

And this is an important disconnect, because there are worthwhile tasks I procrastinate on that don’t have deadlines. And these are often really important! Things that will help me towards becoming a happier person, helpful to my career, and generally improving the world. It is a really big deal if I never do them. But nobody is holding me accountable, there is no urgency, and so nothing ever happens. I call this problem macro-procrastination.

Macro-procrastination was recently made salient to me, because I realised it was happening in a lot of areas of my life. Some of the most noticeable:

Exercise is obviously important to my lifespan, healthspan, general happiness, mental energy and appearance. And is obviously an insanely good use of time. But I just wasn’t in the habit of doing any

I’ve since started Couch to 5K, which I’d highly recommend

Careers: I’m pretty seriously considering a career in AI Safety. But I’d put approximately 0 effort into just re

...
What are good ice breaker questions for meeting people in this community?
by Mati_Roy — 2020-09-28T15:07:16.798Z

I was praising the Petrov Day event and saying I liked the Icebreaker app.

parsley said:

icebreaker.video allows users to use their own questions. I didn't have the time to come up with my own, so I just resorted to the boilerplate ones provided by the website. I wonder if there's a "rationalist ice breaker" list somewhere.

That's a question I've had in mind for years, and I'm now asking it to y'all ^_^
On Destroying the World
by Chris_Leong — 2020-09-28T07:38:38.319Z

Yesterday I blew up the front page. This was unintentional. I was tricked by one of my friends:

petrov_day_admin_account September 26, 2020 11:26 AM Hello Chris_Leong,

You are part of a smaller group of 30 users who has been selected for the second part of this experiment. In order for the website not to go down, at least 5 of these selected users must enter their codes within 30 minutes of receiving this message, and at least 20 of these users must enter their codes within 6 hours of receiving the message. To keep the site up, please enter your codes as soon as possible. You will be asked to complete a short survey afterwards.

In retrospect, this was quite silly of me. I actually noticed that the account was different from the one that sent the first message, which should have given it away, but the message really did feel legit so I trusted it anyway.

But beyond this, there were further details that should have made the message somewhat suspicious. The first is that this experiment occurred after midnight for San Fransisco. Given that most of the users on this site are based in the US, they wouldn't have been awake. While they might have specifically chosen users from suitable timezones, it would have made much more sense for them to just wait until more users woke up. Secondly, 20/30 users within 6 hours seems a bit high given that users weren't told in advance that the game was going on, so it's not clear how many would be available even if they knew.

One thing that greatly surprised me was how much the following comment was misunderstood:

Should I press the button or not? I haven't pressed the button at the current time as it would be disappointing to people if they received the email, but someone pressed it while they were still asleep.

People read the comment and assumed I was intending the press the button and the only different the trick meant was that it occurred earlier. One of the risks when writing comments quickly is that the meaning might not be very clear at all. I hadn't actually made up my mind about whether to push the button or not as I was waiting for comments to come in. All I had decided was that I didn't want the site to blow up while people were still asleep because I thought it'd be less fun for them. That said, I was entirely open to blowing up the site if I thought that the argument for was stronger than the argument against.

Ruby pointed...
"Win First" vs "Chill First"
by lionhearted (Sebastian Marshall) — 2020-09-28T06:48:21.511Z

Around a month ago, I found an incredibly insightful quote deep in the Reddit comments about a particular basketball player who had recently changed teams.

People who try hard to win first and foremost make it uncomfortable when people are trying to just have a good time and do well. And this is aside from whether they're assholes or not, unscrupulous or not. It's about win first vs chill first.

At some point, there's always a conflict between the two types. Someone has to chill, someone has to turn up the intensity, or someone has to leave.

- Source, minor formatting cleanup and emphasis added

Now, I'm not a particularly avid follower of sports. But this was a rather unusually fascinating case.

To simplify a very long story, there's a professional basketball team — the Philadelphia 76ers — that on paper have had a lot of really good players the last few years, yet have consistently underperformed expectations.

Last year, they traded for a player who is known as being super-crazy-hardcore-intense. That player was Jimmy Butler, who was an unheralded quite low draft pick (the 30th player chosen his year, meaning almost every team passed on him at least once) who worked very, very hard to turn himself into a star.

He wasn't one of those players who was really good right when joined the League — he didn't start in pro basketball until he was 22 years old, and wasn't really good until he was 25.

His career stats are here; you don't need to know much about basketball to see the trend of going from scoring 2.6 points per game your first year in the league, to 8.6 your second, to 13.1 your third year, to 20.0 your fourth year in the league is (1) someone who was not-at-all "anointed" or had an easy path for himself, and (2) showed really incredible year-over-year improvements.

Eventually, Jimmy became a consistent All-Star.

Last year, he joined the Philadelphia 76ers.

And it didn't go very well.

Although much of the story is secondhand and hearsay, apparently Jimmy Butler didn't get along with everyone else on the 76ers. During a film session to prepare for an upcoming opponent, there were reports that other players were sleeping or goofing around and Jimmy shouted at them.

Jimmy would yell at people who weren't training hard at practice, get under his teammate's skins, etc. If anyone wanted to relax and refused to go full-out competitive in pursuit of being th...
On "Not Screwing Up Ritual Candles"
by Raemon — 2020-09-27T21:55:10.793Z

I do a lot of experimentation with rational ritual. Rituals often use lots of candles. I used to just buy Random Cheap Candles. Then I kept running into problems. Now I have very strong opinions about what kinds of candles to buy.

The first problem was simply that I wanted impressive photos for Secular Solstice – I wanted to show off how it felt to people who'd never seen it before. I found myself constantly re-using the photos from Bay Solstice 2013, where the organizer (Ben Landau Taylor) had splurged on large, satisfying-looking-from-a-distance candles:

The second problem was... candles are actually pretty logistically complicated. They drip wax. Rituals often require you to light one candle off another, but only certain shapes of candle can do that. Some candles burn really quickly and flicker out halfway through your ritual.

I spent several years buying lots of candles and trying them out. I kinda became this guy:

Okay, I don't actually spend $3,600 a month, I've spent maybe $250 total. And I'm going to pitch you on spending... maybe $50, which I claim is Worth It.

I've tried to tell people "man, you should really get some nice candles for your rituals", but other people hadn't experimented with different types of candles and they were like "really? does it matter? idk man, they're candles. Are you sure you're not just getting absurdly into candles and having the Joe Biden Sandwhich Photo effect?"

I can't be 100% confident my aesthetic opinions are valid. But, I am confident there are at least some serious logistical concerns:

The Logistical Concerns

Problems you'll run into with candles:

1. You often need each candle to light other candles. This works much better if they are thin/long taper candles rather than fatter votive, tealight or pillar candles. It is basically impossible if the candle is in a jar.

2. Because you need thin candles, you need candle holders so they stay upright. Woe to the person who buys a bunch of taper candles and then realizes they have to awkwardly jury-rig candeholders out of aluminum foil at the last minute.

3. You need candles to not run out, so you want fairly large and/or slow-burning ones. Woe to the person's important candle symbolizing human civilization flickers out halfway through the Petrov Day ceremony.

4. Candles can drip wax, which can be annoying. They work better with a candleholder that can catch the wax as it falls. (Ca...
What to do with imitation humans, other than asking them what the right thing to do is?
by Charlie Steiner — 2020-09-27T21:51:36.650Z

This question is about whether you have clever ideas about how to use AI imitations of humans for AI safety. The two main ideas I'm familiar with only seem to interface with these imitations as if they're humans.

The most obvious thing one might do with a good predictor of a human is just to write software that queries the imitation human about what the right thing to do is, and then does it.
The less obvious thing to do is to try and amplify it - e.g. use teams of them working together to try to choose good actions. Or maybe even an IDA loop - use your learner that learned to imitate a human, and train it to imitate the teams working together. Then make teams of teams, etc.

But can we use human imitations to increase the effectiveness of value learning in a way other than amplification/distillation? For example, is there some way of leveraging queries to human imitations to train a non-human AI that has a human-understandable way of thinking about the world?

Keep in mind the challenge that these are only imitation humans, not oracles for the best thing to do, and not even actual humans. So we can't give them problems that are too weird, or heavily optimized by interaction with the imitation humans, because they'll go off-distribution.

Another possible avenue is ways to "look inside" the imitation humans. One analogy would be how if you have an image-generating GAN, you can increase the number of trees in your image by finding the parameters associated with trees and then turning them up. Can you do the same thing with human-imitating GAN, but turning up "act morally" or "be smart?"
What are good rationality exercises?
by Ben Pace — 2020-09-27T21:25:24.574Z

I want to know what are good rationality exercises.

I was just on a call with Liron and PhilH, hanging out after the weekly LessWrong weekend event, and we discussed exercises that could happen on LessWrong.

Here is the list we generated:

Thinking Physics
Fermi Estimates
Project Euler
Calibration Training
Basic probabilistic reasoning
Basic have-you-read-the-sequences knowledge test (e.g. "Which of the following is an example of 'belief as attire'?")

Another user on the call (whose name I forget) suggested it could be fun to have a daily Fermi Estimate on LessWrong, where everyone submits their number and the model they used to reach the number. I think this would be quite exciting.

Please write answers with other exercises that you think are or might be great for rationality training, some explanation of why you think it could be good, and a suggestion of how it could be incorporated into LessWrong. I'll probably add some of the above myself.
Puzzle Games
by Scott Garrabrant — 2020-09-27T21:14:13.979Z

I like puzzle games. I have aesthetic opinions about puzzle games. This post is meant to be a resource for people whose preferences about puzzle games have a lot in common with mine. It includes a list of game recommendations.

Rules about Spoilers

This post will be very spoiler-free. The only things information in this post about individual games will be about how much I like them. Exceptions to this rule are Tetris, Sudoku, and Sokoban. I also combine puzzle games with sequels/downloadable content/other games by the same developer, only if I think the sequels are similar, so I might spoil what games have sequels, and whether sequels are similar to the originals. Additionally one game in the list is a physical book, and one game is not primarily a puzzle game, but has a puzzle mode, and I say that in the list.

I also intend to enforce comments avoiding spoilers. The things you can say about individual games without spoiler tags are:

How much you like the game
Who developed it/What other things they developed
What platforms it is on/When it came out
If there are (good) Sequels/Prequels/DLC
If you believe you finished it
If you believe you reached the end of the main part of the game (e.g. the credits screen)
How long is the game (how long it took you, NOT the number of puzzles)
How difficult is the game, in general (you can compare to other games)
How the puzzles fit into the Deduction/Efficiency/Technical/Linchpin categories

Everything else about individual games should use the spoiler black boxes. You may NOT say specifics about whether a specific game violates the rules of the genre I outline, unless you use spoiler box. For spoilers about individual games, please list what games might be spoiled outside of the spoiler box.

I encourage mods to help me enforce spoiler rules, and encourage others to send me a message if they think a comment violates the rules and should be deleted.

Defining the Genre

I will define the genre of games I am pointing to in a way similar to Berlin interpretation of the roguelike genre. There will be a list of properties of the pure form of the puzzle game. Many great puzzle games will violate some of these properties. We can use this list to measure how close a game is to a pure puzzle game.

Puzzle games under my definition usually: are deterministic, are self-contained, have a clear end, have no time elements, have simple rules, give compl...
What hard science fiction stories also got the social sciences right?
by Mati_Roy — 2020-09-27T20:37:44.256Z

Robin Hanson says that those stories are rare.

Most discussion of the non-immediate future seems to take place in science fiction, and the small subset of science fiction where authors try hard to remain realistic is called hard science fiction. Loosely associated with hard science fiction is an intellectual community of people who try to make projections which are true to our best understanding of the world. They work out the broad science and engineering of plausible future space colonies, starships, virtual reality, computer networks, survellience, software assistants, genetically engineered people, tiny machines made to atomic accuracy, and much more.

Unfortunately, few if any people of these people know much social science. So their projections often combine reasonable physics or computers with laughable economic assumptions. This often seriously compromises their ability to make useful projections.
(source: The Economics of Science Fiction)

This is different from social science fiction, which is deliberate exploration of other possible forms of sociological organisation.
Tips for the most immersive video calls
by benkuhn — 2020-09-27T20:36:51.422Z

I spend a lot of my day on video calls. Wave is a distributed company, so they’re the main way we communicate.
But compared to talking in person, they feel unnatural:

Most people have low-quality microphones and webcams that make them look and sound bad.

There’s a lag between when you say something and when the other person hears it, making it hard to navigate
conversational turn-taking.

If you’re using headphones, you can’t hear your own voice very well.

Because of echo
cancellation, you often can’t talk when someone else is also talking, which makes the conversation
flow less well.

I started wondering how much nicer video calls would feel if I fixed these problems. So I spent way too much time
fiddling with gear and software. This post summarizes what I’ve learned. Collectively, I think these
recommendations have a pretty big impact: when talking one-on-one to friends with equally good setups, I’ve been
able to go 4+ hours without feeling fatigued.

Epistemic status: best guess; not a professional; almost certainly contains wrong bits. Tell me which
ones by comment or email!

omg ben don’t make me read
your 4500 word doorstopper, just tell me what to do

Here’s how I would stack-rank my advice for my past self. (Of course, your personal ranking might be different
depending on your situation.)

($depends) Don’t work in a space where your noise can bother other people, or vice versa.

($10-30) If you ever have network issues, run a
cable between your computer and router. You’ll probably need an adapter. (Contrary to popular belief that a bad
connection is your ISP’s fault, it’s more likely to be flaky wifi.)

(~$100) Buy open-back headphones, which let you
hear your own voice normally and are extremely comfortable.

(~$30) Switch from your built-in computer mic to a headset mic (and pop
filter), which will sound much better and pick up less noise. Note this requires a headset with
detachable cable, like the one I linked above.

You can now leave yourself unmuted! If the other person also has hea

...
A long reply to Ben Garfinkel on Scrutinizing Classic AI Risk Arguments
by Søren Elverlin — 2020-09-27T17:51:31.712Z

In July, Ben Garfinkel scrutinized the classic AI Risk arguments in a 158 minute-long interview with 80000 hours, which I strongly recommend.

I have formulated a reply, and recorded 80 minutes of video, as part of two presentations in the AISafety.com Reading Group:

196. Ben Garfinkel on Scrutinizing Classic AI Risk Arguments

197. Ben Garfinkel on Scrutinizing Classic AI Risk Arguments 2

I strongly recommend turning subtitles on. Also consider increasing the playback speed.

"I have made this longer than usual because I have not had time to make it shorter."

-Blaise Pascal

The Podcast/Interview format is less well suited for critical text analysis, compared to a formal article or a LessWrong post, for 3 reasons:

Lack of precision. It is a difficult skill to place each qualifier carefully and deliberately when speaking, and at several points I was uncertain if I was parsing Ben's sentences correctly.

Lack of references. The "Classic AI Risk Arguments" are expansive, and critical text analysis require clear pointers to the specific arguments that are being criticized.

Expansiveness. There are a lot of arguments presented, and many of them deserve formal answers. Unfortunately, this is a large task, and I hope you will forgive me for replying in the form of a video.

tl;dw: A number of the arguments Ben Garfinkel criticize are in fact not present in "Superintelligence" and "The AI Foom Debate". (This summary is incomplete.)