trentmkelly/LessWrong-43k huggingface dataset:
Hammertime Day 8: Sunk Cost Faith
by alkjash — 2018-02-05T01:00:00.503Z

[Author’s note: I will be moving future Hammertimes to my personal page to avoid cluttering the frontpage. This one is sufficiently short and probably controversial to leave here.]

This is part 8 of 30 in the Hammertime Sequence. Click here for the intro.

It pains me to begin a post about planning with an announcement about two slight changes of plans for Hammertime:

First, I will be travelling the week after next, so there will be a week-and-a-half intermission between the first and second cycles.

Second, when I sat down to write a post about Focusing, I found myself unable to add anything productive to this excellent post: Focusing, for Skeptics. Focusing is probably the second most powerful technique I learned from CFAR, so I will return to it in future cycles after more thought.

Instead, I want to write three posts on planning. These are the first steps to becoming the kind of person who can make thoughtful long-term plans and follow through with them.

Day 8: Sunk Cost Faith

One of my main motivations ever since writing The Solitaire Principle is to solve the Control Problem in humans: the problem of making and following through with long-term plans and habits despite new information and, even worse, value drift. I propose that what is commonly known and vilified as the Sunk Cost Fallacy actually exists for a good reason and is a useful first-order solution to the Control Problem.

The Uncanny Valley of Sunk Cost

Related: Sunk Costs Fallacy Fallacy.

Here is the uncanny valley one falls into when one naively cancels out Sunk Cost Fallacy:

You suck at making plans, but follow through with them anyway. You get a moderate amount done by making overconfident and poorly-thought-out plans, and just doing them despite contradictory information.
One day, you learn about the Sunk Cost Fallacy. You decide to be a Good Rationalist and categorically abandon ship on projects that no longer appeal to you. You still suck at making plans. All your plans fail and you get nothing done.
Over time, you learn that you’re not the kind of person who can follow through with long-term projects. You jump from bright light to bright light, captured by the briefest caprice. You don’t remember what it’s like to be diachronic. Your time horizon shrinks and you stop bothering with plans at all.

There’s an extremely insidious demon hiding at Step 2, related to adverse selection. Over the course of a m...
Hammertime Day 7: Aversion Factoring
by alkjash — 2018-02-04T16:10:00.667Z

This is part 7 of 30 in the Hammertime Sequence. Click here for the intro.

As we move into the introspective segment of Hammertime, I want to frame our approach around the set of (unoriginal) ideas I laid out in The Solitaire Principle. The main idea was that a human being is best thought of as a medley of loosely-related, semi-independent agents across time, and also as governed by a panel of relatively antagonistic sub-personalities à la Inside Out.

An enormous amount of progress can therefore be made simply by articulating the viewpoints of one’s sub-personalities so as to build empathy and trust between them. This is the aim of the remainder of the first cycle.

Day 7: Aversion Factoring

Goal factoring is a CFAR technique with a lot of parts. The most resonant sub-skill for me was Aversion Factoring, so we’ll start there. I highly recommend Critch’s TedX talk on the subject, where I first learned this way of thinking.

Setup

Pick from your Bug List a habit you want to start but haven’t, or that you’ve been forcing yourself to do but remains a drag. What’s happening?

For concreteness, let’s say the habit is “blog every day.”

At some level, you want to blog. You have good ideas. Writing helps you think clearly. You’d reap the benefits of being publicly wrong. If you blogged, other human beings might benefit. But if you really wanted to blog then why does it cost so much willpower every time? Why aren’t you leaping into it every day the way you leap into deep fried ice cream?

Aversion factoring is about noticing and removing the subconscious roadblocks keeping System 1 from wanting the same things System 2 wants.

1. Articulate Aversions

The first step to Aversion Factoring is to articulate the aversions that are holding you back. Begin by listing all the reasons you don’t like about doing the thing. Two things to keep in mind:

Be honest.

“I’m afraid my ideas aren’t original, my writing hasn’t improved since fifth grade, and I’m terrified of people on the internet.”

Being honest is difficult. However, there’s a second category of insidious aversions: trivial, repetitive annoyances that leave a bad taste surrounding the whole experience. See Beware Trivial Inconveniences. Finding such aversions requires attention to detail:

“I hate blogging because of the awful LaTeX support, because every time I want to include a picture I get anxious about copyright issues, and because I re...
Explicit Expectations when Teaching
by Hazard — 2018-02-04T14:12:09.903Z

Epistemic Status: Casual thoughts on small ways to improve teaching flows.

In the lectures for my current classes, things don't always make sense. Sometimes the not making sense comes from a feeling of incompleteness. The definition seems incomplete. I notice that I don't have enough info to fully explain the concept.

Generally I've noticed two reasons for this. This first is that there was some assumed prior knowledge. Sometimes that can be explicit. Maybe I was supposed to know multivariate calculus for this course, yet there was some concept I missed that is being used here. Other times the assumption of prior knowledge is implicit. This is when the professor didn't realize that there was something to explain (i.e unaware of inferential distance).

In these cases, the best route is to ask a question.

In other cases, the professor purposefully gave an incomplete explanation, which they proceed to expand on in the following slide.

In this case, the best route is to put a mental astericks next to new concept and hold tight.

A problem arises from the fact that almost all of the professors I've had (and most people just trying to casually teach me something) talk in a way where I can't distinguish what is assumped prior knowledge and what is a placeholder to be filled in later. This leads to wasted movement. I might ask a lot of questions that get "Hold on for two minutes" as a reply, or I might never explore an idea myself because I figure it will be explained at some point. I think that being explicit can make it much easier for one trying to learn.

In conversation, I can imagine a simple habit of tagging a word or concept with a mental astericks and making clear that you currently are presenting a "working explanation". Something like "A linear seperator divides n-dimensional space in to two region. We'll circle back to how it does that in a sec."

If you teach in any sort of formal setting and use visual aids, it could be worthwhile to introduce a convention like, "Things I'm going to explain later will be highlighted in blue, things that you are expected to know are highlighted in green."

If anyone tries this out in a formal setting, let me know if it had any effect.
Read Backwards
by Allen Kim — 2018-02-04T08:13:07.666Z

Sometimes if I'm reading something difficult to understand or written in an uninteresting way I'll skip ahead to the end of the chapter, or perhaps just skim ahead several paragraphs until something looks interesting. Then I'll read one paragraph at a time in reverse order. I think this helps you focus by forcing you to guess what the writer's thought process is, and it can also help to know in advance what point they're working towards.
Factorio, Accelerando, Empathizing with Empires and Moderate Takeoffs
by Raemon — 2018-02-04T02:33:42.863Z

I started planning this post before Cousin It's post on a similar subject. This is a bit of a more poetic take on moderate, peaceful AI takeoffs.

Spoilers for the game Factorio, and the book Accelerando. They are both quite good. But, if you're not going to get around to play/reading them for awhile, I'd just go ahead and read this thing (I think the game and story are still good if you go in knowing some plot elements).

i. Factorio

Factorio is a computer game about automation.

It begins with you crash landing on a planet. Your goal is to go home. To go home, you need to build a rocket. To build a rocket powerful enough to get back to your home solar system, you will need advanced metallurgy, combustion engines, electronics, etc. To get those things, you'll need to bootstrap yourself from the stone age to the nuclear age.

To do this all by yourself, you must automate as much of the work as you can.

To do this efficiently, you'll need to build stripmines, powerplants, etc. (And, later, automatic tools to build stripmines and powerplants).

One wrinkle you may run into is that there are indigenous creatures on the planet.

They look like weird creepy bugs. It is left ambiguous how sentient the natives are, and how they should factor into your moral calculus. But regardless, it becomes clear that the more you pollute, the more annoyed they will be, and they will begin to attack your base.

If you're like me, this might make you feel bad.

During my last playthrough, I tried hard not to kill things I didn't have to, and pollute as minimally as possible. I built defenses in case the aliens attacked, but when I ran out of iron, I looked for new mineral deposits that didn't have nearby native colonies. I bootstrapped my way to solar power as quickly as possible, replacing my smog-belching furnaces with electric ones.

I needed oil, though.

And the only oil fields I could find were right in the middle of an alien colony.

I stared at the oil field for a few minutes, thinking about how convenient it would be if that alien colony wasn't there. I stayed true to my principles. "I'll find another way", I said. And eventually, at much time cost, I found another oil field.

But around this time, I realized that one of my iron mines was near some native encampments. And those natives started attacking me on a regular basis. I built...
Logical counterfactuals and differential privacy
by Nisan — 2018-02-04T00:17:43.000Z

Edit: This article has major flaws. See my comment below.

This idea was informed by discussions with Abram Demski, Scott Garrabrant, and the MIRIchi discussion group.

Summary

For a logical inductor P, define logical counterfactuals by

Pn(ϕ|ψ):=∑yPk(ϕ|ψ∧Y=y)Pn(Y=y)

for a suitable k<n and a random variable Y independent of ψ with respect to Pk. Using this definition, one can construct agents that perform well in ASP-like problems.

Motivation

Recall the Agent Simulates Predictor problem:

Un=106Pn−1(An=1)+1031(An=2)

Naively, we want to solve this by argmaxing:

An=argmaxaEn[Un|An=a]

Hopefully, Pn(An=1)≈1, Pn−1(An=1)≈1, and En[Un|An=1]≈106. Also, two-boxing should be less attractive than one-boxing:

En[Un|An=2]≈103

However, if we make this well-defined with ε-exploration, we'll get

En[Un|An=2]≈106+103

and then the agent will two-box, contradiction. Instead we'd like to use predictable exploration and set

En[Un|An=2]:=Ek[Un|An=2]

for k small enough that the right-hand side is sensible. Let's see how.

Predictable exploration

Choose k≪n so that Pk(An=2)≫0. Our agent decides whether to explore at stage k, and uses its beliefs at stage k as a substitute for counterfactuals:

explore0:=Pk(explore0)<εexplore1:={1Pk(explore1=1)<122otherwise∀aEn(ϕ|A=a):=Ek(ϕ|A=a∧explore0) if Pn(A=a)<δAn:={explore1if explore0argmaxaEn[Un|An=a]otherwise

Here ε,δ are small positive numbers. It's easy to see that, under reasonable assumptions, this agent 1-boxes on Agent Simulates Predictor. But it can't use the full strength of Pn in its counterfactual reasoning, and this is a problem.

Differential privacy

To illustrate the problem, add a term to the utility function that sometimes rewards two-boxing:

Un=106Pn−1(An=1)+1031(An=2)+1061(An=2∧Xn−1)Xn−1:=Pn−1(Xn−1)<12

The agent should two-box if and only if X. Assuming that's the case, and Pn−1 knows this, we have:

Pn−1(An=1)=12¬Xn−1→En[Un|An=1]=12106¬Xn−1→En[Un|An=2]=Ek[Un|An=2∧explore0]=103+12106

So if ¬Xn−1, two-boxing is the more attractive option, which is a contradiction. (I'm rounding ε to zero for simplicity.)

The problem is that the counterfactual has to rely on Pk's imperfect knowledge of Xn−1. We want to combine Pk's ignorance of explore0 with Pn's knowledge of Xn−1.

If X is independent of A conditioned on explore0 with respect to Pk, then we can do this:

Ek[U|A=a∧explore0]=∑xEk[U|A=a∧explore0∧X=x]Pk(X=x|A=a∧explore0)=∑xEk[U

...
Hammertime Day 6: Mantras
by alkjash — 2018-02-03T22:00:00.474Z

This is part 6 of 30 in the Hammertime Sequence. Click here for the intro.

I’d like to demarcate the line between the two natural halves of Hammertime (fast and interactive vs. slow and introspective) with an experimental post, more reflective than actionable.

Motivation

The seed of this post was planted in my mind by a conversation with Zvi. In said conversation, he invited me to read the rulebook of Mage: The Ascension and take it as literally as possible. The particular magic mechanic that struck me from Mage was the Paradox phenomenon, which (roughly speaking) causes magic to backfire in the presence of Muggles.

When it is performed ineptly, or is vulgar, and especially if it is vulgar and witnessed by sleepers, magic can cause Paradox, a phenomenon in which reality tries to resolve contradictions between the consensus and the Mage’s efforts. Paradox is difficult to predict and almost always bad for the mage. The most common consequences of paradox include physical damage directly to the Mage’s body, and paradox flaws, magic-like effects which can for example turn the mage’s hair green, make him mute, make him incapable of leaving a certain location, and so on. In more extreme cases paradox can cause Quiet (madness that may leak into reality), Paradox Spirits (nebulous, often powerful beings which purposefully set about resolving the contradiction, usually by directly punishing the mage), or even the removal of the Mage to a paradox realm, a pocket dimension from which it may be difficult to escape.

The upshot is not dissimilar from the rather commonplace observation that extraordinary people seem to distort the reality around them and also have a difficult time imparting their reality-distortion field to others.

My foray into the fantastical world of Mage led me to consider taking other mechanics of magic more seriously. Among the infinite variety of ways the human mind might break the laws of physics, only a very few magic mechanics have lasted in the public imagination. Again and again, fantasy writers return to the incantation: words that effect transformation by their mere utterance. What is so psychologically fascinating about incantation?

And if a single utterance can effect magic, what might the repetition of words of power accomplish, over the course of many years?

Day 6: Mantras

Epistemic status: true story.

I was not a particularly well-socialized child...
The Super Bowl Prop Gambling Game 2018
by Jacob Falkovich — 2018-02-03T18:10:00.413Z
Link: https://putanumonit.com/2018/02/03/super-bowl-2018/ 

The Super Bowl Prop Gambling Game for people who don't know anything about football. 20% more degenerate gambling, 50% less saturated fat, 100% money back guarantee!
How I see knowledge aggregation
by zulupineapple — 2018-02-03T10:31:25.359Z

After reading the Arbital postmortem, I remembered some old ideas regarding a tool for claim and prediction aggregation.

First, the tool would have the basic features. There would be a list of claims. Each claim is a clear and concise statements that could be true or false, perhaps with a short explanation. For each claim, the users could vote on its likelihood. All these votes would be aggregated into a single number for each claim.

Second, the tool would allow the creation of composite claims by combining two existing claims. In particular, a conditional claim IF B THEN A would represent the conditional probability P(A|B). For every claim, it should be easy to find the conditionals it participates in, or the claims it is composed of. Conditionals are voted on same as simple claims (I would even consider a version where only conditionals are voted on).

Third, the tool would understand the basic probability laws and use this to direct the users' attention. For example, if three claims don't satisfy the law P(A|B) P(B) < P(A), users might be alerted about this error. On the other hand, if P(A|B) = P(B|A) = 1, the two claims might be merged or one could be discarded, to reduce the clutter.

Fourth, given a claim the tool might collect every other claim that supports it, follow every chain of argument and assemble them into a single graph, or even a semi-readable text, with the strongest arguments and counterarguments most visible.

Let's consider a possible workflow. Suppose you browse the list of claims, and find a ridiculous claim X assigned a high likelihood. You could just vote to decrease the likelihood and perhaps leave an offensive comment, however this is unlikely to have much effect. Instead you could find a convincing counterargument Y, then add both P(Y) = 1 and P(X|Y) = 0 to the list of claims. Now other users would be notified of the resulting inconsistency and would reply by voting on one of these claims, changing their vote on X, or by creating additional arguments that contradict Y or support X. In turn you would attack these new arguments, eventually creating a large graph of reasoning. Perhaps at some point there would be enough general claims that people from different debates could reuse some, instead of duplicating them, and only create new conditional claims, making the graph dense.

I took some time to write a small prototype....
Beware arguments from possibility
by cousin_it — 2018-02-03T10:21:12.914Z

For any claim X, exactly one of these is true:

1) X is possible - compatible with the evidence so far.

2) X is impossible - there's a contradiction between X and the evidence so far.

That means asserting the possibility of something is harder than you think. For example, if Bob says: "Epiphenomenalism is possible, therefore <far reaching conclusions>"

Wait, Bob, did you just say it's impossible to find a contradiction between epiphenomenalism and anything else you know? That's an awfully strong claim! You got any evidence?

Bob pulls back: "I meant only that X sounds plausible to me." But that's a fact about Bob's limits of reasoning, it doesn't support the far-reaching conclusions anymore. For that you need to justify (1) over (2), not just assert it.
Canberra Rationality Exercises
by nBrown — 2018-02-03T02:55:27.183Z

I run the Canberra Rationality group. We're mostly uni students we meet once a week. Here are some of the exercises I've run that went down really well.

Circles

Have a question and take turns answering it by going around the circle. One prompt that worked well was: Say three things about yourself that make you different from the group. The little differences between us and the people we surround ourselves with can be very interesting. "I care less about grades", "I identify more with my system 1 values" or "I get less pleasure from socialising". What some people say can be very surprising. This might expose gaps in your model of them.

Some other questions include: What experiences have you had that you don't think others have? What things are you addicted to? [1] What experiences are you missing?

Mind reading

Free writing is an exercise where you write down whatever comes to mind, you try and capture as much of your stream of consciousness as you can. Even if it doesn't make sense. You try to write everything down. your aim is to fill a whole page with writing. It's okay for it to feel like gibberish. [2]

Free-writing is fine by itself. However if you want to take it a step further; For all the people who are comfortable sharing, pass the piece of paper to your right. Enjoy seeing other peoples thoughts, you might what to imagine what kind of person would generate these thoughts, it's sometimes strange how different other's thoughts can be.

Brainstorms

I use brainstorms a fair bit. We made a list of things that are great. It was good fun trying to discover pleasurable things. You can also enjoy the suggestions of everyone else. Once someone says "pyjamas" or "after rain smell" you get a little buzz from imagining it which is nice.

Other topics have included: tiny projects that are neat, actions I can totally execute.

Cracks in the model

Sometimes I lead with a weird personal discovery. For example where "I" am is located behind my eyes [3]. However if I'm in the blackness of a bath and I close my eyes it feels like "I" move further back into my head. This can sometimes open up the discussion and move us in the right direction to actually look at the world in all it's strageness.

Terminal Values

Pull up a facebook list of your closest friends. One by one n...
Update from Vancouver
by Evan_Gaensbauer — 2018-02-03T00:35:19.172Z

I've wanted to do a more detailed update from Vancouver, as the Post-Mortem on the Craft and the Community on here from a few months ago seems to have inspired Vancouver to get organized more than is true of other rationality communities, and is kickstarting an unprecedented reinvogiration of passion for local rationality community organization. The time it'd take me to do that has been better spent actually doing the organizing for Vancouver meetups. Below is quick update from Vancouver. I wrote it as a Facebook post for an audience of community members in Vancouver as part of a strategic review of local rationality organization in the last 6 weeks. The context is there were 6 Vancouver rationality meetups of a wide diversity of topics in January, which is a high a frequency of ratioanality meetups as there has ever been for Vancouver. The rationale for doing so I hope to post later; I thought I'd update the rest of you on what we've been doing.

I can frame what I'm trying to get us in Vancouver to do better having learned how meetup organization in Seattle works. In Seattle, there's complete integration of the rationality and effective altruism community organization. Each week they have a rationality reading group, which are more structured, and an effective altruism meetup, which is more variable week-to-week. However, what's made their meetups successful is the consistency. What they have in Seattle is regular meetups, of the same kind repeating, occurring on a weekly cycle. However, I think doing the same wouldn't work as well in Vancouver, as being eclectic as we are expect most of us desire more dynamism than is typical for rationalists.

So what I'm trying to do is getting meetup organization in Vancouver to work on a monthly cycle.

In January, we organized 6 meetups in total, 4 of which are a kind of meetup I want to see repeated in January:
- Constructive Contrarian Challenge Club (C4)
- spirituality
- global coordination and AI reading/discussion group
- rationality/discussion group (e.g., LW, SSC, etc.)

Other meetups for February which could be integrated into a system of meetups on a monthly cycle:
- biosecurity reading group
- Consciousness Appreciation Club
- bureaucracy day
- authentic relating
- Dungeons & Discourse
- rationalist game nights

These meetups are mostly still of the type "sitting around and...