trentmkelly/LessWrong-43k huggingface dataset:
A Rationalist Argument for Voting
by Jameson Quinn — 2018-06-07T17:05:42.668Z

The argument that voting is irrational is commonplace. From your point of view as a single voter, the chance that your one vote will sway the election are typically miniscule, and the direct effect of your vote is null if it doesn't sway the election. Thus, even if the expected value of your preferred candidate to you is hugely higher than that of the likely winner without your vote, when multiplied by the tiny chance your vote matters, the overall expected value isn't enough to justify the small time and effort of voting. This problem at the heart of democracy has been noted by many — most prominently, Condorcet, Hegel, and Downs.

There have been various counterarguments posed over the years:

Voters get some kind of intrinsic utility from the act of voting expressively.

Voters get some utility because changing the margin of the election affects how the resulting government behaves. (I personally find this argument highly implausible; the level of effects that would be necessary seem visibly lacking.)

If voters have a sufficiently altruistic utility function, the expected utility of a better government for all citizens could be sufficient to make voting worth it.

Voting itself is irrational, but having a policy of voting is rational.

This could be true if, for instance, paying attention to politics were intrinsically good for one's mental health, and yet akrasia would prevent paying sufficient attention without a policy of voting. I suspect most readers here will decisively reject that idea.

This could also be true if there were some kind of iterated or outrospective prisoners dilemma [Ed: actually, more like stag hunt] involved, in which voting was cooperation and not-voting was betrayal.

Of all of the above, I find the last bullet most interesting. But I am not going to pursue that here. Here, I'm going to propose a different rationale for voting; one that, as far as I know, is novel.

Participating in democratic elections is a group skill that requires practice. And it's worth practicing this skill because there is an appreciable chance that a future election will have a significant impact on existential risk, and thus will have a utility differential so high so as to make a lifetime of voting worth it.

Let's build a toy model with the following variables:

c: The cost of voting, in utilons.

i: "importance", the probability that your highest val

...
How to intro Effective Altruism
by ChaosHufflepuff — 2018-06-07T10:24:59.072Z

Hey,

So I am about to go be one of the camp leaders on a winter camp of a youth movement. I have been given the opportunity to plan a set of late evening activities for the older kids (targeted at the 13-16 year olds). The format of this will be 30 minute sessions over three nights where we split up the kids into groups of mixed ages (usually they're in school-grade groups throughout the day).

I'm planning on making it lightly educational (since we're running these for half an hour starting at 10pm), but want to make them all on the topic of effective altruism.

So does anyone have any suggestions about how I can introduce a group of teenagers to the idea of effective altruism in 1.5 hours spread across three evenings?
Glide #1.5: New Criticism and Rationality
by musicmage4114 — 2018-06-07T00:36:52.111Z

Glide Meditation #1.5: A Reflection on Reflecting on the Original Sequences

When I started to write (what I anticipated to be) my second Glide Meditation, I almost immediately hit a wall. Now that I had something concrete to work with, I couldn't simply smile, cheerily quip "Death of the Author!", then forge blithely ahead. Now the text was saying things explicitly, which meant that _I_ needed to say things explicitly! And frankly, the thought of doing so was (and largely still is) somewhat scary. If I don't prove that I "get it" before veering off the beaten path, will people even take me seriously? If the map is annotated in Tagalog, and I only speak English, is it even possible for me to stumble onto something interesting?

In my first-ever Glide Meditation, I (somewhat unintentionally) revealed that my primary analytical approach is heavily influenced by New Criticism, which originates from my collegiate study of literature and literary theory. However, I came to the realization that such a strategy is (or at least should be) almost wholly incompatible with something like the discussion of rationality. Such a discussion depends on one being clear in what one is saying, and on one's intention and the meaning of one's words being synonymous to the greatest degree possible. If this is true, then any differences in audience interpretation must indicate communicative deficiency on the part of the author, rather than actual openness to philosophical interpretation.

Now, my analytical style is by no means strictly New Criticism; I don't go out of my way to avoid learning about an author's intentions, nor do I seek it out, and I think it can be very useful to understand the general context in which a text was created. However, there's a socially-conscious part of me that still worries that analysis of a philosophical work in relative isolation might be seen as insincere or unworthy of consideration. On the other hand, the rebellious part of me feels that approaching such works without a pre-constructed framework might be key to making interesting or novel insights. Perhaps approaching philosophical texts using a primarily New Criticism-inspired methodology is only possible when the critic is, like me, largely ignorant of related and supporting texts.

Indeed, the New Critical approach eschews context, and what could be more contextual to a given ph...
Hufflepuff Leadership and Fighting Entropy
by Raemon — 2018-06-07T00:28:41.528Z

(In which my mild annoyance at a particular garbage bin outputs an essay that I think is pretty generalizable and important, but maybe is just a rant. *shrug*)

Previously:

Project Hufflepuff [note: sort of deprecated in favor of Archipelago]
Melting Gold

There's an important skill, early on in the Hufflepuff Skill Tree, which is something like "Collaborative Leadership."

The Hufflepuff strategy of "everyone pitching in to keep things nice" requires a mechanism to cause there to be a lot of people pitching in. If you're going to attempt to keep a place nice this way, you need such a mechanism. This requires a certain kind of leadership.

It doesn't need to feel like bossing people around – it can feel like "people making friends and helping each other out". But it does require a certain kind of assertiveness.

If you're pitching in and helping out just because you like to and okay with the notion that others might not do so, coolio. But if your goal is to keep a place nice, instead of making it nice for this particular afternoon, this skill is really important.

Background on Fighting Entropy

AFAICT, there are roughly two good strategies for making sure your environment can regularly fight entropy (i.e. keep a room/office/area clean, organized, both physically and interpersonally).

Specialist/Systemization Strategy – Have a dedicated person/people create a formal, automated system that keeps entropy at bay. (This includes things like hiring maids, designing the space such that people tend to automatically put things back in the right space without even thinking about it, and having recurring, automated deliveries that keep things well stocked)

Teamwork Pitch-In Strategy (aka the Hufflepuff Strategy) – People are expected to help out, clean up after themselves (erring on the side of cleaning up a bit more than they think is fair to account for bias) and proactively notice areas where they can improve things.

(The intersection between these two strategies looks something like "each person on the team focuses on specific roles. One person makes sure food is stocked, another person does dishes, etc")

When you can afford to do the Specialized/Automated strategy, I think you should. It's worth the investment. but there are a couple reasons you might not be able to:

The start-up costs are high. For new orgs or spa

...
Bug Report
by Evan_Gaensbauer — 2018-06-06T21:41:15.182Z

I've been experiencing a few bugs on LW2.0 in the last few months.

1. I can't get to LessWrong by typing in the url as "lesswrong.com". It only works if I do it as "www.lesswrong.com".

2. Maybe it's just I'm using a DuckDuckGo extension in Google Chrome, but when I type in the search bar "lesswrong.com" and the search bar suggests I hit 'Tab' to search "lesswrong.com", and I try to do so through the search bar, it never works. Ditto when try this with "www.lesswrong.com".

3. Whenever I want to submit a post, the time for how long the page takes to load varies wildly and randomly. Sometimes it's only a few seconds, but occasionally, including for submitting this post, the text box for submitting takes several minutes to load.
Monty Hall in the Wild
by Jacob Falkovich — 2018-06-06T18:03:22.593Z

Cross-posted from Putanumonit.com.

I visited a friend yesterday, and after we finished our first bottle of Merlot I brought up the Monty Hall problem, as one does. My friend confessed that she has never heard of the problem. Here’s how it goes:

You are faced with a choice of three doors. Behind one of them is a shiny pile of utilons, while the other two are hiding avocados that are just starting to get rotten and maybe you can convince yourself that they’re still OK to eat but you’ll immediately regret it if you try. The original formulation talks about a car and two goats, which always confused me because goats are better for racing than cars.

Anyway, you point to Door A, at which point the host of the show (yeah, it’s a show, you’re on TV) opens one of the other doors to reveal an almost-rotten avocado. The host knows what hides behind each door, always offers the switch, and never opens the one with the prize because that would make for boring TV.

You now have the option of sticking with Door A or switching to the third door that wasn’t opened. Should you switch?

After my friend figured out the answer she commented that:

The correct answer is obvious.
The problem is pretty boring.
Monty Hall isn’t relevant to anything you would ever come across in real life.

Wrong, wrong, and wrong.

According to Wikipedia, only 13% of people figure out that switching doors improves your chance of winning from 1/3 to 2/3. But even more interesting is the fact that an incredible number of people remain unconvinced even after seeing several proofs, simulations, and demonstrations. The wrong answer, that the doors have an equal 1/2 chance of avocado, is so intuitively appealing that even educated people cannot overcome that intuition with mathematical reason.

I’ve written a lot recently about decoupling. At the heart of decoupling is the ability to override an intuitive System 1 answer with a System 2 answer that’s arrived at by applying logic and rules. This ability to override is what’s measured by the Cognitive Reflection Test, the standard test of rationality. Remarkably, many people fail to improve on the CRT even after taking it multiple times.

When rationalists talk about “me” and “my brain”, the former refers to their System 2 and the latter to their System 1 and their unconscious. “I wanted to get some work done, but my braindecided it needs to scroll through Twitter for an hour...
Simplified Poker Strategy
by Zvi — 2018-06-06T11:10:00.636Z

Previously in Sequence (Required): Simplified Poker

I spent a few hours figuring out my strategy. This is what I submitted.

If you start with a 2, you never want to bet, since your opponent will call with a 3 but fold with a 1. So we can assume no one who bets ever has a 2. But you might want to call a bet.

If you start with a 1, you never call a bet, but sometimes want to bet as a bluff.

If you start with a 3 in first position, sometimes you may want to check to allow your opponent to bet with a 1. If you have a 3 in second position, you have no decisions.

Thus, a non-dominated strategy can be represented by five probabilities: The chance you bet with a 1 in first position, chance you bet with a 3 in first position, chance you bet with a 1 in second position, chance you call with a 2 in first position, and chance you call with a 2 in second position. Call a set of these five numbers a strategy.

There were likely to be a few players bad enough to bet with a 2 or perhaps make the other mistakes, but I chose for complexity reasons not to worry about that, assuming I’d still do something close to optimal. If I was confident complexity was free, I’d have included a check to see if we ever caught the opponent doing something crazy, and adjust accordingly.

If you know the opposing strategy, what to do is obvious. Thus, I defined a function called ‘best response’ that takes a strategy, and outputs the strategy that maximizes against that strategy.

My goal was to derive the opponents’ strategy, then play the best response to that strategy.

As a safeguard against opponents who were anticipating such a strategy, I included an escape hatch: If at any point, my opponent got ahead by 10 or more chips, assume they were a level ahead of me, and playing the best response to what I would otherwise do. So derive what that is, and play the best response to that!

That skipped over the key puzzle, which is figuring out what the opponent is doing. On the first turn, I guessed opponents would pursue reasonable mixed strategies: bet a 1 about a third of the time, bet a 3 in first position about two thirds of the time, call with a 2 about half the time. I represented this with a virtual hand history that I included until I had enough real ones.

On subsequent turns, I looked at the hand history.

If the opponents’ card was revealed, that was a pure data point &...
Resource-Limited Reflective Oracles
by Diffractor — 2018-06-06T02:50:42.000Z

Reflective oracles accurately answer questions about what arbitrary halting probabilistic oracle machines output. It is possible to make a variant of a reflective oracle that accurately answers questions about what sufficiently short-running Turing machines with access to the same oracle output.

These oracles are explicitly computable, fairly powerful in a computational complexity sense, and can probably be used to make a reflective version of AIXItl, (which will not happen in this particular post)

The theorems in this post are not exhaustive at all, just the stuff I was able to figure out in ~2 hours, so there is almost certainly low-hanging fruit in figuring out how these interact with the standard arsenal of theorems in computational complexity theory.

#Motivation:

Let's say we want to make an oracle that is reflective over all TM's that run in polynomial time. How do we do that?

The obvious approach is to let a query consist of a tuple of a turing machine T, a number n, a bitstring x, and a probability p. If you build the tree of all possible execution paths of the turing machine that are poly(n) steps long(for some polynomial), and come up with the probability of every possible output (including a probability for timing out) the oracle must return 0 if the true probability of x is less than p, 1 if the true probability of x is greater than p, and randomize otherwise.

This obvious approach doesn't work, because it's possible to find out whether any halting turing machine, no matter how long-running, returns x, by recursive oracle calls. Let A be some inputless turing machine that runs for a very long number of turns, like the BB(6) machine. Let Em(A) be the turing machine that takes a number n as input, simulates A for n turns, and if A halts in n turns, outputs what A would. If A doesn't halt in n turns, this algorithm queries the oracle about whether Em(A)(2n) outputs x, and if so, outputs x.

It is possible to emulate n steps of a turing machine in O(nlogn) time, and it is possible to write down 2n in O(n) time, and it is possible to write down x in constant time, so Em(A) runs in poly(n) time. Therefore, the oracle should be accurate about what this turing machine outputs for any given n. However, if you look at what this turing machine is doing, it is giving itself an exponentially longer time bound on each oracle call, so it will eventually be able to fu

...
Disambiguating "alignment" and related notions

by David Scott Krueger (formerly: capybaralet) — 2018-06-05T15:35:15.091Z

EDIT: renamed CAIS --> TAIS to avoid acronym class with Drexler's Comprehensive AI Services.

I recently had an ongoing and undetected inferential gap with someone over our use of the term "value aligned".

Holistic alignment vs. parochial alignment

I was talking about "holistic alignment":
Agent R is holistically aligned with agent H iff R and H have the same terminal values.
This is the "traditional AI safety (TAIS)" (as exemplified by Superintelligence) notion of alignment, and the TAIS view is roughly: "a superintelligent AI (ASI) that is not holistically aligned is an Xrisk"; this view is supported by the instrumental convergence thesis.

My friend was talking about "parochial alignment". I'm lacking a satisfyingly crisp definition of parochial alignment, but intuitively, it refers to how you'd want a "genie" to behave:
R is parochially aligned with agent H and task T iff R's terminal values are to accomplish T in accordance to H's preferences over the intended task domain.
We both agreed that a parochially aligned ASI is not safe by default (it might paperclip), but that it might be possible to make one safe using various capability control mechanisms (for instance, anything that effectively restrict it to operating within the task domain).

Sufficient alignment

We might further consider a notion of "sufficient alignment":
R is sufficiently aligned with H iff optimizing R's terminal values would not induce a nontrivial Xrisk (according to H's definition of Xrisk).
For example, an AI whose terminal values are "maintain meaningful human control over the future" is plausibly sufficiently aligned. A sufficiently aligned ASI is safe in the absence of capability control. It's worth considering what might constitute sufficient alignment, short of holistic alignment. For instance, Paul seems to argue that corrigible agents are sufficiently aligned. As another example, we don't know how bad of a perverse instantiation to expect from an ASI whose values are almost correct (e.g. within epsilon in L-infinity norm over possible futures).

Intentional alignment and "benignment"

Paul Christiano's version of alignment, I'm calling "intentional alignment&...
Prisoners' Dilemma with Costs to Modeling
by Scott Garrabrant — 2018-06-05T04:51:30.700Z

We consider a modification to the open source prisoners' dilemma in which agents must pay some resources to model each other. We will use the modal combat framework, but where agents pay a cost proportional to the depth of boxes in their code. Even a small modeling penalty makes the FairBot-FairBot outcome no longer an equilibrium, since the best response to FairBot is to be CooperateBot and not pay the modeling penalty. The best response to CooperateBot is to be DefectBot, and the pure DefectBot-DefectBot outcome is a stable Nash equilibrium. In fact, I believe that DefectBot-DefectBot is the unique pure strategy Nash equilibrium.

Amazingly, this turns out to be okay! For small modeling penalties, there is a mixed strategy equilibrium which mixes between CooperateBot, FairBot, and PrudentBot! Both players get exactly the same utility in expectation as the FairBot-FairBot outcome.

Further, if you consider an evolutionary system where populations reproduce in proportion to how well they do in prisoners' dilemmas with each other, it appears that as the modeling penalty gets small, the basin of the defect equilibrium also gets small, and nearly all initial conditions cycle around CooperateBot, FairBot, and PrudentBot!

This post came out of conversations with Sam Eisenstat, Abram Demski, Tsvi Benson-Tilsen, and Andrew Critch. It is a first draft that could use a coauthor to carefully check everything, expand on it, and turn it into a paper. If you think you could do that with minimal guidance from me, let me know.

Formalism

We will be using the modal combat framework, and identifying ⊤ with cooperation and ⊥ with defection. Agents are defined to formulas that combine the other agent X run on various agents using propositional calculus and a modal operator □. The □ represents provability, and every instance of X run on an agent in the formula must be contained within a □. Recall some common modal agents:

CooperateBot is defined by CB(X)↔⊤.

DefectBot is defined by DB(X)↔⊥.

FairBot is defined by FB(X)↔□(X(FB)).

PrudentBot is defined by PB(X)↔□(X(PB)∧(X(DB)→□⊥)).

These 4 agents interact with each other as follows: CooperateBot cooperates with everyone. DefectBot defects against everyone. FairBot defects against only DefectBot. PrudentBot defects against CooperateBot and DefectBot and cooperates with itself and FairBot.

We will say that the depth of an agent is the m...
A line of defense against unfriendly outcomes: Grover's Algorithm
by Gurkenglas — 2018-06-05T00:59:46.993Z

Epistemic status: armchair computer science

Note: I've been told this is hard to understand. I've rewritten it.

Searching for tools for my toolbox, I investigated the tool of quantum computing. To more easily decide its applicability, I looked for subtools that do not take complex numbers to describe to a computer scientist. Two such are its approach to the hidden subgroup problem, a generalization of the hack that invalidates some cryptography, and amplitude amplification, a generalization of Grover's Algorithm, which provides a quadratic speedup on the boolean satisfiability problem.

Reading [Deutsch 1985, Section 8] inspired me to consider the interaction between quantum computers and algorithms of moral weight.

The next few paragraphs attempt to carve toolspace down to a neighborhood of quantum computing for your moral intuition.

A classical deterministic computer with n bits can be in one of 2^n states. A representation that will prove convenient is that its state is represented by a function that maps each state to 0 or 1, such that exactly one state maps to 1. Such a function can be written as a vector with 2^n entries of 0 or 1. Vectors with one entry 1 and the rest 0 are called unit vectors.

Each program without side effects is then a function between such unit vectors.

This state representation sure looks like a probability distribution. Indeed, the state of a classical probabilistic computer can be seen as a probability distribution over which of the 2^n states one is in, or a vector with 2^n entries between 0 and 1, such that they sum to one.

Each program without side effects is then still completely determined by its effects on unit vectors. Its effects on non-unit vectors will be a weighted sum of its effects on the relevant unit vectors.

The state of a quantum computer with n qubits can be represented as a vector with 2^n entries, each of which is a complex number, such that their squared magnitudes sum to one. In this post, only the subset of vectors with real-numbered entries will be relevant.

Each program without side effects is then still completely determined by its effects on unit vectors. Its effects on non-unit vectors will again be a weighted sum of its effects on the relevant unit vectors.

We can choose the initial vector freely.

Given a vector, we can do a measurement on it. To do this, we specify a set of entry positions. With p...