trentmkelly/LessWrong-43k huggingface dataset:
Late Great Filter Is Not Bad News
by Wei Dai — 2010-04-04T04:17:39.243Z

But I hope that our Mars probes will discover nothing. It would be good news if we find Mars to be completely sterile. Dead rocks and lifeless sands would lift my spirit.

Conversely, if we discovered traces of some simple extinct life form—some bacteria, some algae—it would be bad news. If we found fossils of something more advanced, perhaps something looking like the remnants of a trilobite or even the skeleton of a small mammal, it would be very bad news. The more complex the life we found, the more depressing the news of its existence would be. Scientifically interesting, certainly, but a bad omen for the future of the human race.

— Nick Bostrom, in Where Are They? Why I hope that the search for extraterrestrial life finds nothing

This post is a reply to Robin Hanson's recent OB post Very Bad News, as well as Nick Bostrom's 2008 paper quoted above, and assumes familiarity with Robin's Great Filter idea. (Robin's server for the Great Filter paper seems to be experiencing some kind of error. See here for a mirror.)

Suppose Omega appears and says to you:

(Scenario 1) I'm going to apply a great filter to humanity. You get to choose whether the filter is applied one minute from now, or in five years. When the designated time arrives, I'll throw a fair coin, and wipe out humanity if it lands heads. And oh, it's not the current you that gets to decide, but the version of you 4 years and 364 days from now. I'll predict his or her decision and act accordingly.

I hope it's not controversial that the current you should prefer a late filter, since (with probability .5) that gives you and everyone else five more years of life. What about the future version of you? Well, if he or she decides on the early filter, that would constitutes a time inconsistency. And for those who believe in multiverse/many-worlds theories, choosing the early filter shortens the lives of everyone in half of all universes/branches where a copy of you is making this decision, which doesn't seem like a good thing. It seems clear that, ignoring human deviations from ideal rationality, the right decision of the future you is to choose the late filter.

Now let's change this thought experiment a little. Omega appears and instead says:

(Scenario 2) Here's a button. A million years ago I hid a doomsday device in the solar system and predicted whether you would press this button or not....
Free copy of Feynman's autobiography for best corny rationalist joke
by GreenRoot — 2010-04-04T00:32:45.546Z

I have an extra copy of Richard Feyman's autobiography, "Surely You're Joking, Mr. Feynman!": Aventures of a Curious Character, which I want to give away here.

This is one of two autobiographies (along with Ben Franklin's) to actually change my life. I've seen it quoted often on LessWrong, as Feynman has a point of view on life that fits well with the ideas we explore here. In addition to his rationalist side, Feynman also exhibited a wonderfully free sense of humor. Even when working at the Manhattan Project, he joked around and never took himself too seriously. I think our community would benefit if the rationalism here were likewise leavened by some self-deprecating humor.

I will mail the autobiography, at my expense, to whomever posts the best corny rationalist joke in the comments below, as judged by karma voting. Anything goes. Here's a little inspirational prompting:

How many rationalists does it take to screw in a lightbulb? ...

Two rationalists walk into a bar. ...

You might be a rationalist if ...

Edit (April 12th): The winner of the corny rationalist joke contest is this one-liner by SilasBarta, which collected 17 net up-votes:

Rationalist pick-up line: "I would never cheat on you if and only if you would never cheat on me if and only if I would never cheat on you."

The runner-up (and my personal favorite) is this exchange by Bo102010, which collected 14 net up-votes. The full comment thread for this one has an explanation and suggested refinements.

A rationalist walks into a bar with two bartenders. The rationalist asks "What's the best drink to get tonight?"

The first bartender says "The martini."
The second bartender says "The gin and tonic."
The first bartender repeats "The martini."
The second bartender repeats "The gin and tonic."
The first says again "The martini."
The second says again "The gin and tonic."
Then the first says "The gin and tonic."
The rationalist smiles and says, "I'm glad you could come to an agreement."

Thanks to everybody who contributed and voted on corny jokes.
Bayesian Collaborative Filtering
by JGWeissman — 2010-04-03T23:29:30.507Z

I present an algorithm I designed to predict which position a person would report for an issue on TakeOnIt, through Bayesian updates on the evidence of other people's positions on that issue. Additionally, I will point out some potential areas of improvement, in the hopes of inspiring others here to expand on this method.

For those not familiar with TakeOnIt, the basic idea is that there are issues, represented by yes/no questions, on which people can take the positions Agree (A), Mostly Agree (MA), Neutral (N), Mostly Disagree (MD), or Disagree (D). (There are two types of people tracked by TakeOnIt: users who register their own opinions, and Experts/Influencers whose opinions are derived from public quotations.)

The goal is to predict what issue a person S would take on a position, based on the positions registered by other people on that question. To do this, we will use Bayes' Theorem to update the probability that person S takes the position X on issue I, given that person T has taken position Y on issue I:

Really, we will be updating on several people Tj taking positions Ty on I:

To compute this, let us first figure out the prior probability P(S takes X on I). I use for this a generalization of Laplace's Law of Succession (representing my theory that a person will take each position with a particular frequency, and that there is no reason, before seeing their actual position, to suppose that one position in particular is more frequent than the others), that the odds that S takes the position A : MA : N : MD : D on I is given by:

1 + count of issues S has taken position A on : 1 + count of issues S has taken position MA on : 1 + count of issues S has taken position N on : 1 + count of issues S has taken position MD on : 1 + count of issues S has taken position D on

Thus, the probability

Likewise the probability

This leaves one term in Bayes' Theorem to figure out: P(Tj takes Yj on I | S takes X on I)

For this, I will again use the Generalized Laplace's Law of Succession, looking at issues on which both S and Tj have taken positions:

We now know how to compute, from the records of people's positions on issues, all the terms that Bayes' Theorem requires to compute the posterior probability that person S will take position X on issue I.

So, how well does this work? At this time, I have coded up a SQL script to be run against TakeOnIt's database, that predict...
Less Wrong London meetup, tomorrow (Sunday 2010-04-04) 16:00
by Paul Crowley — 2010-04-03T09:36:05.289Z

UPDATE: Backup plan is to meet at the Starbucks across the road (16 Piccadilly, London W1J 0DE, 020 7287 8311). I've been trying to ring the Waterstones and the coffee shop for a while now and waited several minute for an answer with no success, so I think it's very likely that it is closed. I've called the Starbucks and it's open. If I know you on here, mail me (paul at ciphergoth dot org) and I'll give you my mobile number.

In the grand tradition of giving almost no notice for London meetups, I bring to your attention that a meetup is planned for tomorrow (Sunday 2010-04-04), at 16:00, in the 5th View cafe on top of Waterstone's bookstore. Nearest Tube Piccadilly Circus. Yvain, taw, RichardKennaway, and myself at least hope to be there, doubtless others too!

We should try to give more notice for the next one. This is the first Sunday in April; how about the first Sunday in June for the next one, 2010-06-06? I'd prefer an earlier time and it might be worth experimenting with a different venue, but if we can fix a date we can vary other details closer to the time.
Announcing the Less Wrong Sub-Reddit
by alyssavance — 2010-04-02T01:17:44.603Z

Announcing: the Less Wrong Sub-Reddit, at http://reddit.com/r/LessWrong. This Reddit is intended as a partial replacement for/complement to the Open Thread, which has gotten somewhat unwieldy and overcrowded as of late. I (Thomas McCabe) will be posting things that appear on the April Open Thread to this Reddit, to aid in starting conversation. We'll see how it goes.

This Reddit is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets very long/involved, celebrate by turning it into a top-level post.

To anyone who is worried about the discussion quality devolving to Reddit level: I retain moderator power over the sub-Reddit, and can delete things and ban people from it. If this gets to be too much work for me, I will be happy to give mod power to other interested Less Wrong readers with a track record of good posts and comments.

This is purely my creation, and not that of Eliezer or the Less Wrong admins. If anything goes horribly wrong, don't blame them.

This is completely not an April Fool's joke. I want to start it now (on the first day of the month) because the Open Thread "only" has 52 comments on it.

If you don't have a Reddit account, or want to create a new account to post under your Less Wrong username, you can click "Register" in the upper-right-hand corner. It only takes fifteen seconds.

For those who don't look at the bottom of the website very often, Less Wrong is originally powered by the Reddit codebase.

Good luck, everyone, and may the best discussions win.

Edited for clarity: I'm proposing that we set up a new discussion community such that Less Wrongers have a place to talk about off-topic stuff other than Open Thread (which is hugely overcrowded). If either LW or the subreddit crashes, it should have no effect on the other.
Rationality quotes: April 2010
by wnoise — 2010-04-01T20:41:39.003Z

This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote comments/posts on LW/OB.

No more than 5 quotes per person per monthly thread, please.
What is Rationality?
by Roko — 2010-04-01T20:14:09.309Z

This article is an attempt to summarize basic material, and thus probably won't have anything new for the experienced crowd.

Related: 11 Core Rationalist Skills, What is Bayesianism?

Less Wrong is a blog devoted to refining the art of human rationality, but what is rationality? Rationality is unlike any subject I studied at school or university, and it is probably the case that the synthesis of subjects and ideas here on Less Wrong is fairly unique.

Fundamentally, rationality is the study of general methods for good decision-making, especially where the decision is hard to get right. When an individual is considering whether to get a cryonics policy, or when a country is trying to work out what to do about global warming, one is within the realm of decision-making that we can use rationality to improve. People do badly on hard decision problems for a variety of reasons, including: that they are not born with the ability to deal with the scientific knowledge and complex systems that our modern world runs on, that they haven't been warned that they should think critically about their own reasoning, that they belong to groups that collectively hold faulty beliefs, and that their emotions and biases skew their reasoning process.

Rationality is the ability to do well on hard decision problems.

Another central theme of rationality is truth-seeking. Truth-seeking is often used as an aid to decision-making: if you're trying to decide whether to get a cryonics policy, you might want to find out whether the technology has any good evidence suggesting that it might work. We can make good decisions by getting an accurate estimate of the relevant facts and parameters, and then choosing the best option according to our understanding of things; if our understanding is more accurate, this will tend to work better.

Rationality is also the art of how to systematically come to know what is true.

Often, the processes of truth-seeking and decision-making, both on the individual level and the group level are subject to biases: systematic failures to get to the truth or to make good decisions. Biases in individual humans are an extremely serious problem - most people make important life-decisions without even realizing the extent and severity of the cognitive biases they were born with. Therefore rational thought requires a good deal of critical thinking - analyzing and r...
The human problem
by PhilGoetz — 2010-04-01T19:41:44.735Z

You've fiddled with your physics constants until you got them just right, pushed matter into just the right initial configuration, given all the galaxies a good spin, and tended them carefully for a few billion years. Finally, one of the creatures on one of those planets in one of those galaxies looks up and notices the stars. Congratulations! You've evolved "humans", the term used for those early life forms that have mustered up enough just brain cells to wonder about you.

Widely regarded as the starting point of interest in a universe, they're too often its ending point as well. Every amateur god has lost at least one universe to humans. They occupy that vanishingly-narrow yet dangerous window of intelligence that your universe must safely navigate, in which your organisms are just smart enough to seize the helm of evolution, but not smart enough to understand what they're really doing.

The trouble begins when one of these humans decides, usually in a glow of species pride shortly after the invention of the wheel or the digital watch or some such knicknack, that they are in fact pretty neat, and that it's vitally important to ensure that all future intelligent life shares their values.

At that point, they invent a constrained optimization process, that ensures that all new complex agents (drives, minds, families, societies, etc.) and all improvements to existing agents, have a good score according to some agreed-on function.

If you're lucky, your humans will design a phenomenological function, which evaluates the qualia in the proposed new mind. This will be an inconvenience to your universe, as it will slow down the exploration of agent-design space; but it's not so bad that you have to crumple your universe up and throw it out. It doesn't necessarily cut off all the best places in agent space from ever being explored.

But remember these are humans we're talking about. They've only recently evolved the ability to experience qualia, let alone understand and evaluate them. So they usually design computationalist functions instead. All functions perform computation; by "computationalist" we mean that they evaluate an agent by the output of its computations, rather than by what it feels like to be such an agent.

Before either kind of function can be evaluated, the agent design is abstracted into a descript...
Open Thread: April 2010
by Unnamed — 2010-04-01T15:21:03.777Z

An Open Thread: a place for things foolishly April, and other assorted discussions.

This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.

Update: Tom McCabe has created a sub-Reddit to use for assorted discussions instead of relying on open threads. Go there for the sub-Reddit and discussion about it, and go here to vote on the idea.
The role of neodeconstructive rationalism in the works of Less Wrong
by thomblake — 2010-04-01T14:17:36.357Z

Summary: Yudkowsky's fiction emphasizes neodeconstructive rationalism, which serves as a bridge between class and sexual identity. Materialist libertarianism (in the metaphysical sense) implies quantum nonrealism, but examining the works of Vinge, Gibson, and especially Egan in this light generates the discourse of semitoic consciousness.

1. Precapitalist textual theory and neodeconstructive rationalism

In the works of Yudkowsky, a predominant concept is the concept of cultural reality. It is not enough to believe in belief; one must make beliefs pay rent It could be said that Salamon’s model of neodeconstructive rationalism implies that class has significance, given that the premise of materialist libertarianism is invalid. Given that absence of evidence is evidence of absence, an abundance of discourses concerning Pearlean absurdity may be revealed.

The main theme of the works of Alicorn is not depatriarchialism, but postdepatriarchialism. Thus, Yvain suggests the use of neoconstructive narrative to modify culture. After all, guessing the teacher's password is merely a route to more semantic stopsigns. The defining characteristic, and subsequent dialectic, of materialist libertarianism intrinsic to Yudkowsky’s Three Worlds Collide is also evident in The Sword of Good, although in a more mythopoetical sense.

It could be said that the primary theme of Jaynes's analysis of neodeconstructive rationalism is the bridge between class and sexual identity. pjeby promotes the use of the cultural paradigm of consensus to deconstruct class divisions.

Thus, if neodeconstructive rationalism holds, we have to choose between postdialectic conceptualist theory and subcapitalist theory. But would that take place on a level greater than merely disputing definitions? Several appropriations concerning the stasis of dialectic art exist.

But the characteristic theme of the works of Bayes is a postpatriarchial reality. Hanson’s critique of materialist libertarianism holds that the establishment is meaningless. But is it really just an empty label?

2. Expressions of futility

“Sexual identity is part of the stasis of language,” says Vinge. Thus, Dennett states that we have to choose between Sartreist absurdity and capitalist libertarianism; taw's critique brings this into sharp focus. If neodeconstructive rationalism holds, the works of Yudkowsk...
Loleliezers
by dclayh — 2010-04-01T04:04:27.480Z

Previously: Eliezer Yudkowsky facts, and Kevin's prediction.

A bit of silliness for the day. Below the fold to spare those with delicate sensibilities.

Please contribute your own in the comments. (Lolrobinhansons, etc., would also be welcome.) Unfortunately I have no special source of Eliezer photos to offer beyond Google Images.
City of Lights
by Alicorn — 2010-03-31T23:30:03.011Z

Sequence index: Living Luminously
Previously in sequence: Highlights and Shadows
Next in Sequence: Lampshading
Pretending to be multiple agents is a useful way to represent your psychology and uncover hidden complexities.

You may find your understanding of this post significantly improved if you read the sixth story from Seven Shiny Stories.
When grappling with the complex web of traits and patterns that is you, you are reasonably likely to find yourself less than completely uniform. You might have several competing perspectives, possess the ability to code-switch between different styles of thought, or even believe outright contradictions. It's bound to make it harder to think about yourself when you find this kind of convolution.
Unfortunately, we don't have the vocabulary or even the mental architecture to easily think of or describe ourselves (nor other people) as containing such multitudes. The closest we come in typical conversation more resembles descriptions of superficial, vague ambivalence ("I'm sorta happy about it, but kind of sad at the same time! Weird!") than the sort of deep-level muddle and conflict that can occupy a brain. The models of the human psyche that have come closest to approximating this mess are what I call "multi-agent models". (Note: I have no idea how what I am about to describe interacts with actual psychiatric conditions involving multiple personalities, voices in one's head, or other potentially similar-sounding phenomena. I describe multi-agent models as employed by psychiatrically singular persons.)
Multi-agent models have been around for a long time: in Plato's Republic, he talks about appetite (itself imperfectly self-consistent), spirit, and reason, forming a tripartite soul. He discusses their functions as though each has its own agency and could perceive, desire, plan, and act given the chance (plus the possibility of one forcing down the other two to rule the soul unopposed). Not too far off in structure is the Freudian id/superego/ego model. The notion of the multi-agent self even appears in fiction (warning: TV Tropes). It appears to be a surprisingly prevalent and natural method for conceptualizing the complicated mind of the average human being. Of course, talking about it as something to do rather than as a way to push your psychological theorie...
Disambiguating Doom
by steven0461 — 2010-03-29T18:14:12.075Z

Analysts of humanity's future sometimes use the word "doom" rather loosely. ("Doomsday" has the further problem that it privileges a particular time scale.) But doom sounds like something important; and when something is important, it's important to be clear about what it is.

Some properties that could all qualify an event as doom:

Gigadeath: Billions of people, or some number roughly comparable to the number of people alive, die.

Human extinction: No humans survive afterward. (Or, modified: no human-like life survives, or no sentient life survives, or no intelligent life survives.)

Existential disaster: Some significant fraction, perhaps all, of the future's potential moral value is lost. (Coined by Nick Bostrom, who defines an existential risk as one "where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential", which I interpret to mean the same thing.)

"Doomsday argument doomsday": The total number of observers (or observer-moments) in existence ends up being small – not much larger than the total that have existed in the past. This is what we should believe if we accept the Doomsday argument.

Great filter: Earth ends up not colonizing the stars, or doing anything else widely visible. If all species are filtered out, this explains the Fermi paradox.

Examples to illustrate that these properties are fundamentally different:

If billions die (1), humanity may still recover and not go extinct (2), retain most of its potential future value (3), spawn many future observers (4), and colonize the stars (5). (E.g., nuclear war, but also aging.)

If cockroaches or Klingon colonists build something even cooler afterward, human extinction (2) isn't an existential disaster (3), and conversely, the creation of an eternal dystopia could be an existential disaster (3) without involving human extinction (2).

Human extinction (2) doesn't imply few future observers (4) if it happens too late, or if we're not alone; and few future observers (4) doesn't imply human extinction (2) if we all live forever childlessly. (It's harder to find an example of few observer-moments without human extinction, short of p-zombie infestations.)

If we create an AI that converts the galaxy to paperclips, humans go extinct (2) and it's an existential disaster (3), but it isn't part of

...