trentmkelly/LessWrong-43k huggingface dataset:
Eternal Sunshine of the Rational Mind
by Cyan — 2009-04-07T15:10:27.725Z

What if you could choose which memories and associations to retain and which to discard? Using that capability rationally (whatever that word means to you) would be a significant challenge -- and that challenge has just come one step closer to being a reality.

Dr. Fenton had already devised a clever way to teach animals strong memories for where things are located. He teaches them to move around a small chamber to avoid a mild electric shock to their feet. Once the animals learn, they do not forget. Placed back in the chamber a day later, even a month later, they quickly remember how to avoid the shock and do so.

But when injected — directly into their brain — with a drug called ZIP that interferes with PKMzeta, they are back to square one, almost immediately. “When we first saw this happen, I had grad students throwing their hands up in the air, yelling,” Dr. Fenton said. “Well, we needed a lot more than that” one study.

They now have it. Dr. Fenton’s lab repeated the experiment, in various ways...
What isn't the wiki for?
by Paul Crowley — 2009-04-07T10:36:49.348Z

The new wiki is off to a flying start - it's less than 15 hours old, and already it has over 20 articles and five authors. It's probably about time we worked out what it was for.

I created it because as things stand, I can't point my friends to Less Wrong and say "come and learn about rationality and take part in these fascinating and potentially important discussions!" The discussions we have here assume years of reading Overcoming Bias and close attention to what's been said there and here; it must be practically impenetrable to newcomers. So for me the primary goal is simply to provide a glossary, to give newcomers a fighting chance of understanding what on Earth we are talking about and why. I think it can do more than that, but before I come to that, let me say a little about what I think it's not for.

The way I would currently like to see it, the wiki is not there to duplicate what is already done elsewhere. So it's not a place for discussion - that's what this site is for, and the features to support discussion here are far stronger than they are there, what with voting, threading and so forth. By the same token, it's not a place to advance your ideas - it's better to do that here, where people can comment on them and where it's clearly tagged as the work of one author rather than some sort of collective conclusion.

I'd like to avoid duplication in other areas, too. Anything that can go in Wikipedia instead of our wiki should do: we will get better results if we and they are editing the same biography of Eliezer Yudkowsky, rather than creating a fork. To that end, I've created a {{wikilink}} template that can go at the top of an article, linking to the article with the same name in Wikipedia. Have a look at our current article on Newcomb's paradox - there is far more detail in the linked Wikipedia article, but there are some things we carry because they (rightly) won't: the sometimes non-standard vocabulary we tend to use around it (eg "Omega") and links to related articles in Overcoming Bias/Less Wrong on the subject, which they might not choose to keep (since Wikipedia is not a link farm).

Similarly, we don't want to provide our own index of heuristics and biases, since there's one on Wikipedia and another on the Psychology wiki, and most of what they lack on the subject we can fix there rather than trying to address by duplication.

It's often easier to ...
Newcomb's Problem vs. One-Shot Prisoner's Dilemma
by Wei Dai — 2009-04-07T05:32:37.012Z

Continuation of: http://lesswrong.com/lw/7/kinnairds_truels/i7#comments

Eliezer has convinced me to one-box Newcomb's problem, but I'm not ready to Cooperate in one-shot PD yet. In http://www.overcomingbias.com/2008/09/iterated-tpd.html?cid=129270958#comment-129270958, Eliezer wrote:

PDF, on the 100th [i.e. final] move of the iterated dilemma, I cooperate if and only if I expect the paperclipper to cooperate if and only if I cooperate, that is:

Eliezer.C <=> (Paperclipper.C <=> Eliezer.C)

The problem is, the paperclipper would like to deceive Eliezer into believing that Paperclipper.C <=> Eliezer.C, while actually playing D. This means Eliezer has to expend resources to verify that Paperclipper.C <=> Eliezer.C really is true with high probability. If the potential gain from cooperation in a one-shot PD is less than this cost, then cooperation isn't possible. In Newbomb’s Problem, the analogous issue can be assumed away, by stipulating that Omega will see through any deception. But in the standard game theory analysis of one-shot PD, the opposite assumption is made, namely that it's impossible or prohibitively costly for players to convince each other that Player1.C <=> Player2.C.

It seems likely that this assumption is false, at least for some types of agents and sufficiently high gains from cooperation. In http://www.nabble.com/-sl4--prove-your-source-code-td18454831.html, I asked how superintelligences can prove their source code to each other, and Tim Freeman responded with this suggestion:

Entity A could prove to entity B that it has source code S by consenting to be replaced by a new entity A' that was constructed by a manufacturing process jointly monitored by A and B. During this process, both A and B observe that A' is constructed to run source code S. After A' is constructed, A shuts down and gives all of its resources to A'.

But this process seems quite expensive, so even SIs may not be able to play Cooperate in one-shot PD, unless the stakes are pretty high. Are there cheaper solutions, perhaps ones that can be applied to humans as well, for players i

...
On Comments, Voting, and Karma - Part I
by thomblake — 2009-04-07T02:44:26.333Z

There has been a great deal of discussion here about the proper methods of voting on comments and on how karma should be assigned. I believe it's finally reached the point where a post is warranted that covers some of the issues involved. (This may be just because I find myself frequently in disagreement with others about it.)

The Automatic Upvote

First, there is the question of whether one should be able to upvote one's own comment. This actually breaks apart into two related concerns:

(1) One is able to upvote one's own comments, and

(2) One gains a point of karma just for posting a comment.

These need not be tied. We could have (2) without (1) by awarding a point of karma for commenting, without changing the comment's score. We could also have (1) without (2) by simply not counting self-upvotes for karma.

I am in favor of (2). The main argument against (2) is that it rewards quantity over quality. The main argument for (2) is that it offers an automatic incentive to post comments; that is, it rewards commenting over silence. As we're community-building, I think the latter incentive is more important than the former. But I'm not sure this is worth arguing further - it serves as a distraction from the benefits of (1).

I am also in favor of (1). As a default, all comments have a base rating of 0. Since one is allowed to vote on one's own comments, and upvoting is the default for one's own comments, this makes comments effectively start at a rating of 1. The argument against this is that it makes more sense for comments to start with a rating of 0, so that someone else liking a comment gives it a positive rating, while someone disliking it gives it a negative rating. I disagree with this assessment.

If I post a comment, it's because it's the best comment I could think of to add to the discussion. I will usually not bother saying something if I don't think it's the sort of thing that I would upvote. When I see someone else's comment that I don't think is very good, I downvote it. Since they already upvoted it, I'm in effect disagreeing that this was something worth saying. The score now reflects this - a score of 0 shows that one person thought it was a worthwhile comment, and one person did not.

Furthermore, if I was not able to vote on my own comments, I would be m...
Extenuating Circumstances
by Eliezer Yudkowsky — 2009-04-06T22:57:31.701Z

Followup to: Tsuyoku Naritai

"Just remember, there but for a massive genetic difference, environmental factors, and conscious choices, go you or I." -- Justin Corwin

Failures don't have single causes. We choose single causes to focus on, but nothing in the universe emerges from a single parent event. Every assassination ever committed is the fault of every asteroid that wasn't in the right place to hit the assassin.

What good, then, does it do to blame circumstances for your failure? What good does it do? - to look over a huge causal lattice in which your own decisions played a part, and point to something you can't control, and say: "There is where it failed." It might be that a surgical intervention on the past, altering some node outside yourself, would have let you succeed instead of fail. But what good does this counterfactual do you? Will you choose that outside reality be different on your next try?

And yet... when I look at other people, not myself, I find myself taking "extenuating circumstances" into account a great deal. I go to great lengths to "save the world" (as I believe from my epistemic vantage point). When I consider doing less, I consider that this would make me a horrible awful unforgivable person. And then I cheerfully shake hands with others who aren't trying at all to save the world. I seem to want to have my cake and eat it too - to instantiate Goetz's Paradox: "Society tells you to work to make yourself more valuable. Then it tells you that when you reason morally, you must assume that all lives are equally valuable. You can't have it both ways."

Is this an inherent subjective asymmetry - does morality just look different from the outside than inside? If so, is that okay, or is it a sign of self-contradiction? Or is it condescension on my part - that I think less of others and so hold them to lower standards?

I've pondered this question for a while, and this is the main defense I can offer against the charge of condescension:

I wouldn't tell others to take into account "extenuating circumstances" in judging themselves.

Indeed, that would feel like an act of sabotage - like slashing their tires. Too much of life consists of holding ourselves to a high enough standard.

There are people who blame themselves too easily - people depress...
What do fellow rationalists think about Mensa?
by taw — 2009-04-06T22:08:13.104Z

It's not a typical OB/LW subject, but Robin correctly pointed out that most rationalists are outside OB/LW, and so I'm asking about one of the organizations that might hold many of them.

A couple of weeks ago I took a supervised IQ test by Mensa due to curiosity and for some CV padding (cheap signaling is a perfectly rational thing to do). Now I got a letter back from them that I'm in top whatever %, and they'd like me to join. I wasn't really planning joining Mensa, or anything else, so I'm wondering - does any of fellow rationalists have any experience with them? Is it worth bothering?

As a bonus here's a quick description of their supervised IQ testing process:

First you get Catell scale B test, which is a mix of English word puzzles, picture puzzles, and logic puzzles. It obviously discriminates against non-native speakers. It has somewhat tight per-page timing. It seems to have stddev 24.

Then after a break you get Catell Culture Fair test, which is pure pictures, on extremely tight and stressful per-page timing. It seems to have stddev 16.

They compute percentile based on both tests separately, and higher of two counts as the result. So you can has 0 points on one (if at all possible), and respectively 148 / 132 on the other, and you're in (2 stddev above mean, or top 2%). The tests obviously check knowledge of obscure English words and meanings and ability to deal with pressure in addition to intelligence as such. Well, I guess no test is perfect.

So Mensa - good or bad?
Rationalist wiki, redux
by Paul Crowley — 2009-04-06T19:24:43.022Z

This site is very likely impenetrable to the newcomer. You one-box and defect on the True Prisoner's Dilemma, but is that just because of a cached thought, or is it your Tsuyoku Naratai? So I've created the LessWrong Wiki on Wikia. I'd like this to become a respository of useful definitions and links: it can support our discussions here, and create something lasting from the ephemerality of a blog.

badger already created a Wiki, but as you can see in the updates to that article badger and others pretty quickly concluded that TiddlyWiki wouldn't be up to the job. MediaWiki, the software Wikipedia and Wikia use, is the monster of them all, and will give us good support for practically anything we want to do, including mathematical notation. I've ported across a couple of articles from the old wiki onto the new, but many more are needed. The "download" link in TiddlyWiki and a text editor may help.

EDIT: Usernames are global across all of Wikia, so you may not be able to use the same name there as here. Sorry.
Average utilitarianism must be correct?
by PhilGoetz — 2009-04-06T17:10:02.598Z

I said this in a comment on Real-life entropic weirdness, but it's getting off-topic there, so I'm posting it here.

My original writeup was confusing, because I used some non-standard terminology, and because I wasn't familiar with the crucial theorem. We cleared up the terminological confusion (thanks esp. to conchis and Vladimir Nesov), but the question remains. I rewrote the title yet again, and have here a restatement that I hope is clearer.

We have a utility function u(outcome) that gives a utility for one possible outcome. (Note the word utility. That means your diminishing marginal utility, and all your preferences, and your aggregation function for a single outcome, are already incorporated into this function. There is no need to analyze u further, as long as we agree on using a utility function.)

We have a utility function U(lottery) that gives a utility for a probability distribution over all possible outcomes.

The von Neumann-Morgenstern theorem indicates that, given 4 reasonable axioms about U, the only reasonable form for U is to calculate the expected value of u(outcome) over all possible outcomes. This is why we constantly talk on LW about rationality as maximizing expected utility.

This means that your utility function U is indifferent with regard to whether the distribution of utility is equitable among your future selves. Giving one future self u=10 and another u=0 is equally as good as giving one u=5 and another u=5.

This is the same ethical judgement that an average utilitarian makes when they say that, to calculate social good, we should calculate the average utility of the population; modulo the problems that population can change and that not all people are equal. This is clearer if you use a many-worlds interpretation, and think of maximizing expected value over possible futures as applying average utilitarianism to the population of all possible future yous.

Therefore, I think that, if the 4 axioms are valid when calculating U(lottery), they are probably also valid when calculating not our private utility, but a social utility function s(outcome), which sums over people in a similar way to how U(lottery) sums over possible worlds. The theorem then shows that we should set s(outcome) = the average value of all of the utilities for the different people involved. (In other words, ave

...
Newcomb's Problem standard positions
by Eliezer Yudkowsky — 2009-04-06T17:05:23.522Z

Marion Ledwig's dissertation summarizes much of the existing thinking that's gone into Newcomb's Problem.

(For the record, I myself am neither an evidential decision theorist, nor a causal decision theorist in the current sense. My view is not easily summarized, but it is reflectively consistent without need of precommitment or similar dodges; my agents see no need to modify their own source code or invoke abnormal decision procedures on Newcomblike problems.)
Rationalists should beware rationalism
by Kaj_Sotala — 2009-04-06T14:16:30.733Z

Rationalism is most often characterized as an epistemological position. On this view, to be a rationalist requires at least one of the following: (1) a privileging of reason and intuition over sensation and experience, (2) regarding all or most ideas as innate rather than adventitious, (3) an emphasis on certain rather than merely probable knowledge as the goal of enquiry. -- The Stanford Encyclopedia of Philosophy on Continental Rationalism.
By now, there are some things which most Less Wrong readers will agree on. One of them is that beliefs must be fueled by evidence gathered from the environment. A belief must correlate with reality, and an important part of that is whether or not it can be tested - if a belief produces no anticipation of experience, it is nearly worthless. We can never try to confirm a theory, only test it.

But yet, we seem to have no problem coming up with theories that are either untestable or that we have no intention of testing, such as evolutionary psychological explanations for the underdog effect.

I'm being a bit unfair here. Those posts were well thought out and reasonably argued, and Roko's post actually made testable predictions. Yvain even made a good try at solving the puzzle, and when he couldn't, he reasonably concluded that he was stumped and asked for help. That sounds like a proper use of humility to me.

But the way that ev-psych explanations get rapidly manufactured and carelessly flung around on OB and LW has always been a bit of a pet peeve for me, as that's exactly how bad evpsych gets done. The best evolutionary psychology takes biological and evolutionary facts, applies those to humans and then makes testable predictions, which it goes on to verify. It doesn't take existing behaviors and then try to come up with some nice-sounding rationalization for them, blind to whether or not the rationalization can be tested. Not every behavior needs to have an evolutionary explanation - it could have evolved via genetic drift, or be a pure side-effect from some actual adaptation. If we set out by trying to find an evolutionary reason for some behavior, we are assuming from the start that there must be one, when it isn't a given that there is. And even a good theory need not explain every observation.

Obviously I'm not saying that we should never come up with such theories. Be wary of those who speak of being open-minded and modes...
Heuristic is not a bad word
by HughRistik — 2009-04-06T06:55:10.539Z

An insect tries to escape through the windowpane, tries the same again and again, and does not try the next window which is open and through which it came into the room. A man is able, or at least should be able, to act more intelligently. —George Polya, How To Solve It
Intelligence makes humans capable of many impressive feats. Unlike flies and birds, we don't bang up against windows multiple times trying to get out of our houses. We can travel to the moon. We have taken over the planet. Why? Because intelligence enables us to solve problems.
All problems start the same way. They start unsolved. Each fact humans have figured out was initially unfigured out by us. Then we did something, which converted the unknown fact into a known fact, changed the state of a problem from unsolved to solved.
I emphasize the unknown starting state of problems to make a point: problem solving, the basis of human achievement, depends on a process of discovery, discovery of new facts, new possibilities, new methods, and new ways of thought.
Heuristic—the art and science of discovery—has been integral for human progress. The word "heuristic" is related to "Eureka!"

Heuristics and biases

Unfortunately, heuristic is a bad word. At least, that's the impression you might get, seeing it hand-in-hand with "bias" in the psychological literature. In Judgment under Uncertainty: Heuristics and Biases, Tversky and Kahneman acknowledge that "in general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors." On Overcoming Bias, heuristics seem primarily discussed as resulting in biases.

Bias-reduction is a form of skepticism that is a critical part of rationality. Due to the uncertain nature of the territory of reality, many notions of the territory are wrong. Rational skepticism helps us identify false assumptions, areas where our map will depart from the territory.
While bias-reduction is necessary in the search for rationality, it is not sufficient. It's a mistake in cartography to have areas of your map that are filled in wrong, but it's also a mistake to have areas on your map blank that you could have filled in, at least with something approximate. A map with wrong patches will not take you to your destination, but neither will a map with blank patches. Believing things that are false is one error which will prevent us from finding the...
Rationality Toughness Tests
by RobinHanson — 2009-04-06T01:12:31.928Z

(Epistemic) rationality has two major components:

Smarts: An ability to, by attending, infer truth from info under ideal circumstances.

Toughness: An ability to limit performance degradation as circumstances worsen.

Attending takes time, energy, quiet, etc. Circumstances where human rationality degrades include when:

We expect the truth to long remain hidden.

The stakes are very low, or very high, to us.

Others see our opinions, and prefer certain ones.

The topics are where humans oft self-deceive.

It seems relatively easy to test rationality smarts; repeatedly give folks info and time to work new problems and measure their accuracy, calibration, etc. And I have an idea for testing for rationality toughness: compare performance on info-similar pairs of good/bad-circumstance problems.
For example, assume people are better at evaluating if a spouse is cheating when considering an acquaintance in their social circle, relative to a stranger or their own spouse. If so, we could pose them a pair of problems with very similar info structure, one with an easy spouse and one with a hard spouse. The closeness of their response in these two cases would then be a measure of their rationality toughness.
Of course this test may fail if the similarity is too obvious, or the pair are asked too closely in time. But maybe we don't even need to ask the same person the two questions; perhaps we could usefully compare someone's answer on a hard question to answers from a pool of similar people on matched easy questions.
While I haven't thought this through, it already suggests a training technique: consider matched hard/easy circumstance problems and compare your answers, separated by enough time that you forget most of your previous analysis.
Rationalist Wiki
by badger — 2009-04-06T00:19:25.192Z

Some (including myself) have suggested that a rationality wiki would be a useful supplement to this site. In the spirit of getting things done, I set one up here: http://rationality.tiddlyspot.com/ The password to save edits is omega.

The TiddlyWiki framework it uses is very lightweight and won't be satisfactory as a long-term solution. I do think it has potential as a minimalist beginner's guide though, and could serve us well for the time being. I am not very knowledgable about wiki software in general, but TiddlyWiki has served me well for multiple personal wikis. I planned on developing it a little further before revealing it to the community, but other commitments demand my attention. Please feel free to contribute.

Comments, suggestions? Is it better to start with something that can handle a significant user base and future growth, or should it stay small and self-contained to remain accessible to beginners?

Update: I think it's becoming clear this can't serve as more than a short-term hack, even for a minimalist beginner's guide. At least it is provoking discussion. I'm still hoping for contributions so we have a leg up once an official solution emerges. If you do contribute, please try to keep markup to a minimum to facilitate a future conversion.