trentmkelly/LessWrong-43k huggingface dataset:
Deception by cherry picking
by AlephNeil — 2011-02-20T05:11:54.770Z

[This is an idea I've had 'kicking around' for a long time - may as well see what LW makes of it.]

The Bayesian update procedure tacitly presupposes that the event we're updating on is not itself random. Indeed, naively updating on a random event, that turns out to be correlated with the variable of interest, is how people get Monty Hall wrong.

A false statement can often be made to seem plausible if you naively update on a set of misleading, 'cherry picked' facts.

To make this concrete, imagine a biased coin which we know has probability 1/3 or 2/3 of landing heads - in fact it's 1/3 but we don't know that. Say it's tossed 2000 times. Then someone who wanted to mislead us could cherry pick a sample of 100 in which, say, 70 of the coin tosses landed heads, and hope we assume they picked their sample randomly. (More insidiously, using Derren Brown's variety of dark arts they could even trick us into choosing that sample ourselves, believing that we're choosing 'of our own free will'.)

But now here's the thing: That sample of 100 probably has a high minimum description length. If it had a sufficiently low minimum description length - like if it consisted of 100 contiguous tosses - then even if we suspected "Derren Brown" was trying to manipulate us, our sample would still give us evidence that heads has probability 2/3.

I think there should be a theorem which looks like:

"The largest x such that we'd be irrational not to increase our subjective log-odds of event E by at least x, even if our data was provided by an adversary" = log[P(data|E) / P(data|¬E)] - "The minimum description length of the data" - O(1)

Anyone familiar with "the theory of how to update on evidence provided by adversaries" (assuming it exists)?
Is science the worst field for learning science in?
by KatjaGrace — 2011-02-20T04:29:15.000Z

I wrote that science classes usually demonstrate the art of not significantly changing your beliefs when evidence conflicts with an answer you are sure of, at the expense of the central idea of science: experimenting for the purpose of changing your mind.

Chris Chang asked me how it could be taught better:

I can imagine the occasional chemistry assignment that gets around this by having mystery reagants whose identities you need to discover, and the occasional physics assignment where you have anonymous materials and need to figure out some of their properties. But it seems hard to me to extend this to an entire course worth of labs; any more ideas?

I doubt it’s hard to keep children in the dark about the entities in a whole course’s worth of labs, but it may conflict with wanting to teach them more reliably discovered information on the same topics, or at least not wanting to mislead them. Fortunately though, science is applicable to any subject matter within the realm of reality. The topics on which we have collected a lot of knowledge via science are also called ‘science’, which makes it easy to forget they are not the only possible, or even necessarily the best, subjects of science.

We want to teach our children some of this knowledge we have gained from science. But there seems little reason to combine understanding what we already know, about say chemistry, with learning about the process of doing science. If you want to learn to do science, with all the thrills of actually discovering anything, you are probably best to pick an area where people don’t already know all of the cheap answers (those not requiring monkeys, brain scanners or large chunks of subterranean Europe).

The most obvious place to find undiscovered cheap answers available for scientific investigation is in topics that are not very important to people who pay for science. A notable example is the many small scale questions that are relevant to a given student but not to society as a whole. Does decreasing the length of my skirt increase the propensity of the cool students to talk to me? Does learning the piano as a child really make people happier later in life? Does Father Christmas exist? Do the other children hate me or are they just indifferent? What factors best cause my brothers to leave me alone? How much do my grades change if I do half an hour more o...
Social ethics vs decision theory
by AlexMennen — 2011-02-20T04:14:54.389Z

It seems to me that usually, when someone says "ethics" on lesswrong, ey usually means something along the lines of decision theory. When an average person says "ethics", ey is usually referring to a system of intuitions and social pressures designed to influence the behavior of members of a group. I think that a lot of the disagreement regarding ethics (i.e. consequentialism vs deontology) is rooted in a failure to properly distinguish between decision theory and what society pressures people to do. Most lesswrong users probably understand the distinction fairly clearly, but we only ever talk about decision theory. Why don't we talk about the social meaning of ethics?
Request for Widely Applicable Quantitative Methods
by atucker — 2011-02-20T04:08:11.357Z

I'm going to be competing in the Moody's Mega Math Challenge, and I was wondering if there was anything in particular I should brush up on.
If you look at previous problems, you can see that they're pretty varied. I want to know if there's any widely applicable math that we could study (in a fairly short amount of time) to maximize the odds of us knowing something useful for the competition.

Our math backgrounds include:

Statistics (taught by a frequentist, mostly just probability theory and p, z, chi-squared, etc. tests)

Calculus (single variable and multivariable)

Linear Algebra

Numerical Methods

We're also pretty competent at programming in various programming languages, and LaTeX.
Currently we're looking into Causality by Judea Pearl, and Linear Programming. Should we look at these? Anything else we should know?

Edit:
I suppose we could also use a genetic algorithm, but those don't seem particularly suited to the competition.
Do biases matter?
by Student_UK — 2011-02-20T01:10:24.142Z

It occurred to me that our biases might not matter very much lots of the time. They matter if you want to maximize your chances of finding the truth, but not if you are interested in maximizing the chances the chances of someone finding the truth.

Take science as an example, scientists aren't always free from biases when it comes to thinking about their own theories. They look for ways to confirm what they believe, not discomfirm it. But does that matter? As long as their rivals are looking for ways to falsify those theories then overall the system should work well.

In fact it might work better:

Imagine that there is a prize (a really big prize) at the end of a maze. Several of you are sent into the maze and the first person to find the prize gets to keep it. You all head down the corridor and come to three doors, there are some clues written on the wall, but before you can even begin to read them, someone dashes through door 1. Someone else follows then another through door 2 and a few take door 3. What do you do? You could be methodical and try to solve the clues. This would maximize your chance of finding the right path. However, it will not maximize your chances of being the first one to find the prize. For this, you need to pick a door and run.

Likewise, in science, if you want to get the prize (nobel prize, or a good job, or fame, or a best-selling book), then you might be better off coming up with a new theory and running with it (anecdotally, this seems to be what a lot of successful scientists have done). Having lots of people making leaps in different directions might also make science progress faster overall.

What does all this mean for biases? Are they best thought of on an individual level or group level? Is it really a bias if it is the best thing to do? Can you think of other examples where individual biases might produce better results for the group?
New blog
by chronophasiac — 2011-02-20T00:07:48.657Z

Today I'm launching a new blog: http://www.trajectoryofeverything.com

Subjects are mainly rationality and tech. Feedback of any sort is welcomed. Some of the posts relate to ideas that I've considered for LW but felt unworthy. I've been lurking in the LW/OB community since inception and one big goal for the blog is to get myself writing. Hopefully I'll work up to LW level at some point.
LW was started to help altruists
by RHollerith — 2011-02-19T21:13:00.020Z

The following excerpt from a recent post, Recursively Self-Improving Human Intelligence, suggests to me that it is time for a reminder of the reason LW was started.

"[C]an anyone think of specific ways in which we can improve ourselves via iterative cycles? Is there a limit to how far we can currently improve our abilities by improving our abilities to improve our abilities? Or are these not the right questions; the concept a mere semantic illusion[?]"

These are not the right questions -- not because the concept is a semantic illusion, but rather because the questions are a little too selfish. I hope the author of the above words does not mind my saying that. It is the hope of the people who started this site (and my hope) that the readers of LW will eventually turn from the desire to improve their selves to the desire to improve the world. How the world (i.e., human civilization) can recursively self-improve has been extensively discussed on LW.

Eliezer started devoting a significant portion of his time and energy to non-selfish pursuits when he was still a teenager, and in the 12 years since then, he has definitely spent more of his time and energy improving the world than improving his self (where "self" is defined to include his income, status, access to important people and other elements of his situation). About 3 years ago, when she was 28 or 29, Anna Salamon started spending most of her waking hours trying to improve the world. Both will almost certainly devote the majority of rest of their lives to altruistic goals.

Self-improvement cannot be ignored or neglected even by pure altruists because the vast majority of people are not rational enough to cooperate with an Eliezer or an Anna without just slowing them down and the vast majority are not rational enough to avoid catastrophic mistakes were they to try without supervision to wield the most potent methods for improving the world. In other words, self-improvement cannot be ignored because now that we have modern science and technology, it takes more rationality than most people have just to be able to tell good from evil where "good" is defined as the actions that actually improve the world.

One of the main reasons Eliezer started LW is to increase the rationality of altruists and of people who will become altruists. In other words, of people committed to improving the world. (The other main reason is ...
Rationalist Hobbies
by Wei Dai — 2011-02-19T08:24:38.599Z

This is a place to talk about hobbies that teach rationality lessons. I'll start with a few:

Programming - Lets you practice math and logic, and gain an intuitive understanding of computation. Also teaches that sometimes you can't argue with reality, but you can fix it. (Also suggested recently by ciphergoth.)

Cryptography - That is, designing/breaking cryptosystems and security protocols. Lets you practice math, logic, and probability theory. Also teaches that almost all human ideas are wrong. After doing this for a while, whenever you encounter a new idea (including your own), you'll instinctively think "If I can't find anything wrong with this, it's probably because I'm not smart or knowledgeable enough or haven't tried hard enough."

Science fiction - Reduces status quo bias and gives interesting insights. Also teaches that the way a society is organized depends a lot on the set of technologies it has access to, so if you don't like how your society works, one lever you have is to change that set.

Video games - Teaches that conventional "success" in life is not much less arbitrary than "winning" in a video game. They're both fine for a diversion, but there are more interesting goals to pursue.

A couple others I've seen suggested in recent comments:

Chess - According to JGWeissman, it "teaches you to carefully consider the consequences of your available actions and choose the action with the best consequences". (I've only played a few games of Chinese Chess, and for me, the lesson was that I don't like competition, and I should look for things to do that nobody else is doing.)

Poker - Lets you practice statistics and overcoming emotional biases.
preferences:decision theory :: data:code
by ArthurB — 2011-02-19T07:45:22.119Z

I'd like to present a couple thoughts. While I am somewhat confident in my reasonning, my conclusions strongly contradict what I perceive (possibly incorrectly) to be the concensus around decision theory on LessWrong. This consensus has been formed by people who have spent more time than me thinking about it, and are more intelligent than I am. I am aware of that, this is strong evidence that I am mistaken or obvious. I believe nonetheless the argument I'm about to make is valuable and should be heard.

It is argued that the key difference between Newcomb's problem and Solomon's problem is that precommitment is useful in the former and useless in the latter. I agree that the problems are indeed different, but I do not think that is the fundamental reason. The devil is in the details.

Solomon's problem states that

- There is a gene that causes people to chew gum and to develop throat cancer
- Chewing gum benefits everyone

It is generally claimed that EDT would decide not to chew gum, because doing so would place the agent in a state where its expected utility is reduced. This seems incorrect to me. The ambiguity is in what is meant by "causes people to chew gum". If the gene really causes people to chew gum, then that gene by definition affects that agent's decision theory, and the hypothesis that it is also following EDT is contradictory. What is generally meant is that having this gene induces a preference to chew gum, which is generally acted upon by whatever decision algorithm is used. An EDT agent must be fully aware of its own preferences, otherwise it could not calculate its own utility, therefore, the expected utility of chewing gum must be calculated conditional on having a preexisting or non preexisting taste for gum. In a nutshell, an EDT agent updates not on his action to chew gum, but on his desire to do so.

I've established here a distinction between preferences and decision theory. In fact, the two are interchangeable. It is always possible to hard code preferences in the decision theory, and vice versa. The distinction is very similar to the one drawn between code and data. It is an arbitrary but useful distinction. Intuitively, I believe hard coding preferences in the decision algorithm is poor design, though I do not have a clear argument why that is.

If we insist on preferences being part of the decision algorithm,...
Spoiled Discussion of Permutation City, A Fire Upon The Deep, and Eliezer's Mega Crossover
by JenniferRM — 2011-02-19T06:10:15.258Z

Permutation City is an awesome novel that was written in 1994. Even if the author, Greg Egan, used a caricature of this community as a bad guy in a more recent novel, his work is still a major influence on a lot of people around these parts who have read it. It dissolves so many questions around uploading and simulation that it's hard for someone who has read the book to talk about simulationist metaphysics without wanting to reference the novel... but doing that runs into constraints imposed by spoiler etiquette.

So go read Permutation City if you haven't read it already because it's philosophically important and a reasonably fun read.

In the meantime, if you haven't then you should also read A Fire Upon The Deep by Vernor Vinge (of "singularity" coining fame) and then read Eliezer's fan fic The Finale of the Ultimate Meta Mega Crossover which references both of them in interesting ways to make substantive philosophical points and doesn't take too long to read.

In the comments below there will be discussion that has spoilers for all three works.
Alt text of today's xkcd addresses akrasia
by Cyan — 2011-02-19T02:03:20.482Z

The alt text of today's xkcd reads:

After years of trying various methods, I broke [the habit of clicking on my favorite distractions every 5 minutes] by pitting my impatience against my laziness. I decoupled the action and the neurological reward by setting up a simple 30-second delay I had to wait through, in which I couldn't do anything else, before any new page or chat client would load (and only allowed one to run at once). The urge to check all those sites magically vanished -- and my 'productive' computer use was unaffected.

Anyone have ideas about how to implement this? On Firefox one can always use LeechBlock. On *nix systems there would be a number of ways of implementing this, but not all of us use that OS or have the necessary savvy.

(I'm kind of surprised no one's made a discussion post on this yet.)
Fun and Games with Cognitive Biases
by Cosmos — 2011-02-18T20:38:28.192Z

You may have heard about IARPA's Sirius Program, which is a proposal to develop serious games that would teach intelligence analysts to recognize and correct their cognitive biases. The intelligence community has a long history of interest in debiasing, and even produced a rationality handbook based on internal CIA publications from the 70's and 80's. Creating games which would systematically improve our thinking skills has enormous potential, and I would highly encourage the LW community to consider this as a potential way forward to encourage rationality more broadly.

While developing these particular games will require thought and programming, the proposal did inspire the NYC LW community to play a game of our own. Using a list of cognitive biases, we broke up into groups of no larger than four, and spent five minutes discussing each bias with regards to three questions:

How do we recognize it?

How do we correct it?

How do we use its existence to help us win?

The Sirius Program specifically targets Confirmation Bias, Fundamental Attribution Error, Bias Blind Spot, Anchoring Bias, Representativeness Bias, and Projection Bias. To this list, I also decided to add the Planning Fallacy, the Availability Heuristic, Hindsight Bias, the Halo Effect, Confabulation, and the Overconfidence Effect. We did this Pomodoro style, with six rounds of five minutes, a quick break, another six rounds, before a break and then a group discussion of the exercise.

Results of this exercise are posted below the fold. I encourage you to try the exercise for yourself before looking at our answers.

Caution: Dark Arts! Explicit discussion of how to exploit bugs in human reasoning may lead to discomfort. You have been warned.

Confirmation Bias

Notice if you (don't) want a theory to be true

Don't be afraid of being wrong, question the outcome that you fear will happen

Seek out people with contrary opinions and be genuinely curious why they believe what they do

How do we make people genuinely curious? Maybe try encouraging childlike behavior generally?

If your theory is true every test should come back positive, so don't worry and make a game of disproving your hypothesis

Commit yourself to which directions you will update on the different outcomes of an experiment before running it

Be more suspicious of confirmatory results when you d

...
Ability to react
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2011-02-18T19:19:34.806Z

*Note: this post is based on my subjective observations of myself and a small, likely biased sample of people I know. It may not generalize to everyone.

A few days ago, during my nursing lab, my classmates and I were discussing the provincial exam that we’ll have to sit two years from now, when we’re done our degree, in order to work as registered nurses. The Quebec exam, according to our section prof, includes an entire day of simulations, basically acted-out situations where we’ll have to react as we would in real life. The Ontario exam is also a day long, but entirely written.

I made a comment that although the Quebec exam was no doubt a better test of our knowledge, the Ontario exam sounded a lot easier and I was glad I planned to work in Ontario.

“Are you kidding?” said one of the boys in my class. “Simulations are so much easier!”

I was taken aback, reminded myself that my friends and acquaintances are probably weirder than my models of them would predict (thank you AnnaSalamon for that quote), and started dissecting where exactly the weirdness lay. It boiled down to this:

Some people, not necessarily the same people who can ace tests without studying or learn math easily or even do well in sports, are still naturally good at responding to real-life, real-time events. They can manage their stress, make decision on the spot, communicate flexibly, and even have fun while doing it.

This is something I noticed years ago, when I first started taking my Bronze level lifesaving certifications. I am emphatically not good at this. I found doing “sits” (simulated situations) stressful, difficult, and unpleasant, and I dreaded my turn to practice being the rescuer. I had no problem with the skills we learned, as long as they were isolated, but applying them was harder than the hardest tests I’d had at school.

I went on to pass all my certifications, without any of my instructors specifically saying I had a problem. Occasionally I was accused of having “tunnel vision”; they meant that during a sit, treating my victim and simultaneously communicating with my teammates was more multitasking than my brain could handle.

Practice makes perfect, so I joined the competitive lifeguard team (yes, this exists, see https://picasaweb.google.com/lifeguardpete for photos of competitions). We compete in teams o...
Cryonics and Pascal’s wager
by Timwi — 2011-02-18T18:36:43.757Z

The Cambridge UK meet-up on Saturday 12 February went really well. Many thanks to everyone who came and provided a wonderful and entertaining discussion.

One of the topics that came up was that of cryonics. This is the idea of having your body (or maybe just your brain) frozen after death, to be thawed and revived in the far future when medical technology has advanced to the point where it can heal you. Is this a rational thing to do?

The argument I heard from some of the other attendants effectively boils down to “what have you got to lose?” In other words, have yourself frozen just in case it works and you can be resurrected.

This struck me as awfully reminiscent of Pascal’s Wager, which is similarly a “what have you got to lose?” type argument. Cited in its original form, it is about belief in a god and goes something like this:

You can either believe in God or not. If you do, you will either be rewarded with eternity in heaven (if you’re right) or nothing happens (if you’re wrong). But if you don’t believe, you will either be punished by eternal torture (if you’re wrong) or nothing happens (if you’re right). It’s a no-brainer! You’re better off believing.

This argument falls down on many counts, but I’ll concentrate on a specific one. It makes a far-fetched assumption about the set of possible outcomes. It assumes that there are only the two possibilities quoted and no others. It ignores the possibility of a god that only rewards sceptical atheists.

Coming back to cryonics, the argument seems to proceed approximately like this:

You can either have yourself frozen or not. If you do, you will either wake up in a wonderful, happy-go-lucky utopian future with amazing technological advances (if cryonics works) or nothing happens (if it doesn’t). But if you don’t have yourself frozen, nothing happens either way. It’s a no-brainer! You’re better off in cryopreservation.

If I haven’t already made it abundantly clear, the assumption that the future you wake up in is at all desirable for you is a far-fetched one. It ignores the possibi

...