trentmkelly/LessWrong-43k huggingface dataset:
How thoroughly do you research your dreams?
by KatjaGrace — 2011-04-15T01:35:33.000Z

Eliezer Yudkowsky:

Have I ever remarked on how completely ridiculous it is to ask high school students to decide what they want to do with the rest of their lives and give them nearly no support in doing so?

Support like, say, spending a day apiece watching twenty different jobs and then another week at their top three choices, with salary charts and projections and probabilities of graduating that subject given their test scores? The more so considering this is a central allocation question for the entire economy?

I’ve been meaning to remark how surprised I am that not even the students themselves seem interested in researching such things, or to even think of it. Similarly for their families. It’s not expensive to phone a few people who are doing your dreamed of career, and I expect many would be happy to talk or show you the ropes. Even if high school students aren’t that strategic or farsighted, I would expect a given one to have at least one e.g. great aunt who would think to say ‘Stop! This is a really important decision! Maybe you should research it beyond your ridiculously naive imaginings and a the stereotyped impression your lay-parents have given you!”.

It would make sense for not many people to do this entering college because they didn’t have much idea what they wanted to do yet, but it doesn’t look like people do it later either. And even if you are entering college without knowing what you want to do later, it would probably make sense to at least contact some current students doing your proposed degree and find out something about what that is like. I did neither of these things, I’m not sure why. So far I’m confident the latter would have helped. I don’t know of anyone else doing much research, but maybe they just don’t talk about it. What’s going on?
How You Make Judgments: The Elephant and its Rider
by lukeprog — 2011-04-15T01:02:12.625Z

Part of the sequence: Rationality and Philosophy

Whether you're doing science or philosophy, flirting or playing music, the first and most important tool you are using is your mind. To use your tool well, it will help to know how it works. Today we explore how your mind makes judgments.

From Plato to Freud, many have remarked that humans seem to have more than one mind.1 Today, detailed 'dual-process' models are being tested by psychologists and neuroscientists:

Since the 1970s dual-process theories have been developed [to explain] various aspects of human psychology... Typically, one of the processes is characterized as fast, effortless, automatic, nonconscious, inflexible, heavily contextualized, and undemanding of working memory, and the other as slow, effortful, controlled, conscious, flexible, decontextualized, and demanding of working memory.2

Dual-process theories for reasoning,3 learning and memory,4 decision-making,5 belief,6 and social cognition7 are now widely accepted to be correct to some degree,8 with researchers currently working out the details.9 Dual-process theories even seem to be appropriate for some nonhuman primates.10

Naturally, some have wondered if there might be a "grand unifying dual-process theory that can incorporate them all."11 We might call such theories dual-system theories of mind,12 and several have been proposed.13 Such unified theories face problems, though. 'Type 1' (fast, nonconscious) processes probably involve many nonconscious architectures,14 and brain imaging studies show a wide variety of brain systems at work at different times when subjects engage in 'type 2' (slow, conscious) processes.15

Still, perhaps there is a sense in which one 'mind' relies mostly on type 1 processes, and a second 'mind' relies mostly on type 2 processes. One suggestion is that Mind 1 is evolutionarily old and thus shared with other animals, while Mind 2 is recently evolved and particularly developed in humans. (But not fully unique to humans, because some animals do seem to exhibit a distinction between stimulus-controlled and higher-order controlled behavior.16) But this theory faces problems. A standard motivation for dual-process theories of reasoning is the conflict between cognitive biases (from type 1 processes) and logical reasoning (type 2 processes).17 For example, logic and b...
Say More, Justify Less
by paulfchristiano — 2011-04-14T22:41:11.657Z

Related to: Rational Me or We?
Human brains reach decisions using complicated, often opaque, mechanisms. Rational behavior requires learning about systematic failures of these mechanisms, but also requires learning to effectively use judgments for which our brains provide no communicable justification. If we cannot trust our brains just because we don't understand them, our beliefs will be too slow to follow reality. This is a critically important lesson for individual rationality, but there is an even more important analog for collective rationality.
Aumann's agreement theorem provides conditions under which rationalists cannot agree to disagree. This result is often cited to justify protracted arguments regarding contentious facts. These arguments generally proceed as an exchange of well-specified evidence and reasoned justification. Aumann's agreement theorem says nothing about the effectiveness of such exchanges, and in my experience such arguments almost always end either quickly or unsuccessfully.
Why don't rational arguments end well? Sometimes our beliefs depend on a few pieces of easily identified evidence, but often they depend on the balance of a large quantity of (potentially forgotten) evidence colored by our experience and aggregated using intuition. Sometimes our beliefs depend on simple easily articulated deductions, but often they depend on subtle inferences we don't consciously understand. Arguments fail because the real sources of our beliefs cannot be easily communicated, or even well understood. If we rely only on reasoning and evidence that can be communicated in clean and conclusive arguments, our beliefs will be too slow to follow reality.
Why can't we reach agreement by updating on the beliefs of others, as per Aumann's theorem? Why do we need to describe how we arrived at our beliefs, instead of simply stating them? In fact in ordinary conversation no one even tries to use Aumann-style negotiation, and from my perspective this seems completely rational. Even when talking to people whose rationality I trust, assuming common knowledge of rationality or honesty is never close to accurate; if we abandon skepticism we can expect to be consistently wrong (even when we aren't deliberately manipulated).
The real sources of our belief are hard to communicate and we aren't sufficiently perfect rationalists to use Aumann agreement: if we want to ...
Brief Query- An Idea
by Carinthium — 2011-04-14T19:55:04.308Z

I had an idea- has it been done before, and if not shouldn't somebody try to do it? I live in Melbourne where LWers aren't organised, but perhaps the New York branch or some other group could try this? (IF it hasn't been done, that is- I haven't seen in mentioned, which is why I'm checking)

Idea:

-Recruit a non-rationalist scientist (or better group of scientists) either by persuading friends or perhaps getting some money together to pay somebody (or finding a helpful volenteer) or perhaps several.

-Have THEM come up with a series of tests to test rationalists relative to a control group.

If 'sucessful' (in the sense of a significant difference between rationalists and non-rationalists, the result we on LW would presumably predict), it would provide enough evidence to justify a formal test (there would likely be a few weaknesses, such as recruiting friends of rationalists), which could then (again, assuming such a result) persuade scientists to become rationalists (benefits should be obvious on a bit of thought) and generate publicity. If a 'failure' or indecisive, it justifies a serious reevaluation of site methodology.
Confused about the doomsday argument, please help
by cousin_it — 2011-04-14T18:56:55.856Z

The doomsday argument says I have only a 10% chance of being within the first 10% of humans ever born, which gives nonzero information about when humanity will end. The argument has some problems with the choice of reference class; my favorite formulation (invented by me, I'm not sure if it's well-known) is to use the recursive reference class of "all people who are considering the doomsday argument with regard to humanity". But this is not the issue I want to discuss right now.

Imagine your prior says the universe can contain 10, 1000 or 1000000 humans, with probability arbitrarily assigned to these three options. Then you learn that you're the 50th human ever born. As far as I can understand, after receiving this information you're certain to be among the first 10% of humans ever born, because it's true in every possible universe where you receive such information. Also learning your index doesn't seem to tell you very much about the date of the doomsday: it doesn't change the relative probabilities of doomsday dates that are consistent with your existence. (This last sentence is true for any prior, not just the one I gave.) Is there something I'm missing?
First Phoenix, AZ Less Wrong Meetup 4/17 from 12-2pm
by Danny_Hintze — 2011-04-14T18:38:03.459Z

Many of my friends and I are frequent readers of Less Wrong and aspiring rationalists, and we'd like to get to know anyone else in the area who is interested in getting together to make the world a little brighter. Absolutely anyone is welcome! (Even if you want to make the world darker, that certainly makes for some interesting discussion!).

I will be at a table near the Starbucks in the ASU Memorial Union from 12 to 2 pm. I'll probably be wearing a bright orange shirt, will have a Less Wrong sign of some kind, and will likely be reading some economics.

There are many places in the MU where we can get food and drinks, and also plenty of places to sit and talk.

The address is 1290 S Normal Ave, Tempe, AZ 85281, and here is a map centered on it. Parking in the ASU garages is free on Sundays.

My phone number is 602-501-9420, feel free to call or text me.
The Bias You Didn't Expect
by Psychohistorian — 2011-04-14T16:20:53.424Z

There are few places where society values rational, objective decision making as much as it values it in judges. While there is a rather cynical discipline called legal realism that says the law is really based on quirks of individual psychology, "what the judge had for breakfast," there's a broad social belief that the decision of judges are unbiased. And where they aren't unbiased, they're biased for Big, Important, Bad reasons, like racism or classism or politics.

It turns out that legal realism is totally wrong. It's not what the judge had for breakfast. It's how recently the judge had breakfast. A a new study (media coverage) on Israeli judges shows that, when making parole decisions, they grant about 65% after meal breaks, and almost all the way down to 0% right before breaks and at the end of the day (i.e. as far from the last break as possible). There's a relatively linear decline between the two points.

Think about this for a moment. A tremendously important decision, determining whether a person will go free or spend years in jail, appears to be substantially determined by an arbitrary factor. Also, note that we don't know if it's the lack of food, the anticipation of a break, or some other factor that is responsible for this. More interestingly, we don't know where the optimal result occurred. It's probably not the near 0% at the end of each work period. But is it the post-break high of 65%? Or were judges being too nice? We know there was bias, but we still don't know when bias occurred.

There are at least two lessons from this. The little, obvious one is to be aware of one's own physical limitations. Avoid making big decisions when tired or hungry - though this doesn't mean you should try to make decisions right after eating. For particularly important decisions, consider contemplating them at different times, if you can. Think about one thing Monday morning, then Wednesday afternoon, then Saturday evening, going only to the point of getting an overall feel for an answer, and not to the point of really making a solid conclusion. Take notes, and then compare them. This may not work perfectly, but it may help you realize inconsistencies, which could help. For big questions, the wisdom of crowds may be helpful - unless it's been a while since most of the crowd had breakfast.

The bigger lesson is one of...
Link: Why and how to debate charitably
by RobinZ — 2011-04-14T16:14:21.224Z

Even though this was written by a current Less Wrong poster (hi, pdf23ds!), I don't think it has been posted here: Why and how to debate charitably (pg. 2, comments). (Edit: The original pdf23ds.net site has sadly been lost to entropy – Less Wrong poster MichaelBishop found a repost on commonsenseatheism.com. He also provides this summary version.)

I was linked to this article from a webcomic forum which had a low-key flamewar smouldering in the "Serious Business" section. (I will not link to it here; if you can tell from the description which forum it is, I would thank you not to link it either.) Three things struck me about it:

I have been operating under similar rules for years, with great success.

The participants in the flamewar on the forum where it was posted were not operating under these rules.

Less Wrong posters generally do operate under these rules, at least here.

The list of rules is on pg. 2 - a good example is the rule titled "You cannot read minds":

As soon as you find someone espousing seemingly contradictory positions, you should immediately suspect yourself of being mistaken as to their intent. Even if it seems obvious to you that the person has a certain intent in their message, if you want to engage them, you must respond being open to the possibility that where you see contradictions (or, for that matter, insults), none were intended. While you keep in mind what the person’s contradictory position seems to be, raise your standards some, and ask questions so that the person must state the position more explicitly—this way, you can make sure whether they actually hold it. If you still have problems, keep raising your standards, and asking more specific questions, until the person starts making sense to you.

If part of their position is unclear or ambiguous to you, say that explicitly. Being willing to show uncertainty is an excellent way to defuse the person’s, and your own, defensiveness. It also helps them to more easily understand which aspects of their position they are not making clear enough.

The less their position makes sense to you, the more you should rely on interrogative phrase and the less on declarative. Questions defuse defensiveness and are much more pointed and communicative than statements, because they force you to think more about the person’s arguments, and to really articulate

...
Book reviews
by PhilGoetz — 2011-04-14T13:50:36.445Z

I'd like to see book reviews of books of interest to LW. Some suggestions:

Dan Ariely (2010). The Upside of Irrationality: The unexpected benefits of defying logic at work and at home.

Sam Harris (2010). The Moral Landscape: How science can determine human values.

Dan Ariely (2009). Predictably Irrational: The Hidden Forces That Shape Our Decisions.

Timothy Harris (2010). The Science of Liberty: Democracy, Reason, and the Laws of Nature.

Joel Garreau (2005). Radical Evolution. Book about genetic mods, intelligence enhancement, and the singularity.

ADDED: I don't mean I'd like to see reviews in this thread. I'd like each review to have its own thread. In discussion or on the "new" page is up to you.
Three consistent positions for computationalists
by dfranke — 2011-04-14T13:15:55.175Z

Yesterday, as a followup to We are not living in a simulation, I posted Eight questions for computationalists in order to obtain a better idea of what exactly my computationalist critics were arguing. These were the questions I asked:

As it is used in the sentence "consciousness is really just computation", is computation:
a) Something that an abstract machine does, as in "No oracle Turing machine can compute a decision to its own halting problem"?
b) Something that a concrete machine does, as in "My calculator computed 2+2"?
c) Or, is this distinction nonsensical or irrelevant?

If you answered "a" or "c" to question 1: is there any particular model, or particular class of models, of computation, such as Turing machines, register machines, lambda calculus, etc., that needs to be used in order to explain what makes us conscious? Or, is any Turing-equivalent model equally valid?

If you answered "b" or "c" to question 1: unpack what "the machine computed 2+2" means. What is that saying about the physical state of the machine before, during, and after the computation?

Are you able to make any sense of the concept of "computing red"? If so, what does this mean?

As far as consciousness goes, what matters in a computation: functions, or algorithms? That is, does any computation that give the same outputs for the same inputs feel the same from the inside (this is the "functions" answer), or do the intermediate steps matter (this is the "algorithms" answer)?

Would an axiomatization (as opposed to a complete exposition of the implications of these axioms) of a Theory of Everything that can explain consciousness include definitions of any computational devices, such as "and gate"?

Would an axiomatization of a Theory of Everything that can explain consciousness mention qualia?

Are all computations in some sense conscious, or only certain kinds?

I got some interesting answers to these questions, and from them I can extract three distinct positions that seem consistent to me.

Consistent Position #1: Qualia skepticism

Perplexed asserted this position in no uncertain terms. Here's my unpacking of it:

"Qualia do not exist. The things that you're confused about and are mistaking for qualia can be made clear to you using an argument phrased in terms of computation. When you talk about consciousness, I think I can understand your meaning, but you aren't referring ...
Specific Fiction Discusion (April 2011)
by Armok_GoB — 2011-04-14T12:29:45.271Z

Seeing some recent comments on my links comment, I think this thread might be warranted.

This is a thread for discussing specific works of fiction; books, movies, TV shows, webcomics, fanfictions, whatever. It's purpose is to provide a rationality perspective on shows that are not necessarily aimed at rationalists (but by the correlation of target audience I predict many of them might be anyway...)

To keep this organized, please follow these guidlines when posting; Top level coments shuld with NO exception (I'll make a single meta comment where discussion about this thread itself can go) fit into one of the following templates:

For a single work, the top level comment should consist of the full title, a link to where the work can be found online if applicable, and the TV tropes page for it OR a short description ONLY if there is no TV tropes page for it.

For certain authors that have written a lot of books popular on LW, such as for example Vernor Vinge, discussion of each one might tend to dominate the thread, therefore there should be one post for ALL the works of such authors, and they can be made entire own threads if discussion grows to big for that. The format for these comments is: Authors name, link to their wikipedia page (or homepage if they don't have a wikipedia page), and a short bibliography to make it easier to avoid making separate top level comments for their books.

Also, pleas refrain from discussing things written by Eliezer or otherwise already having a discussion space on LW, for similar reasons you should avoid discussing a certain institute and because it'd be redundant.

If this thread grows large and popular, I'm thinking this might become a monthly thing, hence the (April) part.
Might whole brain emulation require quantum-level emulation?
by lukeprog — 2011-04-14T06:12:37.620Z

Most experts seem to think that whole brain emulation will not require emulation down to the quantum level, but perhaps at the level of atoms or even molecules, either of which is far more computationally tractable than quantum-level brain emulation. Those who think quantum-level emulation will be required for whole brain emulation are often considered to be cranks.

However, it is worth noting that a few biological processes - including photosynthesis - have recently been found [excellent video lecture] to depend on the particularities of quantum phenomena. This lends no support to Penrose's views on quantum phenomena and the brain, but it may not be so crankish after all to suppose that whole brain emulation may require emulation down to the quantum level.

Thoughts?

Links:

On photosynthesis, see Fleming's papers on the topic.

Here is the quantum bird navigation paper.

Here is some coverage on Turin's controversial theory of quantum smell.
Gödel and Bayes: quick question
by hairyfigment — 2011-04-14T06:12:05.618Z

Kurt Gödel showed that we could write within a system of arithmetic the statement, "This statement has no proof within the system," in such a way that we couldn't dismiss it as meaningless. This proved that if the system (or part of it) could prove the logical consistency of the whole, it would thereby contradict itself. We nevertheless think arithmetic does not contradict itself because it never has.

From what I understand we could write a version of the Gödel statement for the axioms of probability theory, or even for the system that consists of those axioms plus our current best guess at P(axioms' self-consistency). Edit: or not. According to the comments the Incompleteness Theorem does not apply until you have a stronger set of assumptions than the minimum you need for probability theory. So let's say you possess the current source code of an AGI running on known hardware. It's just now reached the point where it could pass the test of commenting extensively on LW without detection. (Though I guess we shouldn't yet assume this will continue once the AI encounters certain questions.) For some reason it tries to truthfully answer any meaningful question. (So nobody mention the Riemann hypothesis. We may want the AI to stay in this form for a while.) Whenever an internal process ends in a Jaynes-style error message that indicates a contradiction, the AI takes this as strong evidence of a contradiction in the relevant assumptions. Now according to my understanding we can take the source code and ask about a statement which says, "The program will never call this statement true." Happily the AI can respond by calling the statement "likely enough given the evidence." So far so good.

So, can we or can't we write a mathematically meaningful statement Q saying, "The program will never say 'P(Q)≥0.5'"? What about, "The program will never call 'P(Q)≥0.5' true (or logically imply it)"? How does the AI respond to questions about variations of these statements?

It seems as if we could form a similar question by modifying the Halting-Oracle Killer program to refute more possible responses to the question of its run-time, assuming the AI will know this simpler program's source code. Though it feels like a slightly different problem because we'd have to address a lot of possible responses directly – with the previous examples, if the AI doesn't kill us...
Arational quotes
by Dr_Manhattan — 2011-04-14T00:58:40.264Z

Jacobi, an eminent algebraist, was fond of saying: “Invert, always Invert”

- Charlie Munger

So based on the above and the regular "rationality quotes" feature I figured why not try a thread for quotes that are not rational? Obviously we do not plain stupidity, but I think quotes that appeal on first listen would qualify. As a side benefit of analyzing specific errors this might serve as an inoculation from rhetoric.

I am using "arational" rather than irrational to highlight this distinction.

PS. Please feel free to reply with meta comments on whether this is a valuable idea.