trentmkelly/LessWrong-43k huggingface dataset:
What is missing from rationality? 
by Roko — 2010-04-27T12:32:06.806Z

"In a sufficiently mad world, being sane is actually a disadvantage"

– Nick Bostrom

Followup to: What is rationality?

A canon of work on "rationality" has built up on Less Wrong; in What is rationality?, I listed most of the topics and paradigms that have been used extensively on Less Wrong, including: simple calculation and logic1, probability theory, cognitive biases, the theory of evolution, analytic philosophical thinking, microeconomics. I defined "Rationality" to be the ability to do well on hard decision problems, often abbreviated to "winning" - choosing actions that cause you to do very well.

However, I think that the rationality canon here on Less Wrong is not very good at causing the people who read it to actually do well at most of life's challenges. This is therefore a criticism of the LW canon.

If the standard to judge methods by is whether they give you the ability to do well on a wide range of hard real-life decision problems, with a wide range of terminal values being optimized for, then Less-Wrong-style rationality fails, because the people who read it seem to mostly only succeed at the goal that most others in society would label as "being a nerd".2 We don't seem to have a broad range of people pursuing and winning at a broad range of goals (though there are a few exceptional people here).

Although the equations of probability theory and expected utility do not state that you have to be a "Spock rationalist" to use them, in reality I see more Spock than Kirk. I myself am not exempt from this critique.

What, then, is missing?

The problem, I think, is that the original motivation for Less Wrong was the bad planning decisions that society as a whole takes3. When society acts, it tends to benefit most when it acts in what I would call the Planning model of winning, where reward is a function of the accuracy of beliefs and the efficacy of explicitly reasoned plans.

But individuals within a society do not get their rewards solely based upon the quality of their plans: we are systematically rewarded and punished by the environment around us by:

Our personality traits and other psychological factors such as courage, happiness set-point, self-esteem, etc.

The group we are a member of, especially our close friends and associates.

Our skill in dealing with people, which we might call "emotional intelligence".

The shibboleths we di

...
Attention Less Wrong: We need an FAQ
by Kevin — 2010-04-27T10:06:48.818Z

Less Wrong is extremely intimidating to newcomers and as pointed out by Academian something that would help is a document in FAQ form intended for newcomers. Later we can decide how to best deliver that document to new Less Wrongers, but for now we can edit the existing (narrow) FAQ to make the site less scary and the standards more evident.

Go ahead and make bold edits to the FAQ wiki page or use this post to discuss possible FAQs and answers in agonizing detail.
Proposed New Features for Less Wrong
by alyssavance — 2010-04-27T01:10:39.138Z

Followup to: Announcing the Less Wrong Sub-Reddit

After the recent discussion about the Less Wrong sub-reddit, me and Less Wrong site designer Matthew Fallshaw have been discussing possible site improvements, and ways to implement them. As far as I can tell, the general community consensus in the previous post was that a discussion section to replace the Open Thread would be a good idea, due to the many problems with Open Thread, but that it would be problematic to host it off-site. For this reason, our current proposal involves modifying the main site to include a separate "Discussion" section in the navigation bar (next to "Wiki | Sequences | About"). What are now Open Thread comments would be hosted in the Discussion section, in a more user-friendly and appropriate format (similar to Reddit's or a BBS forum's). If my impression was mistaken, please do say so. (If you think that this is a great idea, please do say so as well, to avoid Why Our Kind Can't Cooperate.)

We have also identified another potential problem with the site: the high quality standard, heavy use of neologisms, and karma penalties for being wrong might be intimidating to newcomers. To help alleviate this, after much discussion, we have come up with two different proposals. (To avoid bias, I'm not going to say which one is mine and which one is Matthew's.)

- Proposal 1: Posts submitted to Less Wrong can be tagged with a "karma coward" option. Such posts can still be voted on, but votes on them will have no effect on a user's karma total. There will be a Profile option to hide "karma coward" posts from view.

- Proposal 2: A grace period for new users. Votes on comments from new users will have no effect on that user's karma total for a certain period of time, like two weeks or a month.

- Proposal 3: Do nothing; the site remains as-is.

To see what the community consensus is, I have set up a poll here: http://www.misterpoll.com/polls/482996. Comments on our proposals, and alternative proposals, are more than welcome. (To avoid clogging the comments, please do not simply declare your vote without explaining why you voted that way.)

EDIT: Posts and comments in the discussion section would count towards a user's karma total (not withstanding the implementation of proposal 1 and proposal 2), although posts would only earn a user 1 karma per upvote instead of 10.

EDIT 2: To avoid contamination by other p...
Only humans can have human values
by PhilGoetz — 2010-04-26T18:57:38.942Z

Ethics is not geometry

Western philosophy began at about the same time as Western geometry; and if you read Plato you'll see that he, and many philosophers after him, took geometry as a model for philosophy.

In geometry, you operate on timeless propositions with mathematical operators. All the content is in the propositions. A proof is equally valid regardless of the sequence of operators used to arrive at it. An algorithm that fails to find a proof when one exists is a poor algorithm.

The naive way philosophers usually map ethics onto mathematics is to suppose that a human mind contains knowledge (the propositional content), and that we think about that knowledge using operators. The operators themselves are not seen as the concern of philosophy. For instance, when studying values (I also use "preferences" here, as a synonym differing only in connotation), people suppose that a person's values are static propositions. The algorithms used to satisfy those values aren't themselves considered part of those values. The algorithms are considered to be only ways of manipulating the propositions; and are "correct" if they produce correct proofs, and "incorrect" if they don't.

But an agent's propositions aren't intelligent. An intelligent agent is a system, whose learned and inborn circuits produce intelligent behavior in a given environment. An analysis of propositions is not an analysis of an agent.

I will argue that:

The only preferences that can be unambiguously determined are the preferences people implement, which are not always the preferences expressed by their beliefs.

If you extract a set of propositions from an existing agent, then build a new agent to use those propositions in a different environment, with an "improved" logic, you can't claim that it has the same values.

Values exist in a network of other values. A key ethical question is to what degree values are referential (meaning they can be tested against something outside that network); or non-referential (and hence relative).

Supposing that values are referential helps only by telling you to ignore human values.

You cannot resolve the problem by combining information from different behaviors, because the needed information is missing.

Today's ethical disagreements are largely the result of attempting to extrapolate ancestral human values i

...
Who is your favorite rationalist?
by xamdam — 2010-04-25T14:56:39.340Z

Light reading about 'Rationalist Heroes'.

I am not sure how useful people find having personal heroes. I would argue that they are definitely useful for children. Perhaps I haven't really grown up enough yet (growing up without a father possibly contributed), but I like to have some people in my head I label as "I wonder what would X think about this". Many times they've set me straight through their ideas. Other times I've had to reprimand them, though unfortunately they never get the memo.

One living example is Charlie Munger.

He was an early practical adopter of the cognitive biases framework, and moreover he clearly put it into context of "something to protect":

"not understanding human misjudgment was reducing my ability to help everything I loved"

(The quote is from his talk on "Misjudgment" which is worth reading on its own http://vinvesting.com/docs/munger/human_misjudgement.html)

One interesting point is that Charlie is seemingly a Christian. I have a deep suspicion that he believes that religion is valuable, for the time, as a payload delivering mechanism.

“Economic systems work better when there’s an extreme reliability ethos. And the traditional way to get a reliability ethos, at least in past generations in America, was through religion. The religions instilled guilt. … And this guilt, derived from religion, has been a huge driver of a reliability ethos, which has been very helpful to economic outcomes for man.”

Also, judge for yourself from his recommended reading list - looks like something out of an Atheist's Bookshelf.

Charlie Munger's reading recommendations

There might also be other reasons, family or whatever, that help prop up the religious appearance. I myself am still wearing a yarmulke for this category of reasons. Whatever it is, Munger is no trinity worshiper.

Another interesting thing is that it is clear today's Berkshire Hathaway was Buffett and Munger's joint venture, and most likely would not succeed in the same way without Munger. I've done a fair amout of reading on the BRK's investment strategy at one point, but cannot find the appropriate supporting quote at the moment. Basically Munger steered Buffett away from just going after underpriced 'crap' companies ('cigar butts') that Buffett 1) already did well with 2) were recommended as the only approach by Ben Graham, who taught Buffett how to invest and was pe

...
Report from Humanity+ UK 2010
by Roko — 2010-04-25T12:33:33.170Z

“Theosophists have guessed at the awesome grandeur of the cosmic cycle wherein our world and human race form transient incidents. They have hinted at strange survival in terms which would freeze the blood if not masked by a bland optimism.”

– H.P. Lovecraft on transhumanism

Just thought I'd write a quick post to sum up the H+ UK conference and subsequent LW meetup attended by myself, ciphergoth, JulianMorrison, Leon and a few other LW lurkers. My thanks to David Wood for organizing the conference, and Anders Sandberg for putting me up/putting up with me the night before.

I made a poster giving a quick introduction to “Cognitive Bias and Futurism”, which I will put up on my website shortly. The LW crowd met up as advertised – we discussed the potential value of spreading the rationality message to the H+ community.1
One idea was for someone (possibly me) to do a talk at UKH+ on “Rationality and Futurism”, and to get the UK transhumanist crowd involved and on board somewhat more. The NYC Less Wrong guys seem to be doing remarkably well with a meetup group, about a billion members, a group house (?) – do you have any advice for us?

The talks were interesting and provocative – of particular note were:

Aubrey De Grey's talk which was far from his usual fare. He tailored his talk to suit the H+ audience who are more familiar with SENS, and spoke about the pace of recent advances in induced pluripotency in stem cells, progress in migrating mitochondrial DNA from the mitochondria to the cell nucleus, and how the mainstream media can blow a minor paper with a sexy title to undeserved levels of fame whilst passing over much more important but “technical” sounding work. I asked what probability he assigned to his SENS program working, but he did not give an estimate. I think that rationalists could potentially help the life-extension movement by publishing an independent, critical review of the probability of the SENS program succeeding, perhaps in an economics journal.

Nick Bostrom added the definition of “capability potential” and “axiological potential” to his usual existential risk meme.

Other talks were interesting, but showed a lack of appreciation of rationalist methods. For example, though many talks made predictions about the future, only Bostrom's talk gave any pr

...
Navigating disagreement: How to keep your eye on the evidence 
by AnnaSalamon — 2010-04-24T22:47:41.096Z

Heeding others' impressions often increases accuracy. But "agreement" and "majoritarianism" are not magic; in a given circumstance, agreement is or isn't useful for *intelligible* reasons.

You and four other contestants are randomly selected for a game show. The five of you walk into a room. Each of you is handed a thermometer drawn at random from a box; each of you, also, is tasked with guessing the temperature of a bucket of water. You’ll each write your guess at the temperature on the card; each person who is holding a card that is within 1° of the correct temperature will win $1000.

The four others walk to the bucket, place their thermometers in the water, and wait while their thermometers equilibrate. You follow suit. You can all see all of the thermometers’ read-outs: they’re fairly similar, but a couple are a degree or two off from the rest. You can also watch, as each of your fellow-contestants stares fixedly at his or her own thermometer and copies its reading (only) onto his or her card.

Should you:

Write down the reading on your own thermometer, because it’s yours;

Write down an average* thermometer reading, because probably the more accurate thermometer-readings will cluster;

Write down an average of the answers on others’ cards, because rationalists should try not to disagree;

Follow the procedure everyone else is following (and so stare only at your own thermometer) because rationalists should try not to disagree about procedures?

Choice 2, of course. Thermometers imperfectly indicate temperature; to have the best possible chance of winning the $1000, you should consider all the information you have, from all the (randomly allocated, and so informationally symmetric) thermometers. It doesn’t matter who was handed which thermometer.

Forming accurate beliefs is *normally* about this simple. If you want the most accurate beliefs you can get, you’ll need to pay attention to the evidence. All of the evidence. Evenly. Whether you find the evidence in your hand or mind, or in someone else’s. And whether weighing all the evidence evenly leaves you with an apparently high-status social claim (“My thermometer is better than yours!”), or an apparently deferential social claim (&ldqu...
The role of mathematical truths
by SilasBarta — 2010-04-24T16:59:28.316Z

Related to: Math is subjunctively objective, How to convince me that 2+2=3

Elaboration of points I made in these comments: first, second

TL;DR Summary: Mathematical truths can be cashed out as combined claims about 1) the common conception of the rules of how numbers work, and 2) whether the rules imply a particular truth. This cashing-out keeps them purely about the physical world and eliminates the need to appeal to an immaterial realm, as some mathematicians feel a need to.

Background: "I am quite confident that the statement 2 + 3 = 5 is true; I am far less confident of what it means for a mathematical statement to be true." -- Eliezer Yudkowsky

This is the problem I will address here: how should a rationalist regard the status of mathematical truths? In doing so, I will present a unifying approach that, I contend, elegantly solves the following related problems:

- Eliminating the need for a non-physical, non-observable "Platonic" math realm.

- The issue of whether "math was true/existed even when people weren't around".

- Cashing out the meaning of isolated claims like "2+2=4".

- The issue of whether mathematical truths and math itself should count as being discovered or invented.

- Whether mathematical reasoning alone can tell you things about the universe.

- Showing what it would take to convince a rationalist that "2+2=3".

- How the words in math statements can be wrong.

This is an ambitious project, given the amount of effort spent, by very intelligent people, to prove one position or another regarding the status of math, so I could very well be in over my head here. However, I believe that you will agree with my approach, based on standard rationalist desiderata.

Here’s the resolution, in short: For a mathematical truth (like 2+2=4) to have any meaning at all, it must be decomposable into two interpersonally verifiable claims about the physical world:

1) a claim about whether, generally speaking, people's models of "how numbers work” make certain assumptions

2) a claim about whether those assumptions logically imply the mathematical truth (2+2=4)

(Note that this discussion avoids the more narrowly-constructed class of mathematical claims that take the form of saying that some admittedly arbitrary set of assumptions en...
23andme genome analysis - $99 today only
by  — 2010-04-23T19:33:10.312Z

I suspect this might interest some people here: for today only, 23andme is offering their full-package DNA testing for only 99 dollars (the normal price is $499).

23andme uses a genotyping process, which differs from a full gene-sequencing. From their website:

The DNA chip that we use genotypes hundreds of thousands of SNPs at one time. It actually reads 550,000 SNPs that are spread across your entire genome. Although this is still only a fraction of the 10 million SNPs that are estimated to be in the human genome, these 550,000 SNPs are specially selected "tag SNPs." Because many SNPs are linked to one another, we can often learn about the genotype at many SNPs at a time just by looking at one SNP that "tags" its group. This maximizes the information we can get from every SNP we analyze, while keeping the cost low.

In addition, we have hand-picked tens of thousands of additional SNPs of particular interest from the scientific literature and added their corresponding probes to the DNA chip. As a result, we can provide you personal genetic information available only through 23andMe.

I don't have any experience with 23andme (though I seem to recall them having some financial difficulties), but the price was low enough for me to order a test.

An article by Steven Pinker discussing his experience getting tested can be found here. This has also been linked on Hacker News.
Systems and stories
by KatjaGrace — 2010-04-22T15:25:53.000Z

Tyler claims we think of most things in terms of stories, which he says is both largely inevitable and one of our biggest biases.

He includes the abstractions of non fiction as ‘stories’, and recommends ‘messiness’ as a better organising principle for understanding our lives and other things. But the problems with stories that Tyler mentions apply mostly to narrative stories, not other abstractions such as scientific ‘stories’. It looks to me like we think about narrative stories and other abstractions quite differently, so should not lump them together. I suspect we would do better to shift more to thinking in terms of other abstractions than to focus on messiness, but I’ll get to that later. First, let me describe the differences between these styles of thought.

I will call the type of thought we use for narrative stories such as fiction and most social interactions ‘story thought’. I will call the style of thought we use for other abstractions ‘system thought’. This is what we use to think about maths for instance. They are both used by all people, but to different degrees on different topics.

Here are the differences between story thought and system thought I think I see, plus a few from Tyler. It’s a tentative list, so please criticize generously and give me more to add.

Agents

Role of agents

Stories are made out of agents, whereas systems are made out of the math and physics which is intuitive to us. Systems occasionally model agents, but in system thought agents are a pretty complex, obscure thing for a system to have. In story thought we expect everything to be intentional.

Perspective

Stories are usually from an agent’s perspective, systems are understood from an objective outside viewpoint. Even if a story doesn’t have a narrator, there is usually a protagonist or several, plus less detailed characters stretching off into the distance.

Unique identity

The agents that stories are made of always have unique identities, even if there is more than one with basically the same characteristics. In system thought units are interchangeable, except they may have varying quantities of generic parameters. ‘You’ are a set of preferences, a gender, an income level, a location, and some other things. In story thought, any ambiguity about whether someone is the same person as they used to be is a big issue, and the whole story is about w...
Too busy to think about life
by Academian — 2010-04-22T15:14:18.727Z

Many adults maintain their intelligence through a dedication to study or hard work. I suspect this is related to sub-optimal levels of careful introspection among intellectuals.

If someone asks you what you want for yourself in life, do you have the answer ready at hand? How about what you want for others? Human values are complex, which means your talents and technical knowledge should help you think about them. Just as in your work, complexity shouldn't be a curiosity-stopper. It means "think", not "give up now."

But there are so many terrible excuses stopping you...

Too busy studying? Life is the exam you are always taking. Are you studying for that? Did you even write yourself a course outline?

Too busy helping? Decision-making is the skill you are aways using, or always lacking, as much when you help others as yourself. Isn't something you use constantly worth improving on purpose?

Too busy thinking to learn about your brain? That's like being too busy flying an airplane to learn where the engines are. Yes, you've got passengers in real life, too: the people whose lives you affect.

Emotions too irrational to think about them? Irrational emotions are things you don't want to think for you, and therefore are something you want to think about. By analogy, children are often irrational, and no one sane concludes that we therefore shouldn't think about their welfare, or that they shouldn't exist.

So set aside a date. Sometime soon. Write yourself some notes. Find that introspective friend of yours, and start solving for happiness. Don't have one? For the first time in history, you've got LessWrong.com!

Reasons to make the effort:

Happiness is a pairing between your situation and your disposition. Truly optimizing your life requires adjusting both variables: what happens, and how it affects you.

You are constantly changing your disposition. The question is whether you'll do it with a purpose. Your experiences change you, and you affect those, as well as how you think about them, which also changes you. It's going to happen. It's happening now. Do you even know how it works? Put your intelligence to work and figure it out!

The road to harm is paved with ignorance. Using your capability to understand yourself and wha...