trentmkelly/LessWrong-43k huggingface dataset:
The fallacy of work-life compartmentalization
by Morendil — 2010-03-04T22:59:01.986Z

Related to: Outside the Laboratory, Ghosts in the Machine

We've all observed how people can be very smart in some contexts, and stupid in others. People compartmentalize, which has been previously hypothesized as the reason for some epic failures to understand things that should be obvious.

It's also important to remember that we are not immune. To that end, I want to start off by considering some comfortable examples, where someone else is the butt of the joke, and then consider examples which might make you more uneasy.

"The mere presence of a computer can short circuit normally intelligent people's brains." -- Computer Stupidities

The reassuring cases concern smart people who become stupid when confronted with our area of expertise. If you're a software developer, that tends to be people who can't figure out something basic about Windows. "I've tried closing the app and restarting, and I've tried rebooting, and it doesn't work, I still can't find my file." You take a deep breath, refrain from rolling your eyes and asking what the heck their mental model is, what they think closing-and-restarting has to do with a misplaced file, and you go looking for some obvious places, like the Desktop, where they keep all their files but somehow neglected to look this time. If it's not there, chances are it will be in My Documents.

It's sometimes draining to be called on for this kind of thing, but it can be reassuring. My dad is a high calibre mathematician, dealing in abstractions at a level that seems stratospheric compared to my rusty-college-math. But we sometimes get into conversations like the above, and I get a slightly guilty self-esteem boost from them.

Now, the harder question: how do we compartmentalize?

I propose work-life compartmentalization as a case study. "Work-life balance" is how we rationalize that separation. It's OK, we think, to put up with some unpleasantness from 9 to 5, as long as we can look forward to getting home, kicking our shoes off and relaxing, alone or among family or friends. And perhaps that's reasonable enough.

But this logic leads many people to tolerate: stress, taking orders, doing work that we think is meaningless, filling out paperwork that will never actually be read, pouring our energy into projects we're certain are failure-bound but never speaking up about that to avoid being branded "not a team player", being bored in endles...
The Graviton as Aether
by alyssavance — 2010-03-04T22:13:59.800Z

Well, first: Does any collapse theory have any experimental support? No.

With that out of the way...

If collapse actually worked the way its adherents say it does, it would be:

The only non-linear evolution in all of quantum mechanics.

The only non-unitary evolution in all of quantum mechanics.

The only non-differentiable (in fact, discontinuous) phenomenon in all of quantum mechanics.

The only phenomenon in all of quantum mechanics that is non-local in the configuration space.

The only phenomenon in all of physics that violates CPT symmetry.

The only phenomenon in all of physics that violates Liouville's Theorem (has a many-to-one mapping from initial conditions to outcomes).

The only phenomenon in all of physics that is acausal / non-deterministic / inherently random.

The only phenomenon in all of physics that is non-local in spacetime and propagates an influence faster than light.

WHAT DOES THE GOD-DAMNED COLLAPSE POSTULATE HAVE TO DO FOR PHYSICISTS TO REJECT IT? KILL A GOD-DAMNED PUPPY?

- Eliezer Yudkowsky, Collapse Postulates

In the olden days of physics, circa 1900, many prominent physicists believed in a substance known as aether. The principle was simple: Maxwell's equations of electromagnetism had shown that light was a wave, and light followed many of the same equations as sound waves and water waves. However, every other kind of wave- sound waves, water waves, waves in springs- needs some sort of medium for its transmission. A "wave" is not really a physical object; it is just a disturbance of some other substance. For instance, if you throw a rock into a pond, you cannot pluck the waves out of the pond and take them home with you in your backpack, because the "waves" are just peaks and troughs in the puddle of water (the medium). Hence, there should be some sort of medium for light waves, and the physicists named this medium "aether".

However, difficulties soon developed. If you have a jar, you can pump the air out of the jar, and then the jar will no longer transmit sound, demonstrating that the wave medium (the air) has been removed. But, there was no way to remove the aether from a jar; no matter what the experimentalists did, you could still shine light through it. There was, in fact, no way of detecting, altering, or experimenting with aether at all. Physicists knew that aether must be unlike all other matter, because it co...
Priors and Surprise
by MichaelVassar — 2010-03-03T08:27:53.554Z

I don’t want to be too dogmatic about this claim, but Godzilla is unrealistic. I don’t want to be too non-dogmatic about this claim either. OK then, just how dogmatic should I be? I have all sorts of reasons for thinking that skyscraper sized lizards or dinosaurs don’t actually exist. Honestly, the most important of these is probably that none of the people who I imagine would know if they did exist seem to believe in them. I never hear any mention of them in the news, in history books, etc, and I don’t see their effects in the national death statistics. No industries seem to exist to deal with their rampages, and no oil or shipping companies lose stock value from lizard attacks. Casually, at least, Godzilla attacks don’t seem like the sort of basic fact about the world that people could just overlook. How confident should I be that Godzilla type creatures don't exist?

I can also fairly easily recognize good biological reasons not to expect there to be giant rampaging lizards. The square/cube law, in its many manifestations, is the most basic of these, but by itself is not completely decisive. I can imagine physical workarounds that would allow sequoia giganticus sized reptiles, but not without novel bio-machinery that would take a long time to evolve and would surely be found in many other organisms. I can even vaguely imagine ways in which biology might prove resistant to conventional military weaponry and ecological niches and lifestyles that might support both such biology and such size, though much of my knowledge of Earth’s ecosystems would have to be re-written. For all that, if I lived in a world where essentially all authorities did refer to the activities of godzilla giganticus I would probably accept that they were probably correct regarding its existence. What should a hypothetical person who lived in a world where the existence of Godzilla type creatures was common knowledge and was regarded as an ordinary non-numinous fact about the world believe?

Godzilla would be considerably more perplexing than thunderstones, and would have to be considerably better documented to be credible. Even with the strongest documentation I would have substantial unresolved questions, inferring that Godzilla&rs

...
Interesting Peter Norvig interview
by xamdam — 2010-03-03T00:59:04.214Z

(Sorry this is mostly a link instead of a post, but I think it will interesting to the FAI folks here)

I helped arrange this interview with Peter Norvig:

http://www.reddit.com/r/blog/comments/b8aln/peter_norvig_answers_your_questions_ask_me/

I think the answer to the AGI question 4 is telling, but judge for yourself. (BTW, the 'components' Peter referred to are probabilistic relational learning and hierarchical modeling. He singled these two in his singularity summit talk)
Individual vs. Group Epistemic Rationality
by Wei Dai — 2010-03-02T21:46:11.930Z

It's common practice in this community to differentiate forms of rationality along the axes of epistemic vs. instrumental, and individual vs. group, giving rise to four possible combinations. I think our shared goal, as indicated by the motto "rationalists win", is ultimately to improve group instrumental rationality. Generally, improving each of these forms of rationality also tends to improve the others, but sometimes conflicts arise between them. In this post I point out one such conflict between individual epistemic rationality and group epistemic rationality.

We place a lot of emphases here on calibrating individual levels of confidence (i.e., subjective probabilities), and on the idea that rational individuals will tend to converge toward agreement about the proper level of confidence in any particular idea as they update upon available evidence. But I argue that from a group perspective, it's sometimes better to have a spread of individual levels of confidence about the individually rational level. Perhaps paradoxically, disagreements among individuals can be good for the group.

A background fact that I start with is that almost every scientific ideas that humanity has ever come up with has been wrong. Some are obviously crazy and quickly discarded (e.g., every perpetual motion proposal), while others improve upon existing knowledge but are still subtly flawed (e.g., Newton's theory of gravity). If we accept that taking multiple approaches simultaneously is useful for solving hard problems, then upon the introduction of any new idea that is not obviously crazy, effort should be divided between extending the usefulness of the idea by working out its applications, and finding/fixing flaws in the underlying math, logic, and evidence.

Having a spread of confidence levels in the new idea helps to increase individual motivation to perform these tasks. If you're overconfident in an idea, then you would tend to be more interested in working out its applications. Conversely, if you're underconfident in it (i.e., are excessively skeptical), you would tend to work harder to try to find its flaws. Since scientific knowledge is a public good, individually rational levels of motivation to produce it are almost certainly too low from a social perspective, and so these individually irrational increases in motivation would tend to increase group rationality.

Even amongst ...
Meetup: Bay Area: Sunday, March 7th, 7pm
by JustinShovelain — 2010-03-02T21:18:28.332Z

Overcoming Bias / Less Wrong meetup in the San Francisco Bay Area at SIAI House on March 7th, 2010, starting at 7PM.

Eliezer Yudkowsky, Alicorn, and Michael Vassar will be present.

Some other extra guests - Wei Dai, Stuart Armstrong, and Nick Tarleton - will be also be there, following our short Decision Theory mini-workshop.
For progress to be by accumulation and not by random walk, read great books
by MichaelVassar — 2010-03-02T08:11:51.034Z

This recent blog post strikes me as an interesting instance of a common phenomenon. The phenomenon looks like the following; an intellectual, working within the assumption that the world is not mad, (an assumption not generally found outside of the Anglo-American Enlightenment intellectual tradition) notices that some feature of the world would only make sense if the world was mad. This intellectual responds by denouncing as silly one of the few features of this vale of tears to be, while not intelligently designed, at least structured by generalized evolution rather than by entropy. The key line in the post is

"Conversely in all those disciplines where we have reliable quantatative measurements of progress (with the obvious exception of history) returning to the original works of past great thinkers is decidedly unhelpful."

I agree with the above statement, and find that the post makes a compelling argument for it. My only caveat is that we essentially never have quantitative measures of progress. Even in physics, when one regards not the theory but the technique of actually doing physics, tools and modes of thought rise and fall for reasons of fashion, and once widespread techniques that remain useful fall into disuse.

Other important techniques, like the ones used to invent calculus in the first place, are never adequately articulated by those who use them and thus never come into general use. One might argue that Newton didn't use any technique to invent calculus, just a very high IQ or some other unusual set of biological traits. This, however, doesn't explain why a couple of people invented calculus at about the same time and place, especially given the low population of that time and place compared to the population of China over the many centuries when China was much more civilized than Europe.

It seems likely to me that in cases like the invention of calculus, looking at the use of such techniques can contribute to their development in at least crude form. By analogy, even the best descriptions of how to do martial arts are inadequate to provide expertise without practice, but experience watching experts fight is a valuable complement to training by the relatively inept. If one wants to know the Standard Model, sure, study it directly, but if you want to actually understan...
Rationality quotes: March 2010
by Morendil — 2010-03-01T10:26:30.434Z

This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote comments/posts on LW/OB.

No more than 5 quotes per person per monthly thread, please.
Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future
by inklesspen — 2010-03-01T02:32:33.652Z

It is the fashion in some circles to promote funding for Friendly AI research as a guard against the existential threat of Unfriendly AI. While this is an admirable goal, the path to Whole Brain Emulation is in many respects more straightforward and presents fewer risks. Accordingly, by working towards WBE, we may be able to "weight" the outcome probability space of the singularity such that humanity is more likely to survive.

One of the potential existential risks in a technological singularity is that the recursively self-improving agent might be inimical to our interests, either through actual malevolence or "mere" indifference towards the best interests of humanity. Eliezer has written extensively on how a poorly-designed AI could lead to this existential risk. This is commonly termed Unfriendly AI.

Since the first superintelligence can be presumed to have an advantage over any subsequently-arising intelligences, Eliezer and others advocate funding research into creating Friendly AI. Such research must not only reverse-engineer consciousness, but also human notions of morality. Unfriendly AI could potentially require only sufficiently fast hardware to evolve an intelligence via artificial life, as depicted in Greg Egan's short story "Crystal Nights", or it may be created inadvertently by researchers at the NSA or a similar organization. It may be that creating Friendly AI is significantly harder than creating Unfriendly (or Indifferent) AI, perhaps so much so that we are unlikely to achieve it in time to save human civilization.

Fortunately, there's a short-cut we can take. We already have a great many relatively stable and sane intelligences. We merely need to increase their rate of self-improvement. As far as I can tell, developing mind uploading via WBE is a simpler task than creating Friendly AI. If WBE is fast enough to constitute an augmented intelligence, then our augmented scientists can trigger the singularity by developing more efficient computing devices. An augmented human intelligence may have a slower "take-off" than a purpose-built intelligence, but we can reasonably expect it to be much easier to ensure such a superintelligence is Friendly. In fact, this slower take-off will likely be to our advantage; it may increase our odds of being able to abort an Unfriendly singularity.

WBE may also be able to provide us with useful insights into the na...
Great Product. Lousy Marketing.
by BenAlbahari — 2010-02-28T09:33:58.318Z

The product of Less Wrong is truth. However, there seems to be a reluctance of the personality types here - myself included - to sell that product. Here's my evidence:

Yvain said: But the most important reason to argue with someone is to change his mind. ... I make the anecdotal observation that a lot of smart people are very good at winning arguments in the first sense [(logic)], and very bad at winning arguments in the second sense [(persuasion)]. Does that correspond to your experience?

Eliezer said: I finally note, with regret, that in a world containing Persuaders, it may make sense for a second-order Informer to be deliberately eloquent if the issue has already been obscured by an eloquent Persuader - just exactly as elegant as the previous Persuader, no more, no less. It's a pity that this wonderful excuse exists, but in the real world, well...

Robin Hanson said: So to promote rationality on interesting important topics, your overwhelming consideration simply must be: on what topics will the world’s systems for deciding who to hear on what listen substantially to you? Your efforts to ponder and make progress will be largely wasted if you focus on topics where none of the world’s “who to hear on what” systems rate you as someone worth hearing. You must not only find something worth saying, but also something that will be heard.

We actually label many highly effective persuasive strategies that can be used to market our true ideas as "dark arts". What's the justification for this negative branding? A necessary evil is not evil. Even if - and this is a huge if - our future utopia is free of dark arts, that's not the world we live in today. Choosing not to use them is analogous to a peacenik wanting to rid the world of violence by suggesting that police not use weapons.

We treat our dislike of dark arts as if it's a simple corollary of the axiom of the virtue of truth. Does this mean we assume the ends (more people believe the truth) doesn't justify the means (persuasion to the truth via exploiting cognitive biases)? Or are we just worried about being hypocrites? Whatever the reason, such an impactful assumption deserves an explanation. Speaking practically, the successful practice of dark arts requires the psychological skill of switching hats, to use Edward de Bono's terminology. While po...
Splinters and Wooden Beams
by knb — 2010-02-28T08:26:27.931Z

I recently told a friend that I was planning to write (and post online) a paper that rigorously refutes every argument1 I’ve ever heard that homosexuality is inherently immoral. The purpose of this effort was to provide a handy link for people who want to persuade family members or friends who are marginal believers of the homosexuality-is-immoral theory. As a key part of this effort, I intended to demonstrate that the predominant religious arguments against homosexuality cause contradictions within the religion. For example, the tortured reasoning of the Roman Catholic Church2 goes like this:

Sex without marriage is forbidden.

Marriage is only for those who are “open to natural reproduction”.

Gays can’t reproduce (in an acceptably “natural” way) and therefore gay sex is not “open to reproduction”.

Since gays cannot be open to reproduction, they cannot marry.

Since they cannot marry, they can’t have sex.

This argument seems to be logically valid, if you accept the insane assumptions. Bizarrely, though, the Catholic Church also recommends a practice called "Natural Family Planning", in which married couples who want to prevent pregnancy have sex only when the woman is believed to be infertile! To be consistent, the Catholic Church would have to oppose such deliberate efforts to prevent natural reproduction.

My paper was going to be full of little examples like this, of how opposing homosexuality leads to contradictions within Christian Virtue Ethics, established interpretations of the Koran, or whatever. However, my friend told me that he thought my efforts were misguided. Why try curing these folks of the splinter of intolerance, when they still have the wooden beam3 of theism in their eyes?

After all, if someone you know is planning to quit her job and move to Alaska because her horoscope told her that Tauruses need more spontaneity, you shouldn't tell her to stay because she's actually an Aries. You tell her to stay because astrology is provably bogus.

I'm uncertain. Most of those wooden beams are staying right where they are for the foreseeable future. But attitudes toward homosexuality are changing relatively quickly. On the other hand, there is something to be said for striking at the root of the problem. Overall, I'm leaning toward making these smaller arguments instead of trying to convert people to...
Mental Crystallography
by Alicorn — 2010-02-27T01:04:54.333Z

Brains organize things into familiar patterns, which are different for different people. This can make communication tricky, so it's useful to conceptualize these patterns and use them to help translation efforts.

Crystals are nifty things! The same sort of crystal will reliably organize in the same pattern, and always break the same way under stress.

Brains are also nifty things! The same person's brain will typically view everything through a favorite lens (or two), and will need to work hard to translate input that comes in through another channel or in different terms. When a brain acquires new concepts - even really vital ones - the new idea will result in recognizeably-shaped brain-bits. Different brains, therefore, handle concepts differently, and this can make it hard for us to talk to each other.

This works on a number of levels, although perhaps the most obvious is the divide between styles of thought on the order of "visual thinker", "verbal thinker", etc. People who differ here have to constantly reinterpret everything they say to one another, moving from non-native mode to native mode and back with every bit of data exchanged. People also store and retrieve memories differently, form first-approximation hypotheses and models differently, prioritize sensory input differently, have different levels of introspective luminosity1, and experience different affect around concepts and propositions. Over time, we accumulate different skills, knowledge, cognitive habits, shortcuts, and mental filing debris. Intuitions differ - appeals to intuition will only convert people who share the premises natively. We have lots in common, but high enough variance that it's impressive how much we do manage to communicate over not only inferential distances, but also fundamentally diverse brain plans. Basically, you can hit two crystals the same way with the same hammer, but they can still break along different cleavage planes.

This phenomenon is a little like man-with-a-hammer syndrome, which is why I chose that extension of my crystal metaphor. But a person's dependence on their mental crystallography, unlike their wanton use of their hammer, rarely seems to diminish with time. (In fact, youth probably confers some increased flexibility - it seems that you can probably train children to have d...