trentmkelly/LessWrong-43k huggingface dataset:
Image vs. Impact: Can public commitment be counterproductive for achievement?
by patrissimo — 2009-05-28T23:18:28.564Z

The traditional wisdom says that publicly committing to a goal is a useful technique for accomplishment. It creates pressure to fulfill one's claims, lest one lose status. However, when the goal is related to one's identity, a recent study shows that public commitment may actually be counterproductive. Nyuanshin posts:

"Identity-related behavioral intentions that had been noticed by other people were translated into action less intensively than those that had been ignored. . . . when other people take notice of an individual's identity-related behavioral intention, this gives the individual a premature sense of possessing the aspired-to identity."

-- Gollwitzer at al (2009)

This empirical finding flies in the face of conventional wisdom about the motivational effects of public goal-setting, but rings true to my experience. Belief is, apparently, fungible -- when you know that people think of you as an x-doer, you afffirm that self-image more confidently than you would if you had only your own estimation to go on. [info]colinmarshall and myself have already become aware of the dangers of vanity to any non-trivial endeavor, but it's nice to have some empirical corroboration. Keep your head down, your goals relatively private, and don't pat yourself on the back until you've got the job done.

This matches my experience over the first year of The Seasteading Institute. We've received tons of press, and I've probably spent as much time at this point interacting with the media as working on engineering. And the press is definitely useful - it helps us reach and get credibility with major donors, and it helps us grow our community of interested seasteaders (it takes a lot of people to found a country, and it takes a mega-lot of somewhat interested people to have a committed subset who will actually go do it).

Yet I've always been vaguely uncomfortable about how much media attention we've gotten, even though we've just started progressing towards our long-term goals. It feels like an unearned reward. But is that bad? I keep wondering "Why should that bother me? Isn't it a good thing to be given extra help in accomplishing this huge and difficult goal? Aren't unearned rewards the best kind of rewards?" This study suggests the answer.

My original goal was to actually s...
Link: The Case for Working With Your Hands
by Daniel_Burfoot — 2009-05-28T14:16:03.492Z

The NYTimes recently publised a long semi-autobiographical article written by Michael Crawford, a University of Chicago Phd graduate who is currently employed as a motorcycle mechanic. The article is partially a somewhat standard lament about the alienation and drudgery of modern corporate work. But it is also very much about rationality. Here's an excerpt:

As it happened, in the spring I landed a job as executive director of a policy organization in Washington. This felt like a coup. But certain perversities became apparent as I settled into the job. It sometimes required me to reason backward, from desired conclusion to suitable premise. The organization had taken certain positions, and there were some facts it was more fond of than others. As its figurehead, I was making arguments I didn’t fully buy myself. Further, my boss seemed intent on retraining me according to a certain cognitive style — that of the corporate world, from which he had recently come. This style demanded that I project an image of rationality but not indulge too much in actual reasoning. As I sat in my K Street office, Fred’s life as an independent tradesman gave me an image that I kept coming back to: someone who really knows what he is doing, losing himself in work that is genuinely useful and has a certain integrity to it. He also seemed to be having a lot of fun.

I think this article will strike a chord with programmers. A large part of the satisfaction of motorcycle work that Crawford describes comes from the fact that such work requires one to confront reality, however harsh it may be. Reality cannot be placated by hand-waving, Powerpoint slides, excuses, or sweet talk. But the very harshness of the challenge means that when reality yields to the finesse of a craftsman, the reward is much greater. Programming has a similar aspect: a piece of software is basically either correct or incorrect. And programming, like mechanical work, allows one to interrogate and engage the system of interest through a very high-bandwidth channel: you write a test, run it, tweak it, re-run, etc.
Can we create a function that provably predicts the optimization power of intelligences?
by  — 2009-05-28T11:35:28.556Z

Follow up to Efficient Cross-domain Optimization

When I am skeptical that we will ever understand intelligence, I am skeptical that we will ever be able to reliably map a systems description onto its optimization power. This has implications for how well we will create intelligences and how well intelligences will be at self-improving.

Obviously we can't predict the effectiveness of an arbitrary program, due to rice's theorem and intelligence being a non-trivial property. So the best we can hope for is predicting the effectiveness of a set of programs. Is such a function possible? This is my take on the subject.

Let o ( p ) be a function that maps a program p to its optimization power.

Mu, Omegas younger brother has a challenge for you, you get to design a system and put it in a box with 20 red and 20 green balls, it will activate itself after 10 minutes and then have the goal of removing as many red balls from the box as possible in 10 minutes. You have to decide how whether it is going to remove more or less than 5 red balls from the box. You get transported to a nirvana if you predict correctly and your world gets turned into paper clips if you get it wrong.

You whip out your trusty o and make a program and the evaluate it using o and bet according to its evaluation.

Unknown to you Mu also has a copy of your o and runs it on the systems you put in the box. Those that return a high value from the optimization power measure, it destroys before they activate, those that have a low effectiveness it performs their goals for them. In the second case it is still p that causes the goal to be fulfilled as if p were different there would be a different amount that the goal is fulfilled. You can see it as inspiring pity in someone else to make them help, who would not have done otherwise. It is still winning.

So Mu forces o to be wrong, so o was not the reliable predictor of a set of programs optimization power we had hoped for, so we have a contradiction. Is there anyway to salvage it? You could make your effectiveness measure depend upon the environment e as well, however that does not remove the potential for self-reference as o is part of the environment. So we might be able to rescue o by constraining the environment to not have any reference to o in. However we don't control the environment nor do we have perfect knowledge of...
Do Fandoms Need Awfulness?
by Eliezer Yudkowsky — 2009-05-28T06:03:22.092Z

Stephen Bond, "Objects of Fandom":

...my theory is that for something to attract fans, it must have an aspect of truly monumental badness about it.

Raiders of the Lost Ark is a robust potboiler, tongue-in-cheek, very competently done. I think it's enjoyable, but even among those who don't, it's hard to see the film attracting actual derision. Boredom or irritation, probably, but nothing more. Star Wars, on the other hand.... From one perspective, it's an entertaining space opera, but from a slightly different perspective, an imperceptible twist of the glass, it's laughably awful. Utterly ridiculously bad. And it's this very badness that makes so many people take up arms in its defence.

...It's impossible to imagine a fan of Animal Farm, the Well-Tempered Clavier, or the theory of gravity. Such works can defend themselves. But badness, especially badness of an obvious, monumental variety, inspires devotion. The quality of the work, in the face of such glaring shortcomings, becomes a matter of faith -- and faith is a much stronger bond than mere appreciation. It drives fans together, gives them strength against those who sneer... And so the fan groups of Tolkien, Star Trek, Spider-man, Japanese kiddie-cartoons etc. develop an almost cult-like character.

"Uh oh," I said to myself on first reading this, "Is this why my fans are more intense than Robin Hanson's fans? And if I write a rationality book, should I actually give in to temptation and self-indulgence and write in Twelve Virtues style, just so that it has something attackable for fans to defend?"

But the second time I turned my thoughts toward this question, I performed that oft-neglected operation, asking: "I read it on the Internet, but is it actually true?" Just because it's unpleasant doesn't mean it's true. And just because it provides a bit of cynicism that would give me rationality-credit to acknowledge, doesn't mean it becomes true just so I can earn the rationality-credit.

The first counterexample that came to mind was Jack Vance. Jack Vance is a science-fiction writer who, to the best of my knowledge, I've never heard accused of any great sin (or any lesser sin, actually). He is - was - the supremely competent craftsman of SF: his words flow, his plots race, and his human cultures are odder than other authors' aliens, to say nothing of his aliens. Vance d...
Anime Explains the Epimenides Paradox
by Eliezer Yudkowsky — 2009-05-27T21:12:13.086Z

The Epimenides Paradox or Liar Paradox is "This sentence is false." Type hierarchies are supposed to resolve the Epimenides paradox... Using an indefinitely extensible, indescribably infinite, ordinal hierarchy of meta-languages. No meta-language can contain its own truth predicate - no meta-language can talk about the "truth" or "falsity" of its own sentences - and so for every meta-language we need a meta-meta-language.
I didn't create this video and I don't know who did - but it does a pretty good job of depicting how I feel about infinite type hierarchies: namely, pretty much the same way I feel about the original Epimenides Paradox.

Bonus problem: In what language did I write the description of this video?
Eric Drexler on Learning About Everything
by Vladimir_Nesov — 2009-05-27T12:57:21.590Z

Related to: The Simple Math of Everything, Your Strength as a Rationalist, Teaching the Unteachable.

Eric Drexler wrote a couple of articles on the importance and methods of obtaining interdisciplinary knowledge:

How to Understand Everything (and Why)

How to Learn About Everything

Note that the title above isn't "how to learn everything", but "how to learn about everything". The distinction I have in mind is between knowing the inside of a topic in deep detail — many facts and problem-solving skills — and knowing the structure and context of a topic: essential facts, what problems can be solved by the skilled, and how the topic fits with others.

This knowledge isn't superficial in a survey-course sense: It is about both deep structure and practical applications. Knowing about, in this sense, is crucial to understanding a new problem and what must be learned in more depth in order to solve it.

This topic was discussed intermittently on Overcoming Bias. Basic understanding of many fields allows to recognize how well-understood by science a problem is and to see its place in the structure of scientific knowledge; to develop better intuitive grasp on what's possible and what's not; and to adequately perceive the natural world.

The advice he gives for obtaining general knowledge feels right, even for studying the topics that you intend to eventually understand in depth:

Don't drop a subject because you know you'd fail a test — instead, read other half-understandable journals and textbooks to accumulate vocabulary, perspective, and context.
Dissenting Views
by byrnema — 2009-05-26T18:55:17.205Z

Occasionally, concerns have been expressed from within Less Wrong that the community is too homogeneous. Certainly the observation of homogeneity is true to the extent that the community shares common views that are minority views in the general population.

Maintaining a High Signal to Noise Ratio

The Less Wrong community shares an ideology that it is calling ‘rationality’(despite some attempts to rename it, this is what it is). A burgeoning ideology needs a lot of faithful support in order to develop true. By this, I mean that the ideology needs a chance to define itself as it would define itself, without a lot of competing influences watering it down, adding impure elements, distorting it. In other words, you want to cultivate a high signal to noise ratio.

For the most part, Less Wrong is remarkably successful at cultivating this high signal to noise ratio. A common ideology attracts people to Less Wrong, and then karma is used to maintain fidelity. It protects Less Wrong from the influence of outsiders who just don't "get it". It is also used to guide and teach people who are reasonably near the ideology but need some training in rationality. Thus, karma is awarded for views that align especially well with the ideology, align reasonably well, or that align with one of the directions that the ideology is reasonably evolving.

Rationality is not a religion – Or is it?

Therefore, on Less Wrong, a person earns karma by expressing views from within the ideology. Wayward comments are discouraged with down-votes. Sometimes, even, an ideological toe is stepped on, and the disapproval is more explicit. I’ve been told, here and there, one way or another, that expressing extremely dissenting views is: stomping on flowers, showing disrespect, not playing along, being inconsiderate.

So it turns out: the conditions necessary for the faithful support of an ideology are not that different from the conditions sufficient for developing a cult.

But Less Wrong isn't a religion or a cult. It wants to identify and dis-root illusion, not create a safe place to cultivate it. Somewhere, Less Wrong must be able challenge its basic assumptions, and see how they hold up to new and all evidence. You have to allow brave dissent.

Outsiders who insist on hanging around can help by pointing to assumptions that are

...
The Wire versus Evolutionary Psychology
by MrShaggy — 2009-05-25T05:21:23.639Z

In their Evolutionary Psychology Primer, Cosmides and Tooby give an example of a hypothesized adaptation that allows us to detect cheaters in a certain type of logical task (Wason) that we generally fail at. In the Wason selection task (both article and wiki give examples) you are presented a type of logic puzzle that people tend to do poorly at and even formal training in logic helps little, yet when the examples involve cheating (such as "If you are to eat those cookies, then you must first fix your bed" and the task would be to figure out if someone whose eating the cookies did indeed fix the bed) perform much better (25% right in the regular task, 65-80% in this version, according to the article).

In the show The Wire, in season one, episode eight, Wallace, a teen-age drug dealer is asked by a young child to help her with her math homework. It's an addition and subtraction word problem about passengers on a bus (can't remember the numbers, but along the lines of, if the bus has 10 people on it and at the next stop 3 get on and 4 leave, etc.). Wallace rephrases the word problem to be about drugs and the kid gets it right. Wallace frustrated asks why and the kid replies along the lines of: "They beat you if you get the count wrong." (Edit:simpleton gives the quote as "Count be wrong, they fuck you up.")

C&T conclude that there are evolved "algorithms" in our brains that deal with social contract processing that explain why people do better on certain Wason selection tasks. The Wire points out a simpler possible explanation that their experiments did not control for: people do better on tasks they care about, unless one would like to suppose there are special math circuits in the brain for certain "social contract" situations.

Of course, I am not saying a fictional anecdote disproves C&T's claim, but it does point to something they didn't test for, and something that I find rather plausible.

Possible tests: Look at emotionally-motivating things that vary across culture and develop Wason selection tasks to test for that; look at various types of emotionally-motivating things (which I do not presume all emotional responses will affect the test results), and obviously, test The Wire example itself.
This Failing Earth
by Eliezer Yudkowsky — 2009-05-24T16:09:11.621Z

Suppose I told you about a certain country, somewhere in the world, in which some of the cities have degenerated into gang rule. Some such cities are ruled by a single gang leader, others have degenerated into almost complete lawlessness. You would probably conclude that the cities I was talking about were located inside what we call a "failed state".

So what does the existence of North Korea say about this Earth?

No, it's not a perfect analogy. But the thought does sometimes occur to me, to wonder if the camel has two humps. If there are failed Earths and successful Earths, in the great macroscopic superposition popularly known as "many worlds" - and we're not one of the successful. I think of this as the "failed Earth" hypothesis.

Of course the camel could also have three or more humps, and it's quite easy to imagine Earths that are failing much worse than this, epic failed Earths ruled by the high-tech heirs of Genghis Khan or the Catholic Church. Oh yes, it could definitely be worse...

...and the "failed state" analogy is hardly perfect; "failed state" usually refers to failure to integrate into the global economy, but a failed Earth is not failing to integrate into anything larger...

...but the question does sometimes haunt me, as to whether in the alternative Everett branches of Earth, we could identify a distinct cluster of "successful" Earths, and we're not in it. It may not matter much in the end; the ultimate test of a planet's existence probably comes down to Friendly AI, and Friendly AI may come down to nine people in a basement doing math. I keep my hopes up, and think of this as a "failing Earth" rather than a "failed Earth".

But it's a thought that comes to mind, now and then. Reading about the ongoing Market Complexity Collapse and wondering if this Earth failed to solve one of the basic functions of global economics, in the same way that Rome, in its later days, failed to solve the problem of orderly transition of power between Caesars.

Of course it's easy to wax moralistic about people who aren't solving their coordination problems the way you like. I don't mean this to degenerate into a standard diatribe about the sinfulness of this Earth, the sort of clueless plea embodied perfectly by Simon and Garfunkel:

I dreamed I saw a mighty room
The room was filled with men
And the paper they were

...
Saturation, Distillation, Improvisation: A Story About Procedural Knowledge And Cookies
by Alicorn — 2009-05-24T02:38:01.532Z

Most propositional knowledge (knowledge of facts) is pretty easy to come by (at least in principle). There is only one capital of Venezuela, and if you wish to learn the capital of Venezuela, Wikipedia will cooperatively inform you that it is Caracas. For propositional knowledge that Wikipedia knoweth not, there is the scientific method. Procedural knowledge - the knowledge of how to do something - is a different animal entirely. This is true not only with regard to the question of whether Wikipedia will be helpful, but also in the brain architecture at work: anterograde amnesiacs can often pick up new procedural skills while remaining unable to learn new propositional information.

One complication in learning new procedures is that there are usually dozens, if not hundreds, of ways to do something. Little details - the sorts of things that sink into the subconscious with practice but are crucial to know for a beginner - are frequently omitted in casual descriptions. Often, it can be very difficult to break into a new procedurally-oriented field of knowledge because so much background information is required. While there may be acknowledged masters of the procedure, it is rarely the case that their methods are ideal for every situation and potential user, because the success of a procedure depends on a vast array of circumstantial factors.

I propose below a general strategy for acquiring new procedural knowledge. First, saturate by getting a diverse set of instructions from different sources. Then, distill by identifying what all or most of them have in common. Finally, improvise within the remaining search space to find something that works reliably for you and your circumstances.

The strategy is not fully general: I expect it would only work properly for procedures that are widely attempted and shared; that you can afford to try multiple times; that have at least partially independent steps so you can mix and match; and that are in fields you have at least a passing familiarity with. The sort of procedural knowledge that I seek with the most regularity is how to make new kinds of food, so I will illustrate my strategy with a description of how I used it to learn to make meringues. If you find cookies a dreadfully boring subject of discourse, you may not wish to read the rest of this post....
Homogeneity vs. heterogeneity (or, What kind of sex is most moral?)
by PhilGoetz — 2009-05-22T23:25:04.300Z

You've all heard discussions of collective ethics vs. individualistic ethics. These discussions always assume that the organism in question remains constant. Your task is to choose the proper weight to give collective versus individual goals.

But the designer of transhumans has a different starting point. They have to decide how much random variation the population will have, and how much individuals will resemble those that they interact with.

Organisms with less genetic diversity place more emphasis on collective ethics. The amount of selflessness a person exhibits towards another person can be estimated according to their genetic similarity. To a first approximation, if person A shares half of their genes with people in group B, person A will regard saving their own life, versus saving two people from group B, as an even tradeoff. In fact, this generalizes across all organisms, and whenever you find insects like ants or bees, who are extremely altruistic, you will find that they share most of their genes with the group they are behaving altruistically towards. Bacterial colonies and other clonal colonies can be expected to be even more altruistic (although they don't have as wide a behavioral repertoire with which to demonstrate their virtue). Google kin selection.

Ants, honeybees, and slime molds, which share more of their genes with their nestmates than humans do with their family, achieve levels of cooperation that humans would consider horrific if it were required of them. Consider these aphids that explode themselves to provide glue to fill in holes in their community's protective gall.

The human, trying to balance collective ethics vs. individual ethics, is really just trying to discover a balance point that is already determined by their sexual diploidy. The designer of posthumans (for instance, an AI designing its subroutines for a task), OTOH, actually has a decision to make -- where should that balance be set? How much variation should there be in the population (whether of genes, memes, or whatever is most important WRT cooperation)?

A strictly goal-oriented AI would supervise its components and resources so as to optimize the trade-off between "exploration" and "exploitation". (Exploration means trying new approaches; exploitation means re-using approaches that have worked well...