trentmkelly/LessWrong-43k huggingface dataset:
The Craft and the Community
by Eliezer Yudkowsky — 2009-04-26T17:52:21.611Z

This sequence ran from March to April of 2009 and dealt with the topic of building rationalist communities that could systematically improve on the art, craft, and science of human rationality. This is a highly forward-looking sequence - not so much an immediately complete recipe, as a list of action items and warnings for anyone setting out in the future to build a craft and a community.

Raising the Sanity Waterline: Behind every particular failure of social rationality is a larger and more general failure of social rationality; even if all religious content were deleted tomorrow from all human minds, the larger failures that permit religion would still be present. Religion may serve the function of an asphyxiated canary in a coal mine - getting rid of the canary doesn't get rid of the gas. Even a complete social victory for atheism would only be the beginning of the real work of rationalists. What could you teach people without ever explicitly mentioning religion, that would raise their general epistemic waterline to the point that religion went underwater?
A Sense That More Is Possible: The art of human rationality may have not been much developed because its practitioners lack a sense that vastly more is possible. The level of expertise that most rationalists strive to develop is not on a par with the skills of a professional mathematician - more like that of a strong casual amateur. Self-proclaimed "rationalists" don't seem to get huge amounts of personal mileage out of their craft, and no one sees a problem with this. Yet rationalists get less systematic training in a less systematic context than a first-dan black belt gets in hitting people.
Epistemic Viciousness: An essay by Gillian Russell on "Epistemic Viciousness in the Martial Arts" generalizes amazingly to possible and actual problems with building a community around rationality. Most notably the extreme dangers associated with "data poverty" - the difficulty of testing the skills in the real world. But also such factors as the sacredness of the dojo, the investment in teachings long-practiced, the difficulty of book learning that leads into the need to trust a teacher, deference to historical masters, and above all, living in data poverty while continuing to act as if the luxury of trust is possible.
Schools Proliferating Without Evidence: The bra

...
SIAI call for skilled volunteers and potential interns
by AnnaSalamon — 2009-04-26T05:56:14.452Z

Want to increase the odds that humanity correctly navigates whatever risks and promises artificial intelligence may bring? Interested in spending this summer in the SF Bay Area, working on projects and picking up background with similar others, with some possibility of staying on thereafter? Want to work with, and learn with, some of the best thinkers you'll ever meet? – more specifically, some of the best at synthesizing evidence across a wide range of disciplines, and using it to make incremental progress on problems that are both damn slippery and damn important?

If so, drop us an email. Show us your skills; give us a chance to jointly brainstorm what you might be able to do.
We are particularly interested in people who have *any* of the following traits:

Dazzling brilliance at math or philosophy;

A history of successful academic paper-writing; strategic understanding of journal submission processes, grant application processes, etc.

Strong general knowledge of science or social science, and the ability to read rapidly and/or to quickly pick up new fields;

Good interpersonal skills, writing skills, and/or marketing skills;

Organization, strong ability to keep projects going without much supervision, and the ability to get mundane stuff done in a reliable manner;

Skill at implementing (non-AI) software projects, such as web apps for interactive technological forecasting, rapidly and reliably;

A history of successfully pulling off large projects or events;

Unusual competence of some other sort, in some domain we need, but haven’t realized we need.

The only musts are that you be capable, rational, and interested in helping reduce existential risk.
If you’re interested, send an email to annasalamon at gmail dot com, who will be doing the first-pass screening. Include:

Why you’re interested;

What particular skills you would bring, and what evidence makes you think you have those skills (you might include a standard resume);

Optionally, any ideas you have for what sorts of projects you might like to be involved in, or how your skillset could help us improve humanity’s long-term odds.

Our application process is fairly informal, so send us a quick email as initial inquiry and after some correspondence we can decide whether or not to follow up with more application components.

(Background on wher...
Less Meta
by Eliezer Yudkowsky — 2009-04-26T05:38:42.984Z

My recent sequence on the craft and the community is highly forward-looking—not an immediately whole recipe, but a list of action items and warnings for anyone setting out in the future. Having expended this much effort already, it seems worthwhile to try and leverage others' future efforts.

That sequence seemed like an appropriate finale, but putting it last had some side effects that I didn't expect. Thanks to the recency effect, people are now talking as if the entire œuvre had all been anticipation of future awesomeness, with no practical value in the present...

Okay, seriously, if you look over my posts on Overcoming Bias that are not just a couple of months old, you really should see quite a lot of practical day-to-day stuff. Yes, there's a long sequence on quantum mechanics, but there really is plenty of day-to-day stuff, right down to applying biases of evaluability to save money on holiday shopping. (Not to mention that I finally did derive real-world advice out of the QM detour!)

I suspect there may also be a problem here with present schools not teaching people the experience of creating new craft. What they present you with, is what you learn. So when I talk about my belief that we could be doing better, they look around and say: "But we aren't doing that well!" rather than "Hm, how can we make progress on this?"

And then your current accomplishments start to pale in the light of grander dreams, etcetera. One of the great lessons of Artificial Intelligence is that no matter how much progress you make, it can always be made to appear slow and unexciting just by having someone else quoted in the newspapers about much grander promises on which they fail to deliver.

Anyway. I think that discussion here is going a bit too meta, too much about the community, and that was not the final push I had planned to deliver. (I know, I know, obvious in retrospect, yes, but I still didn't see it coming.) Hence the quick add-on about practical advice backed by experimental results or deep theories—thankfully we are going back to those again in recent posts.

So if it's all right with you, dear readers, after the end of April, I will not promote more than one meta post unless at least four nonmeta posts have appeared before it—does that sound fair?

Part of the sequence The Craf...
Where's Your Sense of Mystery?
by Scott Alexander — 2009-04-26T00:45:25.820Z

Related to: Joy in the Merely Real, How An Algorithm Feels From Inside, "Science" As Curiosity-Stopper

Your friend tells you that a certain rock formation on Mars looks a lot like a pyramid, and that maybe it was built by aliens in the distant past. You scoff, and respond that a lot of geological processes can produce regular-looking rocks, and in all the other cases like this closer investigation has revealed the rocks to be completely natural. You think this whole conversation is silly and don't want to waste your time on such nonsense. Your friend scoffs and asks:
"Where's your sense of mystery?"
You respond, as you have been taught to do, that your sense of mystery is exactly where it should be, among all of the real non-flimflam mysteries of science. How exactly does photosynthesis happen, what is the relationship between gravity and quantum theory, what is the source of the perturbations in Neptune's orbit? These are the real mysteries, not some bunkum about aliens. And if we cannot learn to take joy in the merely real, our life will be empty indeed.

But do you really believe it?
I loved the Joy in the Merely Real sequence. But it spoke to me because it's one of the things I have the most trouble with. I am the kind of person who would have much more fun reading about the Martian pyramid than about photosynthesis.
And the one shortcoming of Joy in the Merely Real was that it was entirely normative, and not descriptive. It tells me I should reserve my sense of mystery for real science, but doesn't explain why it's so hard to do so, or why most people never even try.

So what is this sense of mystery thing anyway?
I think the sense of mystery (sense of wonder, curiosity, call it what you want) is how the mind's algorithm for determining what problems to work on feels from the inside. Compare this to lust, how the mind's algorithm for determining what potential mates to pursue feels from the inside. In both cases, the mind makes a decision based on criteria of its own, which is then presented to the consciousness in the form of an emotion. And in both cases, the mind's decision is very often contrary to our best interest - as anyone who's ever fallen for a woman based entirely on her looks can tell you.
What sort of stuff makes us curious? I don't have anything better than introspection to go on, but here are some thoughts:
1. We feel more curious about things that ...
"Self-pretending" is not as useful as we think
by pwno — 2009-04-25T23:01:09.067Z

A few weeks ago I made a draft of a post that was originally intended to be about the same issue addressed in MBlume’s post regarding beneficial false beliefs. Coincidentally, my draft included the same exact hypothetical about entering a club believing you’re the most attractive person in the room in order to increase chances of attracting women. There seems to be a general agreement with MBlume’s “it’s ok to pretend because it’s not self-deception and produces similar results” conclusion. I was surprised to see so much agreement considering that when I made my original draft I reached a completely different conclusion.

I do agree, however, that pretending may have some benefits, but those benefits are much more limited than MBlume makes them out to be. He brings up a time where pretending helped him better fit into his character in a play. Unfortunately, his anecdote is not an appropriate example of overcoming vestigial evolutionary impulses by pretending. His mind wasn’t evolutionarily programmed to “be afraid” when pretending to be someone else, it was programmed to “be afraid” when hitting on attractive women. When I am alone in my room I can act like a real alpha male all day long, but put me in front of attractive women (or people in general) and I will retreat back to my stifled self.

The only way false beliefs can overcome your obsolete evolutionary impulses is to truly believe in those false beliefs. And we all know why that would be a bad idea. Furthermore, pretending can be dangerous just like reading fiction can be dangerous. So the small benefit that pretending might give may not even be worth the cost (at times).

But there is something we can learn from these (sometimes beneficial) false beliefs.

Obviously, there is no direct casual chain that goes from self-fulfilling beliefs to real-world success. Beliefs, per se, are not the key variables in causing success; instead, these beliefs give rise to whatever the key variable is. We should figure out what are the key variables that arise and find a systematic way of getting those variables.

With th

...
Meetup Reminder: UC Santa Barbara, Today @6pm
by Eliezer Yudkowsky — 2009-04-25T19:19:27.876Z

Michael Blume and Anna Salamon have invited you to a Less Wrong meetup at UC Santa Barbara, at 6pm in the college of creative studies building. See previous post. (This post is a temporary reminder and will be deleted after the meetup, so comment there.)
Practical Advice Backed By Deep Theories
by Eliezer Yudkowsky — 2009-04-25T18:52:21.809Z

Once upon a time, Seth Roberts took a European vacation and found that he started losing weight while drinking unfamiliar-tasting caloric fruit juices.

Now suppose Roberts had not known, and never did know, anything about metabolic set points or flavor-calorie associations—all this high-falutin' scientific experimental research that had been done on rats and occasionally humans.

He would have posted to his blog, "Gosh, everyone! You should try these amazing fruit juices that are making me lose weight!" And that would have been the end of it. Some people would have tried it, it would have worked temporarily for some of them (until the flavor-calorie association kicked in) and there never would have been a Shangri-La Diet per se.

The existing Shangri-La Diet is visibly incomplete—for some people, like me, it doesn't seem to work, and there is no apparent reason for this or any logic permitting it. But the reason why as many people have benefited as they have—the reason why there was more than just one more blog post describing a trick that seemed to work for one person and didn't work for anyone else—is that Roberts knew the experimental science that let him interpret what he was seeing, in terms of deep factors that actually did exist.

One of the pieces of advice on OB/LW that was frequently cited as the most important thing learned, was the idea of "the bottom line"—that once a conclusion is written in your mind, it is already true or already false, already wise or already stupid, and no amount of later argument can change that except by changing the conclusion. And this ties directly into another oft-cited most important thing, which is the idea of "engines of cognition", minds as mapping engines that require evidence as fuel.

If I had merely written one more blog post that said, "You know, you really should be more open to changing your mind—it's pretty important—and oh yes, you should pay attention to the evidence too." And this would not have been as useful. Not just because it was less persuasive, but because the actual operations would have been much less clear without the explicit theory backing it up. What constitutes evidence, for example? Is it anything that seems like a forceful argument? Having an explicit probability theory and an explicit causal acc...
Cached Procrastination
by jimrandomh — 2009-04-25T16:22:56.720Z

I have a paper to write. Where do I start? The first time I asked this question, it was easy: just sit down and start typing. I wrote a few hundred words, then got stuck; I needed to think some more, so I took a break and did something else. Over the next few days, my thoughts on the subject settled, and I was ready to write again. So I sat down, and asked: What do I do next? Fortunately, my brain had a cached response ready to answer this question: Solitaire!
So I procrastinated, and every time I asked my brain what to write, I got back an answer like "Don't bother!". Now a deadline's approaching, and I still don't have much written, so I sit down to write again. This time, I'm determined: using my willpower, I will not allow myself to think about anything except for the paper and its topic. So I ask again: Where do I start? (Solitaire!) What thoughts come to mind? I should've started a week ago. Every time I think about this topic I get stuck. Maybe I shouldn't write this paper after all. These, too, are cached thoughts, generated during previous failed attempts to get started. These thoughts are much harder to clear, both because there are more of them, because of their emotional content, but I'm determined to do so anyways; I think through all the cached thoughts, return to the original question (Where do I start?), get my text editor open, start planning a section and... Ping! I have a new e-mail to read, I get distracted, and when I return half an hour later I have to clear those same cached thoughts again.
Many authors say to stop in the middle of a thought when you leave off, so that "Where do I start?" will always have an easy answer. This sounds like a solution, but it ignores the fact that you'll get stuck eventually, so that you have to stop, at a spot that won't be easy to come back to.
In order to stop procrastinating, there are two obstacles to overcome: A question to answer, and a cached answer to clear. The question is "What do I do first?" and the cached answer is "procrastinate more". Knowing that "procrastinate" was a cached answer makes it easier to get past, but the original question is still a problem. Why is deciding what to do first so often difficult?
When I'm programming, I make a long, unorded to-do list for each project, listing all of the features I plan to implement. When I finish one, I go back to the list to pick something ...
Programmatic Prediction markets
by  — 2009-04-25T09:29:31.762Z

I have a problem with "prediction markets" as news view. They just aren't informative enough.

If the price of oil goes down is that due to: A reduction in demand. an increase in supply, a large amount of investors finding a better investment or a large amount of investors wanting cash (due to having to pay creditors/taxes).

I want them to tell me enough information so that I can begin trading in an informed manner. When you see the market expects rain fall on montana is 2 cm in a day, what information is this based upon? If you read about a newly created huge man made lake in the are which you expect to change the micro climate, how do you know whether the simulations people and betting using are running take this into consideration?

If I don't get this information I can't trade with expectation of being able to make a profit and the market doesn't get any information that I may have that it doesn't. I could try and reverse engineer peoples climate models from the way they trade, but that is pretty hard to do. So I would like to try and lower the barrier of entry to the market by giving more information to potential players.

This lack of information is due to the signals each trader sends to the market, they are binary in nature buy or sell, with the traders strategy and information she used a black box that we can't get into. As potential traders we don't know if the market is taking certain information into account with the price we are shown.

The only way I have thought of being able to get at some of the information enclosed in the black box, is to only allow programs to bid in a market. The programs would be run on the markets server and have acces to news sources, financial data and government reports. The programs could be stopped at any time by the trader but otherwise not communicated with or updated. So the would be trader would have to build models of the world and ways of processing the financial data.

How then would we get at the information? The source for each program that wanted to trade would be held in escrow and it would be released as open source code when it was withdrawn from the system. This would create a time lag, where people could profit from a better predicting system, but also allow everyone else to catch up after a certain time. There would also be some significant financial cost to starting a new bot so that people would not ju...
Instrumental vs. Epistemic -- A Bardic Perspective
by MBlume — 2009-04-25T07:41:41.482Z

(This article expands upon my response to a question posed by pjeby here)

I've seen a few back-and-forths lately debating the instrumental use of epistemic irrationality -- to put the matter in very broad strokes, you'll have one commenter claiming that a particular trick for enhancing your effectiveness, your productivity, your attractiveness, demands that you embrace some belief unsupported by the evidence, while another claims that such a compromise is unacceptable, since a true art should use all available true information. As Eliezer put it:

I find it hard to believe that the optimally motivated individual, the strongest entrepreneur a human being can become, is still wrapped up in a blanket of comforting overconfidence. I think they've probably thrown that blanket out the window and organized their mind a little differently. I find it hard to believe that the happiest we can possibly live, even in the realms of human possibility, involves a tiny awareness lurking in the corner of your mind that it's all a lie.

And with this I agree -- the idea that a fully developed rational art of anything would involving pumping yourself with false data seems absurd.

Still, let us say that I am entering a club, in which I would like to pick up an attractive woman. Many people will tell me that I must believe myself to be the most attractive, interesting, desirable man in the room. An outside-view examination of my life thus far, and my success with women in particular, tells me that I most certainly am not. What shall I do?

Well, the question is, why am I being asked to hold these odd beliefs? Is it because I'm going to be performing conscious calculations of expected utility, and will be more likely to select the optimal actions if I plug incorrect probabilities into the calculation? Well, no, not exactly. More likely, it's because the blind idiot god has already done the calculation for me.

Evolution's goals are not my own, and neither are evolution's utility calculations. Most saliently, other men are no longer allowed to hit me with mastodon bones if I approach women they might have liked to pursue. The trouble is, evolution has already done the calculation, using this now-faulty assumption, with the result that, if I do not see myself as dominant, my motor cortex directs the movement of my body and the inflection of my voice in a way which cle...