trentmkelly/LessWrong-43k huggingface dataset:
In Praise of Boredom
by Eliezer Yudkowsky — 2009-01-18T09:03:29.000Z

If I were to make a short list of the most important human qualities—

—and yes, this is a fool's errand, because human nature is immensely complicated, and we don't even notice all the tiny tweaks that fine-tune our moral categories, and who knows how our attractors would change shape if we eliminated a single human emotion—

—but even so, if I had to point to just a few things and say, "If you lose just one of these things, you lose most of the expected value of the Future; but conversely if an alien species independently evolved just these few things, we might even want to be friends"—

—then the top three items on the list would be sympathy, boredom and consciousness.

Boredom is a subtle-splendored thing. You wouldn't want to get bored with breathing, for example—even though it's the same motions over and over and over and over again for minutes and hours and years and decades.

Now I know some of you out there are thinking, "Actually, I'm quite bored with breathing and I wish I didn't have to," but then you wouldn't want to get bored with switching transistors.

According to the human value of boredom, some things are allowed to be highly repetitive without being boring—like obeying the same laws of physics every day.

Conversely, other repetitions are supposed to be boring, like playing the same level of Super Mario Brothers over and over and over again until the end of time. And let us note that if the pixels in the game level have a slightly different color each time, that is not sufficient to prevent it from being "the same damn thing, over and over and over again".

Once you take a closer look, it turns out that boredom is quite interesting.

One of the key elements of boredom was suggested in "Complex Novelty": If your activity isn't teaching you insights you didn't already know, then it is non-novel, therefore old, therefore boring.

But this doesn't quite cover the distinction. Is breathing teaching you anything? Probably not at this moment, but you wouldn't want to stop breathing. Maybe you'd want to stop noticing your breathing, which you'll do as soon as I stop drawing your attention to it.

I'd suggest that the repetitive activities which are allowed to not be boring fall into two categories:

Things so extremely low-level, or with such a small volume of possibilities, that you couldn't avoid repeating them even if you tried; bu

...
Getting Nearer
by Eliezer Yudkowsky — 2009-01-17T09:28:54.000Z

Reply to: A Tale Of Two Tradeoffs

I'm not comfortable with compliments of the direct, personal sort, the "Oh, you're such a nice person!" type stuff that nice people are able to say with a straight face. Even if it would make people like me more - even if it's socially expected - I have trouble bringing myself to do it. So, when I say that I read Robin Hanson's "Tale of Two Tradeoffs", and then realized I would spend the rest of my mortal existence typing thought processes as "Near" or "Far", I hope this statement is received as a due substitute for any gushing compliments that a normal person would give at this point.

Among other things, this clears up a major puzzle that's been lingering in the back of my mind for a while now. Growing up as a rationalist, I was always telling myself to "Visualize!" or "Reason by simulation, not by analogy!" or "Use causal models, not similarity groups!" And those who ignored this principle seemed easy prey to blind enthusiasms, wherein one says that A is good because it is like B which is also good, and the like.

But later, I learned about the Outside View versus the Inside View, and that people asking "What rough class does this project fit into, and when did projects like this finish last time?" were much more accurate and much less optimistic than people who tried to visualize the when, where, and how of their projects. And this didn't seem to fit very well with my injunction to "Visualize!"

So now I think I understand what this principle was actually doing - it was keeping me in Near-side mode and away from Far-side thinking. And it's not that Near-side mode works so well in any absolute sense, but that Far-side mode is so much more pushed-on by ideology and wishful thinking, and so casual in accepting its conclusions (devoting less computing power before halting).

An example of this might be the balance between offensive and defensive
nanotechnology, where I started out by - basically - just liking nanotechnology; until I got involved in a discussion about the particulars of nanowarfare, and noticed that people were postulating
crazy things to make defense win. Which made me realize and say, "Look, the
balance between offense and defense has been...
Seduced by Imagination
by Eliezer Yudkowsky — 2009-01-16T03:10:22.000Z

"Vagueness" usually has a bad name in rationality—connoting skipped steps in reasoning and attempts to avoid falsification. But a rational view of the Future should be vague, because the information we have about the Future is weak. Yesterday I argued that justified vague hopes might also be better hedonically than specific foreknowledge—the power of pleasant surprises.

But there's also a more severe warning that I must deliver: It's not a good idea to dwell much on imagined pleasant futures, since you can't actually dwell in them. It can suck the emotional energy out of your actual, current, ongoing life.

Epistemically, we know the Past much more specifically than the Future. But also on emotional grounds, it's probably wiser to compare yourself to Earth's past, so you can see how far we've come, and how much better we're doing. Rather than comparing your life to an imagined future, and thinking about how awful you've got it Now.

Having set out to explain George Orwell's observation that no one can seem to write about a Utopia where anyone would want to live—having laid out the various Laws of Fun that I believe are being violated in these dreary Heavens—I am now explaining why you shouldn't apply this knowledge to invent an extremely seductive Utopia and write stories set there. That may suck out your soul like an emotional vacuum cleaner.

I briefly remarked on this phenomenon earlier, and someone said, "Define 'suck out your soul'." Well, it's mainly a tactile thing: you can practically feel the pulling sensation, if your dreams wander too far into the Future. It's like something out of H. P. Lovecraft: The Call of Eutopia. A professional hazard of having to stare out into vistas that humans were meant to gaze upon, and knowing a little too much about the lighter side of existence.

But for the record, I will now lay out the components of "soul-sucking", that you may recognize the bright abyss and steer your thoughts away:

Your emotional energy drains away into your imagination of Paradise:

You find yourself thinking of it more and more often.
The actual challenges of your current existence start to seem less interesting, less compelling; you think of them less and less.
Comparing everything to your imagined perfect world heightens your annoyances and diminishes your pleasures.

You go into an affectiv

...
Justified Expectation of Pleasant Surprises
by Eliezer Yudkowsky — 2009-01-15T07:26:51.000Z

I recently tried playing a computer game that made a major fun-theoretic error. (At least I strongly suspect it's an error, though they are game designers and I am not.)

The game showed me—right from the start of play—what abilities I could purchase as I increased in level. Worse, there were many different choices; still worse, you had to pay a cost in fungible points to acquire them, making you feel like you were losing a resource... But today, I'd just like to focus on the problem of telling me, right at the start of the game, about all the nice things that might happen to me later.

I can't think of a good experimental result that backs this up; but I'd expect that a pleasant surprise would have a greater hedonic impact, than being told about the same gift in advance. Sure, the moment you were first told about the gift would be good news, a moment of pleasure in the moment of being told. But you wouldn't have the gift in hand at that moment, which limits the pleasure. And then you have to wait. And then when you finally get the gift—it's pleasant to go from not having it to having it, if you didn't wait too long; but a surprise would have a larger momentary impact, I would think.

This particular game had a status screen that showed all my future class abilities at the start of the game—inactive and dark but with full information still displayed. From a hedonic standpoint this seems like miserable fun theory. All the "good news" is lumped into a gigantic package; the items of news would have much greater impact if encountered separately. And then I have to wait a long time to actually acquire the abilities, so I get an extended period of comparing my current weak game-self to all the wonderful abilities I could have but don't.

Imagine living in two possible worlds. Both worlds are otherwise rich in challenge, novelty, and other aspects of Fun. In both worlds, you get smarter with age and acquire more abilities over time, so that your life is always getting better.

But in one world, the abilities that come with seniority are openly discussed, hence widely known; you know what you have to look forward to.

In the other world, anyone older than you will refuse to talk about certain aspects of growing up; you'll just have to wait and find out.

I ask you to contemplate—not just which world you might...
Building Weirdtopia
by Eliezer Yudkowsky — 2009-01-12T20:35:27.000Z

"Two roads diverged in the woods. I took the one less traveled, and had to eat bugs until Park rangers rescued me."
—Jim Rosenberg

Utopia and Dystopia have something in common: they both confirm the moral sensibilities you started with. Whether the world is a libertarian utopia of the non-initiation of violence and everyone free to start their own business, or a hellish dystopia of government regulation and intrusion—you might like to find yourself in the first, and hate to find yourself in the second; but either way you nod and say, "Guess I was right all along."

So as an exercise in creativity, try writing them down side by side: Utopia, Dystopia, and Weirdtopia. The zig, the zag and the zog.

I'll start off with a worked example for public understanding of science:

Utopia: Most people have the equivalent of an undergrad degree in something; everyone reads the popular science books (and they're good books); everyone over the age of nine understands evolutionary theory and Newtonian physics; scientists who make major contributions are publicly adulated like rock stars.
Dystopia: Science is considered boring and possibly treasonous; public discourse elevates religion or crackpot theories; stem cell research is banned.
Weirdtopia: Science is kept secret to avoid spoiling the surprises; no public discussion but intense private pursuit; cooperative ventures surrounded by fearsome initiation rituals because that's what it takes for people to feel like they've actually learned a Secret of the Universe and be satisfied; someone you meet may only know extremely basic science, but they'll have personally done revolutionary-level work in it, just like you. Too bad you can't compare notes.

Disclaimer 1: Not every sensibility we have is necessarily wrong. Originality is a goal of literature, not science; sometimes it's better to be right than to be new. But there are also such things as cached thoughts. At least in my own case, it turned out that trying to invent a world that went outside my pre-existing sensibilities, did me a world of good.

Disclaimer 2: This method is not universal: Not all interesting ideas fit this mold, and not all ideas that fit this mold are good ones. Still, it seems like an interesting technique.

If you're trying to write scien...
Eutopia is Scary
by Eliezer Yudkowsky — 2009-01-12T05:28:34.000Z

Followup to: Why is the Future So Absurd?

"The big thing to remember about far-future cyberpunk is that it will be truly ultra-tech. The mind and body changes available to a 23rd-century Solid Citizen would probably amaze, disgust and frighten that 2050 netrunner!"
—GURPS Cyberpunk

Pick up someone from the 18th century—a smart someone. Ben Franklin, say. Drop them into the early 21st century.

We, in our time, think our life has improved in the last two or three hundred years. Ben Franklin is probably smart and forward-looking enough to agree that life has improved. But if you don't think Ben Franklin would be amazed, disgusted, and frightened, then I think you far overestimate the "normality" of your own time. You can think of reasons why Ben should find our world compatible, but Ben himself might not do the same.

Movies that were made in say the 40s or 50s, seem much more alien—to me—than modern movies allegedly set hundreds of years in the future, or in different universes. Watch a movie from 1950 and you may see a man slapping a woman. Doesn't happen a lot in Lord of the Rings, does it? Drop back to the 16th century and one popular entertainment was setting a cat on fire. Ever see that in any moving picture, no matter how "lowbrow"?

("But," you say, "that's showing how discomforting the Past's culture was, not how scary the Future is." Of which I wrote, "When we look over history, we see changes away from absurd conditions such as everyone being a peasant farmer and women not having the vote, toward normal conditions like a majority middle class and equal rights...")

Something about the Future will shock we 21st-century folk, if we were dropped in without slow adaptation. This is not because the Future is cold and gloomy—I am speaking of a positive, successful Future; the negative outcomes are probably just blank. Nor am I speaking of the idea that every Utopia has some dark hidden flaw. I am saying that the Future would discomfort us because it is better.

This is another piece of the puzzle for why no author seems to have ever succeeded in constructing a Utopia worth-a-damn. When they are out to depict how marvelous and wonderful the world could be, if only we would all be Marxists or Randians o...
Continuous Improvement
by Eliezer Yudkowsky — 2009-01-11T02:09:01.000Z

When is it adaptive for an organism to be satisfied with what it has? When does an organism have enough children and enough food? The answer to the second question, at least, is obviously "never" from an evolutionary standpoint. The first proposition might be true if the reproductive risks of all available options exceed their reproductive benefits. In general, though, it is a rare organism in a rare environment whose reproductively optimal strategy is to rest with a smile on its face, feeling happy.

To a first approximation, we might say something like "The evolutionary purpose of emotion is to direct the cognitive processing of the organism toward achievable, reproductively relevant goals". Achievable goals are usually located in the Future, since you can't affect the Past. Memory is a useful trick, but learning the lesson of a success or failure isn't the same goal as the original event—and usually the emotions associated with the memory are less intense than those of the original event.

Then the way organisms and brains are built right now, "true happiness" might be a chimera, a carrot dangled in front of us to make us take the next step, and then yanked out of our reach as soon as we achieve our goals.

This hypothesis is known as the hedonic treadmill.

The famous pilot studies in this domain demonstrated e.g. that past lottery winners' stated subjective well-being was not significantly greater than that of an average person, after a few years or even months. Conversely, accident victims with severed spinal cords were not as happy as before the accident after six months—around 0.75 sd less than control groups—but they'd still adjusted much more than they had expected to adjust.

This being the transhumanist form of Fun Theory, you might perhaps say: "Let's get rid of this effect. Just delete the treadmill, at least for positive events."

I'm not entirely sure we can get away with this. There's the possibility that comparing good events to not-as-good events is what gives them part of their subjective quality. And on a moral level, it sounds perilously close to tampering with Boredom itself.

So suppose that instead of modifying minds and values, we first ask what we can do by modifying the environment. Is there enough fun in the universe, sufficiently accessible, for a transhuman to jog off th...
Rationality Quotes 23
by Eliezer Yudkowsky — 2009-01-10T00:12:24.000Z

"This year I resolve to lose weight... to be nicer to dogs... and to sprout wings and fly."
-- Garfield

"People understand instinctively that the best way for computer programs to communicate with each other is for each of them to be strict in what they emit, and liberal in what they accept. The odd thing is that people themselves are not willing to be strict in how they speak and liberal in how they listen. You'd think that would also be obvious."
-- Larry Wall

"One never needs enemies, but they are so much fun to acquire."
-- Eluki bes Shahar, Archangel Blues

"I'm accusing you of violating the laws of nature!"
"Nature's virtue is intact. I just know some different laws."
-- Orson Scott Card, A Planet Called Treason

"The goal of most religions is to preserve and elaborate on that concept of the stars as a big painted backdrop. They make Infinity a 'prop' so you don't have to think about the scary part."
-- The Book of the SubGenius

<silverpower> Is humanity even worth saving?
<starglider> As opposed to what?
<silverpower> ...hmm.
-- #sl4
Serious Stories
by Eliezer Yudkowsky — 2009-01-08T23:49:35.000Z

Every Utopia ever constructed—in philosophy, fiction, or religion—has been, to one degree or another, a place where you wouldn't actually want to live. I am not alone in this important observation: George Orwell said much the same thing in "Why Socialists Don't Believe In Fun", and I expect that many others said it earlier.

If you read books on How To Write—and there are a lot of books out there on How To Write, because amazingly a lot of book-writers think they know something about writing—these books will tell you that stories must contain "conflict".

That is, the more lukewarm sort of instructional book will tell you that stories contain "conflict". But some authors speak more plainly.

"Stories are about people's pain." Orson Scott Card.

"Every scene must end in disaster." Jack Bickham.

In the age of my youthful folly, I took for granted that authors were excused from the search for true Eutopia, because if you constructed a Utopia that wasn't flawed... what stories could you write, set there? "Once upon a time they lived happily ever after." What use would it be for a science-fiction author to try to depict a positive Singularity, when a positive Singularity would be...

...the end of all stories?

It seemed like a reasonable framework with which to examine the literary problem of Utopia, but something about that final conclusion produced a quiet, nagging doubt.

At that time I was thinking of an AI as being something like a safe wish-granting genie for the use of individuals. So the conclusion did make a kind of sense. If there was a problem, you would just wish it away, right? Ergo—no stories. So I ignored the quiet, nagging doubt.

Much later, after I concluded that even a safe genie wasn't such a good idea, it also seemed in retrospect that "no stories" could have been a productive indicator. On this particular occasion, "I can't think of a single story I'd want to read about this scenario", might indeed have pointed me toward the reason "I wouldn't want to actually live in this scenario".

So I swallowed my trained-in revulsion of Luddism and theodicy, and at least tried to contemplate the argument:

A world in which nothing ever goes wrong, or no one ever experiences any pain or sorrow, is a world containing no stories worth reading about.

A world that you wouldn

...
Rationality Quotes 22
by Eliezer Yudkowsky — 2009-01-07T12:00:50.000Z

"Two roads diverged in the woods. I took the one less traveled, and had to eat bugs until Park rangers rescued me."
-- Jim Rosenberg

"Lying to yourself about specific actions is easier than re-defining the bounds of your imagined identity... When I see once-ethical men devolve into moral grey, they still identify as upstanding."
-- Ben Casnocha

"Every year buy a clean bed sheet, date it, and lay it over the previous layer of junk on your desk."
-- Vernor Vinge, The Blabber

"Like first we tossed out the bath water, then the baby, and like finally the whole tub."
-- John R. Andrews

"I have no wisdom. Yet I heard a wise man - soon to be a relative of marriage - say not long ago that all is for the best. We are but dreams, and dreams possess no life by their own right. See, I am wounded. (Holds out his hand.) When my wound heals, it will be gone. Should it with its bloody lips say it is sorry to heal? I am only trying to explain what another said, but that is what I think he meant."
-- Gene Wolfe, The Claw of the Conciliator

"On a grand scale we simply want to save the world, so obviously we're just letting ourselves in for a lot of disappointment and we're doomed to failure since we didn't pick some cheap-ass two-bit goal like collecting all the Garbage Pail Kids cards."
-- Nenslo

"He promised them nothing but blood, iron, and fire, and offered them only the choice of going to find it or of waiting for it to find them at home."
-- John Barnes, One For the Morning Glory
Emotional Involvement
by Eliezer Yudkowsky — 2009-01-06T22:23:24.000Z

Followup to: Evolutionary Psychology, Thou Art Godshatter, Existential Angst Factory

Can your emotions get involved in a video game? Yes, but not much. Whatever sympathetic echo of triumph you experience on destroying the Evil Empire in a video game, it's probably not remotely close to the feeling of triumph you'd get from saving the world in real life. I've played video games powerful enough to bring tears to my eyes, but they still aren't as powerful as the feeling of significantly helping just one single real human being.

Because when the video game is finished, and you put it away, the events within the game have no long-term consequences.

Maybe if you had a major epiphany while playing... But even then, only your thoughts would matter; the mere fact that you saved the world, inside the game, wouldn't count toward anything in the continuing story of your life.

Thus fails the Utopia of playing lots of really cool video games forever. Even if the games are difficult, novel, and sensual, this is still the idiom of life chopped up into a series of disconnected episodes with no lasting consequences. A life in which equality of consequences is forcefully ensured, or in which little is at stake because all desires are instantly fulfilled without individual work—these likewise will appear as flawed Utopias of dispassion and angst. "Rich people with nothing to do" syndrome. A life of disconnected episodes and unimportant consequences is a life of weak passions, of emotional uninvolvement.

Our emotions, for all the obvious evolutionary reasons, tend to associate to events that had major reproductive consequences in the ancestral environment, and to invoke the strongest passions for events with the biggest consequences:

Falling in love... birthing a child... finding food when you're starving... getting wounded... being chased by a tiger... your child being chased by a tiger... finally killing a hated enemy...

Our life stories are not now, and will not be, what they once were.

If one is to be conservative in the short run about changing minds, then we can get at least some mileage from changing the environment. A windowless office filled with highly repetitive non-novel challenges isn't any more conducive to emotional involvement than video games; it may be part of real life, but it's a very flat part. The oc...