trentmkelly/LessWrong-43k huggingface dataset:
Morality is subjective preference, but it can be objectively wrong
by KatjaGrace — 2009-06-17T18:09:00.000Z

People are often unwilling to think of ethics as their own preferences, rather than demands from something more transcendent. For instance it’s normal to claim that one really wants to make one choice, but it’s only ethical to make the other. My feelings agree, but my thoughts don’t. If I follow something I call ethics, that demonstrates that I want to. It’s not a physical law. So what’s the difference?

Just that. Ethics is a preference for fulfilling preferences attributed to some other source. Popular external sources of values include Gods, nature, other people, transcendent moral truth, group norms, and leaders. If I prefer for your house not to burn down I will turn on the hose. If I think it’s moral to stop your house burning down I will turn off the hose if I find out that you want to burn it down to collect insurance money. I care about your values, not the house.

One demonstration that having an external source is important for ethics is the fact that invented ethical systems (such as, ‘playing video games is virtuous’) seem illegitimate and cheaty. Crazy seeming practices can be ordained by religion and culture, but if you decide independently that it’s only ethical to eat cereal on Thursdays and most will feel you are missing the point and some marbles.

While ethics is a matter of choice then, it implies the existence of your preferred outside source of values. This means it can be wrong. The outside source of values might not exist, or might not have values. This is why evidence about evolution can influence whether a person likes gays marrying, despite it being an apparent value judgement.

This means moral intuitions aren’t as useful as they seem for information about how to be moral. Gut reactions are handy for working out what you like, but if you find that you like serving someone else’s purposes there is factual information about whether they exist or care to take into account. We have better ways to deal with facts than our emotional responses in most realms, so why not use the same here?

The only things that exist and care that I know of are other people and animals. Gods and transcendent values don’t exist, and society as a whole and the environment don’t care, as far as I know. So if I want to be ethical, preference utilitarianism (caring about other pe

...
Applied Picoeconomics
by Scott Alexander — 2009-06-17T16:08:29.757Z

Related to: Akrasia, Hyperbolic Discounting, and Picoeconomics, Fix It And Tell Us What You Did
A while back, ciphergoth posted an article on "picoeconomics", the theory that akrasia could be partially modeled as bargaining between present and future selves. I think the model is incomplete, because it doesn't explain how the analogy is instantiated in the real world, and I'd like to investigate that further sometime1 - but it's a good first-order approximation.
For those of you too lazy to read the article (come on! It has pictures of naked people! Well, one naked person. Suspended from a graph of a hyperbolic curve) Ainslie argues that "intertemporal bargaining" is one way to overcome preference reversal. For example, an alcoholic has two conflicting preferences: right now, he would rather drink than not drink, but next year he would rather be the sort of person who never drinks than remain an alcoholic. But because his brain uses hyperbolic discounting, a process that pays more attention to his current utility than his future utility, he's going to hit the whiskey.
This sticks him in a sorites paradox. Honestly, it's not going to make much of a difference if he has one more drink, so why not hit the whiskey? Ainslie's answer is that he should set a hard-and-fast rule: "I will never drink alcohol". Following this rule will cure his alcoholism and help him achieve his dreams. He now has a very high preference for following the rule; a preference hopefully stronger than his current preference for whiskey.
Ainslie's other point is that this rule needs to really be hard-and-fast. If his rule is "I will drink less whiskey", then that leaves it open for him to say "Well, I'll drink some whiskey now, and none later; that counts as 'less'", and then the whole problem comes back just as bad as before. Likewise, if he says "It's my birthday, I'll let myself break the rule just this once," then soon he's likely to be saying "It's the Sunday before Cinco de Mayo, this calls for a celebration!" Ainslie has some much more formal and convincing ways of framing this, which is why you should read the article instead of just trusting this summary.
The stuff by Ainslie I read (I didn't spring for any of his dead-tree books) didn't offer any specific pointers for increasing your willpower2, but it's pretty easy to read between the lines and figure out what applied picoec...
Don't Count Your Chickens...
by thomblake — 2009-06-17T15:21:31.616Z

A blog post by Derek Sivers links to evidence that stating one's goals makes one less likely to accomplish them.

Excerpt:

Announcing your plans to others satisfies your self-identity just enough that you're less motivated to do the hard work needed.

Link: Shut up! Announcing your plans makes you less motivated to accomplish them.
Be your conformist, approval seeking, self
by KatjaGrace — 2009-06-17T10:17:00.000Z

People recommend that one another ‘be themselves’ rather than being influenced by outside expectations and norms. Nobody suggests others should try harder to follow the crowd. They needn’t anyway; we seem fairly motivated by impressing others and fitting in. Few seem interested in ‘being themselves’ in the sense of behaving as they would if nobody was ever watching. The ‘individuality’ we celebrate usually seems designed for observers. What do people do when there’s only themselves to care? Fart louder and leave their dirty cups around. This striving for unadulterated selfhood is not praised. Yes, it seems in most cases you can get more approval if you tailor your actions to getting approval. So why do we so commonly offer this same advice, that we don’t follow, and don’t approve of any real manifestation of?
Ask LessWrong: Human cognitive enhancement now?
by taw — 2009-06-16T21:16:03.029Z

Transhumanists have high hopes for enhancing human cognitive abilities in the future. But what realistic steps can we take to enhance them now? On the one hand Flynn effect suggests IQ (which is a major factor in human cognition) can be increased a lot with current technology, on the other hand review of existing drugs seems rather pessimistic - they seem to have minor positive effect on low performers, and very little effect on high performers, what means they're mostly of therapeutic not enhancing use.

So, fellow rationalists, how can we enhance our cognition now? Solid research especially welcome, but consistent anecdotal evidence is also welcome.
Rationalists lose when others choose
by PhilGoetz — 2009-06-16T17:50:07.749Z

At various times, we've argued over whether rationalists always win. I posed Augustine's paradox of optimal repentance to argue that, in some situations, rationalists lose. One criticism of that paradox is that its strongest forms posit a God who penalizes people for being rational. My response was, So what? Who ever said that nature, or people, don't penalize rationality?

There are instances where nature penalizes the rational. For instance, revenge is irrational, but being thought of as someone who would take revenge gives advantages.1

EDIT: Many many people immediately jumped on this, because revenge is rational in repeated interactions. Sure. Note the "There are instances" at the start of the sentence. If you admit that someone, somewhere, once faced a one-shot revenge problem, then cede the point and move on. It's just an example anyway.

Here's another instance that more closely resembles the God who punishes rationalism, in which people deliberately punish rational behavior:

If rationality means optimizing expected utility, then both social pressures and evolutionary pressures tend, on average, to bias us towards altruism. (I'm going to assume you know this literature rather than explain it here.) An employer or a lover would both rather have someone who is irrationally altruistic. This means that, on this particular (and important) dimension of preference, rationality correlates with undesirability.2

<ADDED>: I originally wrote "optimizing expected selfish utility", merely to emphasize that an agent, rational or not, tries to maximize its own utility function. I do not mean that a rational agent appears selfish by social standards. A utility-maximizing agent is selfish by definition, because its utility function is its own. Any altruistic behavior that results, happens only out of self-interest. You may argue that pragmatics argue against this use of the word "selfish" because it thus adds no meaning. Fine. I have removed the word "selfish".

However, it really doesn't matter. Sure, it is possible to make a rational agent that acts in ways that seem unselfish. Irrelevant. Why would the big boss settle for "unselfish" when he can get "self-sacrificing"? It is often possible to find an irrational agent that acts more in your...
Intelligence enhancement as existential risk mitigation
by Roko — 2009-06-15T19:35:07.530Z

Here at Less Wrong, the Future of Humanity Institute and the Singularity Institute, a recurring theme is trying to steer the future of the planet away from disaster. Often, the best way to avert a particular disaster is quite hard for ordinary people to understand as it requires one to think through an argument in a cool, unemotional way; more often than not the best solution will be lost in a mass of low signal-to-noise ratio squabbling and/or emoting. Whatever the substance of the debate, the overall meta-problem is quite well captured by this catch from this month's rationality quotes:

"People are mostly sane enough, of course, in the affairs of common life: the getting of food, shelter, and so on. But the moment they attempt any depth or generality of thought, they go mad almost infallibly.

Attempting to target the meta-problem of getting people to be slightly less mad when it comes to abstract or general thought, especially public policy, is a tempting option. Robin Hanson's futarchy proposal is one way to combat this madness (which it does by removing most people from the policymaking loop). However, another important route to combating human idiocy is to find technologies that make humans smarter. Nick Bostrom proposed that we should work hard looking for ways to enhance the cognition of research scientists, because even a small increase in the average intelligence of research scientists would increase research output by a large amount, as there are lots of scientists. But improving the decisionmaking process of our society would probably have an even more profound effect; if we could improve the intelligence of the average voter by about one standard deviation, it is easy to speculate that the political decisionmaking process would work much better. For example, understanding simple logical arguments and simple quantitative analyses is stretching the capabilities of someone at IQ 100, so it seems that the marginal effect of overall IQ increases would be quite a large marginal increases in the probability that a politician was incentivized to focus on a logical argument over an emotionally appealing slander as the main focus of their campaign.

As a concrete example, consider the initial US reaction to rising oil prices and the need for US-produced energy: pushing corn ethanol, because a strong farming lobby liked the idea of having extra revenue. Now...
The Laws of Magic
by Annoyance — 2009-06-15T19:13:08.743Z

People are always telling you that "we have always done thus", and then you find that their "always" means a generation or two, or a century or two, at most a millennium or two. Cultural ways and habits are blips compared to the ways and habits of the body, of the race. There really is very little that human beings on our plane have always done, except find food and drink, sing, talk, procreate, nurture the children, and probably band together to some extent.

- Ursula K. Le Guin, "Seasons of the Ansarac", Changing Planes

Human cultures vary wildly and dicursively, so it is worth noting which things all known human societies have in common. Several generations ago, anthropologists noted that cultures' beliefs about a suite of concepts crudely describable as 'magic' had certain principles in common.

Humans seem to naturally generate a series of concepts known as "Sympathetic Magic", a host of theories and practices which have certain principles in common, two of which are of overriding importance. These principles can be expressed as follows: the Law of Contagion holds that two things which have interacted, or were once part of a single entity, retain their connection and can exert influence over each other; the Law of Similarity holds that things which are similar or treated the same establish a connection and can affect each other.

These principles are grossly, obviously, in contradiction with everyday experience. Thusly many cultures restrict the phenomena to which the laws supposedly apply to non-standard, special cases, most especially to individuals who it is asserted have unusual powers or ritual actions that are not commonly replicated in normal life. Examples range from African sorcerers could supposedly bring death to their enemies by stabbing their footprints, the Imperial City of ancient China which was designed to function as a stylized representation of the whole of the country and induce peace as long as the Emperor sat in his throne facing south, and all manner of witchcraft superstitions in which a discarded body part or tiny doll could be used to work magic at a distance.

Yet the laws themselves do not seem to be doubted, and the phenomena which they supposedly describe were historically (and often presently) widely believed despite a complete lack of actual evidence. There are even technical spec...
The two meanings of mathematical terms
by JamesCole — 2009-06-15T14:30:35.501Z

[edit: sorry, the formatting of links and italics in this is all screwy. I've tried editing both the rich-text and the HTML and either way it looks ok while i'm editing it but the formatted terms either come out with no surrounding spaces or two surrounding spaces]

In the latest Rationality Quotes thread, CronoDAS quoted Paul Graham:

It would not be a bad definition of math to call it the study of terms that have precise meanings.

Sort of. I started writing a this as a reply to that comment, but it grew into a post.

We've all heard of the story of epicycles and how before Copernicus came along the movement of the stars and planets were explained by the idea of them being attached to rotating epicycles, some of which were embedded within other larger, rotating epicycles (I'm simplifying the terminology a little here).

As we now know, the Epicycles theory was completely wrong. The stars and planets were not at the distances from earth posited by the theory, or of the size presumed by it, nor were they moving about on some giant clockwork structure of rings.

In the theory of Epicycles the terms had precise mathematical meanings. The problem was that what the terms were meant to represent in reality were wrong. The theory involved applied mathematical statements, and in any such statements the terms don’t just have their mathematical meaning -- what the equations say about them -- they also have an ‘external’ meaning concerning what they’re supposed to represent in or about reality.

Lets consider these two types of meanings. The mathematical, or ‘internal’, meaning of a statement like ‘1 + 1 = 2’ is very precise. ‘1 + 1’ is defined as ‘2’, so ‘1 + 1 = 2’ is pretty much the pre-eminent fact or truth. This is why mathematical truth is usually given such an exhaulted place. So far so good with saying that mathematics is the study of terms with precise meanings.

But what if ‘1 + 1 = 2’ happens to be used to describe something in reality? Each of the terms will then take on a second meaning -- concerning what they are meant to be representing in reality. This meaning lies outside the mathematical theory, and there

...
Readiness Heuristics
by jimrandomh — 2009-06-15T01:53:48.804Z

Followup to: How Much Thought

A trolley is hurtling towards three people. It will kill them, unless you pull a lever that diverts it onto a different track. However, if you do this, it will hit a small child and kill her. Do you pull the lever, and kill a child, or do nothing, and let three adults die? This question is used to test moral systems and theories; an answer reveals how you value lives and culpability. Or at least, it's supposed to. It's hard to get a straight answer, because everyone wants to take a third option. Why waste time thinking about whose life is more valuable, when you could be looking for a way to save everyone?

In philosophy, decisions are hardened by saying that there are no other options. The real world doesn't work that way. Every decision has an implied extra option: don't decide. Instead, put it off, gather more information, ask a friend, or think more. You might come up with new information that affects the decision or a new option that's better than the old ones. It could be that there is nothing to find, but it takes a lot of thought and investigation to be sure. Or you could find the perfect solution, if only you wait a few more seconds before deciding.

We can't think about both a decision and a meta-decision at the same time, so we have a set of readiness heuristics to tell us whether we're ready to call our current-best option a final decision. Normal heuristics determine what we decide; if they go awry, we choose poorly. The readiness heuristics determine when and whether we decide. If they go awry, we choose hastily or not at all. Broken readiness heuristics cause decision paralysis, writer's block, and procrastination.

Given a set of known options, decision theory tells us that we can crunch some numbers to get an expected outcome value for each choice, and choose whichever gives the best expectation. We can treat putting off the decision as an extra option, and estimate its utility like we do for the other known options. However, it so different from the other options that it makes more sense to treat it separately. Instead, we split every decision into two: Decide now? And decide what? Formally, you should put off making a decision if the probability of changing your mind times the expected improvement from doing so is greater than the cost of indecision; that is, if

P(change) * E(improvement) >...
Rationality Quotes - June 2009
by pjeby — 2009-06-14T22:00:28.697Z

(Since there didn't seem to be one for this month, and I just ran across a nice quote.)

A monthly thread for posting any interesting rationality-related quotes you've seen recently on the Internet, or had stored in your quotesfile for ages.

Please post all quotes separately (so that they can be voted up (or down) separately) unless they are strongly related/ordered.

Do not quote yourself.

Do not quote comments/posts on LW/OB - if we do this, there should be a separate thread for it.

No more than 5 quotes per person per monthly thread, please.
Explain explanations for choosing by choice
by KatjaGrace — 2009-06-14T21:50:00.000Z

A popular explanation of why it’s worse to seem stupid than lazy is that lazy seems like more of a choice, so not permanent. Similarly it seems more admired and desired to have innate artistic talent than to try hard despite being less naturally good. Being unable to stand by and let a tragedy occur (‘I had no choice!’) is more virtuous than making a calm, reasoned decision to avoid a tragedy.

On the other hand, people usually claim to prefer being liked for their personality over their looks. When asked they also relate it to their choice in the matter; it means more to be liked for something you ‘had a say in’. People are also proud of achievements they work hard on and decisions they make, and less proud of winning the lottery and forced moves.

The influence of apparent choice on our emotions is opposite in these cases, yet we often use it in the explanation for both. Is percieved level of choice really relevant to anything? If so, why does it explain effects in opposite directions? If not, why do we think of it so soon when questioned on these things?
Why safety is not safe
by rwallace — 2009-06-14T05:20:55.442Z

June 14, 3009
Twilight still hung in the sky, yet the Pole Star was visible above the trees, for it was a perfect cloudless evening.
"We can stop here for a few minutes," remarked the librarian as he fumbled to light the lamp. "There's a stream just ahead."
The driver grunted assent as he pulled the cart to a halt and unhitched the thirsty horse to drink its fill.
It was said that in the Age of Legends, there had been horseless carriages that drank the black blood of the earth, long since drained dry. But then, it was said that in the Age of Legends, men had flown to the moon on a pillar of fire. Who took such stories seriously?
The librarian did. In his visit to the University archive, he had studied the crumbling pages of a rare book in Old English, itself a copy a mere few centuries old, of a text from the Age of Legends itself; a book that laid out a generation's hopes and dreams, of building cities in the sky, of setting sail for the very stars. Something had gone wrong - but what? That civilization's capabilities had been so far beyond those of his own people. Its destruction should have taken a global apocalypse of the kind that would leave unmistakable record both historical and archaeological, and yet there was no trace. Nobody had anything better than mutually contradictory guesses as to what had happened. The librarian intended to discover the truth.
Forty years later he died in bed, his question still unanswered.
The earth continued to circle its parent star, whose increasing energy output could no longer be compensated by falling atmospheric carbon dioxide concentration. Glaciers advanced, then retreated for the last time; as life struggled to adapt to changing conditions, the ecosystems of yesteryear were replaced by others new and strange - and impoverished. All the while, the environment drifted further from that which had given rise to Homo sapiens, and in due course one more species joined the billions-long roll of the dead. For what was by some standards a little while, eyes still looked up at the lifeless stars, but there were no more minds to wonder what might have been.

Were I to submit the above to a science fiction magazine, it would be instantly rejected. It lacks a satisfying climax in which the hero strikes down the villain, for it has neither hero nor villain. Yet I ask your indulgence for a short time, for it may yet possess one import...