trentmkelly/LessWrong-43k huggingface dataset:
Simplified Humanism, Positive Futurism & How to Prevent the Universe From Being Turned Into Paper Clips
by Kevin — 2010-07-22T10:03:52.517Z

Simplified Humanism, Positive Futurism & How to Prevent the Universe From Being Turned Into Paper Clips

Michael Anissimov recently did an interview with Eliezer for h+ magazine. It covers material basic to those familiar with the Less Wrong rationality sequences but is worth reading.

The list of questions:

1. Hi Eliezer. What do you do at the Singularity Institute?
2. What are you going to talk about this time at Singularity Summit?
3. Some people consider “rationality” to be an uptight and boring intellectual quality to have, indicative of a lack of spontaneity, for instance. Does your definition of “rationality” match the common definition, or is it something else? Why should we bother to be rational?
4. In your recent work over the last few years, you’ve chosen to focus on decision theory, which seems to be a substantially different approach than much of the Artificial Intelligence mainstream, which seems to be more interested in machine learning, expert systems, neural nets, Bayes nets, and the like. Why decision theory?
5. What do you mean by Friendly AI?
6. What makes you think it would be possible to program an AI that can self-modify and would still retain its original desires? Why would we even want such an AI?
7. How does your rationality writing relate to your Artificial Intelligence work?
8. The Singularity Institute turned ten years old in June. Has the organization grown in the way you envisioned it would since its founding? Are you happy with where the Institute is today?
Book Review: The Root of Thought
by Scott Alexander — 2010-07-22T08:58:18.873Z

Related to: Brain Breakthrough! It's Made of Neurons!

I can't really recommend Andrew Koob's The Root of Thought. It's poorly written, poorly proofread, lacking much more information than is in the Scientific American review, and comes across as about one part neuroscience to three parts angry rant. But it does present an interesting hypothesis and an interesting case study on a major failure of rationality.
Only about ten percent of the brain is made of neurons; the rest is a diverse group of cells called "glia". "Glia" is Greek for glue, because the scientists who discovered them decided that, since they were in the brain and they weren't neurons, they must just be there to glue the neurons together. Since then, new discoveries have assigned glial cells functions like myelination, injury repair, immune defense, and regulation of blood flow: all important, but mostly things only a biologist could love. The Root of Thought argues that glial cells, especially a kind called astrocytes, are also important in some of the higher functions of thought, including memory, cognition, and maybe even creativity. This is interesting to neuroscientists, and the story of how it was discovered is also interesting to us as aspiring rationalists.

Glial cells involved in processing

Koob's evidence is indirect but suggestive. He points out that more intelligent animals have a higher astrocyte to neuron ratio than less intelligent animals, all the way from worms with one astrocyte per thirty neurons, to humans with an astrocyte: neuron ratio well above one. Within the human brain, the areas involved in higher thought, like the cortex, are the ones with the highest astrocyte:neuron ratio, and the most down-to-earth, like the cerebellum, have barely any astrocytes at all. Especially intelligent humans may have higher ratios still: one of the discoveries made from analyzing Einstein's brain was that he had an unusually large number of astrocytes in the part of his brain responsible for mathematical processing. And learning is a stimulus for astrocyte development. When canaries learn new songs, new astrocytes grow in the areas responsible for singing.
Astrocytes have a structure especially suited for learning and cognition. They have their own gliotransmitters, similar in function to neurotransmitters, and they communicate with one another, sparking waves of astrocyte activity across are...
Missed opportunities for doing well by doing good
by multifoliaterose — 2010-07-21T07:45:26.301Z

Related to: Fight zero-sum bias

According to the U.S. Department of State:

In 2006, Americans donated 2.2 percent of their average disposable, or after-tax income.

The Department of State report commends the charitable giving practices of Americans as follows:

“The total amount of money that was given to nonprofit institutions is remarkable,” [Richard Jolly, chairman of Giving US] said. “What we see is when people feel engaged, when they feel a need is legitimate, when they are asked to support it, they do.”

Americans have a long tradition of charitable giving and volunteerism -- the donation of time and labor on behalf of a cause. When disasters happen or a social need arises, government clearly has a responsibility, Jolly said. “But it’s also obvious Americans believe they, too, can make a difference, and they reflect that in terms of giving away a lot of money.”

The United States is “a land of charity,” says Arthur Brooks, an expert on philanthropy and a professor at Syracuse University’s Maxwell School, who sees charitable giving and volunteerism as the signal characteristic of Americans.

For my own part, I think that what Jolly, what Brooks, and what the Department of State report have to say about American charitable giving is absurd. I think that the vast amount of American "charitable giving" should not be conceptualized as philanthropy because the donors do not aspire to maximize their positive social impact. Even aside from that, from a utilitarian point of view, in view of world economic inequality and existential risk, a donation rate of 2.2% looks paltry. As the title of Peter Unger's book Living High and Letting Die: Our Illusion of Innocence suggests, there's a sense in which despite appearances, many Americans are guilty of a moral atrocity.

In light of my last few sentences, you may be surprised to know that I don't think that Americans should sacrifice their well-being for the sake of others. Even from a utilitarian point of view, I think that there are good reasons for thinking that it would be a bad idea to do this. The reason that I say that many Americans are guilty of a moral atrocity is because I think that many Americans could be giving away a lot more of their money with a view toward maximizing their positive social impact and lead more fulfilling lives as a result. ...
Floating Festival Jul 22-25
by Eliezer Yudkowsky — 2010-07-21T06:36:02.408Z

The Seasteading Institute had to cancel their annual on-water get-together, Ephemerisle, after being quoted ridiculous insurance costs, and it came back as the unofficial Floating Festival - no tickets, no organizers, you just show up with a boat. (Or find someone else who already has or is renting a boat and still has a spare spot, etc.) Jul 22-25 with an unconference (i.e., show up and give a talk) on Saturday the 24th. Posting here because of the large de facto overlap in the communities. The location is about a two-hour drive from the Bay Area.

Main info page.
A speculation on Near and Far Modes
by MichaelVassar — 2010-07-21T06:24:56.684Z

Katja's recent post on cryonics elicited this comment from Steven Kaas,

"If cryonics is super-far and altruism is seen as more important in far mode, why isn’t buying cryonics for others seen as especially praiseworthy? Your list of ways in which cryo is far-mode seems too much of a coincidence unless cryo was somehow optimized for distance."

...a comment which finally caused the following hypothesis to click into sharp resolution for me.

My guess is that it's cryonics advocates who are optimized for distance. Most people are basically natives of near mode, using far mode only casually and occasionally for signaling, and never reasoning about its contents. Even those who reason about its contents usually do so and then ignore their reasoning, acting on near mode motivations and against their explicit beliefs. Children, however, actually need to use far mode to guide their actions because they lack the rich tacit knowledge that makes near mode functional.

Some people get stuck in a child-like behavioral pattern, probably due to a mix of neurological bugs which prevent near-mode from gelling (aspergers and schizotype) and internalization of explicit (far mode) rules condemning near-mode (obsessive compulsive personality disorder). They become Shaw's "unreasonable men" and push for the universal endorsement of far-mode ideas. Since other adults don't care about the contents of far mode much anyway except to avoid being condemned for saying the wrong things, others go along with this, and there's a long-term drift towards explicit societal endorsement of altruistic norms with an expanded circle. This does matter, in the long run, because such norms provide convenient nuclei for the emergence of forms of mostly-arbitrary identity markers.

Unreasonable men also develop artificial ways of reasoning, methods of rationality, which work even when the minds innate tendencies towards reason aren't engaged by near mode. These methods don't necessarily suffer from the bugs that near-mode reasoning suffers from. Once the right set of methods, most importantly math, science, nation-states, cosmopolitan liberalism (actions permitted by default rather than banned by default) and capitalism, are developed, they enable scientific and technological evolution to jump over low memetic fitness regions of the memetic ...
(One reason) why capitalism is much maligned
by multifoliaterose — 2010-07-19T03:48:43.524Z

Related to: Fight zero-sum bias

Disclaimer: (added response to comments by Nic_Smith, SilasBarta and Emile) - The point of this post is not to argue in favor of free markets. The point of this post is to discuss an apparent bias in people's thinking about free markets. Some of my own views about free markets are embedded within the post, but my reason for expressing them is to place my discussion of the bias that I hypothesize in context, not to marginalize the adherents to any political affiliation. When I refer to "capitalism" below, I mean "market systems with a degree of government regulation qualitatively similar to the degree present in the United States, Western Europe and Japan in the past 50 years."

The vast majority of the world's wealth has been produced under capitalism. There's a strong correlation between a country's GDP and its citizens' self reported life satisfaction. Despite the evidence for this claims there are some very smart people who are or have been critics of capitalism. There may be legitimate arguments against capitalism, but all too often, these critics of capitalism selectively focus on the negative effects of capitalism, failing to adequately consider the counterfactual "how would things be under other economic systems?" and apparently oblivious to large scale trends.

There are multiple sources of irrational bias against capitalism. Here I will argue that a major factor is zero-sum bias, or perhaps, as hegemonicon suggests, the "relativity heuristic." Within the post I quote Charles Wheelan's Naked Economics several times. I believe that Wheelan has done a great service to society by writing an accessible book which clears up common misconceptions about economics and hope that it becomes even more widely read than it has been so far. Though I largely agreed with the positions that he advocates before taking up the book, the clarity of his writing and focus on the most essential points of economics helped me sharpen my thinking, and I'm grateful to him for this.

A striking feature of the modern world is economic inequality. According to Forbes Magazine, Amazon founder Jeff Bezos made $7.3 billion in 2009. That's more than the (nominal) 2009 earnings of all of the 10 million residents of Chad combined. Many people find such stark inequality troubling. Since humans exhibit diminishing marginal utility, all else being equal, economic...
Politicians stymie human colonization of space to save make-work jobs
by Roko — 2010-07-18T12:57:47.388Z

An example of the collective action failures that happen when millions of not-so-bright humans try to cooperate. From the BBC:

US President Barack Obama had laid out his vision for the future of human spaceflight. He was certain that low-Earth orbit operations should be handed to the commercial sector - the likes of SpaceX and Orbital Sciences Corp. As for Nasa, he believed it should have a much stronger R&D focus. He wanted the agency to concentrate on difficult stuff, and take its time before deciding on how America should send astronauts to distant targets such as asteroids and Mars.

This vision invited fury from many in Congress and beyond because of its likely impact in those key States where the re-moulding of the agency would lead to many job losses - in Florida, Texas, Alabama and Utah.

The continued provision of seed funding to the commercial sector to help it develop low-cost "space taxis" capable of taking astronauts to and from the ISS. The funding arrangements would change, however. Instead of the White House's original request for $3.3bn over three years, the Committee's approach would provide $1.3bn. (Obama had wanted some $6bn in total over five years; the Committee says the total may still be possible, but over a longer period)

Make-work bias and pork-barrel funding are not exactly news, but in this case they are exerting a direct negative influence on the human race's chances of survival.

Opinion in singularitarian circles has gradually shifted to under-emphasizing the importance of space colonization for the survival of the human race. The justification is that if a uFAI is built, we're all toast, and if an FAI is built, it can build spacecraft that make the Falcon 9 look like a paper aeroplane.

However, the development of any kind of AI may be preceded by a period where humanity has to survive nano- or bio-disasters, which space colonization definitely helps to mitigate. Before or soon after we develop cheap, advanced nanotechnology, we could already have a self-sustaining colony on the moon (though this would require NASA to get its ass in gear).

I leave you with an artist's impression of the physical embodiment of government inefficiency, a spacecraft optimized to make work rather than to advance the prospects of the future of the human race:

The Space Shuttle cost $1.5 billion ...
Fight Zero-Sum Bias
by multifoliaterose — 2010-07-18T05:57:11.759Z

This is the first part of a mini-sequence of posts on zero-sum bias and the role that it plays in our world today.

One of the most pernicious of all human biases is zero-sum bias. A situation involving a collection of entities is zero-sum if one entity's gain is another's loss, whereas a situation is positive-sum if the entities involved can each achieve the best possible outcome by cooperating with one another. Zero-sum bias is the tendency to systematically assume that positive-sum situations are zero-sum situations. This bias is arguably the major obstacle to a Pareto-efficient society. As such, it's very important that we work to overcome this bias (both in ourselves and in broader society).

Here I'll place this bias in context and speculate on its origin.

Where this bias comes from

It's always a little risky to engage in speculation about human evolution. We know so little about our ancestral environment that our mental images of it might be totally wrong. Nevertheless, the predictions of evolutionary speculation sometimes agree with empirical results, so it's not to be dismissed entirely. Also, the human mind has an easier time comprehending and remembering information when the information is embedded in a narrative, so that speculative stories can play a useful cognitive role even when wrong.

Anatomically modern humans appear to have emerged 200,000 years ago. In the context of human history, economic growth is a relatively recent discovery, only beginning in earnest several thousand years ago. The idea that it was possible to create wealth was probably foreign to our ancestors. In The Bottom Billion, former director of Development Research at the World bank speculates on the motivation of rebels in the poorest and slowest growing countries in the world who start civil wars (despite the fact that there's a high chance of being killed as a rebel and the fact that civil wars are usually damaging to the countries involved)

[In the portion of the developing world outside of the poorest and slowest growing countries...] growth rates may not sound sensational, but they are without precedent in history. They imply that children in these countries will grow up to have lives dramatically different from those of their parents. Even when people are still poor, these societies can be suffused with hope: time is on their side...If low income and slow growth make a co

...
Is cryonicists’ selfishness distance induced?
by KatjaGrace — 2010-07-17T18:40:52.000Z

Tyler‘s criticism of cryonics, shared by others including me at times:

Why not save someone else’s life instead?

This applies to all consumption, so is hardly a criticism of cryonics, as people pointed out. Tyler elaborated that it just applies to expressive expenditures, which Robin pointed out still didn’t pick out cryonics over the the vast assortment of expressive expenditures that people (who think cryonics is selfish) are happy with. So why does cryonics instinctively seem particularly selfish?

I suspect the psychological reason cryonics stands out as selfish is that we rarely have the opportunity to selfishly splurge on something so far in the far reaches of far mode as cryonics, and far mode is the standard place to exercise our ethics.

Cryonics is about what will happen in a *long time* when you *die* to give you a *small chance* of waking up in a *socially distant* society in the *far future*, assuming you *widen your concept* of yourself to any *abstract pattern* like the one manifested in your biological brain and also that technology and social institutions *continue their current trends* and you don’t mind losing *peripheral features* such as your body (not to mention cryonics is *cold* and seen to be the preserve of *rich* *weirdos*).

You’re not meant to be selfish in far mode! Freeze a fair princess you are truly in love with or something. Far mode livens our passion for moral causes and abstract values. If Robin is right, this is because it’s safe to be ethical about things that won’t affect you yet it still sends signals to those around you about your personality. It’s a truly mean person who won’t even claim someone else a long way away should have been nice fifty years ago. So when technology brings the potential for far things to affect us more, we mostly don’t have the built in selfishness required to zealously chase the offerings.

This theory predicts that other personal expenditures on far mode items will also seem unusually selfish. Here are some examples of psychologically distant personal expenditures to test this:

space tourism

donating to/working on life extension because you want to live forever

traveling in far away socially distant countries without claiming you are doing it to benefit or respect the locals somehow

astronomy for personal gain

buying naming rights to stars

...
Book Club Update, Chapter 3 of Probability Theory
by Morendil — 2010-07-16T08:25:20.056Z

Previously: Book Club introductory post - Chapter 1 - Chapter 2

We will shortly move on to Chapter 3 (I have to post this today owing to vacation - see below). I have updated the previous post with a summary of chapter 2, with links to the discussion as appropriate. But first, a few announcements.

How to participate

This is both for people who have previously registered interest, as well as newcomers. This spreadsheet is our best attempt at coordinating 90+ Less Wrong readers interested in participating in "earnest study of the great literature in our area of interest".

If you are still participating, please let the group know - all you have to do is fill in the "Active (Chapter)" column. Write in an "X" if you are checked out, or the number of the chapter you are currently reading. This will let us measure attrition, as well as adapt the pace if necessary. If you would like to join, please add yourself to the spreadsheet. If you would like to participate in live chat about the material, please indicate your time zone and preferred meeting time. As always, your feedback on the process itself is more than welcome.

Refer to the Chapter 1 post for more details on how to participate and meeting schedules.

Facilitator wanted

I'm taking off on vacation today until the end of the month. I'd appreciate if someone wanted to step into the facilitator's shoes, as I will not be able to perform these duties in a timely manner for at least the next two weeks.

Chapter 3: Elementary Sampling Theory

Having derived the sum and product rules, Jaynes starts us in on a mainstay of probability theory, urn problems.

Readings for the week of 19/07: Sampling Without Replacement - Logic versus Propensity. Exercises: 3.1

Discussion starts here.
Chicago/Madison Meetup
by steven0461 — 2010-07-15T23:30:15.576Z

After a successful first Chicago meetup, Airedale and I are now looking forward to the first nonfirst meetup. There are a few different options:

Hold the meetup in Chicago on a weekend afternoon, like last time. That could mean the 31st or 1st, or maybe the 24th or 25th. Unless there's massive support for another Hyde Park meetup, we propose trying out a different part of town, such as the North side or the Loop.

Hold the meetup in Chicago on the evening of Tuesday the 27th. The upside is LW/SIAI regulars Will Newsome and Kevin plan to be in town.

Will and Kevin plan to visit Madison on the 28th and some other cities in the days after, where they'll want to hold meetups if there's interest. They will post details. (Airedale and I won't be able to attend, though we may be able to attend a future weekend Madison meetup.)

Any thoughts on what's convenient for most people, and on what's a good venue?

If you're interested, there's a Google Group where you can sign up for further updates. Hope to see you soon.
Financial incentives don't get rid of bias? Prize for best answer.
by Roko — 2010-07-15T13:24:59.276Z

I'm trying to better understand the relationship between incentivization and rationality, and it occurred to me that it is a "folk fact" around here that large financial incentives don't make cognitive biases go away.

However, I can't seem to find any papers that actually say this. It's not easy to google for (I have tried) so I wonder if the Less Wrong collective memory knows how to find the papers?

Is there a pattern to which biases go away with incentivization? Do we have at least 5 examples of biases that go away with incentivization and 5 examples that don't go away with incentivization?

As an incentive, I'll paypal $10 to the commenter whose answer is least biased and most useful.