trentmkelly/LessWrong-43k huggingface dataset:
[MORESAFE] Prevention of the global catastrophe and human values
by turchin — 2011-10-27T21:12:27.372Z

The chanses of prevention of the global catastrophe are growing if humans have the goal of it. This is semi-trivial conclusion. But the main question is who should have such goal?
Of course, if we have global government, its main goal should be prevention of global catastrophe. But we do not have global government and most people hate the idea. I find it irrational. But discussion about global goverment is pure theoretical one, because I do not see peaceful ways of its creation.
If friendly AI take over the world ve will became global government de facto.

Or if imminent global risk will be recognized (asteroid is near), UN could temporaly transform in some kind of global government.
But some people think that global government itself will be or will soon lead to the global catastrophe - because it could easily implement global measures - and predicate "global" is nessesary to global catastrophes, as I am going to show in one of next posts. For example it could implement global total vaccination that lately will have dangerous consciences.
So we see that idea of global government is very closely connected with idea of global catastrophe. One could lead to another and back.
But as we do not have global government we could only speak about goals of separate people and separate organizations.
People do not have goals. The only think that they have goals, but these are only declarations, which rarely regulate real people's behavior. This is because human beings are not born as rational subjects and their behavior is mostly regulated by unconscious programs known as instincts.
These instincts are culturally adapted as values. Values are the real reasons of human behavior. "Goals" are what people say about their reasons to others and to themselves.
The problem of how human values influent the course of human history is difficult one. Last year I wrote a book "Futurology. 21 century: immortality or global catastrophe" in Russian together with M.Batin. And the chapter about values was the most difficult one.
Values are always based on instincts , pleasure and collective behavior (values help to form groups of people who share them). Value is always emotion, it has energy to move a person.
But self-preservation is basic human instinct and so prevention of death and global catastrophe could be human value.

Each value need a group of supporters to exist (val...
A pessimistic view of quantum immortality
by anotheruser — 2011-10-27T18:04:38.606Z

You have probably read about the idea of quantum immortality before. The basic idea seems to be that, as anything that can happen does happen (assuming either that the many-worlds interpretation of quantum theory is true, or that there is an infinite number of parallel universes wherein "you" exist) and it is impossible to remember your own death, every living thing is immortal.

Take a game of russian roulette as an example: In those universes in which you die, you are no longer alive enough to care about that fact, leaving only those universes relevant in which you survive. This would make playing russian roulette for money a valid financial strategy, by the way.

However, I think that this view ignores a very important fact: death is not binary. You are not either alive or dead, but may exist in various intermediate forms of suffering and reduced cognitive abilities. This means that what actually happens when you play russian roulette is the following:

In those universes in which you win, everything is fine. In those in which you lose, however, you now have a gaping head wound. I assume that this hurts a lot, at least in those instances where you still have enough mental capacity to actually feel anything. Due to some fluke however (remember that absolutely all possible scenarios happen), you may still be alive and in a lot of pain. Most instances of you will then die from bloodloss or something, but for every timestep afterwards there will alway be an infinite number of universes wherein you continue to live, in most of them in complete agony.

The instances of you in the other worlds that were never shot will be blissfully unaware of this fact.

Now consider that you will also reach such a state of perpetual-agony-close-to-death-but-never-quite-reaching-it in everyday live. In fact, an infinite number of alternate "you"s, having split off from your everett branch just a second ago, are now suffering through this.

The ratio of "you"s in your current state to those in the one described above is very high, as the probability of continued survival in such a state for any amount of time is infinitesimal, but it does exist. Consider however, that this ratio decreases massively as you age and that virtually all instances of you will be in such a state 200 years from now unless immortality is achieved.

There is one bright spot, however:

As time marches on, the continuous e...
Vasili Arkhipov Day
by David Althaus — 2011-10-27T06:27:20.018Z

Stanislav Petrov is a rather famous person (of course only on Lesswrong, not in the real world).

But there is another Russian who saved the world: Vasili Alexandrovich Arkhipov.

On this day in 1962, at the height of the Cuban Missile Crisis Vasili Arkhipov prevented the launch of a nuclear torpedo and thus a possible nuclear war.

It's strange that Petrov attracts much more attention than Arkhipov. E.g. googling "Stanislav Petrov" produces 101.000 results, "Vasili Arkhipov" only 9.040 results. By contrast searching for "Britney Spears" generates about 295.000.000 results. Sorta depressing.

Anyway, let this day be the Vasili Arkhipov Day.
Subjective and Objective Reality: My Essay
by TheatreAddict — 2011-10-27T03:18:07.808Z

I want to start out by saying that I'm an ultra-beginner when it comes to stuff like this, but I had to do a definition essay in English, and I chose to contrast objective and subjective reality. So, if any of this seems wrong to you, please let me know. I'm sort of stuck right now, because I'm not really sure what to write about next. I'm thinking maybe the history of how the terms came to be or something. Also, for citations, I'm struggling with MLA formatting, so I might not have something in quotations cited, let me know if you want the cites for something.

Regardless of whether or not one watches a phenomenon occur, it still, in fact, occurs. Or does it? This argument is known as objective versus subjective reality. While objective reality initially seems to be the more obvious and substantial answer, fairly recent discoveries in the world of quantum physics have caused new questions to spring up regarding this ancient belief. Whether or not reality occurs outside of the human mind or exists only inside of the consciousness is a question that, despite countless arguments, remains unresolved. Despite arguments that insist on promoting the idea that reality is entirely dependent on our fallible perceptions (subjective reality), the evidence remains in favor that there is indeed a real world that exists independent of our observations (objective reality).
Subjective reality is defined as “relating to or of the nature of an object as it is known in the mind as distinct from a thing in itself.” Proponents of this theory generally believe that reality is relative, and so if someone cannot see an object in front of them, it ceases to exist. Objective reality, in contrast, means “existing independent of thought or an observer as part of reality.” Objective reality usually implies that everything can be proven using rationality, science, and mathematics. Although people use their mind to interpret and to put into practice this information, it still exists outside of the mind to be observed.
If there existed a picture, in a room, two people could agree that the picture is objective, and occurs outside of their mind. However, one could find the picture to be quite lovely, and another could detest it with a passion. These two ideas are not necessarily wrong, but are subjective. Based on this thought, some would suggest that reality is merel

...
Meetup : Fort Collins, Colorado Meetup Wednesday 7pm
by EvelynM — 2011-10-27T03:03:22.774Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wednesday 7pm

WHEN:
02 November 2011 07:00:00PM (-0600)

WHERE:
Bean Cycle, 144 North College Avenue, Fort Collins, CO 80524

Another week, another weekly meetup. Stay warm with community, now that winter is here.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wednesday 7pm
[SEQ RERUN] Terminal Values and Instrumental Values
by MinibearRex — 2011-10-27T02:20:39.086Z

Today's post, Terminal Values and Instrumental Values was originally published on 15 November 2007. A summary (taken from the LW wiki):

Discusses a formalism for a discussion of the relationship between terminal and instrumental values. Terminal values are world states that we assign some sort of positive or negative worth to. Instrumental values are links in a chain of events that lead to desired world states.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Thou Art Godshatter, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Rival formalizations of a decision problem
by lukeprog — 2011-10-27T02:03:48.857Z

Decision theory is not one of my strengths, and I have a question about it.

Is there a consensus view on how to deal with the problem of "rival formalizations"? Peterson (2009) illustrates the problem like this:

Imagine that you are a paparazzi photographer and that rumour has it that actress Julia Roberts will show up in either New York (NY), Los Angeles (LA) or Paris (P). Nothing is known about the probability of these states of the world. You have to decide if you should stay in America or catch a plane to Paris. If you stay and [she] shows up in Paris you get $0; otherwise you get your photos, which you will be able to sell for $10,000. If you catch a plane to Paris and Julia Roberts shows up in Paris your net gain after having paid for the ticket is $5,000, and if she shows up in America you for some reason, never mind why, get $6,000. Your initial representation of the decision problem is visualized in Table 2.13.

Table 2.13

P
LA
NY

Stay
$0
$10k
$10k

Go to Paris
$5k
$6k
$6k

Since nothing is known about the probabilities of the states in Table 2.13, you decide it makes sense to regard them as equally probable [see Table 2.14].

Table 2.14

P (1/3)
LA (1/3)
NY (1/3)

Stay
$0
$10k
$10k

Go to Paris
$5k
$6k
$6k

The rightmost columns are exactly parallel. Therefore, they can be merged into a single (disjuntive) column, by adding the probabilities of the two rightmost columns together (Table 2.15).

Table 2.15

P (1/3)
LA or NY (2/3)

Stay
$0
$10k

Go to Paris
$5k
$6k

However, now suppose that you instead start with Table 2.13 and first merge the two repetitious states into a single state. You would then obtain the decision matrix in Table 2.16.

Table 2.16

P
LA or NY

Stay
$0
$10k

Go to Paris
$5k
$6k

Now, since you know nothing about the probabilities of the two states, you decide to regard them as equally probable... This yields the formal representation in Table 2.17, which is clearly different from the one suggested above in Table 2.15.

Table 2.17

P (1/2)
LA or NY (1/2)

Stay
$0
$10k

Go to Paris
$5k
$6k

Which formalisation is best, 2.15 or 2.17? It seems question begging to claim that one of them must be better than the other — so perhaps they are equally reasonable? If they are, we have an example of rival formalisations.

Note that the principle of maximising expect

...
Meetup : First Zurich LW Meetup
by Wakefield — 2011-10-26T20:18:50.267Z

Discussion article for the meetup : First Zurich LW Meetup

WHEN:
12 November 2011 03:00:00PM (+0200)

WHERE:
near Sihlcity, Zurich

For any Zurich-based LWers - let's meet up. I suggest resilience as a topic for discussion. Afterwards, we can figure out whether to do something more regularly.

Meet in our offices near Sihlcity between 2-3pm, then head out for food/drink depending on interests.

PM me for exact address and contact details if you would like to attend.

Discussion article for the meetup : First Zurich LW Meetup
Rhetoric for the Good
by lukeprog — 2011-10-26T18:52:24.123Z

The topics of rationality and existential risk reduction need their own Richard Dawkins. Their own Darwin. Their own Voltaire.

Rhetoric moves minds.

Students and masochists aside, people read only what is exciting. So: Want to make an impact? Be exciting. You must be heard before you can turn heads in the right direction.

Thus, I've decided to try harder and actually put effort into the quality of my writing instead of just cranking stuff out quickly so I can fill in inferential gaps and get to the cutting edge of the research subjects I care about.

That's why I asked LWers for their picks of best nonfiction writing on Less Wrong.

It's also why I've been reading lots of good science writing, focusing on those who manage to be exciting while covering fairly complex subjects: Dawkins, Sagan, Gleick, Zimmer, Shermer, Ramachandran, Roach, Sacks, Hawking, Greene, Hofstadter, Penrose, Wilson, Feynman, Kaku, Gould, Bryson, Pinker, Kurzban, and others.

I've also been re-reading lots of books and articles on how to write well: Keys to Great Writing, Style: Lessons in Clarity and Grace, Elements of Style, On Writing Well, The Classic Guide to Better Writing, The Book on Writing, Telling True Stories, Writing Tools, Ideas into Words, The Chicago Guide to Communicating Science, A Field Guide for Science Writers, Six Rules for Rewriting, Writing, Briefly, and Singularity Writing Advice. (Conversations with Eliezer also helped.)

I don't know if I can become the Voltaire of rationality and existential risk reduction, but it seems worth a shot. Every improvement in writing style is beneficial even if my starry goal is never met. Also, it appears I produce better writing without really trying than most people produce with trying. (If you've ever had to grade essays by honors English seniors, you'll know what I mean.) I expect to gain more by striving where I already excel than by pushing where I have little natural talent.

(I won't try to write everything well. Sometimes I should just crank things out. To be honest, I didn't spend much time optimizing this post.)

My other hope is that a few other writers decide they would like to be the Voltaire of rationality and/or existential risk reduction. May this post be useful to them. It's a list of recommendations on writing style pulled from many sources, in no particular order.

Begin in m

...
[LINK] Non-Conformists Better At Working Toward Common Good
by  — 2011-10-26T17:38:57.622Z

First read about this on Para Pundit, relevant link here.

If you follow the pack are you more likely to co-operate with others in it? Not necessarily according to research into social behaviour by academics at the University of East Anglia.

The study, published in the August issue of the journal Personality and Individual Differences, shows that people who do not conform are most likely to work together for the greater good, while conforming to social norms can actually make people less likely to co-operate – a finding which surprised the researchers and could have implications in the workplace for team design and operations management.

"Here we've got a measure of people's co-operation, which could apply to any situation where you've two or more people who are trying to co-operate in an activity. For example in a work setting, if you are part of a team working on a project you expect everyone to put the same effort in to the task. The expectation is that people who are high in social desirability will conform to the effort other people are putting into the task, but actually the conforming people may be less helpful because they take their cue from the less helpful members of the team. They are conforming to the person who is not necessarily working that hard."
[MORESAFE] Starting global risk discussion
by turchin — 2011-10-26T16:38:16.977Z

I am writing to propose an online discussion of global risk within the discussion part of Less Wrong. We might call this discussion "More Safe". In future it could be a site where anyone interested could discuss global risks, possibly to aggregate all existing information about global risks. The idea comes from discussions at the Singularity summit that I had with Anna Solomon, Seth Baum, and others.

I propose labeling such discussions "More Safe". Less wrong means more safe. Fewer mistakes means fewer catastrophes.

At Seth's suggestion, we should be careful to follow safety guidelines for such discussions. For example, no technical detail should be posted online about topics which could be used by potential terrorists especially in biology. The point of the discussion is to try to reduce risk, not to have open discussion of risky ideas.

Here are some further thoughts on my idea:

Intelligence appeared as an instrument of adaptation which lead to longer survival of an individual or a small group. Unfortunately it was not adapted as an instrument for survival of technological civilizations. So we have to somehow update our intelligence. One way to do it is to reduce personal cognitive biases.

Another way is to make our intelligence collective. Collective intelligence is more effective in finding errors and equilibrium - democracy and free markets are examples. Several people and organizations dedicated themselves to preventing existential risks.
But we do not see a place which is accepted as main discussion point on existential risks.

Lifeboat Foundation has a mailing list and blog, but its themes are not strictly about existential risks (a lot of star travel) and no open forum exists.

Future of Humanity Institute has excellent collection of articles and the book but it's not a place where people could meet online.

Less Wrong was not specially dedicated to existential risks and many risks are out of its main theme (nuclear, climate and so on).

Immortality Institute has a subforum about global risks but it is not a professional discussing point.

J.Hughes has a mailing list on existential risks [x-risk].

The popular blog "extinction protocol" is full of fear mongering and focuses mostly on natural disasters like earthquakes and floods.

There are several small organizations which are created by one person and limited to...
Review of Lakoff & Johnson, 'Philosophy in the Flesh'
by lukeprog — 2011-10-26T11:57:59.441Z

Lakoff & Johnson's Philosophy in the Flesh (1999) is an ambitious 550-page attempt to rewrite philosophy from scratch given what we now know from the cognitive sciences about how human reasoning works. After reading the first page, I had some hope it would be the book I could hand to somebody who wanted to study philosophy without being corrupted by studying 2500 years worth of bad methods and wrong answers.

Yet, while I agree with the book in broad strokes and in many particular details, it has several weaknesses that make it difficult to engage at a technical level. These problems may be part of why the book has 1400+ citations on Google Scholar even though there are almost no books or articles that engage its contents in detail.

The biggest problem is that Lakoff and Johnson cover too much ground, and therefore don't have the space to defend or even explain the precise nature of the thousands of claims they make in the book. E.g.: Several times per chapter, they claim that philosophers assume X without citing a single philosopher in the business of claiming X, or even someone else claiming that philosophers often assume X.

The majority of the book (chapters 9-24) engages directly in what Eliezer calls "dissolving the question" — at least, the part where one explains the cognitive science of how particular philosophical confusions and debates arise. While these chapters are enlightening, they depend too heavily on the earlier account of metaphor, rarely draw upon other findings in cognitive science that are likely relevant, are sparse in scientific citations, and (as I've said) rarely cite actual philosophers claiming the things they say that philosophers claim.

The authors are also too unclear about their positive project: "embodied philosophy." Nowhere do they state the propositions that make up what they call "embodied philosophy," nowhere do they explicate those propositions in precise detail, and nowhere do they defend those propositions in detail from misinterpretation and objection. The reader is left with only a vague sense of what they mean by embodied philosophy, or why it should be favored over other forms of naturalistic philosophy that have been defined and defended more precisely by their proponents. (Compare, for example, Bishop & Trout's clear explication and defense of "strategic reliabilism" in Epistemology and th...
On maximising expected value
by  — 2011-10-26T11:15:44.269Z

One thing that tends to bug me about discourse round here is the notion that maximising one's expectation is the be all and end all of decisions one should make. Given any problem, one should look at it, and pick the course that maximising one's expectation. This usually ignores two major problems: what if my utility is non-linear, and what about risk aversion?

Let’s take an example: I bump into Omega, who offers me a choice: I can take a certain 1 unit of utility, or have a 1 in 10 million chance of getting 1 billion utility. The naive expectation maximiser will take that chance: after all, their expectation will be 100 units of utility, which is much better than a measly one! In all likelihood, our maximiser will walk away with nothing. It's certainly true that if this is repeated enough then we would expect our maximiser to be doing better... but a simple calculation reveals that it will have to occur around 7 million times for our maximiser to have a greater than 0.5 chance of having actually won (once or more times).

This is a problem with tiny probabilities and large utilities: some justifications of cryonics have run along the lines of a Pascal’s wager, where a small monetary investment gives one a massive utility, so large, in fact, that no matter how small the probability of cryonics it makes sense to invest. But if the probability becomes small enough, then I've probably just wasted a lot of money. After all, we only get to live once (Note that I am aware that some people have much higher probability estimates for cryonics, which is fine: I'm addressing those who do not).

Without multiple repetition, risk aversion is, I would argue, an extremely sensible strategy for utility maximisation. Of course if one believes that one will be faced with a similar choice multiple times, then one can revert back to utility maximisation. As to when one should do this, I would probably encourage one to revert when the number of repetitions, N, is large enough so that the probability of an event occurring at least once has passed some threshold, p, decided by the user. Certainly p should probably be higher than 0.5.

Lets now take another example: I am on Deal or No Deal, and there are three boxes left: $100000, $25000 and $.01. The banker has just given me a deal of $20000 (no doubt to much audience booing). Should I take that? Expected gains maximisation s...