trentmkelly/LessWrong-43k huggingface dataset:
Uploading: what about the carbon-based version?
by NancyLebovitz — 2012-07-23T08:49:29.722Z

In this video, long about 48:00, Eliezer talks about uploading and about how it wouldn't be murder if his meat body were anesthetized before the upload and killed without regaining consciousness.

It's arguable that it wouldn't be murder, but I'm not clear about why Eliezer would want to do it that way. I've got some guesses about why one might want to not let the meat body wake up (legal and practical complications of a double but diverging identity, the meat version feeling hopelessly envious), but I'm not sure whether either of them apply.

On the other hand, I can think of a couple of reasons for *not* eliminating the meat version-- one is that two Eliezers would presumably be better than one, though I don't have a strong intuition about the optimum number of Eliezers. The other, which I consider to be more salient, is that the meat version is a backup in case the upload isn't as good as hoped.

More generally, what would folks here consider to be good enough evidence that uploading was worth doing?
[SEQ RERUN] Contaminated by Optimism
by MinibearRex — 2012-07-23T03:36:18.842Z

Today's post, Contaminated by Optimism was originally published on 06 August 2008. A summary (taken from the LW wiki):

Avoid situations, as much as you possibly can, in which optimistic thinking suggests ideas for conscious consideration. In real life problems, if you've done that, you've probably already screwed up.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Anthropomorphic Optimism, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Washington DC Biased Boardgames meetup
by rocurley — 2012-07-23T02:10:21.208Z

Discussion article for the meetup : Washington DC Biased Boardgames meetup

WHEN:
05 August 2012 03:00:00PM (-0400)

WHERE:
Arlington, VA

The next meetup will be Sunday, August 12th, at 3pm, at a private residence. PM me or email the list for details.

The topic will be biased boardgames . If you don't want to read the link, basically we will be assigning biases for people to exaggerate while we play a game (most likely pandemic, but maybe others). Suggestions for games would be fantastic, as would be bringing them.

Discussion article for the meetup : Washington DC Biased Boardgames meetup
The Perception-Action Cycle
by royf — 2012-07-23T01:44:25.808Z

Would readers be interested in a sequence of posts offering an intuitive explanation of my underway thesis on the application of information theory to reinforcement learning? Please also feel free to comment on the quality of my presentation.

In this first post I offer a high-level description of the Perception-Action Cycle as an intuitive explanation of reinforcement learning.

Imagine that the world is divided into two parts: one we shall call the agent and the rest - its environment. Imagine that the two interact in turns. One moment the agent receives information from its environment in the form of an observation. Then the next moment the agent sends out information to its environment in the form of an action. Then it makes another observation, then another action, and so on.

To break down the cycle, we start with the agent having a belief about the state of its environment. This is actually the technical term: the belief is the probability that the agent assigns, implicitly, to each possible state of the environment. The cycle then proceeds in 4 phases.

In the first phase, the agent makes an observation. Since the observation conveys information of the environment, the agent needs to update its belief, ideally using Bayes' theorem. The agent now has more information about the environment.

In the second phase, the agent uses this new information to update its plan. Note the crucial underlying principle that information about the environment is useful in making better plans. This gives a desired fusion between Bayesian updates and decision making.

In the third phase, the agent executes a step of its plan - a single action. This changes the environment. Some of the things that the agent knew about the previous state of the environment may no longer be true, and the agent is back to having less information.

In the fourth phase, the agent makes a prediction about future observations. The importance of making a prediction before a scientific experiment is well understood by philosophers of science. But the importance of constantly making predictions of all of our sensory inputs as a functional part of our cognition, is only now dawning on neuroscientists and machine learning researchers.

The Perception-Action Cycle is an intuitive explanation of the technical setting of reinforcement learning. Reinforcement learning...
Meetup : Brussels meetup
by Axel — 2012-07-22T21:50:24.055Z

Discussion article for the meetup : Brussels meetup

WHEN:
11 August 2012 12:00:00PM (+0200)

WHERE:
Rue des Alexiens 55 1000 Bruxelles

We're trying out a new location this time as the museum cafeteria can be a bit loud sometimes. We're meeting at 'La Fleur en Papier Doré' close to the Brussels Central station (this should also improve train travel) We'll be having some nice intelligent discussions and I'll hopefully have a second version of the learning-Bayes-game.

If you're in the neighborhood consider dropping by!

EDIT: I realize the google maps marker is in a slightly different place then the 'La Fleur en Papier Doré' marker, this comes from a spelling error on their site. The red marker is the correct one, you can check with street-view.

Discussion article for the meetup : Brussels meetup
Meetup : Cambridge, MA Meetup
by jimrandomh — 2012-07-22T15:05:10.642Z

Discussion article for the meetup : Cambridge, MA Meetup

WHEN:
22 July 2012 02:00:00PM (-0400)

WHERE:
25 Ames St, Cambridge

The usual schedule is to have meetups on the first and third Sundays of each month, but some cool Less Wrongers are in town who've been away for awhile, so let's also have one tomorrow! (Sunday the 22nd). The usual time and place (2:00pm, 25 Ames St, Cambridge, room number posted at the entrance based on availability). No formal agenda, just socializing and intellectual conversations.

Discussion article for the meetup : Cambridge, MA Meetup
Meetup : Dublin, Ireland Meetup
by Laoch — 2012-07-22T13:52:13.521Z

Discussion article for the meetup : Dublin, Ireland Meetup

WHEN:
12 August 2012 03:00:28PM (+0100)

WHERE:
College Green

For anybody in Ireland, more specifically in Dublin, interested in meeting up.
College Green Starbucks beside the wax museum.
This has been postponed a week.

Discussion article for the meetup : Dublin, Ireland Meetup
[SEQ RERUN] Anthropomorphic Optimism
by MinibearRex — 2012-07-22T06:10:07.374Z

Today's post, Anthropomorphic Optimism was originally published on 04 August 2008. A summary (taken from the LW wiki):

You shouldn't bother coming up with clever, persuasive arguments for why evolution will do things the way you prefer. It really isn't listening.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was No Logical Positivist I, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
HP:MOR and the Radio Fallacy
by RichardChappell — 2012-07-21T19:55:33.199Z

No plot spoilers here, just wanted to flag a bit of poor reasoning that shows up in Chapter 39:

I shouldn't have believed it even for all of thirty seconds! Because if people had souls there wouldn't be any such thing as brain damage, if your soul could go on speaking after your whole brain was gone, how could damage to the left cerebral hemisphere take away your ability to talk?

This is a surprisingly common fallacy. Just because X depends on Y, it doesn't follow that X depends on nothing but Y. A phenomenon may involve more than just its most obvious failure point.

To illustrate: Suppose I'm trapped in a box, and my only way to communicate with the outside world is via radio communication. Someone on the other end argues that I don't really exist -- "There's no person beyond the radio receiver, for if there was then there wouldn't be any such thing as damaged radios!" Pretty silly, huh? But people say this kind of thing in defense of physicalism all the time.

(N.B. This is not to defend the existence of souls. It's just to point out that this particular argument against them is invalid.)
Work on Security Instead of Friendliness?
by Wei Dai — 2012-07-21T18:28:44.692Z

So I submit the only useful questions we can ask are not about AGI, "goals", and other such anthropomorphic, infeasible, irrelevant, and/or hopelessly vague ideas. We can only usefully ask computer security questions. For example some researchers I know believe we can achieve virus-safe computing. If we can achieve security against malware as strong as we can achieve for symmetric key cryptography, then it doesn't matter how smart the software is or what goals it has: if one-way functions exist no computational entity, classical or quantum, can crack symmetric key crypto based on said functions. And if NP-hard public key crypto exists, similarly for public key crypto. These and other security issues, and in particular the security of property rights, are the only real issues here and the rest is BS.

-- Nick Szabo

Nick Szabo and I have very similar backrounds and interests. We both majored in computer science at the University of Washington. We're both very interested in economics and security. We came up with similar ideas about digital money. So why don't I advocate working on security problems while ignoring AGI, goals and Friendliness?

In fact, I once did think that working on security was the best way to push the future towards a positive Singularity and away from a negative one. I started working on my Crypto++ Library shortly after reading Vernor Vinge's A Fire Upon the Deep. I believe it was the first general purpose open source cryptography library, and it's still one of the most popular. (Studying cryptography led me to become involved in the Cypherpunks community with its emphasis on privacy and freedom from government intrusion, but a major reason for me to become interested in cryptography in the first place was a desire to help increase security against future entities similar to the Blight described in Vinge's novel.)

I've since changed my mind, for two reasons.

1. The economics of security seems very unfavorable to the defense, in every field except cryptography.

Studying cryptography gave me hope that improving security could make a difference. But in every other security field, both physical and virtual, little progress is apparent, certainly not enough that humans might hope to defend their property rights against smarter intelligences. Achieving "security against malware as strong as we can achieve for symmetric key cryptography" seems quite ...
[SEQ RERUN] No Logical Positivist I
by MinibearRex — 2012-07-21T05:26:56.807Z

Today's post, No Logical Positivist I was originally published on 04 August 2008. A summary (taken from the LW wiki):

Logical positivism was based around the idea that the only meaningful statements were those that could be verified by experiment. Unfortunately for positivism, there are meaningful statements that are very likely true and very likely false, and yet cannot be tested.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Comedy of Behaviorism, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
[LINK] Inferring the rate of psychopathy from roadkill experiment
by JaneQ — 2012-07-20T20:10:23.755Z

Pardon the sensationalist headline of that article:

Mark says that "one thing that might explain the higher numbers here—in case people question my methods—is that I used a tarantula." Apparently, people seemed pretty eager about hitting a spider. "If you take that out it goes to 2.8% which is closer to the other turtle vs. snake studies I ended up finding."

It is still quite a surprisingly high number. At least compared to a 2008 study using the Psychopathy Checklist, which discovered that 1.2 percent of the US population were potential psychopaths. 1.2 vs 2.8 is a huge difference.

I was not aware of the other turtle and snake studies.

Note that with turtle this is the lower bound on percentage of evil; a perfectly amoral person that could e.g. kill for modest and unimportant sum of money or any other reason would still have no incentive to steer to drive over a turtle; and a significant percentage of people would simply fail to notice the turtle entirely.

This gives interesting prior for mental model of other people. Even at couple percent, psychopathy is much more common than notable intelligence or many other situations considered 'rare' or 'unlikely'. It appears to me that due to the politeness and the necessary good-until-proven-evil strategy, many people act as if they have an incredibly low prior for psychopathy, which permits easy exploitation by psychopaths. There may also be signaling reasons for pretending to have very low prior for psychopathy as one of the groups of people with high prior for psychopathy is psychopaths themselves; pretending easily becomes too natural, though.

Perhaps adjusting the priors could improve personal safety and robustness with regards to various forms of exploitation, whenever the priors are set incorrectly.
Meetup : Fort Collins, Colorado Meetup Thursday 7pm *New Place*
by EvelynM — 2012-07-20T19:08:59.468Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Thursday 7pm *New Place*

WHEN:
26 July 2012 07:00:00PM (-0600)

WHERE:
112 S. College Ave, Fort Collins, CO 80524

Flinching, try bad, project management, Game Theory.
And, for those who still eat carbs, the best cake in town is not a lie.

Discussion article for the meetup : Fort Collins, Colorado Meetup Thursday 7pm *New Place*
Mass-murdering neuroscience Ph.D. student
by James_Miller — 2012-07-20T17:02:52.624Z

A Ph.D student in neuroscience shot at least 50 people at a showing of the new Batman movie. He also appears to have released some kind of gas from a canister. Because of his educational background this person almost certainly knows a lot about molecular biology. How long will it be (if ever) before a typical bio-science Ph.D will have the capacity to kill, say,a million people?

Edit: I'm not claiming that this event should cause a fully informed person to update on anything. Rather I was hoping that readers of this blog with strong life-science backgrounds could provide information that would help me and other interested readers assess the probability of future risks. Since this blog often deals with catastrophic risks and the social harms of irrationality and given that the events I described will likely dominate the U.S. news media for a few days I thought my question worth asking. Given the post's Karma rating (currently -4), however, I will update my beliefs about what constitutes an appropriate discussion post.
Weekly LW Meetups: Brussels, Sydney
by FrankAdamek — 2012-07-20T15:16:38.128Z

This summary was posted to LW main on July 13th, and has now been moved to discussion.

There are upcoming irregularly scheduled Less Wrong meetups in:

Brussels meetup: 14 July 2012 12:00PM
Less Wrong Sydney 16th July Event for Less Wrong: 16 July 2012 06:35PM
Washington DC Calibration Games meetup: 22 July 2012 03:00AM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

(NYC) A Game of Nomic: 21 July 2012 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, Chicago, Madison WI, Melbourne, Mountain View, New York, Ohio, Oxford, Portland, Salt Lake City, Seattle, Toronto, Waterloo, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Berlin, Helsinki, London, Marin CA, Ottawa, Pittsburgh, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver, Washington, DC.

If your meetup has a mailing list that you'd like mentioned here, or has become regular and isn't listed as such, let me know!

Want to help out the common good? If one of the meetups listed as regular has become inactive, let me know so we can present more accurate information to newcomers.
[link] Prepared to wait? New research challenges the idea that we favour small rewards now over bigger later
by Kaj_Sotala — 2012-07-20T08:17:27.145Z

http://bps-research-digest.blogspot.com/2012/07/prepared-to-wait-new-research.html

The old idea that we make decisions like rational agents has given way over the last few decades to a more realistic, psychologically informed picture that recognises the biases and mental short-cuts that sway our thinking. Supposedly one of these is hyperbolic discounting - our tendency to place disproportionate value on immediate rewards, whilst progressively undervaluing distant rewards the further in the future they stand. But not so fast, say Daniel Read at Warwick Business School and his colleagues with a new paper that fails to find any evidence for the phenomenon.
SI's Summer 2012 Matching Drive Ends July 31st
by lukeprog — 2012-07-20T05:48:12.760Z

The Singularity Institute's summer 2012 matching drive ends on July 31st! Donate by the end of the month to have your gift matched, dollar for dollar.

As of this posting, SI has raised $70,000 of the $150,000 goal.

The announcement says:

Since we published our strategic plan in August 2011, we have achieved most of the near-term goals outlined therein...

In the coming year, the Singularity Institute plans to do the following:

Hold our annual Singularity Summit, this year in San Francisco!

Spin off the Center for Applied Rationality as a separate organization focused on rationality training, so that the Singularity Institute can be focused more exclusively on Singularity research and outreach.

Publish additional research on AI risk and Friendly AI.

Eliezer will write an "Open Problems in Friendly AI" sequence for Less Wrong. (For news on his rationality books, see here.)

Finish Facing the Singularity and publish ebook versions of Facing the Singularity and The Sequences, 2006-2009.

And much more! For details on what we might do with additional funding, see How to Purchase AI Risk Reduction.

If you're planning to earmark your donation to CFAR (Center for Applied Rationality), here's a preview of what CFAR plans to do in the next year:

Develop additional lessons teaching the most important and useful parts of rationality. CFAR has already developed and tested over 18 hours of lessons so far, including classes on how to evaluate evidence using Bayesianism, how to make more accurate predictions, how to be more efficient using economics, how to use thought experiments to better understand your own motivations, and much more.

Run immersive rationality retreats to teach from our curriculum and to connect aspiring rationalists with each other. CFAR ran pilot retreats in May and June. Participants in the May retreat called it “transformative” and “astonishing,” and the average response on the survey question, “Are you glad you came? (1-10)” was a 9.4. (We don't have the June data yet, but people were similarly enthusiastic about that one.)

Run SPARC, a camp on the advanced math of rationality for mathematically gifted high school students. CFAR has a stellar first-year class for SPARC 2012; most students admitted to the program placed in the top 50 on the USA Math Olympiad (or performed equivalently in a similar contest).

Coll

...
[SEQ RERUN] The Comedy of Behaviorism
by MinibearRex — 2012-07-20T04:10:15.107Z

Today's post, The Comedy of Behaviorism was originally published on 02 August 2008. A summary (taken from the LW wiki):

The behaviorists thought that speaking about anything like a mind, or emotions, or thoughts, was unscientific. After all, they said, you can't observe anger. You can just observe behavior. But, it is possible, using empathy, to correctly predict wide varieties of behavior, which you can't account for by pavlovian conditioning.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Detached Lever Fallacy, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
[LINK] Anchoring Bias In Medicine (NYT)
by  — 2012-07-20T03:31:31.705Z

http://well.blogs.nytimes.com/2012/07/19/falling-into-the-diagnostic-trap/

For the doctors, this was a harrowing lesson in the trap of anchoring bias. It is so easy to slip into it without even knowing. But this case reminded us to keep reciting the mantra: if something doesn’t fit, don’t try to make it fit. Ask what else might be going on. Don’t fall into the trap.

(Wikipedia's article: http://en.wikipedia.org/wiki/Anchoring )