trentmkelly/LessWrong-43k huggingface dataset:
AI risk bibliography (draft)
by lukeprog — 2012-04-07T18:46:36.295Z

Generally, only sources including an extended analysis of AI risk are included, though there are some exceptions among the earliest sources. Listed sources discuss either the likelihood of AI risk or they discuss possible solutions. (This does not include most of the "machine ethics" literature, unless an article discusses machine ethics in the explicit context of artificial intelligence as an existential risk.)

Please let me know what i missed!

Butler, Samuel [Cellarius, pseud.]. 1863. Darwin among the machines. Christchurch Press, June 13. http: //www.nzetc.org/tm/scholarly/tei-ButFir-t1-g1-t1-g1-t4-body.html.

Good, Irving John. 1959. Speculations on perceptrons and other automata. Research Lecture, RC-115. IBM, Yorktown Heights, New York, June 2. http://domino.research.ibm.com/library/cyberdig.nsf/papers/58DC4EA36A143C218525785E00502E30/$File/rc115.pdf.

Good, Irving John. 1965. Speculations concerning the first ultraintelligent machine. In Advances in computers, ed. Franz L. Alt and Morris Rubinoff, 31–88. Vol. 6. New York: Academic Press. doi:10.1016/S0065-2458(08)60418-0.

Good, Irving John. 1970. Some future social repercussions of computers. International Journal of Environmental Studies 1 (1–4): 67–79. doi:10.1080/00207237008709398.

Versenyi, Laszlo. 1974. Can robots be moral? Ethics 84 (3): 248–259. http://www.jstor.org/stable/2379958.

Good, Irving John. 1982. Ethical machines. In Machine intelligence, ed. J. E. Hayes, Donald Michie, and Y.-H. Pao, 555–560. Vol. 10. Intelligent Systems: Practice and Perspective. Chichester: Ellis Horwood.

Minsky, Marvin. 1984. Afterword to Vernor Vinge’s novel, “True Names.” Oct. 1. http://web.media.mit.edu/~minsky/papers/TrueNames.Afterword.html (accessed Mar. 26, 2012).

Moravec, Hans P. 1988. Mind children: The future of robot and human intelligence. Cambridge, MA: Harvard University Press.

Crevier, Daniel. 1993. The silicon challengers in our future. Chap. 12 in AI: The tumultuous history of the search for artificial intelligence. New York: Basic Books.

Vinge, Vernor. 1993. The coming technological singularity: How to survive in the post-human era. In Vision- 21: Interdisciplinary science and engineering in the era of cyberspace, 11–22. NASA Conference Publication 10129. NASA Lewis Research Center. http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19

...
[link] TEDxYale - Keith Chen - The Impact of Language on Economic Behavior 
by Grognor — 2012-04-07T17:20:27.624Z

http://www.youtube.com/watch?v=CiobJhogNnA

The short version is that if the language you speak requires different verbs for the present and the future, it causes you to think about it differently. Depending on the magnitude of the effect, this has important implications for construal level theory. If your language allows you to think about the future in Near mode, it may allow you to think about it more rationally.

Previous discussion on one of Keith Chen's papers here.
AI Risk & Opportunity: Strategic Analysis Via Probability Tree
by lukeprog — 2012-04-07T08:25:58.208Z

Part of the series AI Risk and Opportunity: A Strategic Analysis.

(You can leave anonymous feedback on posts in this series here. I alone will read the comments, and may use them to improve past and forthcoming posts in this series.)

There are many approaches to strategic analysis (Bishop et al. 2007). Though a morphological analysis (Ritchey 2006) could model our situation in more detail, the present analysis uses a simple probability tree (Harshbarger & Reynolds 2008, sec. 7.4) to model potential events and interventions.

A very simple tree

In our initial attempt, the first disjunction concerns which of several (mutually exclusive and exhaustive) transformative events comes first:

"FAI" = Friendly AI.

"uFAI" = UnFriendly AI, not including uFAI developed with insights from WBE.

"WBE" = Whole brain emulation.

"Doom" = Human extinction, including simulation shutdown and extinction due to uFAI striking us from beyond our solar system.

"Other" = None of the above four events occur in our solar system, perhaps due to stable global totalitarianism or for unforeseen reasons.

Our probability tree begins simply:

Each circle is a chance node, which represents a random variable. The leftmost chance node above represents the variable of whether FAI, uFAI, WBE, Doom, or Other will come first. The rightmost chance nodes are open to further disjunctions: the random variables they represent will be revealed as we continue to develop the probability tree.

Each left-facing triangle is a terminal node, which for us serves the same function as a utility node in a Bayesian decision network. The only utility node in the tree above assigns a utility of 0 (bad!) to the Doom outcome.

Each branch in the tree is assigned a probability. For the purposes of illustration, the above tree assigns .01 probability to FAI coming first, .52 probability to uFAI coming first, .07 probability to WBE coming first, .35 to Doom coming first, and .05 to Other coming first.

How the tree could be expanded

The simple tree above could be expanded "downstream" by adding additional branches:

We could also make the probability tree more actionable by trying to estimate the probability of desirable and undesirable outcomes given certain that certain shorter-term goals are met. In the example below, "private push" means that a non-state actor passionate about safety invests $30 bill...
[SEQ RERUN] No Individual Particles
by MinibearRex — 2012-04-07T01:34:45.069Z

Today's post, No Individual Particles was originally published on 18 April 2008. A summary (taken from the LW wiki):

One of the chief ways to confuse yourself while thinking about quantum mechanics, is to think as if photons were little billiard balls bouncing around. The appearance of little billiard balls is a special case of a deeper level on which there are only multiparticle configurations and amplitude flows. It is easy to set up physical situations in which there exists no fact of the matter as to which electron was originally which.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Feynman Paths, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
binomial variance problem
by nerfhammer — 2012-04-06T22:59:55.015Z

Found in an old Kahneman & Tversky paper:

There are two programs in a high school. Boys are a majority (65%) in program A, and a minority (45%) in program B. There is an equal number of classes in each of the two programs.

You enter a class at random, and observe that 55% of the students are boys. What is your best guess -- does the class belong to program A or to program B?
Decision Theories: A Semi-Formal Analysis, Part II
by orthonormal — 2012-04-06T18:59:35.787Z

Or: Causal Decision Theory and Substitution

Previously:

0. Decision Theories: A Less Wrong Primer
1. The Problem with Naive Decision Theory

Summary of Post: We explore the role of substitution in avoiding spurious counterfactuals, introduce an implementation of Causal Decision Theory and a CliqueBot, and set off in the direction of Timeless Decision Theory.

In the last post, we showed the problem with what we termed Naive Decision Theory, which attempts to prove counterfactuals directly and pick the best action: there's a possibility of spurious counterfactuals which lead to terrible decisions. We'll want to implement a decision theory that does better; one that is, by any practical definition of the words, foolproof and incapable of error...

I know you're eager to get to Timeless Decision Theory and the others. I'm sorry, but I'm afraid I can't do that just yet. This background is too important for me to allow you to skip it...

Over the next few posts, we'll create a sequence of decision theories, each of which will outperform the previous ones (the new ones will do better in some games, without doing worse in others0) in a wide range of plausible games.

Let's formalize the setup a bit more: we've written a program X, which is paired with another program Y in a game G. Both X and Y have access to the source codes of X, Y, and G, and G calculates the payouts to the programmers of X and of Y in terms of their outputs alone. That is, X and Y should be functions of two inputs, and G should do the following:

Calculate the output of X(code G, 1, code Y).1

Calculate the output of Y(code G, 2, code X).

From these values, calculate the payoffs to the programmers.

We write the expected payout to X's programmer as U, by which we mean U(output X, output Y), by which we really mean U(output X(code G, 1, code Y), output Y(code G, 2, code X)); similarly, we'll write V as the expected payout to Y's programmer. We'll need to use quining in order for G to feed its own source code into X and Y, but this is straightforward. (I said earlier that X should have access to its own source code as well, but the programmer can do this via quining, with no help from G.)

We're also assuming that X is equipped with a valid inference module that deduces some true statements and no false ones; it can return "true", "false" or "unknown" when given a statement. (One such example is an automated ...
Meetup : London
by Paul Crowley — 2012-04-06T16:42:02.277Z

Discussion article for the meetup : London

WHEN: 15 April 2012 02:00:00PM (+0100)

WHERE: LShift, Hoxton Point, 6 Rufus St, London, N1 6PE

First London meetup in a while, will be good to see everyone again! To talk about how we'll use the time, join the London Less Wrong mailing list: http://groups.google.com/group/lesswronglondon

Discussion article for the meetup : London
Meetup : Budapest Meetup
by  — 2012-04-06T16:11:37.513Z

Discussion article for the meetup : Budapest Meetup

WHEN:
07 April 2012 05:30:00PM (+0100)

WHERE:
Szent Istvan ter 4-5, Budapest

Meeting at California Coffee Company Basilica (coffee shop), Szent Istvan ter 4-5. link.

Please come and bring friends. If you have questions, contact katyusha.

BudLW mailing list: link.

Discussion article for the meetup : Budapest Meetup
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-04-06T15:49:49.621Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
11 April 2012 07:00:00PM (-0600)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

And again with the conversational coolness.

New skill acquisition: slack lining, yoga, soldering, negotiation.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm
Meetup : Longmont  Sparkfun Soldering Competition Field Trip
by EvelynM — 2012-04-06T15:39:00.316Z

Discussion article for the meetup : Longmont Sparkfun Soldering Competition Field Trip

WHEN:
28 April 2012 11:00:00AM (-0600)

WHERE:
1555 S Hover Rd Longmont, Colorado 80501

Sparkfun is sponsoring a Soldering Competition from 11am to 5pm April 28 in Longmont CO.
http://www.sparkfun.com/products/11126
Soldering starts at 12.

Still lots of spots open to enter.

Discussion article for the meetup : Longmont Sparkfun Soldering Competition Field Trip
Weekly LW Meetups: Berkeley, Madison, Melbourne, Norwich
by FrankAdamek — 2012-04-06T15:00:07.382Z

There are upcoming irregularly scheduled Less Wrong meetups in:

First Norwich UK Meetup Sunday 1 April 11am: 01 April 2012 11:00AM
Brussels meetup: 14 April 2012 11:00AM
Shanghai Less Wrong Meetup: 15 April 2012 10:36PM
Sydney meetup - Biased pandemic and other games: 17 April 2012 07:30PM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Monday Madison Meetup: 02 April 2012 06:30PM
Big Berkeley meetup: 04 April 2012 06:30PM
Melbourne, practical rationality: 06 April 2012 08:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, London, Madison WI, Melbourne, Mountain View, New York, Ohio, Ottawa, Oxford, Portland, Seattle, Toronto, Waterloo, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Helsinki, London, Marin CA, Pittsburgh, Salt Lake City, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver, Washington, DC.

If your meetup has a mailing list that you'd like mentioned here, or has become regular and isn't listed as such, let me know!

Want to help out the common good? If one of the meetups listed as regular has become inactive, let me know so we can present more accurate information to newcomers.

Also: Nickolai Leschov is collecting...
Meetup : Twin Cities, MN (for real this time)
by FrankAdamek — 2012-04-06T14:50:54.773Z

Discussion article for the meetup : Twin Cities, MN (for real this time)

WHEN:
15 April 2012 01:00:00PM (-0700)

WHERE:
Purple Onion Coffeeshop, 1301 University Avenue Southeast, Minneapolis, MN

[This meetup is really being put together by Helloses, but while his karma is just getting started I'm putting the text into a post on his behalf.]

Hi. Let's make this work.

Suggested discussion topics would be:

What do we want this group to do? Rationality practice? Skill sharing? Mastermind group?

Acquiring guinea pigs for the furtherance of mad science (testing Center for Modern Rationality material)

Fun - what it is and how to have almost more of it than you can handle

If you'd like to suggest a location closer to you or a different time, please comment to that effect. If you know a good coffeeshop with ample seating in Uptown or South Minneapolis, we could meet there instead. Also comment if you'd like to carpool.

If you're even slightly interested in this, please join up or at least comment.

Folks, let's hang out and take it from there.

Discussion article for the meetup : Twin Cities, MN (for real this time)
Peter Singer and Tyler Cowen transcript
by jefftk — 2012-04-06T12:25:27.082Z

In March 2009, Tyler Cowen (blog) interviewed Peter Singer about morality, giving, and how we can most improve the world. They are both thinkers I respect a lot, and I was excited to read their debate. Unfortunately the interview was available only as a video. I wanted a transcript, so I made one:

Cowen:
This is Tyler Cowen of George Mason University. I'm doing a BloggingHeads with Peter Singer, the world-famous philosopher from Princeton. This is a forum on Peter's latest book, which he'll start off by telling you a bit about.

Singer:
Hi. The book's called "The Life You Can Save: Acting Now To End World Poverty". It begins with an argument that I've used many times in articles about a child drowning in a pond, and suggests that if you saw a child drowning in a pond that you would jump in and save that child, and you think that is what you ought to do, even if it meant that you ruined an expensive pair of shoes that you were wearing.

From there I pull back to saying "what does this mean about the problem of world poverty, given that there are, according to Unicef, ten million children dying of avoidable poverty-related causes every year?" We could save some of them, and probably it wouldn't cost us much more than the cost of an expensive pair of shoes if we find an effective aid agency that is doing something to combat the causes of world poverty, or perhaps to combat the deaths of children from simple conditions like diarrhea or measles, conditions that are not that hard to prevent or to cure. We could probably save a life for the cost of a pair of shoes. So why don't we? What's the problem here? Why do we think it's ok to live a comfortable, even luxurious, life while children are dying? In the book I explore various objections to that view, I don't find any of them really convincing. I look at some of the psychological barriers to giving, and I acknowledge that they are problems. And I consider also some of the objections to aid and questions raised by economists as to whether aid really works. In the end I come to a proposal by which I want to change the culture of giving.

The aim of the book in a sense is to get us to internalize the view that not to do anything for those living in poverty, when we are living in luxury and abundance, is ethically wrong, that it's not just not a nice thing to do but that a part of living an ethically decent life is a

...
Looking for reductionism help
by  — 2012-04-06T06:34:18.932Z

I have read the sequences on reductionism and quantum physics some time ago now and I was hoping for some help finding the right places to go back and re-read there to address a question. If the way I describe my question reveals other ignorance on my part, please feel free to add comments above and beyond sequence references.

When trying to talk a little about reductionism, most (non-LW) people I speak to seem to want to play the following game:
What's an airplane made out of? Molecules and atoms that comprise materials like metal, plastic, glass, rubber, etc. What are molecules and atoms made out of? Well, molecules are collections of atoms bonded together, and atoms are made up of three basic particles: protons, neutrons, and electrons. What are basic particles made out of? Well, here things start to get a little more dicey. Some of the basic particles are known to be made up of even smaller sub-atomic elementary particles, such as quarks, leptons, and bosons. Some of the basic particles are examples of these elementary particles. Well, what's an elementary particle made of? Well, that's a pretty tough one, but basically there's this sort of fabric of stuff underlying everything called quantum amplitude, and a certain configuration of quantum amplitude corresponds to an elementary particle. So what's quantum amplitude made up of? Well, I'm not sure that is a coherent question. It just sort of is. A ha! I've caught you. So ultimately way down at the bottom of it all, you're telling me that some something "just exists" (i.e. is ontologically basic). But then why do you call it reductionism if it ultimately boils down to a Platonistic ideal of quantum amplitude (no one actually says this, but it's my translation of the objections I tend to face).
Is it more or less right to say that, as far as we can tell, the only reasonable thing to which we can attribute ontologically basic status is quantum amplitude? Given that amplitude is a mathematical device that allows calculation of probability, and probability describes my ignorance about the world (i.e. my best guess as to what the territory is, as opposed to the actual territory), do we view quantum amplitude as some sort of pre-states-of-knowledge concept? How does that mesh up with "what is an elementary particle made of?" It makes me want to call it "the residue of intrinsic uncertainty" or some...
A Poisson process paradox
by nerfhammer — 2012-04-06T05:20:01.130Z

From Steven Pinker's latest book:

Suppose you live in a place that has a constant chance of being struck by lightning at any time throughout the year. Suppose that the strikes are random: every day the chance of a strike is the same, and the rate works out to one strike a month. Your house is hit by lightning today, Monday. What is the most likely day for the next bolt to strike your house?

.

.

.

.

.

.

The answer is “tomorrow,” Tuesday. That probability, to be sure, is not very high; let’s approximate it at 0.03 (about once a month). Now think about the chance that the next strike will be the day after tomorrow, Wednesday. For that to happen, two things have to take place. First lightning has to strike on Wednesday, a probability of 0.03. Second, lightning can’t have struck on Tuesday, or else Tuesday would have been the day of the next strike, not Wednesday. To calculate that probability, you have to multiply the chance that lightning will not strike on Tuesday (0.97, or 1 minus 0.03) by the chance that lightning will strike on Wednesday (0.03), which is 0.0291, a bit lower than Tuesday’s chances. What about Thursday? For that to be the day, lightning can’t have struck on Tuesday (0.97) or on Wednesday either (0.97 again) but it must strike on Thursday, so the chances are 0.97×0.97×0.03, which is 0.0282. What about Friday? It’s 0.97×0.97×0.97×0.03, or 0.274. With each day, the odds go down (0.0300 … 0.0291 … 0.0282 … 0.0274), because for a given day to be the next day that lightning strikes, all the previous days have to have been strike-free, and the more of these days there are, the lower the chances are that the streak will continue. To be exact, the probability goes down exponentially, accelerating at an accelerating rate. The chance that the next strike will be thirty days from today is 0.97^29x0.03, barely more than 1 percent.

Almost no one gets this right. I gave the question to a hundred Internet users, with the word next italicized so they couldn’t miss it. Sixty-seven picked the option “every day has the same chance.” But that answer, though intuitively compelling, is wrong. If every day were equally likely to be the next one, then a day a thousand years from now would be just as likely as a day a month from now. That would mean that the hou

...
[SEQ RERUN] Feynman Paths
by MinibearRex — 2012-04-06T04:58:25.173Z

Today's post, Feynman Paths was originally published on 17 April 2008. A summary (taken from the LW wiki):

Instead of thinking that a photon takes a single straight path through space, we can regard it as taking all possible paths through space, and adding the amplitudes for every possible path. Nearly all the paths cancel out - unless we do clever quantum things, so that some paths add instead of canceling out. Then we can make light do funny tricks for us, like reflecting off a mirror in such a way that the angle of incidence doesn't equal the angle of reflection. But ordinarily, nearly all the paths except an extremely narrow band, cancel out - this is one of the keys to recovering the hallucination of classical physics.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Quantum Arena, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.