trentmkelly/LessWrong-43k huggingface dataset:
AI Risk and Opportunity: A Strategic Analysis
by lukeprog — 2012-03-04T06:06:01.435Z

Suppose you buy the argument that humanity faces both the risk of AI-caused extinction and the opportunity to shape an AI-built utopia. What should we do about that? As Wei Dai asks, "In what direction should we nudge the future, to maximize the chances and impact of a positive intelligence explosion?"

This post serves as a table of contents and an introduction for an ongoing strategic analysis of AI risk and opportunity.

Contents:

Introduction (this post)

Humanity's Efforts So Far

A Timeline of Early Ideas and Arguments

Questions We Want Answered

Strategic Analysis Via Probability Tree

Intelligence Amplification and Friendly AI

...

Why discuss AI safety strategy?

The main reason to discuss AI safety strategy is, of course, to draw on a wide spectrum of human expertise and processing power to clarify our understanding of the factors at play and the expected value of particular interventions we could invest in: raising awareness of safety concerns, forming a Friendly AI team, differential technological development, investigating AGI confinement methods, and others.

Discussing AI safety strategy is also a challenging exercise in applied rationality. The relevant issues are complex and uncertain, but we need to take advantage of the fact that rationality is faster than science: we can't "try" a bunch of intelligence explosions and see which one works best. We'll have to predict in advance how the future will develop and what we can do about it.

Core readings

Before engaging with this series, I recommend you read at least the following articles:

Muehlhauser & Salamon, Intelligence Explosion: Evidence and Import (2013)

Yudkowsky, AI as a Positive and Negative Factor in Global Risk (2008)

Chalmers, The Singularity: A Philosophical Analysis (2010)

Example questions

Which strategic questions would we like to answer? Muehlhauser (2011) elaborates on the following questions:

What methods can we use to predict technological development?

Which kinds of differential technological development should we encourage, and how?

Which open problems are safe to discuss, and which are potentially dangerous?

What can we do to reduce the risk of an AI arms race?

What can we do to raise the "sanity waterline," and how much will this help?

What can we do to attract more funding, support, and research to x-risk reduction and to specific sub-problems of successful Singularit

...
Request for input: draft of my "coming out" statement on religious deconversion
by jwhendy — 2012-03-03T21:58:20.038Z

Edited 3/4/2012: I shortened up the summary a bit and add the following update:

Thanks for the lively comments. As a preliminary summary of things I've found quite useful/helpful:

Shorten/transform the document (David_Gerard)

Remove/postpone any reasons (TimS)

Don't be so prosy/fake sounding (orthonormal)

Show to religious people, not just LW (AlexMennen) -- doing that, btw

Give reasons (Will_Newsome)

Overall, I was initially a bit discouraged. This took a lot of effort and so it's frustrating to contemplate changing directions. Due to reading a very generously shared similar PDF document (from someone I'll leave anonymous unless they'd like to be named), I'm much more upbeat about doing so. I plan to:

Create two documents. One very simple, plain-language, frank relating of the fact that I no longer believe in god. I'd like to write it just as though I were saying it personally to someone, easing them into hearing this (like Bugmaster suggested, except that actually doing this in person is impractical for me)

The second will be my actual list of reasons. I think it will be valuable to actually spell them out, and many will want to know reasons anyway (and probably ask)

A question that came up in me from the comments below is what worth this actually brings about. I don't find myself compelled to write a dissertation-style document defending my power tool purchases, Linux custom kernel options, or why I listen to the music I do. I am aware of a desire for validation, to feel that I've done enough with respect to my "quest," to prove myself on this topic. I'm still wrestling with whether this is completely irrational and unnecessary, or only partially so, validated by the fact of my social/environmental circumstances that do present some real obstacles that this document could help alleviate.

Open to any thoughts on that last bit as well. Thanks again for the valuable input.

It's almost one year later, and I've finally made tangible progress on some of the input suggested in my post about being non-religious in a primarily religious environment. That is, I have a near-final draft of a "coming out" statement I plan to share with a majority of those who know me.

I was involved in two religious communities for about six years of my life (SPO and CCR). Two years post-deconversion from Catholicism, many of them still do not know I no longer...
[Link] The emotional system (aka Type 1 thinking) might excel at complex decisions
by XiXiDu — 2012-03-03T19:05:59.564Z

For thousands of years, human beings have looked down on their emotions. We’ve seen them as primitive passions, the unfortunate legacy of our animal past. When we do stupid things – say, eating too much cake, or sleeping with the wrong person, or taking out a subprime mortgage – we usually blame our short-sighted feelings. People commit crimes of passion. There are no crimes of rationality.

This bias against feeling has led people to assume that reason is always best. When faced with a difficult dilemma, most of us believe that it’s best to carefully assess our options and spend a few moments consciously deliberating the information. Then, we should choose the alternative that best fits our preferences. This is how we maximize utility; rationality is our Promethean gift.

[...] it’s only in the last few years that researchers have demonstrated that the emotional system (aka Type 1 thinking) might excel at complex decisions, or those involving lots of variables.

[...]

The latest demonstration of this effect comes from the lab of Michael Pham at Columbia Business School. The study involved asking undergraduates to make predictions about eight different outcomes, from the Democratic presidential primary of 2008 to the finalists of American Idol. They forecast the Dow Jones and picked the winner of the BCS championship game. They even made predictions about the weather.

Here’s the strange part: although these predictions concerned a vast range of events, the results were consistent across every trial: people who were more likely to trust their feelings were also more likely to accurately predict the outcome. [...]

Consider the results from the American Idol quiz: while high-trust-in-feelings subjects correctly predicted the winner 41 percent of the time, those who distrusted their emotions were only right 24 percent of the time. The same lesson applied to the stock market, that classic example of a random walk: those emotional souls made predictions that were 25 percent more accurate than those who aspired to Spock-like cognition.

[...] the unconscious brain is able to process vast amounts of information in parallel, thus allowing it to analyze large data sets without getting overwhelmed. (Human reason, in contrast, has a very strict bottleneck and can only process about four bits of data at any given moment.)

[...] how do we gain acc

...
[LINK] Why You Should Keep Your Goals Secret
by Barry_Cotter — 2012-03-03T13:40:48.113Z

Popularisation, extremely short

Original Article [pdf]

When intentions go public: does social reality widen the intention-behavior gap?

Gollwitzer PM, Sheeran P, Michalski V, Seifert AE.

Source

New York University, Psychology Department, New York, NY 10003, USA. peter.gollwitzer@nyu.edu

Abstract

Based on Lewinian goal theory in general and self-completion theory in particular, four experiments examined the implications of other people taking notice of one's identity-related behavioral intentions (e.g., the intention to read law periodicals regularly to reach the identity goal of becoming a lawyer). Identity-related behavioral intentions that had been noticed by other people were translated into action less intensively than those that had been ignored (Studies 1-3). This effect was evident in the field (persistent striving over 1 week's time; Study 1) and in the laboratory (jumping on opportunities to act; Studies 2 and 3), and it held among participants with strong but not weak commitment to the identity goal (Study 3). Study 4 showed, in addition, that when other people take notice of an individual's identity-related behavioral intention, this gives the individual a premature sense of possessing the aspired-to identity.

If you have tags to suggest please do and I'll edit them in.
[SEQ RERUN] Savanna Poets
by MinibearRex — 2012-03-03T08:51:31.651Z

Today's post, Savanna Poets was originally published on 18 March 2008. A summary (taken from the LW wiki):

Equations of physics aren't about strong emotions. They can inspire those emotions in the mind of a scientist, but the emotions are not as raw as the stories told about Jupiter (the god). And so it might seem that reducing Jupiter to a spinning ball of methane and ammonia takes away some of the poetry in those stories. But ultimately, we don't have to keep telling stories about Jupiter. It's not necessary for Jupiter to think and feel in order for us to tell stories, because we can always write stories with humans as its protagonists.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Fake Reductionism, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Rationality Quotes March 2012
by Thomas — 2012-03-03T08:04:55.112Z

Here's the new thread for posting quotes, with the usual rules:

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself

Do not quote comments/posts on LW/OB

No more than 5 quotes per person per monthly thread, please.
Is it obvious that pain is very important?
by KatjaGrace — 2012-03-03T05:04:55.000Z

“Never, for any reason on earth, could you wish for an increase of pain. Of pain you could wish only one thing: that it should stop. Nothing in the world was so bad as physical pain. In the face of pain there are no heroes, no heroes […]. –George Orwell, 1984 via Brian Tomasik , who seems to agree that just considering pain should be enough to tell you that it’s very important.

It seems quite a few people I know consider pain to have some kind of special status of badness, and that preventing it is thus much more important than I think it. I wouldn’t object, except that they apply this in their ethics, rather than just their preferences regarding themselves. For instance arguing that other people shouldn’t have children, because of the possibility of those children suffering pain. I think pain is less important to most people relative to their other values than such negative utilitarians and similar folk believe.

One such argument for the extreme importance of pain is something like ‘it’s obvious’. When you are in a lot of pain, nothing seems more important than stopping that pain. Hell, even when you are in a small amount of pain, mitigating it seems a high priority. When you are looking at something in extreme pain, nothing seems more important than stopping that pain. So pain is just obviously the most important bad thing there is. The feeling of wanting a boat and not having one just can’t compare to pain. The goodness of lying down at the end of a busy day is nothing next to the badness of even relatively small pains.

I hope I do this argument justice, as I don’t have a proper written example of it at hand.

An immediate counter is that when we are not in pain, or directly looking at things in pain, pain doesn’t seem so important. For instance, though many people in the thralls of a hangover consider it to be pretty bad, they are repeatedly willing to trade half a day of hangover for an evening of drunkenness. ‘Ah’, you may say, ‘that’s just evidence that life is bad – so bad that they are desperate to relieve themselves from the torment of their sober existences! So desperate that they can’t think of tomorrow!’. But people have been known to plan drinking events, and even to be in quite good spirits in anticipation of the whole thing.

It is implicit in the argument from ‘pain seems really bad close up’ that pain does not...
Self-locating beliefs across identity fission
by lukeprog — 2012-03-03T03:14:32.333Z

From Wolfgang Schwarz's Belief Dynamics Across Fission:

Fred’s home planet, Sunday, is surrounded by two moons, Monday and Tuesday. Tonight, while Fred is asleep, his body will be scanned and destroyed; then a signal is sent to both Monday and Tuesday where he will be recreated from local matter.

A lot of ink has been spent on how to describe scenarios like this. Should we say that Fred will find himself both on Monday and on Tuesday? Which of the persons awakening on the two moons is identical to the person going to sleep on Sunday? In this paper, I want to look at a different question: what should Fred’s successors believe when they awaken on Monday and on Tuesday? More precisely, how should their beliefs be related to Fred’s beliefs before he went to sleep on Sunday?

Also see: Sleeping Beauty problem.
Meetup : Austin, TX
by Vaniver — 2012-03-03T01:14:10.412Z

Discussion article for the meetup : Austin, TX

WHEN:
03 March 2012 01:30:00PM (-0600)

WHERE:
2222B Guadalupe St Austin, Texas 78705

The Austin Meetup continues! Explore UT will be going on at the same day, so expect pretty heavy traffic around campus.

We'll be at Caffe Medici again, from about 1:30 to about 4:30, sitting on the second floor to the left by the stage. We hope to see you there!

Discussion article for the meetup : Austin, TX
Weekly LW Meetups: Buenos Aires, Cambridge UK, Chicago, Loveland/Fort Collins CO, Melbourne, Tucson, Twin Cities
by FrankAdamek — 2012-03-03T00:04:11.988Z

There are upcoming irregularly scheduled Less Wrong meetups in:

Tucson Meetup: 24 February 2012 07:00PM
Loveland CO (Fort Collins) 3D printer tour: 24 February 2012 02:15PM
Buenos Aires meetup: Saturday, February 25th, 4pm: 25 February 2012 04:00PM
Chicago Meetup at the University of Chicago: 25 February 2012 01:00PM
Tucson Meetup: 02 March 2012 07:00PM
Twin Cities South Metro: 06 March 2012 08:00PM
[Ohio/Washington DC] Interest in Reason Rally meetup?: 24 March 2012 04:14PM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Cambridge UK: 26 February 2012 11:00AM
Melbourne practical rationality meetup: 02 March 2012 07:00PM
Cambridge UK: 04 March 2012 11:00AM
Ohio Monthly: 18 March 2012 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, London, Madison WI, Melbourne, Mountain View, New York, Ohio, Ottawa, Oxford, Portland, San Francisco, Seattle, Toronto, Washington, DC, Waterloo, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Helsinki, London, Marin CA, Pittsburgh, Salt Lake City, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver.

If your meetup has a mailing list that you'd like mentioned here or has...
People who "don't rationalize"?  [Help Rationality Group figure it out]
by Mercurial — 2012-03-02T23:38:56.712Z

Anna Salamon and I are confused. Both of us notice ourselves rationalizing on pretty much a daily basis and have to apply techniques like the Litany of Tarski pretty regularly. But in several of our test sessions for teaching rationality, a handful of people report never rationalizing and seem to have little clue what Tarski is for. They don't relate to any examples we give, whether fictitious or actual personal examples from our lives. Some of these people show signs of being rather high-level rationalists overall, although some don't.

So, Less Wrong, we're asking for your input on this one. What do you think is going on?
[LINK] An amazing breakthrough in wireless communication, or a pipe dream?
by Shmi — 2012-03-02T22:17:42.258Z

Encoding many channels on the same frequency through radio vorticity: first experimental test

"We have shown experimentally, in a real-world setting, that it is possible to use two beams of incoherent radio waves, transmitted on the same frequency but encoded in two different orbital angular momentum states, to simultaneously transmit two independent radio channels. This novel radio technique allows the implementation of, in principle, an infinite number of channels in a given, fixed bandwidth, even without using polarization, multiport or dense coding techniques. This paves the way for innovative techniques in radio science and entirely new paradigms in radio communication protocols that might offer a solution to the problem of radio-band congestion."

"Moreover, our experimental findings demonstrate that the spatial phase signature was preserved even in the far-field region and for incoherent non-monochromatic wave beams. These results open up new perspectives not only for wireless communication but also for physics and astronomy, including the possible detection of Kerr black holes in the test general relativity"

This looks too good to be true, but I cannot see any obvious issues, and they have an experimental confirmation.

If this pans out, it would be a black swan in the making for many of the wireless spectrum allocation/licensing authorities and companies.
Random personal thoughts on workload pessimism as a procrastination problem
by  — 2012-03-02T21:06:48.382Z

I am in one of those moods where I feel extremely burned out and apathetic. What I am noticing about my feelings right now is that there is no short term reward system that could possibly motivate me. I don't want any candy, food, soda, coffee, alcohol, nor even money, entertainment, sexual excitement, sleep, exercise, or play of any kind. I know that's not an exhaustive list of potential short term rewards, but it's fairly exhaustive for the kinds of things that I could reasonably bring about in my own life in the short term. The thing right now that makes me feel bad is the idea that my future work time horizon (FWTH), the perceived amount of future time that will have to be devoted to doing work, is very full. Whenever I perceive a full FWTH, it causes my mind to recoil and become extremely apathetic.

It seems to me that the only kind of reward system that motivates me to feel productive is having completely unabated free time: large blocks of time with literally no named obligations. I'm sure this is not profound or new, but it is interesting to me that this seems to be a dominant part of my psyche. If I think that my marginal effort put towards work right this minute will yield a large expanse of completely and utterly free time in the reasonably near future, then I feel very motivated to work hard. If I perceive that, no matter how hard I work right this minute, my future schedule is already bloated with work that I have yet to even get started on, then I feel very tired, cranky, and just want to lay in bed apathetically and watch cartoons on repeat.

I don't know a good method for dealing with this sort of procrastination effect. It doesn't seem to fit into the normal procrastination equation, but at first approximation there are a few ways that it might. One is that I could be very impulsive about free time. Given that I have been a grad student for the past 3 years, this is plausible. As a grad student, the time when I can truly relax and unwind from work is very random and scattered, and often even when I think that I have legitimate relaxation time, I am interrupted by email requests for work that needs to be done right away, or the revelation of a new, difficult homework assignment that will create very negative consequences if I don't start working diligently on it immediately. Even my undergraduate college life was stressful in this r

...
Journal of Consciousness Studies issue on the Singularity
by lukeprog — 2012-03-02T15:56:16.185Z

...has finally been published.

Contents:

Uziel Awret - Introduction

Susan Blackmore - She Won’t Be Me

Damien Broderick - Terrible Angels: The Singularity and Science Fiction

Barry Dainton - On Singularities and Simulations

Daniel Dennett - The Mystery of David Chalmers

Ben Goertzel - Should Humanity Build a Global AI Nanny to Delay the Singularity Until It’s Better Understood?

Susan Greenfield - The Singularity: Commentary on David Chalmers

Robin Hanson - Meet the New Conflict, Same as the Old Conflict

Francis Heylighen - Brain in a Vat Cannot Break Out

Marcus Hutter - Can Intelligence Explode?

Drew McDermott - Response to ‘The Singularity’ by David Chalmers [this link is a McDermott-corrected version, and therefore preferred to the version that was published in JCS]

Jurgen Schmidhuber - Philosophers & Futurists, Catch Up!

Frank Tipler - Inevitable Existence and Inevitable Goodness of the Singularity

Roman Yampolskiy - Leakproofing the Singularity: Artificial Intelligence Confinement Problem

The issue consists of responses to Chalmers (2010). Future volumes will contain additional articles from Shulman & Bostrom, Igor Aleksander, Richard Brown, Ray Kurzweil, Pamela McCorduck, Chris Nunn, Arkady Plotnitsky, Jesse Prinz, Susan Schneider, Murray Shanahan, Burt Voorhees, and a response from Chalmers.

McDermott's chapter should be supplemented with this, which he says he didn't have space for in his JCS article.
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-03-02T15:26:21.584Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
07 March 2012 07:00:00PM (-0700)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

It's almost spring, though there is sometimes snow on the ground. Come hang out and chat with interesting people.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm