trentmkelly/LessWrong-43k huggingface dataset:
Link: Elon Musk wants gov't oversight for AI
by polymathwannabe — 2014-10-28T02:15:24.842Z

"I'm increasingly inclined to thing there should be some regulatory oversight, maybe at the national and international level just to make sure that we don't do something very foolish."

http://www.cnet.com/news/elon-musk-we-are-summoning-the-demon-with-artificial-intelligence/#ftag=CAD590a51e
Superintelligence 7: Decisive strategic advantage
by KatjaGrace — 2014-10-28T01:01:01.415Z

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the seventh section in the reading guide: Decisive strategic advantage. This corresponds to Chapter 5.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: Chapter 5 (p78-91)

Summary

Question: will a single artificial intelligence project get to 'dictate the future'? (p78)

We can ask, will a project attain a 'decisive strategic advantage' and will they use this to make a 'singleton'?

'Decisive strategic advantage' = a level of technological and other advantages sufficient for complete world domination (p78)

'Singleton' = a single global decision-making agency strong enough to solve all major global coordination problems (p78, 83)

A project will get a decisive strategic advantage if there is a big enough gap between its capability and that of other projects.

A faster takeoff would make this gap bigger. Other factors would too, e.g. diffusion of ideas, regulation or expropriation of winnings, the ease of staying ahead once you are far enough ahead, and AI solutions to loyalty issues (p78-9)

For some historical examples, leading projects have a gap of a few months to a few years with those following them. (p79)

Even if a second project starts taking off before the first is done, the first may emerge decisively advantageous. If we imagine takeoff accelerating, a project that starts out just behind the leading project might still be far inferior when the leading project reaches superintelligence. (p82)

How large would a successful project be? (p83) If the route to superintelligence is not AI, the project probably needs to be big. If it is AI, size is less clear. If lots of insights are accumulated in open resources, and c

...
"Solving" selfishness for UDT
by Stuart_Armstrong — 2014-10-27T17:51:00.615Z

With many thanks to Beluga and lackofcheese.

When trying to decide between SIA and SSA, two anthropic probability theories, I concluded that the question of anthropic probability is badly posed and that it depends entirely on the values of the agents. When debating the issue of personal identity, I concluded that the question of personal identity is badly posed and depends entirely on the values of the agents. When the issue of selfishness in UDT came up recently, I concluded that the question of selfishness is...

But let's not get ahead of ourselves.

A selfish scenario

Using Anthropic Decision Theory, I demonstrated that selfish agents using UDT should reason in the same way that average utilitarians did - essentially behaving 'as if' SSA were true and going for even odds of heads and tail ("halfer") in the Sleeping Beauty problem.

Then Beluga posted an argument involving gnomes, that seemed to show that selfish UDT agents should reason as total utilitarians did - essentially behaving 'as if' SIA were true and going for 2:1 odds of heads and tail ("thirder") in the Sleeping Beauty problem. After a bit of back and forth, lackofcheese then refined the argument. I noticed the refined argument was solid, and incidentally made the gnomes unnecessary.

How does the argument go? Briefly, a coin is flipped and an incubator machine creates either one person (on heads) or two people (on tails), each in separate rooms.

Without knowing what the coin flip was or how many people there were in the universe, every new person is presented with a coupon that pays £1 if the coin came out tails. The question is - assuming utility is linear in money - what amount £x should the created person(s) pay for this coupon?

The argument from Beluga/lackofcheese can be phrased like this. Let's name the people in the tails world, calling them Jack and Roger (yes, they like dressing like princesses - what of it?). Each of them reasons something like this:

"There are four possible worlds here. In the tails world, I, Jack/Roger, could exist in Room 1 or in Room 2. And in the heads world, it could be either me existing in Room 1, or the other person existing in Room 1 (in which case I don't exist). I'm completely indifferent to what happens in worlds where I don't exist (sue me, I'm selfish). So if I buy the coupon for £x, I expect to make utility: 0.25(0) ...
vaccination research/reading
by freyley — 2014-10-27T17:20:09.408Z

Vaccination is probably one of the hardest topics to have a rational discussion about. I have some reason to believe that the author of http://whyarethingsthisway.com/2014/10/23/the-cdc-and-cargo-cult-science/ is someone interested in looking for the truth, not winning a side - at the very least, I'd like to help him when he says this:

I genuinely don’t want to do Cargo Cult Science so if anybody reading this knows of any citations to studies looking at the long term effects of vaccines and finding them benign or beneficial, please, be sure to post them in the comments.

I'm getting started on reading the actual papers, but I'm hoping this finds someone who's already done the work and wants to go post it on his site, or if not, someone else who's interested in looking through papers with me - I do better at this kind of work with social support.
Open thread, Oct. 27 - Nov. 2, 2014
by MrMind — 2014-10-27T08:58:10.828Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
2014 Less Wrong Census/Survey
by Scott Alexander — 2014-10-26T18:05:38.524Z

It's that time of year again.
If you are reading this post and self-identify as a LWer, then you are the target population for the Less Wrong Census/Survey. Please take it. Doesn't matter if you don't post much. Doesn't matter if you're a lurker. Take the survey.
This year's census contains a "main survey" that should take about ten or fifteen minutes, as well as a bunch of "extra credit questions". You may do the extra credit questions if you want. You may skip all the extra credit questions if you want. They're pretty long and not all of them are very interesting. But it is very important that you not put off doing the survey or not do the survey at all because you're intimidated by the extra credit questions.
It also contains a chance at winning a MONETARY REWARD at the bottom. You do not need to fill in all the extra credit questions to get the MONETARY REWARD, just make an honest stab at as much of the survey as you can.
Please make things easier for my computer and by extension me by reading all the instructions and by answering any text questions in the simplest and most obvious possible way. For example, if it asks you "What language do you speak?" please answer "English" instead of "I speak English" or "It's English" or "English since I live in Canada" or "English (US)" or anything else. This will help me sort responses quickly and easily. Likewise, if a question asks for a number, please answer with a number such as "4", rather than "four".
The planned closing date for the survey is Friday, November 14. Instead of putting the survey off and then forgetting to do it, why not fill it out right now?
Okay! Enough preliminaries! Time to take the...
***

[EDIT: SURVEY CLOSED, DO NOT TAKE!]
***
Thanks to everyone who suggested questions and ideas for the 2014 Less Wrong Census/Survey. I regret I was unable to take all of your suggestions into account, because of some limitations in Google Docs, concern about survey length, and contradictions/duplications among suggestions. The current survey is a mess and requires serious shortening and possibly a hard and fast rule that it will never get longer than it is right now.
By ancient tradition, if you take the survey you may comment saying you have done so here, and people will upvote you and you will get karma.
[Link] "The Problem With Positive Thinking"
by CronoDAS — 2014-10-26T06:50:31.836Z

Psychology researchers discuss their findings in a New York Times op-ed piece.

The take-home advice:

Positive thinking fools our minds into perceiving that we’ve already attained our goal, slackening our readiness to pursue it.

...

What does work better is a hybrid approach that combines positive thinking with “realism.” Here’s how it works. Think of a wish. For a few minutes, imagine the wish coming true, letting your mind wander and drift where it will. Then shift gears. Spend a few more minutes imagining the obstacles that stand in the way of realizing your wish.

This simple process, which my colleagues and I call “mental contrasting,” has produced powerful results in laboratory experiments. When participants have performed mental contrasting with reasonable, potentially attainable wishes, they have come away more energized and achieved better results compared with participants who either positively fantasized or dwelt on the obstacles.

When participants have performed mental contrasting with wishes that are not reasonable or attainable, they have disengaged more from these wishes. Mental contrasting spurs us on when it makes sense to pursue a wish, and lets us abandon wishes more readily when it doesn’t, so that we can go after other, more reasonable ambitions.
Alcoholics Anonymous: Much More Than You Wanted To Know
by Scott Alexander — 2014-10-26T06:15:44.000Z

[EDIT 10/27: Slight changes in response to feedback; correcting some definitions. I am not an expert in this field and will continue to make changes as I learn about them. There is a critique of this post here and other worse critiques elsewhere. My only excuse for doing this is that I am failing less spectacularly than other online sources writing about the same topic.]

I’ve worked with doctors who think Alcoholics Anonymous is so important for the treatment of alcoholism that anyone who refuses to go at least three times a week is in denial about their problem and can’t benefit from further treatment.

I’ve also worked with doctors who are so against the organization that they describe it as a “cult” and say that a physician who recommends it is no better than one who recommends crystal healing or dianetics.

I finally got so exasperated that I put on my Research Cap and started looking through the evidence base.

My conclusion, after several hours of study, is that now I understand why most people don’t do this.

The studies surrounding Alcoholics Anonymous are some of the most convoluted, hilariously screwed-up research I have ever seen. They go wrong in ways I didn’t even realize research could go wrong before. Just to give some examples:

– In several studies, subjects in the “not attending Alcoholics Anonymous” condition attended Alcoholics Anonymous more than subjects in the “attending Alcoholics Anonymous” condition.

– Almost everyone’s belief about AA’s retention rate is off by a factor of five because one person long ago misread a really confusing graph and everyone else copied them without double-checking.

– The largest study ever in the field, a $30 million effort over 8 years following thousands of patients, had no untreated control group.

Not only are the studies poor, but the people interpreting them are heavily politicized. The entire field of addiction medicine has gotten stuck in the middle of some of the most divisive issues in our culture, like whether addiction is a biological disease or a failure of willpower, whether problems should be solved by community and peer groups or by highly trained professionals, and whether there’s a role for appealing to a higher power in any public organization. AA’s supporters see it as a scruffy grassroots organization of real people willing to get their hands dirty, who can cure addicts failed time and time again by...
Seán Ó hÉigeartaigh on FHI and CSER
by KatjaGrace — 2014-10-26T02:01:00.000Z

This is the last part of the Cause Prioritization Shallow, all parts of which are available here. Previously in this series, conversations with Owen Cotton-Barratt, Paul Christiano, Paul Penley, Gordon Irlam, and Alexander Berger, and Robert Wiblin.

Nick Beckstead interviewed Seán Ó hÉigeartaigh on the Future of Humanity Institute (FHI) and the Center for the Study of Existential Risk (CSER). The notes are here.
Self-signaling the ability to do what you want
by So8res — 2014-10-26T02:00:00.000Z
Link: https://mindingourway.com/self-signaling-the-ability-to-do-what-you-want/

1

In college, I would often find that I had just a little bit too much food. Either I'd cooked too much or the food I'd ordered was just a bit too large, or whatever.

I'm sure many of you are familiar with the feeling of having four bites of food left, wanting roughly one more bite, but knowing that three bites is too few to justify saving the food for later.

(Then you either apply lots of willpower to save the food for later, or you take another bite, realize that there isn't enough food left to save, and proceed to stuff yourself.)

This is pretty much a standard instance of the sunk cost fallacy, where reasoning of the form "I can't just not eat the food, because I already paid for it" neglects the fact that the costs are already sunk. In these scenarios, the only consideration should be whether or not eating the rest of the food is be better than throwing it away. Your money, which is gone no matter what you choose, shouldn't factor into the decision.

As a student of economics, I understood the sunk cost fallacy well. But extra food didn't quite seem sunk: after all, the food would still give me more calories, and even if it made me overfull for an hour or two, it could lead me to have smaller (and thus cheaper) subsequent meals.

Or, at least, that's the argument that my internal monologue would spin up to distract me long enough for my hands to keep shoving food into my mouth.

The counterargument would go something like

First of all, many of the calories will be either wasted or harmful if I consume them now. Secondly, the cost of dinner is more dependent upon what's available than how hungry I am. Third, even if the cost of dinner is reduced, it will be reduced by maybe a dollar, and a few hours of discomfort is not worth a dollar.

But by then, it would already be too late; the food would be gone and I'd be overfull.

2

Failures of this form can generally be fixed by "just not doing that," which in this case entails forcing yourself to stop eating. I don't like that solution, as it requires an application of willpower, and in general, any solution that requires an application of willpower is a stopgap, not a remedy. I much prefer solutions that get all of myself onto the same page, including the parts that make distracting arguments so they can shovel more food into my mouth while I'm not looking.

(A problem isn't solved until it's solved automatically, without need...
Podcasts?
by Capla — 2014-10-25T23:42:29.637Z

I discovered podcasts last year, and I love them! Why not be hearing about new ideas while I'm walking to where I'm going? (Some of you might shout "insight porn!", and I think that I largely agree. However, 1) I don't have any particular problem with insight porn and 2) I have frequently been exposed to an idea or been recommenced a book through a podcast, on which I latter followed up, leading to more substantive intellectual growth.)

I wonder if anyone has favorites that they might want to share with me.

I'll start:

Radiolab is, hands down, the best of all the podcasts. This seems universally recognized: I’ve yet to meet anyone who disagrees. Even the people who make other podcasts think that Radiolab is better than their own. This one regularly invokes a profound sense of wonder at the universe and gratitude for being able to appreciate it. If you missed it somehow, you're probably missing out.
The Freakonomics podcast, in my opinion, comes close to Radiolab. All the things that you thought you knew, but didn’t, and all the things you never knew you wanted to know, but do, in typical Freakonomics style. Listening to their podcast is one of the two things that makes me happy.
There’s one other podcast that I consider to be in the same league (and this one you've probably never heard of) : The Memory Palace. 5-10 minute stories form history, it is really well done. It’s all the more impressive because while Radiolab and Freakonomics are both made by professional production teams in radio studios, The Memory Palace is just some guy who makes a podcast.
Those are my three top picks (and they are the only podcasts that I listen to at “normal” speed instead of x1.5 or x2.0, since their audio production is so good).
I discovered Rationally Speaking: Exploring the Borderlands Between Reason and Nonsense recently and I’m loving it. It is my kind of skeptics podcast, investigating topics that are on the fringe but not straight out bunk (I don't need to listen to yet another podcast about how astrology doesn't work). The interplay between the hosts, Massimo (who has a PhD in Philosophy, but also one in Biology, which excuses it) and Julia (who I only just realized is a founder of the CFAR), is great.
I also sometimes enjoy the Cracked podcast, which has topics that touch on cognitive bias and statistics but also analysis of (pop) ...
Meetup : Utrecht: Game theory
by Philip_W — 2014-10-25T09:09:58.889Z

Discussion article for the meetup : Utrecht: Game theory

WHEN:
16 November 2014 02:00:00PM (+0100)

WHERE:
La Place Cafe, Rijnkade 5, Utrecht

We have biweekly meetups in a pub in Utrecht, near Central Station. For details, please look on meetup.com which is supposed to be up to date. http://www.meetup.com/LWEANL/events/215180412/

Discussion article for the meetup : Utrecht: Game theory
New LW Meetup: Bath UK
by FrankAdamek — 2014-10-25T00:47:23.005Z

This summary was posted to LW Main on October 17th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Bangalore Meetup: 15 November 2014 04:15PM
Bath: Introduction and PredictionBook: 19 October 2014 02:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

Australia online meetup: 19 October 2014 07:00PM
Bratislava: 20 October 2014 06:00PM
Houston, TX: 18 October 2014 02:00PM
Perth, Australia: Sunday lunch: 19 October 2014 12:00PM
Portland, OR: Improv for Rationalists: 18 October 2014 01:00PM
Urbana-Champaign: Stoicism, anthropics: 19 October 2014 02:00PM
Utrecht: Artificial Intelligence: 19 October 2014 02:00PM
Utrecht: Climate Change: 02 November 2014 03:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 18 October 2025 01:30PM
Boston Meetup - New Location: 19 October 2014 03:30PM
Canberra: Would I Lie To You?: 24 October 2014 06:00PM
Sydney Meetup - October: 22 October 2014 06:30PM
Vienna: 18 October 2014 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before ...
Non-standard politics
by NancyLebovitz — 2014-10-24T15:27:46.496Z

In the big survey, political views are divided into large categories so that statistics are possible. This article is an attempt to supply a text field so that we can get a little better view of the range of beliefs.

My political views aren't adequately expressed by "libertarian". I call myself a liberal-flavored libertarian, by which I mean that I want the government to hurt people less. The possibility that the government is giving too much to poor people is low on my list of concerns. I also believe that harm-causing processes should be shut down before support systems

So, what political beliefs do you have that don't match the usual meaning of your preferred label?