trentmkelly/LessWrong-43k huggingface dataset:
[Resolved] Is the SIA doomsday argument wrong?
by Brian_Tomasik — 2014-12-13T06:01:31.965Z

[EDIT: I think the SIA doomsday argument works after all, and my objection to it was based on framing the problem in a misguided way. Feel free to ignore this post or skip to the resolution at the end.]

ORIGINAL POST:

Katja Grace has developed a kind of doomsday argument from SIA combined with the Great Filter. It has been discussed by Robin Hanson, Carl Shulman, and Nick Bostrom. The basic idea is that if the filter comes late, there are more civilizations with organisms like us than if the filter comes early, and more organisms in positions like ours means a higher expected number of (non-fake) experiences that match ours. (I'll ignore simulation-argument possibilities in this post.)

I used to agree with this reasoning. But now I'm not sure, and here's why. Your subjective experience, broadly construed, includes knowledge of a lot of Earth's history and current state, including when life evolved, which creatures evolved, the Earth's mass and distance from the sun, the chemical composition of the soil and atmosphere, and so on. The information that you know about your planet is sufficient to uniquely locate you within the observable universe. Sure, there might be exact copies of you in vastly distant Hubble volumes, and there might be many approximate copies of Earth in somewhat nearer Hubble volumes. But within any reasonable radius, probably what you know about Earth requires that your subjective experiences (if veridical) could only take place on Earth, not on any other planet in our Hubble volume.

If so, then whether there are lots of human-level extraterrestrials (ETs) or none doesn't matter anthropically, because none of those ETs within any reasonable radius could contain your exact experiences. No matter how hard or easy the emergence of human-like life is in general, it can happen on Earth, and your subjective experiences can only exist on Earth (or some planet almost identical to Earth).

A better way to think about SIA is that it favors hypotheses containing more copies of our Hubble volume within the larger universe. Within a given Hubble volume, there can be at most one location where organisms veridically perceive what we perceive.

Katja's blog post on the SIA doomsday draws orange boxes with humans waving their hands. She has us update on knowing we're in the human-level stage, i.e., that we're one of those orange boxes. But we k...
A forum for researchers to publicly discuss safety issues in advanced AI
by Rob Bensinger — 2014-12-13T00:33:50.516Z

MIRI has an organizational goal of putting a wider variety of mathematically proficient people in a position to advance our understanding of beneficial smarter-than-human AI. The MIRIx workshops, our new research guide, and our more detailed in-the-works technical agenda are intended to further that goal.

To encourage the growth of a larger research community where people can easily collaborate and get up to speed on each other's new ideas, we're also going to roll out an online discussion forum that's specifically focused on resolving technical problems in Friendly AI. MIRI researchers and other interested parties will be able to have more open exchanges there, and get rapid feedback on their ideas and drafts. A relatively small group of people with relevant mathematical backgrounds will be authorized to post on the forum, but all discussion on the site will be publicly visible to visitors.

Topics will run the gamut from logical uncertainty in formal agents to cognitive models of concept generation. The exact range of discussion topics is likely to evolve over time as researchers' priorities change and new researchers join the forum.

We're currently tossing around possible names for the forum, and I wanted to solicit LessWrong's input, since you've been helpful here in the past. (We're also getting input from non-LW mathematicians and computer scientists.) We want to know how confusing, apt, etc. you perceive these variants on 'forum for doing exploratory engineering research in AI' to be:

1. AI Exploratory Research Forum (AIXRF)

2. Forum for Exploratory Engineering in AI (FEEAI)

3. Forum for Exploratory Research in AI (FERAI, or FXRAI)

4. Exploratory AI Research Forum (XAIRF, or EAIRF)

We're also looking at other name possibilities, including:

5. AI Foundations Forum (AIFF)

6. Intelligent Agent Foundations Forum (IAFF)

7. Reflective Agents Research Forum (RARF)

We're trying to avoid names like "friendly" and "normative" that could reinforce someone's impression that we think of AI risk in anthropomorphic terms, that we're AI-hating technophobes, or that we're moral philosophers.

Feedback on the above ideas is welcome, as are new ideas. Feel free to post separate ideas in separate comments, so they can be upvoted individually. We're especially looking for feedback along the lines of: 'I'm a grad student in theoretical computer scienc...
Approval-directed agents
by paulfchristiano — 2014-12-12T22:38:37.402Z

Most concern about AI comes down to the scariness of goal-oriented behavior. A common response to such concerns is “why would we give an AI goals anyway?” I think there are good reasons to expect goal-oriented behavior, and I’ve been on that side of a lot of arguments. But I don’t think the issue is settled, and it might be possible to get better outcomes without them. I flesh out one possible alternative here, based on the dictum "take the action I would like best" rather than "achieve the outcome I would like best."

(As an experiment I wrote the post on medium, so that it is easier to provide sentence-level feedback, especially feedback on writing or low-level comments.)
Harper's Magazine article on LW/MIRI/CFAR and Ethereum
by gwern — 2014-12-12T20:34:45.244Z

Cover title: “Power and paranoia in Silicon Valley”; article title: “Come with us if you want to live: Among the apocalyptic libertarians of Silicon Valley” (mirrors: 1, 2, 3), by Sam Frank; Harper’s Magazine, January 2015, pg26-36 (~8500 words). The beginning/ending are focused on Ethereum and Vitalik Buterin, so I'll excerpt the LW/MIRI/CFAR-focused middle:

…Blake Masters-the name was too perfect-had, obviously, dedicated himself to the command of self and universe. He did CrossFit and ate Bulletproof, a tech-world variant of the paleo diet. On his Tumblr’s About page, since rewritten, the anti-belief belief systems multiplied, hyperlinked to Wikipedia pages or to the confoundingly scholastic website Less Wrong: “Libertarian (and not convinced there’s irreconcilable fissure between deontological and consequentialist camps). Aspiring rationalist/Bayesian. Secularist/agnostic/ ignostic . . . Hayekian. As important as what we know is what we don’t. Admittedly eccentric.” Then: “Really, really excited to be in Silicon Valley right now, working on fascinating stuff with an amazing team.” I was startled that all these negative ideologies could be condensed so easily into a positive worldview. …I saw the utopianism latent in capitalism-that, as Bernard Mandeville had it three centuries ago, it is a system that manufactures public benefit from private vice. I started CrossFit and began tinkering with my diet. I browsed venal tech-trade publications, and tried and failed to read Less Wrong, which was written as if for aliens.

…I left the auditorium of Alice Tully Hall. Bleary beside the silver coffee urn in the nearly empty lobby, I was buttonholed by a man whose name tag read MICHAEL VASSAR, METAMED research. He wore a black-and-white paisley shirt and a jacket that was slightly too big for him. “What did you think of that talk?” he asked, without introducing himself. “Disorganized, wasn’t it?” A theory of everything followed. Heroes like Elon and Peter (did I have to ask? Musk and Thiel). The relative abilities of physicists and biologists, their standard deviations calculated out loud. How exactly Vassar would save the world. His left eyelid twitched, his full face winced with effort as he told me about his “personal war against the u

...
Weekly LW Meetups
by FrankAdamek — 2014-12-12T17:06:48.775Z

This summary was posted to LW Main on December 5th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Australia onling hangout: 14 December 2014 07:00PM
Copenhagen: December Meetup: 20 December 2014 03:00PM
East Coast Solstice Megameetup: 20 December 2014 03:00PM
European Community Weekend 2015: 12 June 2015 12:00PM
Frankfurt: 07 December 2014 02:00PM
Munich Meetup in December: 13 December 2014 12:00PM
Utrecht: Rationality Games: 14 December 2014 02:00PM
Utrecht: 21 December 2014 02:00PM
Utrecht: a critique of effective altruism: 04 January 2015 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Canberra: End of year party: 13 December 2014 06:00PM
London Social Meetup, 07/12/2014: 07 December 2014 02:00PM
[Melbourne] December Rationality Dojo - Online Communication: Conveying Ideas Effectively: 07 December 2014 03:30PM
Moscow Meetup: CBT is back: 07 December 2014 02:00PM
Seattle Secular Solstice: 13 December 2014 05:30PM
Sydney Rationality Dojo - Approaches, Conspecific Interactions, Arguments and Communications: 07 December 2014 04:00AM
Toronto EA meetup: 11 December 2014 07:00PM
[Vienna] Rationality Weekend Vienna: 13 December 2014 03:00PM
Washington, D.C.: Brains: 07 December 2014 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of h...
Beware The Man Of One Study
by Scott Alexander — 2014-12-12T09:04:56.000Z

Aquinas famously said: beware the man of one book. I would add: beware the man of one study.

For example, take medical research. Suppose a certain drug is weakly effective against a certain disease. After a few years, a bunch of different research groups have gotten their hands on it and done all sorts of different studies. In the best case scenario the average study will find the true result – that it’s weakly effective.

But there will also be random noise caused by inevitable variation and by some of the experiments being better quality than others. In the end, we might expect something looking kind of like a bell curve. The peak will be at “weakly effective”, but there will be a few studies to either side. Something like this:

We see that the peak of the curve is somewhere to the right of neutral – ie weakly effective – and that there are about 15 studies that find this correct result.

But there are also about 5 studies that find that the drug is very good, and 5 studies missing the sign entirely and finding that the drug is actively bad. There’s even 1 study finding that the drug is very bad, maybe seriously dangerous.

This is before we get into fraud or statistical malpractice. I’m saying this is what’s going to happen just by normal variation in experimental design. As we increase experimental rigor, the bell curve might get squashed horizontally, but there will still be a bell curve.

In practice it’s worse than this, because this is assuming everyone is investigating exactly the same question.

Suppose that the graph is titled “Effectiveness Of This Drug In Treating Bipolar Disorder”.

But maybe the drug is more effective in bipolar i than in bipolar ii (Depakote, for example)

Or maybe the drug is very effective against bipolar mania, but much less effective against bipolar depression (Depakote again).

Or maybe the drug is a good acute antimanic agent, but very poor at maintenance treatment (let’s stick with Depakote).

If you have a graph titled “Effectiveness Of Depakote In Treating Bipolar Disorder” plotting studies from “Very Bad” to “Very Good” – and you stick all the studies – maintenence, manic, depressive, bipolar i, bipolar ii – on the graph, then you’re going to end running the gamut from “very bad” to R...
Meetup : Washington, D.C.: Tetlock's "Expert Political Judgment"
by RobinZ — 2014-12-12T04:25:23.222Z

Discussion article for the meetup : Washington, D.C.: Tetlock's "Expert Political Judgment"

WHEN:
14 December 2014 03:00:00PM (-0500)

WHERE:
National Portrait Gallery

We will be meeting in the Kogod Courtyard of the National Portrait Gallery (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance) to talk about Philip E. Tetlock's book Expert Political Judgment: How Good Is It? How Can We Know?. Per the norm, we plan to let people congregate from 3:00 to 3:30 before kicking things off.

As with prior informal-discussion meetups, conversation on any subject of interest to attendees (be it in the main conversation or in a side conversation) is both permitted and encouraged, but we suggest taking advantage of the meetup topic as a Schelling point.

Upcoming Meetups:

The Less Wrong DC organizers haven't decided whether to hold meetups on the remaining two Sundays in 2014: the 21st is the day after Brighter Than Today, a secular winter solstice celebration that many regulars will be out of town for, and the 28th falls during the week after Christmas Day, an ostensibly-religious winter solstice celebration that many regulars may be out of town (or hosting out-of-town guests) for. If there is sufficient interest in attending a meetup on either or both of these dates, meetups will occur; otherwise the Fun & Games meetup will be postponed to January 4th.

Discussion article for the meetup : Washington, D.C.: Tetlock's "Expert Political Judgment"
What Peter Thiel thinks about AI risk
by Dr_Manhattan — 2014-12-11T21:22:27.167Z

This is probably the clearest statement from him on the issue:

http://betaboston.com/news/2014/12/10/audio-peter-thiel-visits-boston-university-to-talk-entrepreneurship-and-backing-zuck/

25:30 mins in

TL;DR: he thinks its an issue but also feels AGI is very distant and hence less worried about it than Musk.

I recommend the rest of the lecture as well, it's a good summary of "Zero to One" and a good QA afterwards.
Meetup : Saint-Petersburg Meetup 
by LizzardWizzard — 2014-12-11T12:48:52.891Z

Discussion article for the meetup : Saint-Petersburg Meetup

WHEN:
13 December 2014 04:00:00PM (+0400)

WHERE:
Russia, Saint-Petersburg, Liteyniy Prospekt 57

Experimental Meetup at the new location! All aspiring rationalists are welcome to participate. Discussions on various topics and social skills games will be included

Address of the meetup: Bookstore "Podpisnyie Izdaniya", Liteyniy Prospekt, 57, 2nd floor

If you are lost or have any questions feel free to call me +7 921 967 91 11 Kirill

Discussion article for the meetup : Saint-Petersburg Meetup
Make your own cost-effectiveness Fermi estimates for one-off problems
by owencb — 2014-12-11T12:00:07.145Z

In some recent work (particularly this article) I built models for estimating the cost effectiveness of work on problems when we don’t know how hard those problems are. The estimates they produce aren’t perfect, but they can get us started where it’s otherwise hard to make comparisons.

Now I want to know: what can we use this technique on? I have a couple of applications I am working on, but I’m keen to see what estimates other people produce.

There are complicated versions of the model which account for more factors, but we can start with a simple version. This is a tool for initial Fermi calculations: it’s relatively easy to use but should get us around the right order of magnitude. That can be very useful, and we can build more detailed models for the most promising opportunities.

The model is given by:

This expresses the expected benefit of adding another unit of resources to solving the problem. You can denominate the resources in dollars, researcher-years, or another convenient unit. To use this formula we need to estimate four variables:

R(0) denotes the current resources going towards the problem each year. Whatever units you measure R(0) in, those are the units we’ll get an estimate for the benefit of. So if R(0) is measured in researcher-years, the formula will tell us the expected benefit of adding a researcher year.

You want to count all of the resources going towards the problem. That includes the labour of those who work on it in their spare time, and some weighting for the talent of the people working in the area (if you doubled the budget going to an area, you couldn’t get twice as many people who are just as good; ideally we’d use an elasticity here).

Some resources may be aimed at something other than your problem, but be tangentially useful. We should count some fraction of those, according to how much resources devoted entirely to the problem they seem equivalent to.

B is the annual benefit that we’d get from a solution to the problem. You can measure this in its own units, but whatever you use here will be the units of value that come out in the cost-effectiveness estimate.

p and y/z are parameters that we will estimate together. p is the probability of getting a solution by the time y resources have been dedicated to the problem, if z resources have been dedic

...
Estimating the cost-effectiveness of research
by owencb — 2014-12-11T11:55:01.025Z

At a societal level, how much money should we put into medical research, or into fusion research? For individual donors seeking out the best opportunities, how can we compare the expected cost-effectiveness of research projects with more direct interventions?

Over the past few months I've been researching this area for the Global Priorities Project. We've written a variety of articles which focus on different parts of the question. Estimating the cost-effectiveness of research is the central example here, but a lot of the methodology is also applicable to other one-off projects with unknown difficulty (perhaps including political lobbying). I don't think it's all solved, but I do think we've made substantial progress.

I think people here might be interested, so I wanted to share our work. To help you navigate and find the most appropriate pieces, here I collect them, summarise what's contained in each, and explain how they fit together.

I gave an overview of my thinking at the Good Done Right conference, held in Oxford in July 2014. The slides and audio of my talk are available; I have developed more sophisticated models for some parts of the area since then.

How to treat problems of unknown difficulty introduces the problem: we need to make decisions about when to work more on problems such as research into fusion where we don't know how difficult it will be. It builds some models which allow principled reasoning about how we should act. These models are quite crude but easy to work with: they are intended to lower the bar for Fermi estimates and similar, and provide a starting point for building more sophisticated models.

Estimating cost-effectiveness for problems of unknown difficulty picks up from the models in the above post, and asks what they mean for the expected cost-effectiveness of work on the problems. This involves building a model of the counterfactual impact, as solvable research problems are likely to be solved eventually, so the main effect is to move the solution forwards. This post includes several explicit formulae that you can use to produce estimates; it also explains analogies between the explicit model we derive and the qualitative 'three factor' model that GiveWell and 80,000 Hours have used for cause selection.

Estimating the cost-effectiveness of research into neglected diseases is an investigation by Max Da

...
[link] Etzioni: AI will empower us, not exterminate us
by SeekingEternity — 2014-12-11T08:51:46.364Z

https://medium.com/backchannel/ai-wont-exterminate-us-it-will-empower-us-5b7224735bf3

(Slashdot discussion: http://tech.slashdot.org/story/14/12/10/1719232/ai-expert-ai-wont-exterminate-us----it-will-empower-us)

Not sure what the local view of Oren Etzioni or the Allen Institute for AI is, but I'm curious what people think if his views on UFAI risk. As far as I can tell from this article, it basically boils down to "AGI won't happen, at least not any time soon." Is there (significant) reason to believe he's wrong, or is it simply too great a risk to leave to chance?
Cognitive distortions of founders
by Dr_Manhattan — 2014-12-11T03:19:58.802Z

Interesting take on entrepreneurial success:

http://www.harrisonmetal.com/cognitive-distortions-of-founders/

More in depth here:

http://quarry.stanford.edu/xapm1111126lse/docs/02_LSE_Cognitive.pdf

Curious what people here think of this.
Meetup : Atlanta December Meetup - Game Night!
by Adele_L — 2014-12-11T02:56:58.697Z

Discussion article for the meetup : Atlanta December Meetup - Game Night!

WHEN:
27 December 2014 07:00:00AM (-0500)

WHERE:
2388 Lawrenceville Highway Unit L, Norgate Manor, Decatur, GA 30033

Come join us for a get together! We'll be playing games and having a low key social gathering. Feel free to bring your favorite games and any snacks you would like.

Please park in a spot marked visitor. Parking in a numbered spot can get you towed. There are cats at the location.

Hope to see you!

Discussion article for the meetup : Atlanta December Meetup - Game Night!
Lifehack Ideas December 2014
by Gondolinian — 2014-12-10T00:21:09.486Z

Life hacking refers to any trick, shortcut, skill, or novelty method that increases productivity and efficiency, in all walks of life.

— Wikipedia

This thread is for posting any promising or interesting ideas for lifehacks you've come up with or heard of. If you've implemented your idea, please share the results. You are also encouraged to post lifehack ideas you've tried out that have not been successful, and why you think they weren't. If you can, please give credit for ideas that you got from other people.

To any future posters of Lifehack Ideas threads, please remember to add the "lifehacks_thread" tag.