trentmkelly/LessWrong-43k huggingface dataset:
Room For More Funding In AI Safety Is Highly Uncertain
by Evan_Gaensbauer — 2016-05-12T13:57:49.977Z

(Crossposted to the Effective Altruism Forum)

Introduction
In effective altruism, people talk about the room for more funding (RFMF) of various organizations. RFMF is simply the maximum amount of money which can be donated to an organization, and be put to good use, right now. In most cases, “right now” typically refers to the next (fiscal) year. Most of the time when I see the phrase invoked, it’s to talk about individual charities, for example, one of Givewell’s top-recommended charities. If a charity has run out of room for more funding, it may be typical for effective donors to seek the next best option to donate to.
Last year, the Future of Life Institute (FLI) made the first of its grants from the pool of money it’s received as donations from Elon Musk and the Open Philanthropy Project (Open Phil). Since then, I've heard a few people speculating about how much RFMF the whole AI safety community has in general. I don't think that's a sensible question to ask before we have a sense of what the 'AI safety' field is. Before, people were commenting on only the RFMF of individual charities, and now they’re commenting of entire fields as though they’re well-defined. AI safety hasn’t necessarily reached peak RFMF just because MIRI has a runway for one more year to operate at their current capacity, or because FLI made a limited number of grants this year.

Overview of Current Funding For Some Projects

The starting point I used to think about this issue came from Topher Hallquist, from his post explaining his 2015 donations:

I’m feeling pretty cautious right now about donating to organizations focused on existential risk, especially after Elon Musk’s $10 million donation to the Future of Life Institute. Musk’s donation don’t necessarily mean there’s no room for more funding, but it certainly does mean that room for more funding is harder to find than it used to be. Furthermore, it’s difficult to evaluate the effectiveness of efforts in this space, so I think there’s a strong case for waiting to see what comes of this infusion of cash before committing more money.

My friend Andrew and I were discussing this last week. In past years, the Machine Intelligence Research Institute (MIRI) has raised about $1 million (USD) in funds, and received more than that &nbs...
Newcomb versus dust specks
by ike — 2016-05-12T03:02:29.720Z

You're given the option to torture everyone in the universe, or inflict a dust speck on everyone in the universe. Either you are the only one in the universe, or there are 3^^^3 perfect copies of you (far enough apart that you will never meet.) In the latter case, all copies of you are chosen, and all make the same choice. (Edit: if they choose specks, each person gets one dust speck. This was not meant to be ambiguous.)

As it happens, a perfect and truthful predictor has declared that you will choose torture iff you are alone.

What do you do?

How does your answer change if the predictor made the copies of you conditional on their prediction?

How does your answer change if, in addition to that, you're told you are the original?
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2016-05-11T03:02:27.834Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
15 May 2016 03:30:00PM (-0400)

WHERE:
Reynolds Center

We will be meeting in the courtyard to play games, hang out, and engage in fun conversation.

Upcoming meetups:

May 22: Typical Mind Fallacy

May 29: meeting canceled for Memorial Day

Jun. 5: Intro to Rationality

Jun. 12: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Improving long-run civilisational robustness
by RyanCarey — 2016-05-10T11:15:55.694Z

People trying to guard civilisation against catastrophe usually focus on one specific kind of catastrophe at a time. This can be useful for building concrete knowledge with some certainty in order for others to build on it. However, there are disadvantages to this catastrophe-specific approach:

1. Catastrophe researchers (including Anders Sandberg and Nick Bostrom) think that there are substantial risks from catastrophes that have not yet been anticipated. Resilience-boosting measures may mitigate risks that have not yet been investigated.

2. Thinking about resilience measures in general may suggest new mitigation ideas that were missed by the catastrophe-specific approach.

One analogy for this is that an intrusion (or hack) to a software system can arise from a combination of many minor security failures, each of which might appear innocuous in isolation. You can decrease the chance of an intrusion by adding extra security measures, even without a specific idea of what kind of hacking would be performed. Things like being being able to power down and reboot a system, storing a backup and being able to run it in a "safe" offline mode are all standard resilience measures for software systems. These measures aren't necessarily the first thing that would come to mind if you were trying to model a specific risk like a password getting stolen, or a hacker subverting administrative privileges, although they would be very useful in those cases. So mitigating risk doesn't necessarily require a precise idea of the risk to be mitigated. Sometimes it can be done instead by thinking about the principles required for proper operation of a system - in the case of its software, preservation of its clean code - and the avenues through which it is vulnerable - such as the internet.

So what would be good robustness measures for human civilisation? I have a bunch of proposals:

Disaster forecasting

Disaster research

* Build research labs to survey and study catastrophic risks (like the Future of Humanity Institute, the Open Philanthropy Project and others)

Disaster prediction

* Prediction contests (like IARPA's Aggregative Contingent Estimation "ACE" program)

* Expert aggregation and elicitation

Disaster prevention

General prevention measures

* Build a culture of prudence in groups that run risky scientific experiments

* Lobby for these mitigation measures

* Impr...
Meetup : San Jose Meetup: Park Day (III)
by steven0461 — 2016-05-09T20:10:55.447Z

Discussion article for the meetup : San Jose Meetup: Park Day (III)

WHEN:
15 May 2016 03:00:00PM (-0700)

WHERE:
438 Coleman Ave, San Jose, CA 95110

You're invited to bring whatever you'd like to see at Park Day - balls, frisbees, food, etc. (Also, consider taking measures against the sun.) We'll carry a green blanket so you can recognize us. The idea is to gather outside near the listed address at the park entrance first, but probably walk elsewhere soon. If you can't find us or there's something else, please call me (Steven) at 510-6844565.

Discussion article for the meetup : San Jose Meetup: Park Day (III)
Meetup : Rationality Meetup Vienna
by AnnaLeptikon — 2016-05-09T11:55:14.660Z

Discussion article for the meetup : Rationality Meetup Vienna

WHEN:
14 May 2016 03:00:00PM (+0200)

WHERE:
Kaisermühlenstraße 24, Wien

FB Event: https://www.facebook.com/events/272339816432545/
(join the Rationality Vienna Group to see it)

Discussion article for the meetup : Rationality Meetup Vienna
Open Thread May 9 - May 15 2016
by Elo — 2016-05-09T01:55:09.343Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : LW Sao Paulo - Meetup de maio
by leohmarruda — 2016-05-08T22:07:25.925Z

Discussion article for the meetup : LW Sao Paulo - Meetup de maio

WHEN:
21 May 2016 02:00:00PM (-0300)

WHERE:
Vanilla café - 2o andar, Rua Antônio Carlos, 452. São Paulo, Brazil.

Encontro mensal de racionalistas, novatos são bem vindos! Teremos algumas lições de racionalidade prática, discussões diversas e jogos de tabuleiro no final.

Discussion article for the meetup : LW Sao Paulo - Meetup de maio
Meetup : San Francisco Meetup: Group Debugging
by rocurley — 2016-05-08T03:43:43.524Z

Discussion article for the meetup : San Francisco Meetup: Group Debugging

WHEN:
09 May 2016 06:15:00PM (-0700)

WHERE:
1597 Howard St. San Francisco, CA

We'll be meeting to try and solve each other's problems! This time, we'll be focusing on trying to pick the best problems to debug, in addition to the object level solving of problems.
As usual, call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Group Debugging
May Outreach Thread
by Gleb_Tsipursky — 2016-05-06T20:02:39.520Z

Please share about any outreach that you have done to convey rationality-style ideas broadly, whether recent or not, which you have not yet shared on previous Outreach threads. The goal of having this thread is to organize information about outreach and provide community support and recognition for raising the sanity waterline, a form of cognitive altruism that contributes to creating a flourishing world. Likewise, doing so can help inspire others to emulate some aspects of these good deeds through social proof and network effects.
Meetup : Baltimore / UMBC Weekly Meetup: General Discussion
by iarwain1 — 2016-05-06T18:34:31.609Z

Discussion article for the meetup : Baltimore / UMBC Weekly Meetup: General Discussion

WHEN:
08 May 2016 03:00:00PM (-0400)

WHERE:
Performing Arts and Humanities Bldg Room 456, 1000 Hilltop Cir, Baltimore, MD 21250

Meetup is in Room 456, the philosophy department conference room. As usual, parking restrictions don't apply on weekends so park wherever you want.

Discussion article for the meetup : Baltimore / UMBC Weekly Meetup: General Discussion
Weekly LW Meetups
by FrankAdamek — 2016-05-06T15:55:30.696Z

This summary was posted to LW Main on May 6th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

European Community Weekend: 02 September 2016 03:35PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

[Melbourne] May Rationality Dojo, Melbourne: Stimulating Discussions: 07 May 2016 03:00PM
New Hampshire Meetup: 10 May 2016 07:00PM
Raleigh, NC (RTLW) Discussion Meetup: 07 May 2022 07:30PM
Sydney Rationality Dojo - June: 05 June 2016 04:00PM
Sydney Rationality Dojo - July: 03 July 2016 04:00PM
Washington, D.C.: Survivorship Bias: 08 May 2016 03:30PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Cambridge UK, Chicago, Cincinnati, Cleveland, Frankfurt, Helsinki, Marin CA, Ottawa, Pittsburgh, Portland, Salt Lake City, Southern Cali...
Rationality Quotes May 2016
by Sabiola — 2016-05-06T15:15:26.628Z

Another month, another rationality quotes thread. The rules are:

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.

Post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.
Call for information, examples, case studies and analysis: votes and shareholder resolutions v.s. divestment for social and environmental outcomes
by  — 2016-05-05T00:08:51.334Z

Typology: since not elsewhere disambiguated, divestment will be considered a form of shareholder activism in this article.

The aim of this call for information is to identify under what conditions shareholder activism or divestment is more appropriate. Shareholder activism referrers to the action and activities around proposing and rallying support for a resolution at a company AGM such as reinstatement or impeachment of a director, or a specific action like renouncing a strategic direction (like investment in coal). In contrast, divestment infers to withdrawal of an investment in a company by shareholders, such as a tobacco or fossil fuel company. By identifying the important variables that determine which strategy is most appropriate, activists and shareholders will be able to choose strategies that maximise social and environmental outcomes while companies will be able to maximise shareholder value.

Very little published academic literature exists on the consequences of divestment. Very little published academic literature exists on the social and environmental consequences of shareholder activism other than the impact on the financial performance of the firm, and conventional metrics of shareholder value.

Controversy (1)

One item of non academic literature, a manifestos on a socially responsible investing blog (http://www.socialfunds.com/media/index.cgi/activism.htm) weighs up the option of divestment against shareholder activism by suggesting that divestment is appropriate as a last resort, if considerable support is rallied, the firm is interested in its long term financial sustainability, and responds whereas voting on shareholder resolutions is appropriate when groups of investors are interested in having an impact. It’s unclear how these contexts are distinguished. DVDivest, a divestment activist group (dcdivest.org/faq/#Wouldn’t shareholder activism have more impact than divestment?) contends in their manifesto the shareholder activism is better suited to changing one aspect of a company's operation whereas divestment is appropriate when rejected a basic business model. This answer too is inadequate as a decision model since one companies can operate multiple simultaneous business models, own several businesses, and one element of their operation may not be easily distinguished from the whole system - the business. They also identify no...
Collaborative Truth-Seeking
by Gleb_Tsipursky — 2016-05-04T23:28:22.268Z

Summary: We frequently use debates to resolve different opinions about the truth. However, debates are not always the best course for figuring out the truth. In some situations, the technique of collaborative truth-seeking may be more optimal.

Acknowledgments: Thanks to Pete Michaud, Michael Dickens, Denis Drescher, Claire Zabel, Boris Yakubchik, Szun S. Tay, Alfredo Parra, Michael Estes, Aaron Thoma, Alex Weissenfels, Peter Livingstone, Jacob Bryan, Roy Wallace, and other readers who prefer to remain anonymous for providing feedback on this post. The author takes full responsibility for all opinions expressed here and any mistakes or oversights.

The Problem with Debates

Aspiring rationalists generally aim to figure out the truth, and often disagree about it. The usual method of hashing out such disagreements in order to discover the truth is through debates, in person or online.

Yet more often than not, people on opposing sides of a debate end up seeking to persuade rather than prioritizing truth discovery. Indeed, research suggests that debates have a specific evolutionary function – not for discovering the truth but to ensure that our perspective prevails within a tribal social context. No wonder debates are often compared to wars.

We may hope that as aspiring rationalists, we would strive to discover the truth during debates. Yet given that we are not always fully rational and strategic in our social engagements, it is easy to slip up within debate mode and orient toward winning instead of uncovering the truth. Heck, I know that I sometimes forget in the midst of a heated debate that I may be the one who is wrong – I’d be surprised if this didn’t happen with you. So while we should certainly continue to engage in debates, we should also use additional strategies – less natural and intuitive ones. These strategies could put us in a better mindset for updating our beliefs and improving our perspective on the truth. One such solution is a mode of engagement called collaborative truth-seeking.

Collaborative Truth-Seeking

Collaborative truth-seeking is one way of describing a more intentional approach in which two or more people with different opinions engage in a process that focuses on finding out the truth. Collaborative truth-seeking is a modality that should be used among people with sha...
Anything you can do with n AIs, you can do with two (with directly opposed objectives)
by jessicata — 2016-05-04T23:14:31.000Z

Summary: For any normal-form game, it's possible to cast the problem of finding a correlated equilibrium in this game as a 2-player zero-sum game. This seems useful because zero-sum games are easy to analyze and more resistant to collusion.

Consider the following class of games (equivalent to the class of normal-form games):

There are n players. Player i's action set is Ai. After each player i chooses their action Ai, the state X results from these actions (perhaps stochastically), and each player i receives utility Ui(X).

Often, we are interested in finding the Nash equilibrium of a game like this. One strategy for this is to instantiate the n players as agents. However, this could cause collusion; see here and here for previous writing on collusion. Zero-sum games seem more resistant to collusion (although maybe not 100% resistant). Additionally, 2-player zero-sum games are typically easier to reason about than n-player games.

So we might be interested in finding the Nash equilibrium using a zero-sum game. I don't actually know how to find a mixed Nash equilibrium, so instead I'll present a strategy for finding a correlated equilibrium (a superset of Nash equilibria which are computationally easier to find). Here's how it works:

The actor chooses actions A1,...,An.

The critic chooses a player index I, observes the action AI, and suggests an alternative action A′I.

Flip a fair coin. If it comes up heads, observe the state X that results from actions A1,...,An, and give the actor utility UI(X).

If it comes up tails, observe the state X that results from actions A1,...,AI−1,A′I,AI+1,...,An, and give the actor utility −UI(X).

Either way, the critic's utility is the negation of the actor's utility.

It will be useful to use the concept of an ϵ-correlated equilibrium. While a correlated equilibrium is where no player can gain any expected utility by strategy modification, an ϵ-correlated equilibrium is where no player can gain more than ϵ expected utility by strategy modification.

Note that the critic's policies correspond to mixtures of strategy modifications; the critic can be seen as jointly picking a player I and a strategy modification ϕ:AI→AI for the player. Furthermore, the critic's expected utility is half the expected utility gained by the corresponding player for the average strategy modification in this mixture:

expected critic utility=12

...
Rationality Reading Group: Part Z: The Craft and the Community
by Gram_Stone — 2016-05-04T23:03:03.292Z

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part Z: The Craft and the Community (pp. 1651-1750). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

Z. The Craft and the Community

312. Raising the Sanity Waterline - Behind every particular failure of social rationality is a larger and more general failure of social rationality; even if all religious content were deleted tomorrow from all human minds, the larger failures that permit religion would still be present. Religion may serve the function of an asphyxiated canary in a coal mine - getting rid of the canary doesn't get rid of the gas. Even a complete social victory for atheism would only be the beginning of the real work of rationalists. What could you teach people without ever explicitly mentioning religion, that would raise their general epistemic waterline to the point that religion went underwater?

313. A Sense That More Is Possible - The art of human rationality may have not been much developed because its practitioners lack a sense that vastly more is possible. The level of expertise that most rationalists strive to develop is not on a par with the skills of a professional mathematician - more like that of a strong casual amateur. Self-proclaimed "rationalists" don't seem to get huge amounts of personal mileage out of their craft, and no one sees a problem with this. Yet rationalists get less systematic training in a less systematic context than a first-dan black belt gets in hitting people.

314. Epistemic Viciousness - An essay by Gillian Russell on "Epistemic Viciousness in the Martial Arts" generalizes amazingly to possible and actual problems with building a community around rationality. Most notably the extreme dangers associated with "data poverty" - the difficulty of testing the skills in the real world. But also such factors as the sacredness of the dojo, the investment in teachings long-practiced, the difficulty of book learning that leads into the need to trust a teacher, deference to historical masters, and above all, living in da...
LINK: New clinical trial will try to restore dead brains
by polymathwannabe — 2016-05-04T20:05:28.681Z

"... in an effort to revive the brains of those being kept alive solely through life support. Stem cells will be injected directly into the brain..."

More at:

http://news.discovery.com/tech/biotechnology/dead-could-be-brought-back-to-life-in-medical-trial-160503.htm
Link: Thoughts on the basic income pilot, with hedgehogs
by Jacob Falkovich — 2016-05-04T17:47:38.681Z

I have resisted the urge of promoting my blog for many months, but this is literally (per my analysis) for the best cause.

We have also raised a decent amount of money so far, so at least some people were convinced by the arguments and didn't stop at the cute hedgehog pictures.