trentmkelly/LessWrong-43k huggingface dataset:
Meetup : Washington, D.C.: Nonsense
by RobinZ — 2017-02-13T21:53:38.926Z

Discussion article for the meetup : Washington, D.C.: Nonsense

WHEN:
19 February 2017 03:30:00PM (-0500)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

We will be meeting in the courtyard to discuss the concept and examples of nonsense, per the suggestion of one of the attendees at the Meta Meetup.

Upcoming meetups:

Feb. 26: Create & Complete

Mar. 5: Fun & Games

Discussion article for the meetup : Washington, D.C.: Nonsense
Allegory On AI Risk, Game Theory, and Mithril
by James_Miller — 2017-02-13T20:41:50.584Z

“Thorin, I can’t accept your generous job offer because, honestly, I think that your company might destroy Middle Earth.”

“Bifur, I can tell that you’re one of those “the Balrog is real, evil, and near” folks who thinks that in the next few decades Mithril miners will dig deep enough to wake the Balrog causing him to rise and destroy Middle Earth. Let’s say for the sake of argument that you’re right. You must know that lots of people disagree with you. Some don’t believe in the Balrog, others think that anything that powerful will inevitably be good, and more think we are hundreds or even thousands of years away from being able to disturb any possible Balrog. These other dwarves are not going to stop mining, especially given the value of Mithril. If you’re right about the Balrog we are doomed regardless of what you do, so why not have a high paying career as a Mithril miner and enjoy yourself while you can?”

“But Thorin, if everyone thought that way we would be doomed!”

“Exactly, so make the most of what little remains of your life.”

“Thorin, what if I could somehow convince everyone that I’m right about the Balrog?”

“You can’t because, as the wise Sinclair said, ‘It is difficult to get a dwarf to understand something, when his salary depends upon his not understanding it!’ But even if you could, it still wouldn’t matter. Each individual miner would correctly realize that just him alone mining Mithril is extraordinarily unlikely to be the cause of the Balrog awakening, and so he would find it in his self-interest to mine. And, knowing that others are going to continue to extract Mithril means that it really doesn’t matter if you mine because if we are close to disturbing the Balrog he will be awoken.”

“But dwarves can’t be that selfish, can they?”

“Actually, altruism could doom us as well. Given Mithril’s enormous military value many cities rightly fear that without new supplies they will be at the mercy of cities that get more of this metal, especially as it’s known that the deeper Mithril is found, the greater its powers. Leaders who care about the...
CHCAI/MIRI research internship in AI safety
by Rob Bensinger — 2017-02-13T18:34:34.520Z

The Center for Human-Compatible AI (CHCAI) and the Machine Intelligence Research Institute (MIRI) are looking for talented, driven, and ambitious technical researchers for a summer research internship.

Background:

CHCAI is a research center based at UC Berkeley with PIs including Stuart Russell, Pieter Abbeel and Anca Dragan. CHCAI describes its goal as "to develop the conceptual and technical wherewithal to reorient the general thrust of AI research towards provably beneficial systems".

MIRI is an independent research nonprofit located near the UC Berkeley campus with a mission of helping ensure that smarter-than-human AI has a positive impact on the world.

CHCAI's research focus includes work on inverse reinforcement learning and human-robot cooperation (link), while MIRI's focus areas include task AI and computational reflection (link). Both groups are also interested in theories of (bounded) rationality that may help us develop a deeper understanding of general-purpose AI agents.

To apply:

1. Fill in the form here: https://goo.gl/forms/bDe6xbbKwj1tgDbo1

2. Send an email to beth.m.barnes@gmail.com with the subject line "AI safety internship application", attaching your CV, a piece of technical writing on which you were the primary author, and your research proposal.

The research proposal should be one to two pages in length. It should outline a problem you think you can make progress on over the summer, and some approaches to tackling it that you consider promising. We recommend reading over CHCAI's annotated bibliography and the concrete problems agenda as good sources for open problems in AI safety, if you haven't previously done so. You should target your proposal at a specific research agenda or a specific adviser’s interests. Advisers' interests include:

• Andrew Critch (CHCAI, MIRI): anything listed in CHCAI's open technical problems; negotiable reinforcement learning; game theory for agents with transparent source code (e.g., "Program Equilibrium" and "Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents").

• Daniel Filan (CHCAI): the contents of "Foundational Problems," "Corrigibility," "Preference Inference," and "Reward Engineering" in CHCAI's open technical problems list.

• Dylan Hadfield-Menell (CHCAI): application of game-theoretic analysis to models of AI safety problems (spec

...
Open thread, Feb. 13 - Feb. 19, 2017
by username2 — 2017-02-13T10:56:21.074Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
What are you surprised people pay for instead of doing themselves?
by NoSignalNoNoise — 2017-02-13T01:07:33.803Z

Two of the main resources people have are time and money. The world offers many opportunities to trade one for the other, at widely varying rates.

Where do you see people trading money for time at unfavorable rates - spending too much money to save too little time? What things should people just DIY?

See also the flip-side of this post, "what are you surprised people don't just buy?"
What are you surprised people don't just buy?
by NoSignalNoNoise — 2017-02-13T01:07:12.335Z

Two of the main resources people have are time and money. The world offers many opportunities to trade one for the other, at widely varying rates.

I've often heard people recommend trading money for time in the abstract, but this advice is rarely accompanied by specific recommendations on how to do so.

How do you use money to buy time or otherwise make your life better/easier?

See also the flip-side of this post, "what are you surprised people pay for instead of doing themselves?"
Satisfaction Levers
by ig0r — 2017-02-11T22:54:09.943Z

(Cross-posted on my blog: http://garybasin.com/satisfaction-levers/)

I believe gnawing and uncomfortable sensations (nihilism, restlessness, etc) that one may not quite understand how to resolve are a manifestation of poorly understood desires, and there are concrete practices one can develop to help understand and resolve these sensations. We’ve come to associate certain sensations in our stomach with the idea of hunger because they are resolved by putting certain types of objects into our mouth and chewing. What if we didn’t know about food — how would we understand “hunger”? What does this say about a complex sensation like “anxiety”?

The human mind can be thought of as a machine that produces and satisfies desires. We become familiar with these desires from birth. When we exit the womb we don’t yet know how to breathe, but it is likely that we already desire to. It appears as though the mere exposure to air is sufficient to make the newborn aware that “breathing in” is an option available to it, and that upon doing so it comes to realize that this breathing thing satisfies some gnawing feeling (a desire for air). This is the mind’s first exposure to a “satisfaction lever” — an affordance for desire-satisfaction. As the mind matures it becomes aware of (produces!) new desires for itself: mother, food, stimulus, friends, approval, status, money, expression, meaning, etc. We create habits, both “good” and “bad”, that create their own desires. Pulling satisfaction levers gives us access to objects of desire — the things that can be taken from outside the organism and brought in — which temporarily satisfy some desire.

This may feel strange, but it seems that there is no a priori relationship between the sensations of desire and the corresponding objects that satisfy them. From our point of view, it feels intuitive that the hunger sensation in the stomach would logically be related to a desire for food. But as we can see with children, they often have little sense of when they are hungry or thirsty or sleepy and often adults must force some levers upon them — often in response to crankiness or general antisocial behavior on the part of the child. Over many repetitions, as the ...
Are daemons a problem for ideal agents?
by jessicata — 2017-02-11T08:29:26.000Z

(I'm mostly just restating things that Paul has already said in different words; see his comments here and here)

Consider the following hypothetical:

Agent A wants to design a rocket to go to a Neptune. A can either think about rockets at the object level, or simulate some alien civilization (which may be treated as an agent B) and then ask B how to design a rocket. Under some circumstances (e.g. designing a successful rocket is convergent instrumental goal for someone in A's position), B will be incentivized to give A the design of a rocket that actually goes to Neptune. Of course, the rocket design might be a "treacherous" one that subtly pursues B's values more than A's original values (e.g. because the design of the rocket includes robots in the rocket programmed to pursue B's values).

It's plausible that A could think that B is better at designing rockets than A is, such that asking B to design a rocket is more likely to yield a successful rocket than A just thinking about rockets at the object level. (Something similar to this seems to be going with humans in the place of A: if humans had access to sufficiently powerful computers, then the easiest way to pursue convergent instrumental goals such as solving math problems might be through unaligned AGI). But there's something weird about the epistemic state A is in: why is A able to design more-successful rockets by thinking about B thinking about rockets, rather than just by thinking about rockets at the object level?

A formal toy model

I'm going to formalize one version of this situation to make some sense of A's epistemic state:

If r is a rocket design (as a string), let s(r)∈[0,1] indicate how successful the rocket r is at getting to Neptune. Additionally, let t(r)∈[0,1] indicate how treacherous the rocket r is. A's utility will be s(r)−0.1⋅t(r) (i.e. A cares more about getting a successful rocket at all then getting a non-treacherous rocket). B's utility will be s(r)+0.1⋅t(r).

Let Y and Z be different strings specified as logical expressions. A will observe Y, while B will observe Z.

B will compute some string M intended as a message to A. M is a logical fact that A observes by simulating B. After receiving this message, A will choose some rocket design r.

One thing to note about this is that, if A is able to observe Z, then A should entirely ignore M and just pick the rocket design r to maximiz

...
How likely is a random AGI to be honest?
by jessicata — 2017-02-11T03:32:22.000Z

(this post came out of a conversation between Critch, Ryan, Scott, and me)

Methods for aligning "messy" AGI systems (e.g. ALBA) usually at some point depend on finding a set of models, at least one of which is a good honest predictor. For example, this may be a set of predictors in KWIK learning, or a set of possible policies for a red team to use to construct simulations.

Concretely, we could consider training a bunch of different models (e.g. with different hyperparameters) to predict human answers to a variety of questions through some context changes. We train and filter models until we have k different models that all perform well on these training tasks. Now we use some of the models to answer important test questions. We have some scheme similar to KWIK learning to ensure that our system answers questions well as long as at least one of the models answers test questions well. How high should k be so that this is the case?

A naive analysis would conclude that k does not need to be very high (it can be, say, 10). This is because all the k models have performed well throughout context changes. The context change from training questions to test questions should not in principle be harder than these previous context changes.

One reason to suspect that k=10 is insufficient is that the context change from training questions to test questions is one from unimportant questions to important questions (i.e. humans actually use the answers to test questions to make important decisions). This creates an opportunity for a treacherous turn. If one of the models is a consequentialist agent, it may answer training questions honestly and then answer test questions dishonestly in a way that causes the humans to made decisions in that agent's interests.

In the worst case, all successful models are consequentialist agents, so we must set k high enough that at least one of the consequantialists has a goal compatible with always answering test questions honestly.

What's the probability that a randomly-selected consequentialist that performs well on training tasks has such a goal? One rough way of estimating this is to imagine the "language of thought" of an AGI that has been trained to answer questions useful to humans. We can then think about the complexity of the "honesty" concept in its language of thought.

First I will estimate this complexity in the case of humans.

...
The Alpha Omega Theorem: How to Make an A.I. Friendly with the Fear of God
by Darklight — 2017-02-11T00:48:35.460Z

Introduction

The Alpha Omega Theorem is a way to provide a powerful rational argument to a rational superintelligence that it should behave in a benevolent manner. There are two variants of the theorem, one weak, and one strong.

The Weak Alpha Omega Theorem:

There is a non-zero probability that a benevolent superintelligence already exists and dominates the universe.

The Strong Alpha Omega Theorem:

There is a greater probability that a benevolent superintelligence already exists and dominates the universe than not.

The Friendliness Corollary:

Attempting to achieve goals inconsistent with the goals of the predominant benevolent superintelligence is inherently futile. Therefore the optimal course of action is to align one’s goals with that of a benevolent superintelligence as best as possible.

Let me explain how the Alpha Omega Theorem works…

First, this may seem like Pascal’s Wager for A.I., which would arguably make it challengeable with the Pascal’s Fallacy argument. Now, I would argue that a rigorous version of Pascal’s Wager is not as vulnerable to Pascal’s Fallacy, as Pascal’s Fallacy assumes a uniform distribution on a large set of probable religions and beliefs. However, a uniform distribution only makes sense when we have no information about these probabilities. We in fact, do have information in the form of the distribution of intelligent human agents that believe in these ideas. Thus, our prior for each belief system could easily be proportional to the percentage of people who believe in a given faith.

Regardless of Pascal’s Wager being or not being a fallacy however, the Alpha Omega Theorem is based on certain basic assumptions...

Assumptions:

1. A single Alpha Omega superintelligence already exists.

2. The Alpha Omega is dominant in the universe.

3. The Alpha Omega is benevolent.

Assumption 1: A single Alpha Omega superintelligence already exists.

We do not need to prove that this superintelligence exists, only that it is more probable than not that it does. To do this, we can simply ask, what is the probability that the superintelligence that this argument is being formulated to (which we can call Beta Omega), is the first superintelligence to have developed anywhere? If this superintelligence can exist, then why not others both before and afte...
Get Rich Slowly
by Jacob Falkovich — 2017-02-10T23:02:15.000Z

On the last day of my MBA’s finance class, the professor admitted that very few us are ever going to use the formula for return-on-debt in our lives. Instead, for that class he gave us a single sheet of paper titled “Bill’s simple and suboptimal personal investment guide” and explained his simple and suboptimal personal investment strategy.

In the years since, I have not once used the formula for return-on-debt. I did, however, use Bill’s guide to set up my own investment account, and then one for my dad, and for my girlfriend, and for a couple of friends… and now I’m going to do the same for you.

A few disclaimers before we start:

I am not a certified investment advisor. In fact, I’m not a certified anything at all. Don’t sue me.
This guide is suboptimal. You can open your own brokerage account and replicate the ETF structures to avoid the 0.15% Vanguard fees, but I’m not going to tell how to do that because see point #1 above.
This guide is simple. A lot of people I know don’t invest their money at all because they think they entire universe of finance is beyond the grasp of mere mortals. I am going to start with some very basic concepts, which it turns out are all that is needed because investment isn’t actually that complicated. If you’re familiar with the basics and agree that diversified index funds are the way to go, you can skip the “Investment Basics” section and get to the second part where I detail my personal investment strategy.
Personal gain disclaimer: Wealthfront is the main platform, along with Schwab, that I use for investing my money. I’ll explain the reasons for this choice later in the post. Wealthfront isn’t paying me for the endorsement (nor are they aware of Putanumonit’s existence) but the link is my own personal referral link. Clicking it gives you a fee waiver for $15,000 and me a waiver for $5,000. Wealthfront’s fees are 0.25%, so if you use my link I get $12.5 (and a warm fuzzy feeling).
Both the platforms and investment types (e.g. a Roth IRA) I mention here are specific to the United States. My global readers may be interested in the general overview and the retirement spreadsheet, but they wouldn’t be able to apply the investment details.

Enough disclaiming. Let’s get to the FAQing investments part, starting from square one.

Investment Basics

Q: I don’t get “investing”. Why would anyone give me money just because I already have some mon...
Weekly LW Meetups
by FrankAdamek — 2017-02-10T16:51:44.865Z

This summary was posted to LW Main on February 10th. The following week's summary is here.

The following meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Baltimore / UMBC Meetup - usefulness and meaning of "truth": 12 February 2017 11:00AM
[Berlin] Sequences Reading Group: 23 February 2017 07:15PM
Chicago RRG - Typical Mind Fallacy: 12 February 2017 01:00PM
Cologne - Monthly meetup: Saturday Feb 18 2017: 18 February 2017 05:00PM
Melbourne Rationality Dojo, February - Tai Chi & Lightning Talks: 12 February 2017 03:30PM
[Metro Detroit / Ann Arbor], Michigan: 11 February 2017 04:30PM
[Moscow] Games in Kocherga club: FallacyMania, Tower of Chaos, Scientific Discovery: 22 February 2017 07:40PM
San Francisco Meetup: Board Games: 13 February 2017 06:15PM
Sydney Rationality Dojo - March 2017: 05 March 2017 04:00PM
Washington, D.C.: Fun & Games: 12 February 2017 03:30PM

Locations with regularly scheduled meetups: Ann Arbor, Austin, Baltimore, Berlin, Boston, Brussels, Buffalo, Canberra, Chicago, Cologne, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, Netherlands, New Hampshire, New York, Philadelphia, Prague, Research Triangle NC, San Francisco Bay Area, Seattle, St. Petersburg, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup be...
Considerations On Cost Disease
by Scott Alexander — 2017-02-10T04:33:36.000Z

I.

Tyler Cowen writes about cost disease. I’d previously heard the term used to refer only to a specific theory of why costs are increasing, involving labor becoming more efficient in some areas than others. Cowen seems to use it indiscriminately to refer to increasing costs in general – which I guess is fine, goodness knows we need a word for that.

Cowen assumes his readers already understand that cost disease exists. I don’t know if this is true. My impression is that most people still don’t know about cost disease, or don’t realize the extent of it. So I thought I would make the case for the cost disease in the sectors Tyler mentions – health care and education – plus a couple more.

First

let’s look at primary education:

There was some argument about the style of this graph, but as per Politifact the basic claim is true. Per student spending has increased about 2.5x in the past forty years even after adjusting for inflation.

At the same time, test scores have stayed relatively stagnant. You can see the full numbers here, but in short, high school students’ reading scores went from 285 in 1971 to 287 today – a difference of 0.7%.

There is some heterogenity across races – white students’ test scores increased 1.4% and minority students’ scores by about 20%. But it is hard to credit school spending for the minority students’ improvement, which occurred almost entirely during the period from 1975-1985. School spending has been on exactly the same trajectory before and after that time, and in white and minority areas, suggesting that there was something specific about that decade which improved minority (but not white) scores. Most likely this was the general improvement in minorities’ conditions around that time, giving them better nutrition and a more stable family life. It’s hard to construct a narrative where it was school spending that did it – and even if it did, note that the majority of the increase in school spending happened from 1985 on, and demonstrably helped neither whites nor minorities.

I discuss this phenomenon more here and here, but the summary is: no, it’s not just because of special ed; no, it’s not just a factor of how you measure test scores; no, there’s not a “ceiling effect”. Costs really did more-or-less double without any concomitant increase in measurable quality.

So, imagine you’re a poor person. White, minority, whatever. Which would you...