trentmkelly/LessWrong-43k huggingface dataset:
See the dark world
by So8res — 2015-09-20T03:00:00.000Z
Link: https://mindingourway.com/see-the-dark-world/

Consider fictional Carol, who has convinced herself that she doesn't need to worry about the suffering of people who live far away. She works to improve her local community, and donates to her local church. She's a kind and loving woman, and she does her part, and (she reasons) that's all anyone can be expected to do.

Now consider fictional Dave, who failed a job interview. When telling his friends the story, he emphasizes how the interviewers were biased against him, and how they asked stupid questions.

Meanwhile, driven by hunger, a fox tries to reach some grapes hanging high on the vine but is unable to, although he leaps with all his strength. As he goes away, he remarks "Oh, you aren't even ripe yet! I don't need any sour grapes."

All of these reactions — and many others — share a common kernel. Carol, Dave, and the fox are all inventing reasons why an unpleasant state of affairs is acceptable. They're not inventing reasons why the world is good, by any means; but they are putting forth cognitive effort to make it seem tolerable.

Carol would surely tell you that it's terrible that children are suffering abroad — but only after convincing herself that her duty to help them had been discharged.

The fox would tell you that the world is worse for being full of sour grapes — and yet, he still had to work hard to assure himself that he didn't live in a far worse world, where the grapes were both ripe and inaccessible.

There's a certain type of darkness in the world that most people simply cannot to see. It's not the abstract darkness: people will readily acknowledge that the world is broken, and explain how and why the hated out-group is responsible. And that's exactly what I'm pointing at: upon seeing that the world is broken, people experience an impulse to explain the brokenness in a way that relieves the tension. When seeing that the world is broken, people reflexively feel a need to explain. Carol can acknowledge that there is suffering abroad, but this acknowledgement comes part and parcel with an explanation about why she bears no responsibility. Dave can acknowledge that he failed to pass the interview, but his mind automatically generates reasons why this is an acceptable state of affairs.

This is the type of darkness in the world that most people cannot see: they cannot see a world that is unacceptable. Upon noticing that the world is broken, they reflexiv...
Ideas for rationality slogans?
by Gleb_Tsipursky — 2015-09-19T23:32:03.725Z

As part of my broader project of promoting rationality widely, I'm going to work on making rationality-themed merchandise with slogans. I'd appreciate any ideas on what slogans would be short (5 words or less), engaging, and accessible, and appealing for both aspiring rationalists and smart youth and young adults who are just starting to learn about rationality. As an example, slogans like "Growing Mentally Stronger" or "Updating My Beliefs" are good, but "Tsuyoku Naritai!" is not, however much I personally like that slogan.
Kant's Multiplication
by Vamair0 — 2015-09-19T14:25:20.549Z

In this community there is a respected technique of "shutting up and multiplying". However using it in many realistic ethical dillemas can be difficult. Imagine a situation: there is a company, and each its employee gains utility for pressing buttons. Each employee has a one-use only button that when pressed gives an employee one hundred units of utility, while all the others lose a unit each. They can't communicate about the button and there are no other effects. Is it ethical to press the button?

This is an extremely simple situation. Utilitarianism, no matter which one, would easily say that it's ethical to press the button if there are less than one hundred and one employee and unethical if more than one hundred and one. I believe (the proponents of other ethical theories may correct me if I'm wrong) that both virtue ethics (a person demonstrates a vice by pressing a button) and deontology (that's a kind of stealing and stealing is wrong) as they're usually used (and not as a utiitarianism substitute) would say it's wrong to be the first one to press the button, and so all the eleven employees would lose ninety utils.

But the only reason this situation is so simple under utilitarianism is that we've got a direct access to the employees utility functions. Usually, though, that's not the case. If we want to make a decision in a common question such as "is it ethical to throw a piece of trash on the road, or is it better to carry it to the trash bin" or "is it okay to smoke in a room with other people inside" we had to calculate the utility we gain from throwing it right here versus the utility of all the people. We can also use quick rules, which would say "no" in the both situations. But if there's no rule or two rules, or we don't trust one, then it would be useful to have a method that's more reliable than our Fermi estimations of utility or even money.

I believe there is such a method, and as you probably already figured out, it's the question "what would've happened if everyone does something like this". It's most often used in the context of deontology, but for a utilitarian it allows to feel the shared costs.

What am I talking about? Imagine we have to estimate if we should throw a piece of trash on a road. To calculate we're taking the number of people N that will be travelling this road, calculate their average loss for irritation R of seeing a piece ...
Ask and ye shall be answered
by Stuart_Armstrong — 2015-09-18T21:53:21.540Z

A putative new idea for AI control; index here.

EDIT: the mathematics of this idea can be found here.

It would be useful to have a mechanism for getting an AI to provide an honest answer to a specific question. This post presents such a mechanism.

The question is: what is the impact of X/¬X on the expected utility of a utility function v? Here X/¬X is some well-defined binary outcome. Formally, for a constant c (positive or negative), we want to query the AI as to whether Q(v,X,c) is true, where

Q(v,X,c) = {E(v|X) - E(v|¬X) > c}.

How can we do that? Well, for a start, let M(u) be a u-maximising agent. We'll assume we can feed M() any utility function to maximise. We'll further assume that M() is boxed, that it has a single output bit Y (or ¬Y), and that we have cut any acausal trade possibilities. We'll add another binary process Z, which, if Z=1, erases the bit from Y before we can see it. In actual fact, P(Z=0)≈1.

The remaining assumptions are on the operations we can perform on v (without needing any further understanding of v). We replace v by v#, defined in the following way. First of all, we make v# indifferent to worlds in which X≠Y or Z=0 (note: there are subtleties to defining indifference to events that are partially under the AI's control; I will address these in a later post). Therefore any v# maximiser will behave as if it lived in a world where X=Y (its output and the outcome of X were the same) and Z=1 (its output was erased and never read). Then we add a special, one-off penalty of -c for outputting Y=1 (there is no effect for outputting Y=0). This defines v#, and we give our boxed AI v# to maximise.

So, how will M(v#) behave? Because it acts as if X=Y with certainty, it behaves as if it can determine the value of X by outputting Y. Because it acts as if Z=0 with certainty, it behaves as if the output of Y is never read, so it can determine the value of X without affecting anything else about the universe. It will now behave as if it was a v maximiser with control over X. So if it outputs ¬Y, it expects to get utility E(v|¬X). If it outputs Y, it expects to get utility E(v|X)-c (because of the -c penalty).

Therefore (ignoring ties) M(v#) will output Y iff it estimates Q(v,X,c) to be true. And, since in actual fact Z=0 almost certainly, we get to see its output, hence its conclusion. Voila!

Note: the advan...
Monty Hall Sleeping Beauty
by PhilGoetz — 2015-09-18T21:18:23.137Z

A friend referred me to another paper on the Sleeping Beauty problem. It comes down on the side of the halfers.

I didn't have the patience to finish it, because I think SB is a pointless argument about what "belief" means. If, instead of asking Sleeping Beauty about her "subjective probability", you asked her to place a bet, or take some action, everyone could agree what the best answer was. That it perplexes people is a sign that they're talking non-sense, using words without agreeing on their meanings.

But, we can make it more obvious what the argument is about by using a trick that works with the Monty Hall problem: Add more doors. By doors I mean days.

The Monty Hall Sleeping Beauty Problem is then:

On Sunday she's given a drug that sends her to sleep for a thousand years, and a coin is tossed.

If the coin lands heads, Beauty is awakened and interviewed once.

If the coin comes up tails, she is awakened and interviewed 1,000,000 times.

After each interview, she's given a drug that makes her fall asleep again and forget she was woken.

Each time she's woken up, she's asked, "With what probability do you believe that the coin landed tails?"

The halfer position implies that she should still say 1/2 in this scenario.

Does stating it this way make it clearer what the argument is about?
Matching donation funds and the problem of illusory matching 
by Stefan_Schubert — 2015-09-18T20:05:46.098Z

On average, matching donations supposedly do increase charitable giving (though I want to see more rigorous research on this - tips are welcome). One criticism against them is, though, that they are "illusory" - that is, that the matching donor eventually donates the same amount whether smaller donors match their donations or not. That means that a dollar from a smaller donor doesn't actually cause the matcher to contribute more.

One way to make matching donations real, as opposed to illusory, is this. Suppose that the matching donor is indifferent between donating to two charities (e.g. Against Malaria Foundation and MIRI). At the same time, lots of small donors think that one of them is far better than the other. Also, suppose that the matching donor sets the terms so that it's virtually certain that their whole matching fund will be used up (this could be done, e.g. by making the matching ratio very favourable).

Under these conditions, it will make a difference whether a small donor contributes or not, since if you don't, chances are that your donation will be replaced by a donation to the other charity. That means that a dollar from you as a smaller donor on average does cause the matcher to contribute more to your favourite charity.

This suggest a more general strategy for leveraging charity contributions. You could set up a set of matching funds, to which small donors could contribute. These funds would be "disjunctive" - they would match contributions to, e.g. AMF or MIRI, Open Borders or MSF or The Humane League, etc. The funds would from time to time declare that they match any donations to their target charities, and supporters of the respective target charities would start competing, in effect, for the matching donations.

In the simplest system, only people who are more or less indifferent between the target charities would donate to the matching funds. A somewhat more complex system incentivizes people who prefer one of the target charities, A, to give to the matching fund. Under such a system, an "A-ear-marked" donation to the matching fund would increase the matching donations (e.g. from 1:1 to 3:2) to A, and decrease matching donations to the other target charities the matching fund supports. That will, in turn, incentivize more giving to A relative to the other target charities. It is important that such adjustments are done in th...
Agents detecting agents: counterfactual versus influence
by Stuart_Armstrong — 2015-09-18T16:17:36.920Z

A putative new idea for AI control; index here. Status: still somewhat underdeveloped.

In this post, I want to explore another avenue for controlling the ongoing influence of an AI (through subagents or other means). It is the idea, touched upon here and here, of figuring out whether the AI's existence was important or not for predicting the future. Informally, if you knew the AI was turned on or not, would you consider that information valuable or almost irrelevant?

Agent vs agent

This post rests on two insights. The first is that, though it is hard to define what an agent or subagent is, we can produce examples of them. Or at least, at the point where we are capable of producing AIs that might be dangerous, we can likely produce examples of them. This can give us a scale for comparing the importance or impact of different agents.

The second is that if we produce an agent with certain motivations, it will likely expand and fill its light cone with... stuff (of some type or other). Those who don't agree with the AI risk thesis argument may still accept that we could program universe-optimising AIs (ie "we could make a paperclipper, if we were stupid enough to do so").

Why is this relevant? Well, consider an agent E (expansionist) that fills its light cone with stuff (likely including manufacturing and tracking devices), eventually at very high speed. And imagine that there is another agent H (hidden) inside that light cone. At some point, agent E will either detect agent H (and make plans to deal with it), or find its plans hampered by agent H, or agent H will go into deep hiding that does not visibly affect agent E's plans.

Therefore, an expansionist agent is a perfect tool for detecting the influence of other agents. Either agent E will have difficulty accomplishing its plans, or agent H's influence will be so limited that agent E cannot find any trace of it. Any influential agent will get detected, sooner or later, by an expansionist agent. So if E finds H unimportant, then H almost certainly is.

Of course, we don't really want expansionist agents like E - but these can be counterfactual.

The stage and the players

We'll now try and define precisely how we can use this idea to control the influence of an agent, setting out the axioms and assumptions needed along the way. Like some mathematics paper, it might be useful to skip lightly over ...
Weekly LW Meetups
by FrankAdamek — 2015-09-18T15:53:04.537Z

This summary was posted to LW Main on September 11th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Rochester LessWrong and Transhumanists: 12 September 2015 01:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

NH Meet-up: 15 September 2015 07:00PM
Oslo Lesswrong meetup, September: 27 September 2015 04:00PM
San Antonio Meetup: 13 September 2015 12:30PM
San Francisco Meetup: Short Talks: 14 September 2015 06:15PM
Sao Paulo meetup at Starbucks: 12 September 2015 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Durham, NC (RTLW) Discussion Meetup: 17 September 2026 07:00PM
Sydney Meetup - September: 23 September 2015 06:30PM
[Tel Aviv] Effective Altruism @ LessWrong Tel Aviv: 29 September 2015 07:00PM
Vienna: 26 September 2015 03:00PM
Washington, D.C.: Sorting Hat: 13 September 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in...
The subagent problem is really hard
by Stuart_Armstrong — 2015-09-18T13:06:32.790Z

A putative new idea for AI control; index here.

The first step to solving a problem is to define it. The first first step is to realise how tricky it is to define. This is a stub on a difficult problem.

Subagents and turning AIs off

One of the hardest things with designing safe AIs, is turning them off. Installing an OFF button is trivial; motivating them to let you push it can be done. No, the real problem is the subagent problem. The AI can create one or many subagents to carry out their task; once that's done, turning off the original AI is of little value, we have to control or extinguish the subagents.

A general problem

Subagents are a problem for many other control ideas. For example, if you want to program 'achieve X using only this many computer cycles', then 'create a subagent that wants to achieve X without the computer cycle restrictions' is one way of doing it (as long as that can be done within the restrictions). The same goes for energy use and other restrictions. Satisficers can satisfice by creating a maximiser. A boxed Oracle can remain in the box while cheerfully plotting to create an unboxed copy of itself. In fact, whatever the AI's motivation, "create subagents with the same motivation" is almost always a good idea. And, once created, there is no motivation to want to shut them down. Even if the AI is motivated to shut itself down, there is no reason for it to extend this to subagents.

Pre-corriged agents would at least be motivated to keep control of their subagents (unlike corrigible agents, who have no current desire to create subagents that would obey their future preferences).

What is a subagent?

It's also almost impossible to define a subagent. Is a corporation a subagent? Is a thousand pieces of code that reside on different computers and occasionally communicate a subagent? Is an employee or a seduced human a subagent? A pile of rocks with water running through could be a piece of conceptual art - or a slow computer. How about a virulent meme among the human population, making us all into its subagents?

The best idea I've found for defining subagents is that they are reactive intelligences with planning abilities, so random natural events that should have large influence in the future (say, a meteor heading towards Earth) would instead be re-diverted to have littl...
Meetup : San Francisco Meetup: Fun and Games
by rocurley — 2015-09-17T04:18:05.502Z

Discussion article for the meetup : San Francisco Meetup: Fun and Games

WHEN:
21 September 2015 06:15:00PM (-0700)

WHERE:
1390 Market St, San Francisco, CA 94103

We'll be meeting to hang out and play board games at my apartment (our normal place doesn't reliably have tables). Please bring your own games: I have Dominion, Ticket to Ride, and Avalon. You can call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Fun and Games
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-09-16T23:48:55.299Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
20 September 2015 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

Crossposted from mailing list. Meeting in courtyard, gathering from 3:00pm, hard start 3:30pm - until closing.

We'll be meeting to play games and/or just hang out and talk.

If you want to get people together for a big game, or a specific game, post here looking for players and get an early start because they'll be kicking us out a little before 7pm.

Remember to bring games!

Upcoming Meetups:

Sep 27: Singing (weather permitting)

Oct. 04: Availability Heuristic

Oct. 11: Fun and Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Meetup : Moscow LW experimental group meetup
by Alexander230 — 2015-09-16T20:33:32.104Z

Discussion article for the meetup : Moscow LW experimental group meetup

WHEN:
25 September 2015 07:20:00PM (+0300)

WHERE:
Москва, ул. Дубининская, 20

Welcome to the next experimental group (former "lecture centre") meetup!

Our plan:

A talk about Sunk Cost Fallacy.

FallacyMania game ( https://lesswrong-ru.hackpad.com/Fallacymania--neGfMe9MFjH ).

Meetup schedule is in hackpad: https://lw-msk-lect.hackpad.com/-25--CwbUS5gRHfh

Come to "Nauchka", ul.Dubininskaya, 20. Entrance through the Central children library #14. Nearest metro station is Paveletskaya. Map is here: http://nauchka.ru/contact-information/ . If you are lost, call Sasha at +7-905-527-30-82.

Meetup begins at 19:20, the length is 1.5 hours.

Discussion article for the meetup : Moscow LW experimental group meetup
Cardiologists and Chinese Robbers
by Scott Alexander — 2015-09-16T19:00:00.000Z

I.

It takes a special sort of person to be a cardiologist. This is not always a good thing.

You may have read about one or another of the “cardiologist caught falsifying test results and performing dangerous unnecessary surgeries to make more money” stories, but you might not have realized just how common it really is. Maryland cardiologist performs over 500 dangerous unnecessary surgeries to make money. Unrelated Maryland cardiologist performs another 25 in a separate incident. California cardiologist does “several hundred” dangerous unnecessary surgeries and gets raided by the FBI. Philadelphia cardiologist, same. North Carolina cardiologist, same. 11 Kentucky cardiologists, same. Actually just a couple of miles from my own hospital, a Michigan cardiologist was found to have done $4 million worth of the same. Etc, etc, etc.

My point is not just about the number of cardiologists who perform dangerous unnecessary surgeries for a quick buck. It’s not even just about the cardiology insurance fraud, cardiology kickback schemes, or cardiology research data falsification conspiracies. That could all just be attributed to some distorted incentives in cardiology as a field. My point is that it takes a special sort of person to be a cardiologist.

Consider the sexual harassment. Head of Yale cardiology department fired for sexual harassment with “rampant bullying”. Stanford cardiologist charged with sexually harassing students. Baltimore cardiologist found guilty of sexual harassment. LA cardiologist fined $200,000 for groping med tech. Three different Pennsylvania cardiologists sexually harassing the same woman. Arizona cardiologist suspended on 19 (!) different counts of sexual abuse. One of the “world’s leading cardiologists” fired for sending pictures of his genitals to a female friend. New York cardiologist in trouble for refusing to pay his $135,000 bill at a strip club. Manhattan cardiologist taking naked pictures of patients, then using them to sexually abuse employees. New York cardiologist secretly installs spycam in office bathroom. Just to shake things up, a Florida cardiologist was falsely accused of sexual harassment as part of feud with another cardiologist.

And yeah, you can argue that if you put high-status men in an office with a lot of subordinates, sexual harassment will be depressingly common just as a result of the environment. But there’s also the T

...