trentmkelly/LessWrong-43k huggingface dataset:
Knightian uncertainty: a rejection of the MMEU rule
by So8res — 2014-08-26T03:03:57.026Z

Recently, I found myself in a conversation with someone advocating the use of Knightian uncertainty. He (who I'm anonymizing as Sir Percy) made suggestions that are useful to most bounded reasoners, and which can be integrated into a Bayesian framework. He also claimed preferences that depend upon his Knightian uncertainty and that he's not an expected utility maximizer. Further, he claimed that Bayesian reasoning cannot capture his preferences. Specifically, Sir Percy said he maximizes minimum expected utility given his Knightian uncertainty, using what I will refer to as the "MMEU rule" to make decisions.

In my previous post, I showed that Bayesian expected utility maximizers can exhibit behavior in accordance with his preferences. Two such reasoners, Paranoid Perry and Cautious Caul, were explored. These hypothetical agents demonstrate that it is possible for Bayesians to be "ambiguity averse", e.g. to avoid certain types of uncertainty.

But Perry and Caul are unnatural agents using strange priors. Is this because we are twisting the Bayesian framework to represent behavior it is ill-suited to emulate? Or does the strangeness of Perry and Caul merely reveal a strangeness in the MMEU rule?

In this post, I'll argue the latter: maximization of minimum expected utility is not a good decision rule, for the same reason that Perry and Caul seem irrational. My rejection of the MMEU rule will follow from my rejections of Perry and Caul.

A rejection of Perry

I understand the appeal of Paranoid Perry, the agent that assumes ambiguity is resolved adversarially. Humans often keep the worst-case in mind, avoid big gambles, and make sure that they'll be OK even if everything goes wrong. Perry promises to capture some of this intuitively reasonable behavior.

Unfortunately, this promise is not kept. From the description of the MMEU rule, you might think that Perry is forgoing high utility in the average case to ensure moderate utility in the worst case. But this is not the case: Perry willingly takes huge gambles, so long as those gambles are resolved by "normal" uncertainty rather than "adversarial" uncertainty.

Allow me to reiterate: Maximizing minimum expected utility does not ensure that you do well in the worst case. It merely selects a single type of uncertainty against which to play defensively, and then gambles against the rest. To illustrate, consider the followi...
The immediate real-world uses of Friendly AI research
by ancientcampus — 2014-08-26T02:47:16.924Z

Much of the glamor and attention paid toward Friendly AI is focused on the misty-future event of a super-intelligent general AI, and how we can prevent it from repurposing our atoms to better run Quake 2. Until very recently, that was the full breadth of the field in my mind. I recently realized that dumber, narrow AI is a real thing today, helpfully choosing advertisements for me and running my 401K. As such, making automated programs safe to let loose on the real world is not just a problem to solve as a favor for the people of tomorrow, but something with immediate real-world advantages that has indeed already been going on for quite some time. Veterans in the field surely already understand this, so this post is directed at people like me, with a passing and disinterested understanding of the point of Friendly AI research, and outlines an argument that the field may be useful right now, even if you believe that an evil AI overlord is not on the list of things to worry about in the next 40 years.

Let's look at the stock market. High-Frequency Trading is the practice of using computer programs to make fast trades constantly throughout the day, and accounts for more than half of all equity trades in the US. So, the economy today is already in the hands of a bunch of very narrow AIs buying and selling to each other. And as you may or may not already know, this has already caused problems. In the “2010 Flash Crash”, the Dow Jones suddenly and mysteriously hit a massive plummet only to mostly recover within a few minutes. The reasons for this were of course complicated, but it boiled down to a couple red flags triggering in numerous programs, setting off a cascade of wacky trades.

The long-term damage was not catastrophic to society at large (though I'm sure a couple fortunes were made and lost that day), but it illustrates the need for safety measures as we hand over more and more responsibility and power to processes that require little human input. It might be a blue moon before anyone makes true general AI, but adaptive city traffic-light systems are entirely plausible in upcoming years.

To me, Friendly AI isn't solely about making a human-like intelligence that doesn't hurt us – we need techniques for testing automated programs, predicting how they will act when let loose on the world, and how they'll act when faced ...
Persistent Idealism
by jefftk — 2014-08-26T01:38:20.167Z

When I talk to people about earning to give, it's common to hear worries about "backsliding". Yes, you say you're going to go make a lot of money and donate it, but once you're surrounded by rich coworkers spending heavily on cars, clothes, and nights out, will you follow through? Working at a greedy company in a selfishness-promoting culture you could easily become corrupted and lose initial values and motivation.

First off, this is a totally reasonable concern. People do change, and we are pulled towards thinking like the people around us. I see two main ways of working against this:

Be public with your giving. Make visible commitments and then list your donations. This means that you can't slowly slip away from giving; either you publish updates saying you're not going to do what you said you would, or you just stop updating and your pages become stale. By making a public promise you've given friends permission to notice that you've stopped and ask "what changed?"

Don't just surround yourself with coworkers. Keep in touch with friends and family. Spend some time with other people in the effective altruism movement. You could throw yourself entirely into your work, maximizing income while sending occasional substantial checks to GiveWell's top picks, but without some ongoing engagement with the community and the research this doesn't seem likely to last.

One implication of the "won't you drift away" objection, however, is often that if instead of going into earning to give you become an activist then you'll remain true to your values. I'm not so sure about this: many people who are really into activism and radical change in their 20s have become much less ambitious and idealistic by their 30s. You can call it "burning out" or "selling out" but decreasing idealism with age is very common. This doesn't mean people earning to give don't have to worry about losing their motivation—in fact it points the opposite way—but this isn't a danger unique to the "go work at something lucrative" approach. Trying honestly to do the most good possible is far from the default in our society, and wherever you are there's going to be pressure to do the easy thing, the normal thing, and stop putting so much effort into altruism.
Open thread, 25-31 August 2014
by jaime2000 — 2014-08-25T11:14:52.992Z

Previous open thread

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one.

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Announcing The Effective Altruism Forum
by RyanCarey — 2014-08-24T08:07:31.980Z

The Effective Altruism Forum will be launched at effective-altruism.com on September 10, British time.

Now seems like a good time time to discuss why we might need an Effective Altruism Forum, and how it might compare to LessWrong.

About the Effective Altruism Forum

The motivation for the Effective Altruism Forum is to improve the quality of effective altruist discussion and coordination. A big part of this is to give many of the useful features of LessWrong to effective altruists, including:

Archived, searchable content (this will begin with archived content from effective-altruism.com)

Meetups

Nested comments

A karma system

A dynamically upated list of external effective altruist blogs

Introductory materials (this will begin with these articles)

The Effective Altruism Forum has been designed by Mihai Badic. Over the last month, it has been developed by Trike Apps, who have built the new site using the LessWrong codebase. I'm glad to report that it is now basically ready, looks nice, and is easy to use.

I expect that at the new forum, as on the effective altruist Facebook and Reddit pages, people will want to discuss the which intellectual procedures to use to pick effective actions. I also expect some proposals of effective altruist projects, and offers of resources. So users of the new forum will share LessWrong's interest in instrumental and epistemic rationality. On the other hand, I expect that few of its users will want to discuss the technical aspects of artificial intelligence, anthropics or decision theory, and to the extent that they do so, they will want to do it at LessWrong. As a result, I expect the new forum to cause:

A bunch of materials on effective altruism and instrumental rationality to be collated for new effective altruists

Discussion of old LessWrong materials to resurface

A slight increase to the number of users of LessWrong, possibly offset by some users spending more of their time posting at the new forum.

At least initially, the new forum won't have a wiki or a Main/Discussion split and won't have any institutional affiliations.

Next Steps:

It's really important to make sure that the Effective Altruism Forum is established with a beneficial culture. If people want to help that process by writing some seed materials, to be posted around the time of the site's launch, th...
Meetup : Any one interested in Milwaukee meetup?
by zerotimer — 2014-08-24T04:19:58.661Z

Discussion article for the meetup : Any one interested in Milwaukee meetup?

WHEN:
06 September 2014 05:00:00PM (-0500)

WHERE:
Rock Bottom, 740 N Plankinton Ave, Milwaukee, WI 53203

We can discuss the options for meetup place and date but is anyone here interested in milwaukee meetup. I dont think it has happened before.

Discussion article for the meetup : Any one interested in Milwaukee meetup?
Meetup : Urbana-Champaign: Reconstituting
by Manfred — 2014-08-23T04:19:55.882Z

Discussion article for the meetup : Urbana-Champaign: Reconstituting

WHEN:
31 August 2014 02:00:00PM (-0500)

WHERE:
206 S. Cedar St, 61801

In the puddles and pools formed by the fall rains, meetup groups that seemed dead end their arid hibernation and emerge from the earth.

Come join us for the first meetup of the new school year. Board games and light discussion are likely. A starter topic: increasing exposure to positive black swans.

Google group

Discussion article for the meetup : Urbana-Champaign: Reconstituting
Study: In giving charity, let not your right hand...
by homunq — 2014-08-22T22:23:34.215Z

So, here's the study¹:

It's veterans' day in Canada. As any good Canadian knows, you're supposed to wear a poppy to show you support the veterans (it has something to do with Flanders Field). As people enter a concourse on the university, a person there does one of three things: gives them a poppy to wear on their clothes; gives them an envelope to carry and tells them (truthfully) that there's a poppy inside; or gives them nothing. Then, after they've crossed the concourse, another person asks them if they want to put donations in a box to support Canadian war veterans.

Who do you think gives the most?

...

If you guessed that it's the people who got the poppy inside the envelope, you're right. 78% of them gave, for an overall average donation of $0.86. That compares to 58% of the people wearing the poppy, for an average donation of $0.34; and 56% of those with no poppy, for an average of $0.15.

Why did the envelope holders give the most? Unlike the no-poppy group, they had been reminded of the expectation of supporting veterans; but unlike the poppy-wearers, they hadn't been given an easy, cost-free means of demonstrating their support.

I think this research has obvious applications, both to fundraising and to self-hacking. It also validates the bible quote (Matthew 6:3) which is the title of this article.

¹ The Nature of Slacktivism: How the Social Observability of an Initial Act of Token Support Affects Subsequent Prosocial Action; K Kristofferson, K White, J Peloza - Journal of Consumer Research, 2014
[LINK] Physicist Carlo Rovelli on Modern Physics Research
by Shmi — 2014-08-22T21:46:01.254Z

A blog post in Scientific American, well worth reading. Rovelli is a researcher in Loop Quantum Gravity.

Some quotes:

Horgan: Do multiverse theories and quantum gravity theories deserve to be taken seriously if they cannot be falsified?

Rovelli: No.

Horgan: What’s your opinion of the recent philosophy-bashing by Stephen Hawking, Lawrence Krauss and Neil deGrasse Tyson?

Rovelli: Seriously: I think they are stupid in this. I have admiration for them in other things, but here they have gone really wrong. Look: Einstein, Heisenberg, Newton, Bohr…. and many many others of the greatest scientists of all times, much greater than the names you mention, of course, read philosophy, learned from philosophy, and could have never done the great science they did without the input they got from philosophy, as they claimed repeatedly. You see: the scientists that talk philosophy down are simply superficial: they have a philosophy (usually some ill-digested mixture of Popper and Kuhn) and think that this is the “true” philosophy, and do not realize that this has limitations.

Horgan: Can science attain absolute truth?

Rovelli: I have no idea what “absolute truth” means. I think that science is the attitude of those who find funny the people saying they know something is absolute truth. Science is the awareness that our knowledge is constantly uncertain. What I know is that there are plenty of things that science does not understand yet. And science is the best tool found so far for reaching reasonably reliable knowledge.

Horgan: Do you believe in God?

Rovelli: No. But perhaps I should qualify the answer, because like this it is bit too rude and simplistic. I do not understand what “to believe in God” means. The people that “believe in God” seem like Martians to me. I do not understand them. I suppose this means that I “do not believe in God”. If the question is whether I think that there is a person who has created Heavens and Earth, and responds to our prayers, then definitely my answer is no, with much certainty.

Horgan: Are science and religion compatible?

Rovelli: Of course yes: you can be great in solving Maxwell’s equations and pray to God in the evening. But there is an unavoidable clash between science and certain religions, ...
Weekly LW Meetups
by FrankAdamek — 2014-08-22T15:38:29.362Z

This summary was posted to LW Main on August 15th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

[Atlanta] MIRIxAtlanta - Decision Theory 2: 17 August 2014 06:00PM
Bratislava: 18 August 2014 06:00PM
Helsinki Book Blanket Meetup: 16 August 2014 03:00PM
Houston, TX: 13 September 2014 02:00PM
Perth, Australia: Sunday lunch: 17 August 2014 12:00PM
Perth, Australia: Games night: 02 September 2014 06:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 16 August 2025 01:30PM
[Cambridge MA] The Psychology of Video Games: 17 August 2014 03:30PM
Canberra: Cooking for LessWrongers: 22 August 2014 06:00PM
[Moscow] Regular Moscow Meetup: 17 August 2014 02:00PM
Sydney Meetup - August: 27 August 2014 06:30PM
[Utrecht] Cognitive Biases: 23 August 2014 02:00PM
Washington, D.C.: Fun & Games Meetup: 17 August 2014 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Salt Lake City, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in...
Conservation of Expected Jury Probability
by jefftk — 2014-08-22T15:25:34.102Z

The New York Times has a calculator to explain how getting on a jury works. They have a slider at the top indicating how likely each of the two lawyers think you are to side with them, and as you answer questions it moves around. For example, if you select that your occupation is "blue collar" then it says "more likely to side with plaintiff" while "white collar" gives "more likely to side with defendant". As you give it more information the pointer labeled "you" slides back and forth, representing the lawyers' ongoing revision of their estimates of you. Let's see what this looks like.

Initial

Selecting "Over 30"

Selecting "Under 30"

For several other questions, however, the options aren't matched. If your household income is under $50k then it will give you "more likely to side with plaintiff" while if it's over $50k then it will say "no effect on either lawyer". This is not how conservation of expected evidence works: if learning something pushes you in one direction, then learning its opposite has to push you in the other.

Let's try this with some numbers. Say people's leanings are:

income	probability of siding with plaintiff	probability of siding with defendant

>$50k
50%
50%

<$50k
70%
30%

Before asking you your income the lawyers' best guess is you're equally likely to be earning >$50k as <$50k because $50k's the median [1]. This means they'd guess you're 60% likely to side with the plaintiff: half the people in your position earn over >$50k and will be approximately evenly split while the other half of people who could be in your position earn under <$50k and would favor the plaintiff 70-30, and averaging these two cases gives us 60%.

So the lawyers best guess for you is that you're at 60%, and then they ask the question. If you say ">$50k" then they update their estimate for you down to 50%, if you say "<$50k" they update it up to 70%. "No effect on either lawyer" can't be an option here unless the question gives no information.

[1] Almost; the median income in the US in 2012 was $51k. (pdf)
Memory is Everything 
by Qwake — 2014-08-22T04:48:49.875Z

I have found (there is some (evidence)[http://mentalfloss.com/article/52586/why-do-our-best-ideas-come-us-shower] to suggest this) that showers are a great place to think. While I am taking a shower I find that I can think about things in a whole new perspective and it's very refreshing. Well today, while I was taking a shower, an interesting thing popped into my head. Memory is everything. Your memory contains you, it contains your thoughts, it contains your own unique perception of reality. Imagine going to bed tonight and waking up with absolutely no memory of your past. Would you still consider that person yourself? There is no question that our memories/experiences influence our behavior in every possible way. If you were born in a different environment with different stimuli you would've responded to your environment differently and became a different person. How different? I don't want to get involved in the nature/nurture debate but I think there is no question that humans are influenced by their environment. How are humans influenced by our environment? Through learning from our past experiences, which are contained in our memory. I'm getting off topic and I have no idea what my point is... So I propose a thought experiment!

Omega the supercomputer gives you 3 Options. Option 1 is for you to pay Omega $1,000,000,000 and Omega will grant you unlimited utility potential for 1 week in which Omega will basically provide to your every wish. You will have absolutely no memory of the experience after the week is up. Option 2 is for Omega to pay you $1,000,000,000 but you must be willing to suffer unlimited negative utility potential for a week (you will not be harmed physically or mentally you will simply experience excruciating pain). You will also have absolutely memory of this experience after the week (your subconscious will also not be affected). Finally, Option 3 is simply to refuse Option 1 and 2 and maintain the status quo.

At first glance, it may seem that Option 2 is simply not choosable. It seems insane to subject yourself to torture when you have the option of nirvana. But it requires more thought than that. If you compare Option 1 to Option 2 after the week is up there is no difference between the options except that Option 2 nets you 2 billion dollars compared to Option 1. In both Options you have absolutely no memory of either we...
Meetup : Washington, D.C.: Museums Meetup
by RobinZ — 2014-08-22T04:17:28.493Z

Discussion article for the meetup : Washington, D.C.: Museums Meetup

WHEN:
24 August 2014 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

We will be congregating between 3:00 and 3:30 in the Kogod Courtyard of the National Portrait Gallery (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance), and heading out after that to look at exhibits. Those present will decide on museums to visit (possibly including the National Portrait Gallery itself) and inform anyone joining belatedly of location and direction of travel via the lesswrong-dc Google Group.

Discussion article for the meetup : Washington, D.C.: Museums Meetup