trentmkelly/LessWrong-43k huggingface dataset:
Tonic Judo
by Gram_Stone — 2016-04-02T21:19:06.459Z

(Content note: This is a story about one of the times that I've applied my understanding of rationality to reduce the severity of an affect-laden situation. This may remind you of Bayesian Judo, because it involves the mental availability and use of basic rationality techniques to perform feats that, although simple to perform in hindsight, leave an impression of surprising effectiveness on those who don't know what is generating the ability to perform the feats. However, I always felt dissatisfied with Bayesian Judo because it seemed dishonest and ultimately unproductive. Rationalists should exude not only auras of formidability, but of compassion. Read assured that the participants in this story leave mutually satisfied. I haven't read much about cognitive behavioral therapy or nonviolent communication, but this will probably look like that. Consider moving on to something else if what I've described doesn't seem like the sort of thing that would interest you.)

My friend lost his comb, and it was awful. He was in a frenzy for half an hour, searching the entire house, slamming drawers and doors as he went along. He made two phone calls to see if other people took his comb without asking. Every once in a while I would hear a curse or a drawn-out grunt of frustration. I kind-of couldn't believe it.

It makes more sense if you know him. He has a very big thing about people taking his possessions without asking, and the thing is insensitive to monetary value.

I just hid for a while, but eventually he knocked on my door and said that he 'needed to rant because that was the headspace he was in right now'. So he ranted about some non-comb stuff, and then eventually we got to the point where we mutually acknowledged that he was basically talking at me right now, and not with me, and that he was seriously pissed about that comb. So we started talking for real.

I said, "I can hardly imagine losing any one of my possessions and being as angry as you are right now. I mean, in particular, I never comb or brush my hair, so I can't imagine it in the most concrete possible sense, but even then, I can't imagine anything that I could lose that would make me react that way, except maybe my cellphone or my computer. The only way I can imagine reacting that way is if it was a consistent thing, and someone was consistently overstepping my boundaries by taking my things without a...
[Link] Op-Ed on Brussels Attacks
by Gleb_Tsipursky — 2016-04-02T17:38:18.943Z

Trigger warning: politics is hard mode.

"How to you make America safer from terrorists" is the title of my op-ed published in Sun Sentinel, a very prominent newspaper in Florida, one of the most swingiest of the swing states in the US for the presidential election, and the one with the most votes. The maximum length of the op-ed was 450 words, and it was significantly edited by the editor, so it doesn't convey the full message I wanted with all the nuances, but such is life. My primary goal with the piece was to convey methods of thinking more rationally about politics, such as to use probabilistic thinking, evaluating the full consequences of our actions, and avoiding attention bias. I used the example of the proposal to police heavily Muslim neighborhoods as a case study. Hope this helps Floridians think more rationally and raises the sanity waterline regarding politics!

EDIT: To be totally clear, I used guesstimates for the numbers I suggested. Following Yvain/Scott Alexander's advice, I prefer to use guesstimates rather than vague statements.
Maximizing a quantity while ignoring effect through some channel
by jessicata — 2016-04-02T01:20:57.000Z

Chris Olah originally came up with this idea and I helped to formalize it.

Let A be the AI's action, and B and C quantities possibly affected by the action. For example, suppose C is the amount of reward signal an AI is receiving, and B is the source code of a program that determines the AI's reward signal. We would like to formalize a goal of the form "choose A to maximize U(C), ignoring the effect through B", where U is some utility function over C. In this case, the AI will maximize expected reward signal, but will ignore the source code of the program in its maximization (so it has no incentive to rewrite the code, e.g. to make it output higher rewards).

Here's an attempt to formalize this objective. Let d and d′ be distributions over actions. Define

vd(d′)=EA∼d,A′∼d′,B∼P(B|do(A)),C∼P(C|do(A′),do(B))[U(C)]

A distribution d over actions is optimal iff d∈argmaxd′vd(d′). This is similar to expectation maximization in that an optimal value optimizes an objective that itself depends on the value. Of course, in practice it will usually be computationally difficult to find an optimal d, but this notion of optimality generalizes well to approximate optimality.

A solution d always exists by Kakutani's fixed point theorem; this is because the function mapping the distribution d to the set argmaxd′vd(d′) has a closed graph. In fact, d can be computed using a finite reflective oracle (or equivalently a Nash equilibrium solver).

To see how this works, consider a game where the AI has 4 actions:

Shut down immediately.

Press the shutdown button and shut down after that.

Produce 10 paperclips (causing the AI to get shut down).

Destroy the shutdown button and produce 9 paperclips.

Define B to be whether the shutdown button is pressed, and C to be the outcome. U(C) is the number of paperclips produced if the shutdown button is not pressed. If the shutdown button is pressed, it is 0 for actions 3-4, 5 for action 2, and 10 for action 1. In this case no individual action satisfies the optimality condition, because:

if the AI takes action 1, the shutdown button is not pressed. So the AI could have gotten more utility by taking action 3, controlling for the shutdown button.

if the AI takes action 2, the shutdown button is pressed. So the AI could have gotten more utility by taking action 1, controlling for the shutdown button.

if the AI takes action 3, the shutdown b

...
April 2016 Media Thread
by ArisKatsaris — 2016-04-01T20:02:32.165Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Meetup : San Antonio Meetup
by TheAltar — 2016-04-01T17:16:58.010Z

Discussion article for the meetup : San Antonio Meetup

WHEN:
03 April 2016 03:00:00PM (-0500)

WHERE:
12651 Vance Jackson Rd #118, San Antonio, TX 78230

Bubble tea, frozen yogurt, and discussion at Yumi Berry. All are welcome.

Meetup to discuss rationality and all things LessWrong and meet the local community. Look for the sign that says LessWrong.

Discussion article for the meetup : San Antonio Meetup
Weekly LW Meetups
by FrankAdamek — 2016-04-01T16:52:34.053Z

This summary was posted to LW Main on April 1st. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Baltimore Area / UMBC: Intro to LW and future of the meetup: 03 April 2016 03:00PM
European Community Weekend: 02 September 2016 03:35PM
San Francisco Meetup: Cooking: 04 April 2016 06:15PM
San Jose Meetup: Park Day: 03 April 2016 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

[Melbourne] "What is the Intelligence Explosion?" (and more...) LW Melbourne dojo: 02 April 2016 02:00PM'
Moscow meetup: inspection paradox, common dual process theory fallacies, preparing summer outreach programs: 03 April 2016 02:00PM
New Hampshire Meetup: 12 April 2016 07:00PM
Vienna Meetup: 16 April 2016 02:00PM
Washington, D.C.: Fun & Games: 03 April 2016 03:30PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing li...
Happy Notice Your Surprise Day!
by Vaniver — 2016-04-01T13:02:33.530Z

One of the most powerful rationalist techniques is noticing your surprise.

It ties in to several deep issues. One of them relates to one of my favorite LW comments (the second highest upvoted one in Main):

One of the things that I've noticed about this is that most people do not expect to understand things. For most people, the universe is a mysterious place filled with random events beyond their ability to comprehend or control. Think "guessing the teacher's password", but not just in school or knowledge, but about everything.

Such people have no problem with the idea of magic, because everything is magic to them, even science.

--pjeby

For the universe to make sense to you, you have to have a model; for that model to be useful, you have to notice what it says, and then you need to act on it. I've done many things the wrong way in my life, but the ones I remember as mistakes are the ones where some part of me *knew* it was a problem, and instead of having a discussion with that part of me, I just ignored it and marched on.

It is good to notice your surprise. But that's only the first step.

--Douglas_Knight

So any stories, of tricks you noticed, didn't notice, or successfully pulled?
Consider having sparse insides
by AnnaSalamon — 2016-04-01T00:07:07.777Z

It's easier to seek true beliefs if you keep your (epistemic) identity small. (E.g., if you avoid beliefs like "I am a democrat", and say only "I am a seeker of accurate world-models, whatever those turn out to be".)

It seems analogously easier to seek effective internal architectures if you also keep non-epistemic parts of your identity small -- not "I am a person who enjoys nature", nor "I am someone who values mathematics" nor "I am a person who aims to become good at email" but only "I am a person who aims to be effective, whatever that turns out to entail (and who is willing to let much of my identity burn in the process)".

There are obviously hazards as well as upsides that come with this; still, the upsides seem worth putting out there.

The two biggest exceptions I would personally make, which seem to mitigate the downsides: "I am a person who keeps promises" and "I am a person who is loyal to [small set of people] and who can be relied upon to cooperate more broadly -- whatever that turns out to entail".

Thoughts welcome.
[Link] Peer-Reviwed Piece on Meaning and Purpose in a Non-Religious Setting
by Gleb_Tsipursky — 2016-03-31T22:59:09.740Z

My peer-reviewed article in a psychology journal on the topic of meaning and purpose in a non-religious setting is now accessible without a paywall for a limited time, so get it while it's free if you're interested. I'd be interested in hearing your feedback on it. For those curious, the article is not directly related to my Intentional Insights project, but is a part of my aspiration to raise the sanity waterline regarding religion, the focus of Eliezer's original piece on the sanity waterline.
What makes buying insurance rational?
by rayalez — 2016-03-31T20:50:43.977Z

Hey, everyone! So I've been reading an article about the expected utility, apparently to figure out whether the risk is worth taking you multiply expected value of the outcome by it's probability.

And apparently insurance companies can make money because the expected utility of buying insurance is lower than it's price.

So why would buying insurance be the rational action? I mean intuitively it makes sense(you want to avoid the risk), but it doesn't seem to fit well with this idea. If insurance is almost by definition is worth slightly less than it's price, how is it worth buying?

(sorry if it's a dumb question)
True answers from AI
by Stuart_Armstrong — 2016-03-31T15:00:35.000Z

A putative new idea for AI control; index here.

This post supersedes previous posts (such as this, this, and, partially, this).

This post explains how we can get honest answers from an AI about the expectation of certain utility u, probabilities of specific events, and conditional expectations.

The next post will partially solve one of the main problems with using the approach in general, the problem of very low probabilities.

The setup

Let u be any bounded utility function; by rescaling, we can assume it's bounded between 0 and 1.

Assume we have a smart AI that's boxed (as in the case of reduced impact AI, the boxing is also to protect the AI from us observing it). It has a single output channel, on which it can emit a message M.

A random process E (erasure) will erase M before any human can observe it. The probability of erasure, P(E), is low (thus P(¬E), non-erasure, is high). Assume that the box and the setup (including the AI's internal setup) is such that if E happens, humans cannot figure out what M was.

The AI will be given a utility u# to maximise (ignore the other box with Y coming from it, for the moment).

Honest expectation estimator

Suppose we want to figure out what the expectation of u is. Let Q be this expectation, as estimated by the AI (and presumably unknown to us). Since u is bounded by 0 and 1, so must be Q as well.

Let vE be the indicator function for E: so vE=1 if E happens, and vE=0 if ¬E does.

The AI's message M gets to contain a single number q, bounded between 0 and 1 (any other message is taken to be 0.5, say), and the AI is motivated to maximise:

u#=f(q,u)vE,

where f(q,u)=−q2+2qu.

First of all, note that every term in u# is multiplied by vE. This means the AI will get utility only in worlds where E happens. Thus it will work on the assumption that its message M is never read by any humans (and thus that the contents of this message cannot affect human behaviour - in particular, it has no reason to lie or manipulate us through this message, as such manipulations only succeed in worlds where the AI gets 0 utility anyway).

If we assume that P(E) is independent of u and q, then the AI's expected utility is proportional to

−q2+2qQ.

But the AI can choose q! Differentiating this expression with respect to q gives −2q+2Q, which is zero iff q=Q. Differentiating a second time gives −2, showing that q=Q is a maximum.

Therefore, in the worlds wher

...
Ultimate List of Irrational Nonsense
by AlwaysUnite — 2016-03-30T20:25:31.776Z

A couple of years back I stumbled across this diagram on reddit. Since then I've kept a list of all illogical arguments, quackery, irrational ideas and similar nonsense, to pass the time. The count as of today stands at 1229 irrational ideas (a few could be debated perhaps). Hopefully you'll have a laugh or two! Any additions let me know!

Link to the list:

https://docs.google.com/spreadsheets/d/1T3HQ6TnCg6Q44WzpJPFxLvfqbfvKmT5RqvpGI3ji1y8/edit?usp=sharing

And the original diagram:
Lesswrong 2016 Survey
by Elo — 2016-03-30T18:17:13.552Z

It’s time for a new survey!

Take the survey now

The details of the last survey can be found here. And the results can be found here.

I posted a few weeks back asking for suggestions for questions to include on the survey. As much as we’d like to include more of them, we all know what happens when we have too many questions. The following graph is from the last survey.

http://i.imgur.com/KFTn2Bt.png

(Source: JD’s analysis of 2014 survey data)

Two factors seem to predict if a question will get an answer:

The position

Whether people want to answer it. (Obviously)

People answer fewer questions as we approach the end. They also skip tricky questions. The least answered question on the last survey was - “what is your favourite lw post, provide a link”. Which I assume was mostly skipped for the amount of effort required either in generating a favourite or in finding a link to it. The second most skipped questions were the digit-ratio questions which require more work, (get out a ruler and measure) compared to the others. This is unsurprising.

This year’s survey is almost the same size as the last one (though just a wee bit smaller). Preliminary estimates suggest you should put aside 25 minutes to take the survey, however you can pause at any time and come back to the survey when you have more time. If you’re interested in helping process the survey data please speak up either in a comment or a PM.

We’re focusing this year particularly on getting a glimpse of the size and shape of the LessWrong diaspora. With that in mind; if possible - please make sure that your friends (who might be less connected but still hang around in associated circles) get a chance to see that the survey exists; and if you’re up to it - encourage them to fill out a copy of the survey.

The survey is hosted and managed by the team at FortForecast, you’ll be hearing more from them soon. The survey can be accessed through http://lesswrong.com/2016survey.

Survey responses are anonymous in that you’re not asked for your name. At the end we plan to do an opt-in public dump of the data. Before publication the row order will be scrambled, datestamps, IP addresses and any other non-survey question information will be stripped, and certain questions which are marked private su...
Meetup : Moscow meetup: inspection paradox, common dual process theory fallacies, preparing summer outreach programs
by Slava Matyukhin — 2016-03-30T18:13:56.142Z

Discussion article for the meetup : Moscow meetup: inspection paradox, common dual process theory fallacies, preparing summer outreach programs

WHEN:
03 April 2016 02:00:00PM (+0300)

WHERE:
ул. Большая Дорогомиловская, д.5к2

Note: most our members join meetups via other channels. Still, the correlation between "found out about Moscow meetups via lesswrong.com" and "is a great fit for our community" is very high. So we're posting just a short link to the hackpad document with the schedule here instead of the full translation of the announcement into english.

Pad with the details about 03.04.2016 meetup.

We're meeting at anticafe "Kocherga", as usual.

Discussion article for the meetup : Moscow meetup: inspection paradox, common dual process theory fallacies, preparing summer outreach programs
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2016-03-30T14:58:57.480Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
03 April 2016 03:30:00PM (-0400)

WHERE:
Reynolds Center

We will be meeting to hang out, play games, and engage in fun conversation.

Upcoming meetups:

Apr. 10: Cherry Blossoms

Apr. 17: What If

Apr. 24: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Does Evidence Have To Be Certain?
by Ronny Fernandez — 2016-03-30T10:32:39.632Z

It seems like in order to go from P(H) to P(H|E) you have to become certain that E. Am I wrong about that?
Say you have the following joint distribution:
P(H&E) = a
P(~H&E) = b
P(H&~E) = c

P(~H&~E) = d

Where a,b,c, and d, are each larger than 0.

So P(H|E) = a/(a+b). It seems like what we're doing is going from assigning ~E some positive probability to assigning it a 0 probability. Is there another way to think about it? Is there something special about evidential statements that justifies changing their probabilities without having updated on something else?
Meetup : San Jose Meetup: Park Day
by steven0461 — 2016-03-30T04:39:09.532Z

Discussion article for the meetup : San Jose Meetup: Park Day

WHEN:
03 April 2016 02:00:00PM (-0700)

WHERE:
438 Coleman Ave, San Jose, CA 95110

We thought we'd take advantage of the good weather and have what will hopefully be the first meetup in a series. (There will be at least two people attending.) You're invited to bring whatever you'd like to see at Park Day - balls, frisbees, food, etc.
We'll carry a green blanket so you can recognize us. (Or my CFAR bag, if I can find it. If you want to be sure, please check back here later.) The idea is to gather outside near the listed address at the park entrance first, but probably walk elsewhere soon. If you can't find us or there's something else, please call me (Steven) at 510-6844565.

Discussion article for the meetup : San Jose Meetup: Park Day