Harley-ml/lesswrong huggingface dataset:
# Discussion of Slate Star Codex: "Extremism in Thought Experiments is No Vice"

[Link to Blog Post: "Extremism in Thought Experiments is No Vice"](http://slatestarcodex.com/2015/03/26/high-energy-ethics/)  
  
\_\_\_\_\_

> Phil Robertson is being criticized for a thought experiment in which an atheist’s family is raped and murdered. On a talk show, he accused atheists of believing that there was no such thing as objective right or wrong, then continued:
> 
> > I’ll make a bet with you. Two guys break into an atheist’s home. He has a little atheist wife and two little atheist daughters. Two guys break into his home and tie him up in a chair and gag him.
> > 
> > Then they take his two daughters in front of him and rape both of them and then shoot them, and they take his wife and then decapitate her head off in front of him, and then they can look at him and say, ‘Isn’t it great that I don’t have to worry about being judged? Isn’t it great that there’s nothing wrong with this? There’s no right or wrong, now, is it dude?’
> > 
> > Then you take a sharp knife and take his manhood and hold it in front of him and say, ‘Wouldn’t it be something if \[there\] was something wrong with this? But you’re the one who says there is no God, there’s no right, there’s no wrong, so we’re just having fun. We’re sick in the head, have a nice day.’
> > 
> > If it happened to them, they probably would say, ‘Something about this just ain’t right’.
> 
> The media has completely proportionally described this as Robinson [“fantasizing about”](http://unvis.it/www.rawstory.com/rs/2015/03/duck-dynasty-star-fantasizes-about-atheist-familys-brutal-rape-and-murder-to-make-point-about-gods-law/) raping atheists, and there are the usual calls for him to apologize/get fired/be beheaded.
> 
> So let me use whatever credibility I have as a guy with a philosophy degree to confirm that Phil Robertson is doing moral philosophy exactly right.

\_\_\_\_\_  
  
This is a LW discussion post for [Yvain](/user/Yvain/)'s blog posts at [Slate Star Codex](http://slatestarcodex.com/), as per [tog's suggestion](/r/discussion/lw/lyn/slate_star_codex_alternative_comment_threads_on/):

> Like many Less Wrong readers, I greatly enjoy Slate Star Codex; there's a large overlap in readership. However, the comments there are far worse, not worth reading for me. I think this is in part due to the lack of LW-style up and downvotes. Have there ever been discussion threads about SSC posts here on LW? What do people think of the idea occasionally having them? Does Scott himself have any views on this, and would he be OK with it?

Scott/Yvain's permission to repost on LW was granted (from facebook):

> I'm fine with anyone who wants reposting things for comments on LW, except for posts where I specifically say otherwise or tag them with "things i will regret writing"
# What have we learned from meetups?

We've been running regular, well-attended Less Wrong meetups in London for a few years now, (and irregular, badly-attended ones for even longer than that). In this time, I'd like to think we've learned a few things about having good conversations, but there are probably plenty of areas where we could make gains. Given the number of Less Wrong meetups around the world, it's worth attempting some sort of meetup cross-pollination. It's possible that we've all been solving each other's problems. It's also good to have a central location to make observations and queries about topics of interest, and it's likely people have such observations and queries on this topic.

So, what have you learned from attending or running Less Wrong meetups? Here are a few questions to get the ball rolling:

*   What do you suppose are the dominant positive outcomes of your meetups?
*   What problems do you encounter with discussions involving \[x\] people? How have you attempted to remedy them?
*   Do you have any systems or procedures in place for making sure people are having the sorts of conversations they want to have?
*   Have you developed or consciously adopted any non-mainstream social norms, taboos or rituals? How are those working out?
*   How do Less Wrong meetups differ from other similar gatherings you've been involved with? Are there any special needs idiosyncratic to this demographic?
*   Are there any activities that you've found work particularly well or particularly poorly for meetups? Do you have examples of runaway successes or spectacular failures?
*   Are there any activities you'd _like_ to try, but haven't managed to pull off yet? What's stopping you?

If you have other specific questions you'd like answered, you're encouraged to ask them in comments. Any other observations, anecdotes or suggestions on this general topic are also welcome and encouraged.
# Thinking well

Many people want to know how to live well. Part of living well is thinking well, because if one thinks the wrong thoughts it is hard to do the right things to get the best ends.

We think a lot about how to think well, and one of the first things we thought about was how to not think well. Bad ways of thinking repeat in ways we can see coming, because we have looked at how people think and know more now about that than we used to.

But even if we know how other people think bad thoughts, that is not enough. We need to both accept that we can have bad ways of thinking and figure out how to have good ways of thinking instead.

The first is very hard on the heart, but is why we call this place "Less Wrong." If we had called it something like more right, it could have been about how we're more right than other people instead of more right than our past selves.

The second is very hard on the head. It is not just enough to study the bad ways of thinking and turn them around. There are many ways to be wrong, but only a few ways to be right. If you turn left all the way around, it will point right, but we want it to point up.

The heart of our approach has a few parts:

1.  We are okay with not knowing. Only once we know we don't know can we look. 
2.  We are okay with having been wrong. If we have wrong thoughts, the only way to have right thoughts is to let the wrong ones go. 
3.  We are quick to change our minds. We look at what is when we get the chance. 
4.  We are okay with the truth. Instead of trying to force it to be what we thought it was, we let it be what it is. 
5.  We talk with each other about the truth of everything. If one of us is wrong, we want the others to help them become less wrong. 
6.  We look at the world. We look at both the time before now and the time after now, because many ideas are only true if they agree with the time after now, and we can make changes to check those ideas. 
7.  We like when ideas are as simple as possible. 
8.  We make plans around being wrong. We look into the dark and ask what the world would look like if we were wrong, instead of just what the world would look like if we were right. 
9.  We understand that as we become less wrong, we see more things wrong. We try to fix all the wrong things, because as soon as we accept that something will always be wrong we can not move past that thing. 
10.  We try to be as close to the truth as possible. 
11.  We study as many things as we can. There is only one world, and to look at a part tells you a little about all the other parts. 
12.  We have a reason to do what we do. We do these things only because they help us, not because they are their own reason.
# Futarchy and Unfriendly AI

We have a reasonably clear sense of what "good" is, but it's not perfect. Suffering is bad, pleasure is good, more people living enjoyable lives is good, yes, but tradeoffs are hard. How much worse is it to go blind than to lose your leg? \[1\] How do we compare the death of someone at eighty to the death of someone at twelve? If you wanted to build some automated system that would go from data about the world to a number representing how well it's doing, where you would prefer any world that scored higher to any world scoring lower, that would be very difficult.

Say, however, that you've built a metric that you think matches your values well and you put some powerful optimizer to work maximizing that metric. This optimizer might do many things you think are great, but it might be that the easiest ways to maximize the metric are the ones that pull it apart from your values. Perhaps after it's in place it turns out your metric included many things that only strongly correlated with what you cared about, where the correlation breaks down under maximization.

What confuses me is that the people who warn about this scenario with respect to [AI](http://wiki.lesswrong.com/wiki/Unfriendly_artificial_intelligence) are often the same people in favor of [futarchy](http://mason.gmu.edu/~rhanson/futarchy.html). They both involve trying to define your values and then setting an indifferent optimizer to work on them. If you think AI would be very dangerous but futarchy would be very good, why?

_I also posted this [on my blog](http://www.jefftk.com/p/futarchy-and-unfriendly-ai)._

  
\[1\] This is a question people working in public health try to answer with [Disability Weights](http://www.jefftk.com/p/disability-weights) for DALYs.
# Even when contrarians win, they lose: Jeff Hawkins

**Related**: [Even When Contrarians Win, They Lose](http://www.overcomingbias.com/2007/09/even-when-contr.html)

I had long thought that [Jeff Hawkins](https://en.wikipedia.org/wiki/Jeff_Hawkins) (and the Redwood Center, and Numentia) were pursuing an idea that didn't work, and were continuing to fail to give up for a prolonged period of time. I formed this belief because I had not heard of any impressive results or endorsements of their research. However, I recently read [an interview with Andrew Ng](http://www.forbes.com/sites/roberthof/2014/08/28/interview-inside-google-brain-founder-andrew-ngs-plans-to-transform-baidu/), a leading machine learning researcher, in which he credits Jeff Hawkins with publicizing the "one learning algorithm" hypothesis - the idea that most of the cognitive work of the brain is done by one algorithm. Ng says that, as a young researcher, this pushed him into areas that could lead to general AI. [He still believes](http://fusion.net/story/54583/the-case-against-killer-robots-from-a-guy-actually-building-ai/) that AGI is far though.

I found out about Hawkins' influence on Ng after reading [an old SL4 post by Eliezer](http://www.sl4.org/archive/0509/12394.html) and looking for further information about Jeff Hawkins. It seems that the "one learning algorithm" hypothesis was widely known in neuroscience, but not within AI until Hawkins' work. Based on Eliezer's citation of Mountcastle and his known familiarity with cognitive science, it seems that he learned of this hypothesis independently of Hawkins. The "one learning algorithm" hypothesis is important in the context of intelligence explosion forecasting, since [hard takeoff is vastly more likely if it is true](http://www.overcomingbias.com/2011/07/debating-yudkowsky.html). I have been told that further evidence for this hypothesis has been found recently, but I don't know the details.

This all fits well with Robin Hanson's model. Hawkins had good evidence that better machine learning should be possible, but the particular approaches that he took didn't perform as well as less biologically-inspired ones, so he's not really recognized today. Deep learning would definitely have happened without him; there were already many people working in the field, and they started to attract attention because of improved performance due to a few tricks and better hardware. At least Ng's career though can be credited to Hawkins.

I've been thinking about Robin's hypothesis a lot recently, since many researchers in AI are starting to think about the impacts of their work (most still only think about the near-term societal impacts rather than thinking about superintelligence though). They recognize that this shift towards thinking about societal impacts is recent, but they have no idea why it is occurring. They know that many people, such as Elon Musk, have been outspoken about AI safety in the media recently, but few have heard of _Superintelligence_, or attribute the recent change to FHI or MIRI.
# I'd like advice from LW regarding migraines

So, I read a post a little while ago saying that asking the community for advice on personal problems was okay, and no one seemed to disagree strongly with this. Therefore, I'll just ask for some advice, and hope that I'm not accidentally going past some line. If I do, I apologise

 I have had migraines for quite a while now. They started when I was a child, but were infrequent in those days. They got progressively worse as time went on, and things started to get quite bad when I was about 12. A few years down the line, I would have headaches for months at a time, with migraines popping up for a few days a month. It got worse from there. Now, I have had migraine-like symptoms for 10 months now. I say migraine-like because part of the definition of a migraine is that it lasts from about 3 hours to a few days. According to a neurologist I recently went to, I have transformative migraines, or wording similar to that. So I have all the symptoms of migraines, except they last for inordinate amounts of time. I've had an MRI, and it showed nothing wrong with my brain. According to the World Health Organisation, this is more disabling than blindness, and as bad as acute psychosis: http://www.migrainetrust.org/chronic-migraine You can see why its rather important to me that I get rid of/deal with this.

Now, I've tried quite a lot of things over the years, especially in the last two or so. NSAIDs do very little, and thing like migraleve (paracetamol with codeine) are a little better. Sumatriptan provides some relief, but it doesn't get rid of the migraine. At best it will knock me down to a weak migraine. I've tried taking propanalol (160mg) for half a year, and it does little to help. I was prescribd Amitiptyline (10mg) a week ago, but it hasn't had much effect. I was told to increase by 10mg it every two weeks until I hit 30mg. I've also tried cutting things out like chocolates, and dairy for a month. It didn't have any effect. I also don't have any caffeine. So this eliminates some common causes of migraines. My migraines sometimes respond to heat/cold applied to my head, but this is only some of the time, due to my migraines shifting in nature. Further, it only takes the edge of them. I've also tried taking magnesium supplements, but they had a negative effect on me i.e. strange dreams and insomnia. That just made my problems worse. Also, I've ruled out medication overuse.

 So, does anyone have any recommendations? There should be a few people who have had experience with this level of migraines, and I expect they might be able to provide some advice. I'm not too optimistic, but I really need something that works.
# A pair of free information security tools I wrote

Information security is a pretty big passion of mine; I don't think someone needs to have "something to hide" in order to make use of digital signing, encryption, etc. Another passion of mine is making things easier for other people to do. I've written a couple of tools that I think can be useful for the LW crowd. 

[Online PGP Signature](http://www.2pih.com/pgp.html): This is an online javascript-based tool which allows you to sign messages using your PGP private key. I love the idea of PGP-signed messages (I remember someone under the pseudonym "Professor Quirrell" handing out PGP-verified Quirrell points a few years back). The problem is, I had yet to find an easy way to do this that didn't involve downloading command-line based software. So I wrote this tool that uses open-sourced, javascript-based PGP libraries to let you easily sign messages in your browser.

The whole thing is client-side so your private key is never seen by me, but be smart about security. If you don't trust me, that's fine, just don't use the tool. But also remember that you could have a virus, your computer could be monitored, someone could be watching over your shoulder, etc. so please be smart about your security. But hopefully this can be helpful. 

  
[Decoy: an iPhone App](http://www.decoyapp.com/): I wrote this after "The Fappening", where I was basically appalled at the terrible security practices that pretty much everyone uses when sending pictures back and forth. Decoy uses a combination of steganography and AES encryption to let you send images back and forth without having to sign up for an account or use some outside service that can be hacked or otherwise compromised. 

You take the original picture, then you come up with a passphrase, then you take a "decoy" picture. The original picture is converted to base64 image data, which is then AES-encrypted using your passphrase. The resulting cipher text is then encoded into the pixels of the "decoy" picture, which is what gets saved on your phone and sent out. The "decoy" pictures are indistinguishable from any other picture on your or your recipients' camera rolls, and unless you have the passphrase, then the original image is thoroughly inaccessible. 

If your phone is lost, hacked, stolen, or (more benignly) someone happens to be looking through pictures on your phone, all anyone will see are the "decoy" pictures. Without the password, those pictures are worthless. Although the app is primarily branded for, \*ahem\*, "personal use", there are plenty of other ways to use it. For example, my wife and I use it for things like sending pictures of sensitive physical documents like credit cards, birth certificates, social security cards, etc.  

(full disclosure: although Decoy is free, it is ad-supported so I do financially benefit from people using the app. But on the bright side I'm an avowed rationalist and if I make a quajillion dollars with this app I will spend the vast majority of it on LW-friendly causes!)
# Failing with abandon

This is a short public service announcement: *you don't have to fail with abandon.*

Say you're playing [Civilization](http://en.wikipedia.org/wiki/Civilization_%28video_game%29), and your target is to get to sleep before midnight, and you check the clock, and it's already 12:15. If that happens, you *don't* have to say "too late now, I already missed my target" and then keep playing until 4 in the morning.

Say you're trying to eat no more than 2000 calories per day, and then you eat 2300 by the end of dinner, you *don't* have to say "well I already missed my target, so I might as well indulge."

If your goal was to watch only one episode of that one TV show, and you've already watched three, you don't *have* to binge-watch the whole thing.

Over and over, I see people set themselves a target, miss it by a little, and then throw all restraint to the wind. "Well," they seem to think, "willpower has failed me; I might as well over-indulge." I call this pattern "failing with abandon."

But you don't have to fail with abandon. When you miss your targets, you're allowed to say "dang!" and then *continue trying to get as close to your target as you can.*

You don't have to say dang, either. You're allowed to over-indulge, if that's what you want to do. But for lots and lots of people, the idea of missing by *as little as possible* never seems to cross their mind. They miss their targets, and then suddenly they treat their targets as if they were external mandates set by some unjust authority; the jump on the opportunity to defy whatever autarch set an impossible target in the first place; and then (having already missed their target) they reliably fail with abandon.

So this is a public service announcement: you don't have to do that. When you miss your target, you can take a moment to remember who put the target there, and you can ask yourself whether you want to get as close to the target as possible. If you decide you only want to miss your target by a little bit, you still can.

You don't have to fail with abandon.
# One week left for CSER researcher applications

This is the last week to apply for one of four postdoctoral research positions at the [Centre for the Study of Existential Risk](http://cser.org). We are seeking researchers in disciplines including: economics, science and technology studies, science policy, arms control policy, expert elicitation and aggregation, conservation studies and philosophy.

The application requires a research proposal of no more than 1500 words from an individual with a relevant doctorate.

"We are looking for outstanding and highly-committed researchers, interested in working as part of growing research community, with research projects relevant to any aspect of the project. We invite applicants to explain their project to us, and to demonstrate their commitment to the study of extreme technological risks.

We have several shovel-ready projects for which we are looking for suitable postdoctoral researchers. These include:

1\. Ethics and evaluation of extreme technological risk (ETR) (with Sir Partha Dasgupta;

2\. Horizon-scanning and foresight for extreme technological risks (with Professor William Sutherland);

3\. Responsible innovation and extreme technological risk (with Dr Robert Doubleday and the Centre for Science and Policy).

However, recruitment will not necessarily be limited to these subprojects, and our main selection criterion is suitability of candidates and their proposed research projects to CSER’s broad aims.

More details are available [here](http://cser.org/vacancies/). Applications close on April 24th.

\- Sean OH and Ryan
# A simple exercise in rationality: rephrase an objective statement as subjective and explore the caveats

"This book is awful" => "I dislike this book" => "I dislike this book because it is shallow and is full of run-on sentences." => I dislike this book because I prefer reading books I find deep and clearly written."

"The sky is blue" => ... => "When I look at the sky, the visual sensation I get is very similar to when I look at a bunch of other objects I've been taught to associate with the color blue."

"Team X lost but deserved to win" => ...

"Being selfish is immoral" 

"The Universe is infinite, so anything imaginable happens somewhere"

In general, consider a quick check whether in a given context replacing "is" with "appears to be" leads to something you find non-trivial.

Why? Because it exposes the multiple levels of maps we normally skip. So one might find illuminating occasionally walking through the levels and making sure they are still connected as firmly as the last time. And maybe figuring out where the people who hold a different opinion from yours construct a different chain of maps. Also to make sure you don't mistake a map for the territory.

That is all. ( => "I think that I have said enough for one short post and adding more would lead to diminishing returns, though I could be wrong here, but I am too lazy to spend more time looking for links and quotes and better arguments without being sure that they would improve the post.")
# Replacing guilt

In my experience, many people are motivated primarily by either guilt, shame, or some combination of the two. Some are people who binge-watch television, feel deeply guilty about it, and convert that guilt into a burning need to Actually Do Something on the following day. Others are people who feel guilty whenever they stop working before they *literally fall over from exhaustion,* and in attempts to avoid that guilty feeling, they consistently work themselves weary.

I find that using guilt as a motivation source is both unhealthy and inefficient, but yet, I find it to be a common practice, especially among [effective altruists](http://www.effectivealtruism.org/).

Thus, in the coming series of posts, I'm going to explore a whole slew of tools for removing guilt-based motivation and replacing it with something that is both healthier and stronger.

My goal is to help people remove guilt-based motivation entirely, and replace it with intrinsic motivation. I'm aiming to both reduce the frequency of netflix binges *and* reduce the bad feelings that follow. I'm aiming to help people feel like they're still worthwhile human beings if they stop working before they literally drop. I'd like to help people avoid the failure mode where they feel guilty about something for days (even after learning their lesson), and I'm also hoping to remove some shame-based motivation while I'm in the area.

My first goal will be to address the guilt that comes from a feeling of *listlessness*, the vague feeling of guilt that one might get when they play video games all day, or when they turn desperately towards drugs or parties, in attempts to silence the part of themselves that whispers that there must be something else to life.

This sort of guilt cannot be removed by force of will, in most people. The trick to removing this sort of guilt, I think, is to start exploring that feeling that there must be something else to life, that there must be something more to do — and either find something worth working towards, or find that there really isn't actually anything missing. This first sort of listless guilt, I think, comes from someone who wants to find something else to do, and hasn't yet.

Unfortunately, addressing this sort of guilt isn't as easy as just finding a hobby. In my experience, this listless guilt tends to be found in people who have fallen into the nihilistic trap — people who either believe they can't matter, or who believe that no one can matter. It tends to be found in people who believe that humans only ever do what they want, that nothing is truly "better" than anything else, that there is no such thing as altruism, that "morality" is a pleasant lie — that class of beliefs is the class that I will address first, starting with the Allegory of the Stamp Collector.

I'll post the allegory tomorrow. In the interim, I invite you to devise your own tools for removing the listless guilt: the tools that people develop themselves are often more useful to them than the tools they are given.
# Mental representation and the is-ought distinction

I'm reading Thinking, Fast and Slow. In appendix B I came across the following comment. Emphasis mine:

> Studies of language comprehension indicate that people quickly recode much of what they hear into an abstract representation that no longer distinguishes whether the idea was expressed in an active or in a passive form and _no longer discriminates what was actually said from what was implied, presupposed, or implicated_ (Clark and Clark 1977).

My first thought on seeing this is: holy crap, this explains why people insist on seeing [relevance claims](/lw/k6t/arguments_and_relevance_claims/) in my statements that I didn't put there. If the brain doesn't distinguish statement from implicature, and my conversational partner believes that A implies B when I don't, then _of course_ I'm going to be continually running into situations where people model me as saying and believing B when I actually only said A. At a minimum this will happen any time I discuss any question of seemingly-morally-relevant fact with someone who hasn't trained themselves to make the is-ought distinction. Which is most people.

The next thought my brain jumped to: This process might explain the failure to make the is-ought distinction in the first place. That seems like much more of a leap, though. I looked up the [Clark and Clark cite](http://www.amazon.com/Psychology-Language-An-Introduction-Psycholinguistics/dp/0155728156). Unfortunately it's a fairly long book that I'm not entirely sure I want to wade through. Has anyone else read it? Can someone offer more details about exactly what findings Kahneman is referencing?
# Rational discussion of politics

In a [recent poll](/lw/m0g/open_thread_apr_06_apr_12_2015/c85x), many LW members expressed interest in a separate website for rational discussion of political topics. The website has been created, but we need a group of volunteers to help us test it and calibrate its recommendation system (see below).

If you would like to help (by participating in one or two discussions and giving us your feedback) please sign up [here](http://www.omnilibrium.com/register.php).

* * *

About individual recommendation system

All internet forums face a choice between freedom of speech and quality of debate. In absence of censorship, constructive discussions can be easily disrupted by [the inflow of the mind-killed](/lw/ly4/political_topics_attract_participants_inclined_to/) which causes the more intelligent participants [to leave](/lw/c1/wellkept_gardens_die_by_pacifism/) or [descend to the same level](/lw/gw/politics_is_the_mindkiller/).

Preserving quality thus usually requires at least one of the following methods:

1.  Appointing censors (a.k.a. moderators).
2.  Limiting membership.
3.  Declaring certain topics (e.g., politics) off limits.

On the new website, we are going to experiment with a different method. In brief, the idea is to use an automated recommendation system which sorts content, raising the best comments to the top and (optionally) hiding the worst. The sorting is done based on the individual preferences, allowing each user to avoid what he or she (rather than moderators or anyone else) defines as low quality content. In this way we should be able to enhance quality without imposing limits on free speech.

UPDATE. The discussions are scheduled to start on May 1.
# How to sign up for Alcor cryo

I wrote an article about the process of signing up for cryo since I couldn't find any such accounts online. If you have questions about the sign-up process, just ask.

A few months ago, I signed up for Alcor's brain-only cryopreservation. The entire process took me 11 weeks from the day I started till the day I received my medical bracelet (the thing that’ll let paramedics know that your dead body should be handled by Alcor). I paid them $90 for the application fee. From now on, every year I’ll pay $530 for Alcor membership fees, and also pay $275 for my separately purchased life insurance.

[http://specterdefied.blogspot.com/2015/04/how-to-sign-up-for-alcor-cryo.html](http://specterdefied.blogspot.com/2015/04/how-to-sign-up-for-alcor-cryo.html)
# Nick Bostrom's TED talk on Superintelligence is now online

[http://www.ted.com/talks/nick\_bostrom\_what\_happens\_when\_our\_computers\_get\_smarter\_than\_we_are](http://www.ted.com/talks/nick_bostrom_what_happens_when_our_computers_get_smarter_than_we_are)

> Artificial intelligence is getting smarter by leaps and bounds — within this century, research suggests, a computer AI could be as "smart" as a human being. And then, says Nick Bostrom, it will overtake us: "Machine intelligence is the last invention that humanity will ever need to make." A philosopher and technologist, Bostrom asks us to think hard about the world we're building right now, driven by thinking machines. Will our smart machines help to preserve humanity and our values — or will they have values of their own?

I realize this might go into a post in a media thread, rather than its own topic, but it seems big enough, and likely-to-prompt-discussion enough, to have its own thread.

I liked the talk, although it was less polished than TED talks often are. What was missing I think was any indication of how to solve the problem. He could be seen as just an ivory tower philosopher speculating on something that might be a problem one day, because apart from mentioning in the beginning that he works with mathematicians and IT guys, he really does not give an impression that this problem is already being actively worked on.
# CFAR-run MIRI Summer Fellows program: July 7-26

CFAR will be running a three week summer program this July for MIRI, designed to increase participants' ability to do technical research into the [superintelligence alignment problem](http://futureoflife.org/misc/open_letter).

The intent of the program is to boost participants as far as possible in four skills:

1.  The CFAR “applied rationality” skillset, including both what is taught at our intro [workshops](http://rationality.org/workshops/), and more advanced material from our alumni workshops;
2.  “Epistemic rationality as applied to the foundations of AI, and other philosophically tricky problems” -- i.e., the skillset taught in the [core](http://wiki.lesswrong.com/wiki/Sequences#Core_Sequences) LW Sequences.  (E.g.: reductionism; how to reason in contexts as confusing as anthropics without getting lost in words.)
3.  The long-term impacts of AI, and strategies for intervening (e.g., the content discussed in Nick Bostrom’s book [Superintelligence](http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom-ebook/dp/B00LOOCGB2/ref=dp_kinw_strp_1)).
4.  The basics of AI safety-relevant technical research.  (Decision theory, anthropics, and similar; with folks trying their hand at doing actual research, and reflecting also on the cognitive habits involved.)

The program will be offered free to invited participants, and partial or full scholarships for travel expenses will be offered to those with exceptional financial need.

If you're interested (or possibly-interested), sign up for an admissions interview ASAP at this link (takes 2 minutes): [http://rationality.org/miri-summer-fellows-2015/](http://rationality.org/miri-summer-fellows-2015/)

Also, please forward this post, or the page itself, to anyone you think should come; the skills and talent that humanity brings to bear on the superintelligence alignment problem may determine our skill at navigating it, and sharing this opportunity with good potential contributors may be a high-leverage way to increase that talent.
# Stupid Questions May 2015

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
# Calling all Nigerian rationalists and effective altruists

I'm in Lagos, Nigeria till the end of May and I'd like to hold a LessWrong/EA meetup while I'm here. If you'll ever be in the country in the future (or in the subcontinent), please get in touch so we can coordinate a meetup. I'd also appreciate being put in contact with any Nigerians who may not regularly read this list.

My e-mail address is oge@nnadi.org. I hope to hear from you.
# Saying "Everyone Is Biased" May Create Bias

It looks like telling people "everyone is biased" might make people [not want to change their behavior to overcome their biases](http://digest.bps.org.uk/2015/04/why-message-that-were-all-prone-to.html):

> In initial experiments, participants were simply asked to rate a particular group, such as women, on a series of stereotypical characteristics, which for women were: warm, family-oriented and (less) career-focused. Beforehand, half of the participants were told that "the vast majority of people have stereotypical preconceptions." Compared to those given no messages, these participants produced more stereotypical ratings, whether about women, older people or the obese.
> 
> Another experiment used a richer measure of stereotyping – the amount of clichés used by participants in their written account of an older person’s typical day. This time, those participants warned before writing that “Everyone Stereotypes” were more biased in their writings than those given no message; in contrast, those told that stereotyping was very rare were the least clichéd of all. Another experiment even showed that hearing the “Everyone Stereotypes” message led men to negotiate more aggressively with women, resulting in poorer outcomes for the women.

The authors suggest that telling participants that everyone is biased makes being biased seem like not much of a big deal. If everyone is doing it, then it's not wrong for me to do it as well. However, it looks like the solution to the problem presented here is to give a little white lie that will prompt people to overcome their biases:

> A further experiment suggests a possible solution. In line with the other studies, men given the "Everyone Stereotypes" message were less likely to hire a hypothetical female job candidate who was assertive in arguing for higher compensation. But other men told that everyone tries to overcome their stereotypes were fairer than those who received no information at all. The participants were adjusting their behaviour to fit the group norms, but this time in a virtuous direction.
# Guidelines for Upvoting and Downvoting?

I've only recently joined the LessWrong community, and I've been having a blast reading through posts and making the occasional comment.  So far, I've received a few karma points, and I’m pretty sure I’m more proud of them than of all the work I did in high school put together.

My question is simple, and aimed a little more towards the veterans of LessWrong:

What are the guidelines for upvoting and downvoting?  What makes a comment good, and what makes one bad?  Is there somewhere I can go to find this out (I've looked, but there doesn't seem to be a guide on LessWrong already up.  On the other hand, I lose my glasses while wearing them, so…)

Additionally, why do I sometimes see discussion posts with many comments but few upvotes, and others with many upvotes but few comments?  If a post is worth commenting on, isn't it worth upvoting?  I feel as though my map is missing a few pages here.

Not only would having a clear discussion of this help me review the comments of others better, it would also help me understand what I’m being reinforced for on each of my comments, so I can alter my behaviors accordingly.

I want to help keep this a well-kept garden, but I’m struggling to figure out how to trim the hedges.
# The File Drawer Effect and Conformity Bias (Election Edition)

As many of you may be aware, the UK general election took place yesterday, resulting in a surprising victory for the Conservative Party. The pre-election [opinion polls](http://en.wikipedia.org/wiki/United_Kingdom_general_election,_2015#Opinion_polling) predicted that the Conservatives and Labour would be roughly equal in terms of votes cast, with perhaps a small Conservative advantage leading to a [hung parliament](http://en.wikipedia.org/wiki/Hung_parliament); instead the Conservatives got 36.9% of the vote to Labour's 30.4%, and won the election outright.

There has already been a lot of discussion about why the polls were wrong, from methodological problems to incorrect adjustments. But perhaps more interesting is the possibility that the polls were right! For example, Survation did a poll on the evening before the election, which predicted the correct result (Conservatives 37%, Labour 31%). However, that [poll was never published](http://survation.com/snatching-defeat-from-the-jaws-of-victory/) because the results seemed "out of line." Survation didn't want to look silly by breaking with the herd, so they just kept quiet about their results. Naturally this makes me wonder about the existence of other unpublished polls with similar readings.

This seems to be a case of two well know problems colliding with devastating effect. Conformity bias caused Survation to ignore the data and go with what they "knew" to be the case (for which they have now paid dearly). And then the [file drawer effect](http://en.wikipedia.org/wiki/Publication_bias) meant that the generally available data was skewed, misleading third parties. The scientific thing to do is to publish all data, including "outliers," both so that information can change over time rather than be anchored, and to avoid artificially compressing the variance. Interestingly, the exit poll, which had a methodology agreed beforehand and was previously committed to be published, was basically right.

This is now the [third](http://en.wikipedia.org/wiki/United_Kingdom_general_election,_1970) [time](http://en.wikipedia.org/wiki/United_Kingdom_general_election,_1992) in living memory that opinion polls have been embarrassingly wrong about the UK general election. Each time this has lead to big changes in the polling industry. I would suggest that one important scientific improvement is for polling companies to announce the methodology of a poll and any adjustments to be made _before_ the poll takes place, and commit to publishing _all_ polls they carry out. Once this became the norm, data from any polling company that didn't follow this practice would be rightly seen as unreliable by comparison.
# Wild Moral Dilemmas

_\[CW: This post talks about personal experience of moral dilemmas. I can see how some people might be distressed by thinking about this.\]_

Have you ever had to decide between pushing a fat person onto some train tracks or letting five other people get hit by a train? Maybe you have a more exciting commute than I do, but for me it's just never come up.

In spite of this, I'm unusually _prepared_ for a trolley problem, in a way I'm not prepared for, say, being offered a high-paying job at an unquantifiably-evil company. Similarly, if a friend asked me to lie to another friend about something important to them, I probably wouldn't carry out a utilitarian cost-benefit analysis. It seems that I'm happy to adopt consequentialist policy, but when it comes to personal quandaries where I have to decide for myself, I start asking myself about _what sort of person this decision makes me._ What's more, I'm not sure this is necessarily a bad heuristic in a social context.

It's also noteworthy (to me, at least) that I rarely experience moral dilemmas. They just don't happen all that often. I like to think I have a reasonably coherent moral framework, but do I really need one? Do I just lead a very morally-inert life? Or have abstruse thought experiments in moral philosophy equipped me with broader principles under which would-be moral dilemmas are resolved before they reach my conscious deliberation?

To make sure I'm not giving too much weight to my own experiences, I thought I'd put a few questions to a wider audience:

\- What kind of moral dilemmas do you actually encounter?

\- Do you have any thoughts on how much moral judgement you have to exercise in your daily life? Do you think this is a typical amount?

\- Do you have any examples of pedestrian moral dilemmas to which you've applied abstract moral reasoning? How did that work out?

\- Do you have any examples of personal moral dilemmas on a Trolley Problem scale that nonetheless happened?

The Username/password anonymous account is, as always, available.
# [link] Bayesian inference with probabilistic population codes

[_Bayesian inference with probabilistic population codes_ by Wei Ji Ma et al 2006](http://www.gatsby.ucl.ac.uk/~pel/papers/ppc-06.pdf)

> Recent psychophysical experiments indicate that humans perform near-optimal Bayesian inference in a wide variety of tasks, ranging from cue integration to decision making to motor control. This implies that neurons both represent probability distributions and combine those distributions according to a close approximation to Bayes’ rule. At first sight, it would seem that the high variability in the responses of cortical neurons would make it difficult to implement such optimal statistical inference in cortical circuits. We argue that, in fact, this variability implies that populations of neurons automatically represent probability distributions over the stimulus, a type of code we call probabilistic population codes. Moreover, we demonstrate that the Poissonlike variability observed in cortex reduces a broad class of Bayesian inference to simple linear combinations of populations of neural activity. These results hold for arbitrary probability distributions over the stimulus, for tuning curves of arbitrary shape and for realistic neuronal variability.

Note that "humans perform near-optimal Bayesian inference" refers to the integration of information - not conscious symbolic reasoning. Nonetheless I think this is of interest here.
# We Should Introduce Ourselves Differently

I told an intelligent, well-educated friend about Less Wrong, so she googled, and got "Less Wrong is an online community for people who want to _apply_ the discovery of biases like [the conjunction fallacy](http://lesswrong.com/lw/ji/conjunction_fallacy/), [the affect heuristic](http://lesswrong.com/lw/lg/the_affect_heuristic/), and [scope insensitivity](http://lesswrong.com/lw/hw/scope_insensitivity/) in order to fix their own thinking." and gave up immediately because she'd never heard of the biases.

While hers might not be the best possible attitude, I can't see that we win anything by driving people away with obscure language.  
  
Possible improved introduction: "Less Wrong is a community for people who would like to think more clearly in order to improve their own and other people's lives, and to make major disasters less likely."
# The Best Popular Books on Every Subject

I enjoy reading popular-level books on a wide variety of subjects, and I love getting new book recommendations. In the spirit of lukeprog's [The Best Textbooks on Every Subject](/lw/3gu/the_best_textbooks_on_every_subject/), can we put together a list of the best popular books on every subject?

Here's what I mean by popular-level books:

*   Written very well and clearly, preferably even entertaining.
*   Does not require the reader to write anything (e.g., practice problems) or do anything beyond just reading and thinking, except perhaps on very rare occasions.
*   Cannot be "heavy" reading that requires the reader to proceed slowly and carefully and/or do lots of heavy thinking.
*   Can be understood by anyone with a decent high school education (_not _including calculus). However, sometimes this requirement can be circumvented, if the following additional criteria are met:
    *   There must be other books on this list that cover all the prerequisite information.
    *   When you suggest the book, list any prerequisites.
    *   There shouldn't be more than 2 or 3 prerequisites.

Textbooks are actually ok, as long as they meet all the above criteria.

  

I'm going to start off by also requiring the following, as per lukeprog. But if people prefer I might relax these:

1.  Post the title of your favorite book on a given subject.
2.  You must have read at least two other books on that same subject.
3.  You must briefly name the other books you've read on the subject and explain why you think your chosen book is superior to them.

**ETA:** If you really liked a book but didn't read alternatives, an easy way to fulfill these extra requirements is to look at reviews at Amazon, Goodreads, or similar. Usually you can find reviews that compare the book to one or more alternatives.

  

Finally, the purpose of this list is to try to be as comprehensive as possible. Copying books that have already been recommended on other lists is therefore to be encouraged, even if you yourself haven't read those books.

  

**ETA 2:** It seems everybody has their own ideas about what should be the criteria for this list. So how about everybody just add in books using whatever criteria they would prefer for a list of "The Best Popular Books on Every Subject".