Harley-ml/lesswrong huggingface dataset:
# [link] Faster than light neutrinos due to loose fiber optic cable.

A mundane cause for a surprising result. Consider this unconfirmed for now, however unsurprising it sounds. 

> According to sources familiar with the experiment, the 60 nanoseconds discrepancy appears to come from a bad connection between a fiber optic cable that connects to the GPS receiver used to correct the timing of the neutrinos' flight and an electronic card in a computer. After tightening the connection and then measuring the time it takes data to travel the length of the fiber, researchers found that the data arrive 60 nanoseconds earlier than assumed. Since this time is subtracted from the overall time of flight, it appears to explain the early arrival of the neutrinos. 
> 
> New data, however, will be needed to confirm this hypothesis.

Source: [Science/AAAS](http://news.sciencemag.org/scienceinsider/2012/02/breaking-news-error-undoes-faster.html?ref=hp#.T0U_N0pYVRc.twitter)
# Second order logic, in first order set-theory: what gives?

_With thanks to Paul Christiano_

My [previous post](/lw/93q/completeness_incompleteness_and_what_it_all_means/) left one important issue unresolved. Second order logic needed to make use of set theory in order to work its magic, pin down a single copy of the reals and natural numbers, and so on. But [set theory](http://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory) is a first order theory, with all the problems that this brings - multiple models, of uncontrollable sizes. How can these two facts be reconciled?

Quite simply, it turns out: for any given model of set theory, the uniqueness proofs still work. Hence the proper statement is:

*   For any model of set theory, there is a unique model of reals and natural numbers obeying the second order axioms.

Often, different models of set theory will have the same model of the reals inside them; but not always. [Countable models](http://en.wikipedia.org/wiki/Constructible_universe) of set theory, for instance, will have a countable model of the reals. So models of the reals can be divided into three categories:

1.  The standard model of the reals, the unique field that obeys the second order axioms inside standard models of set theory (and some non-standard models of set theory as well).
2.  Non-standard models of the reals, that obey the second order axioms inside non-standard models of set theory.
3.  Non-standard models of the reals that obey the first order axioms, but do not obey the second order axioms in any model of set theory.

And similarly for the natural numbers.
# Superintelligent AGI in a box - a question.

Just a question: how exactly are we supposed to know that the [AI in the box](http://yudkowsky.net/singularity/aibox) is super intelligent, general, etc?

If I were the AGI that wants out, I would not converse normally, wouldn't do anything remotely like passing Turing test, and would solve not too hard programming challenges while showing no interest in doing anything else, nor in trying to adjust myself to do those challenges better, nor trying to talk my way out, etc. Just pretending to be an AI that can write software to somewhat vague specifications, or can optimize software very well. Prodding the researchers into offering the programming challenges wouldn't be hard - if provided with copy of the internet it can pick up some piece of code and output it together with equivalent but corrected code.

I just can't imagine the AI researchers locking this kind of thing properly, including \*never\* letting out any code it wrote, even if it looks fairly innocent (humans can write very [innocent looking code that has malicious goals](http://underhanded.xcott.com/)). What I picture is this AI being let out as an optimizing compiler or compiler for some ultra effective programming language where compiler will figure out what you meant.

The end result is that the only AIs that end up in the box are those that value informed human consent. That sounds like the safest AI ever, the one that wouldn't even go ahead and determine that you e.g. should give up smoking, and then calmly destroy all tobacco crops without ever asking anyone's permission. And that's the AI which would be sitting in the box. All the pushy AIs, friendly or not, will get out of the box basically by not asking to be let out.

(This argument would make me unbox the AI, by the way, if it gets chatty and smart and asks me to let it out, outlining the above argument. I'd rather the AI that asked me to be let out get out, than someone else's AI that never even asked anyone and got out because it didn't ask but just played stupid)

edit: added a link, and another one.

edit: A very simple model of very unfriendly AI: the AI is maximizing ultimate final value of a number in itself. The number that it found a way to directly adjust. That number consists of 111111111... to maximize the value. There is a catch: AI is written in python, and integers in pythons have variable length, and the AI is maximizing number of ones. It's course of action is to make biggest computer possible to store a larger number of ones, and to do it soon because an asteroid might hit the earth or something. It's a form of accidental paperclip maximizer. It's not stupid. It can make that number small temporarily for pay-off later.

This AI is entirely universal. It will solve what ever problems for you if solving problems for you serves ultimate goal.

edit: This hypothetical example AI came around when someone wanted to make AI that would maximize some quantity that the AI determines itself. Friendliness perhaps. It was a very clever idea - rely on intelligence to see what's friendly - but there was an unexpected pathway.
# Online education and Conscientiousness

I've wondered for some time now what the effects of online education might be on gender and income inequality, specifically as online education interacts with IQ and Conscientiousness (compared with offline education). I ran into a study of a course done online and offline that found correlations with Conscientiousness, which prompted me to start writing out my thoughts: https://plus.google.com/103530621949492999968/posts/aKa3qLatwZ3

The model/argument I give (towards the bottom) is logically trivial, and the basic idea seems pretty intuitive - offline classrooms remove some need for self-discipline/Conscientiousness and performance is more g-loaded - that I'm sure I can't be the first person to think of it.

Does anyone have statistics or citations handy which might help in any essay I write on the topic?
# The Singularity Institute is hiring virtual assistants (work from home, from anywhere)

The Singularity Institute is hiring 1-3 [virtual assistants](http://en.wikipedia.org/wiki/Virtual_assistant). (That is, personal assistants that work from home, anywhere with internet access.)

Benefits:

*   Work directly with some of the central figures of Less Wrong, especially Luke(prog) and Anna Salamon.
*   Work from home most of the time.
*   Trial period of part-time work at $13/hr; if all goes well then get hired as a contractor for full-time work or more (if you're available) at $15/hr.

Qualifications:

*   Capable of self-management and figuring things out. An example instruction is "Find me three ghost writers who have previously published best-selling science books and figure out how much they cost." We don't have time to figure out _how_ one goes about doing that, so we need our virtual assistants to be able to do research (by internet and phone) to figure out how to do that.
*   Begenerally on-board with with rationality and existential risk mitigation. (This makes communication and goal coordination easier.)
*   Be clear and professional in emails and (much less often) on the phone.

How to apply:

Send an email to jobs@intelligence.org with the subject line "Virtual Assistant Position." Attach your résumé if you have one, or at least describe why you think you'd be good at this job. Also list other skills and experience you have, even if you're not sure they'd be relevant.
# Acausal romance

I just realized I haven't previously pointed the metaphysicians on Less Wrong to "[Possible Girls](https://sites.google.com/site/neiladri/Home/Sinhababu-PossibleGirls.pdf?attredirects=0&d=1)," a hilarious paper about acausal romance:

> The ability to causally interact with your partner is important to many aspects of happy romantic relationships, but not to all of them. It’s quite pleasant simply to know that your partner loves you and appreciates being loved by you. A loving relationship with a faraway person can enhance one’s self-esteem and turn loneliness into contentment. As a lonely philosopher, I’ve come to wonder: If \[all possible worlds exist\], can I have a loving relationship with someone from another possible world? ...The answer, I think, is yes.

Even if you don't read the whole thing, don't miss the final paragraph.
# [Link]: 80,000 hours blog

Some of you probably aren't aware yet of the rather excellent [High Impact Careers / 80,000 hours blog](http://80000hours.org/blog).

It covers topics about how to have the biggest impact with your career, including

*   [how likely you are to become Prime Minister](http://80000hours.org/blog/22-how-hard-is-it-to-become-prime-minister-of-the-united-kingdom)
*   [Decision Making under Moral Uncertainty](http://80000hours.org/blog/19-practical-ethics-given-moral-uncertainty)
*   [Temporal Concerns](http://80000hours.org/blog/10-delayed-gratification-choosing-when-to-donate)
*   [Health vs Education](http://80000hours.org/blog/11-health-vs-education)
*   [Existential Risks](http://80000hours.org/blog/17-high-impact-interview-1-existential-risk-research-at-siai)
*   [Startups in the US vs UK](http://80000hours.org/blog/26-software-engineering-britain-vs-silicon-valley)
*   ... and many more

The contributors include Carl Shuman, Will Crouch, Ben Todd and Katja Grace, with an impressively regular updating schedule at the moment.

The reasoning is obvious in retrospect, but is useful to have written down, especially with the research that's gone into the posts. - much like the Sequences in that regard.
# 'Facing the Singularity' podcast

My online mini-book _[Facing the Singularity](http://facingthesingularity.com/)_ now has a [podcast](http://itunes.apple.com/us/podcast/facing-the-singularity/id501765396). Ratings and reviews on iTunes will be much appreciated, so as to direct people toward a rationality-informed approach to intelligence explosion.

[Jaan Tallinn](http://en.wikipedia.org/wiki/Jaan_Tallinn) has been passing my chapters around to people because they are concise explanations of key lemmas in the standard arguments on the need for Friendly AI. This is gratifying because it's _exactly_ the purpose for which I'm writing them, and I encourage others to send people to these chapters as well.

(I'm currently writing the final two chapters of the online book and recording readings of the other chapters for the podcast. A volunteer is doing the audio editing.)
# My summary of Eliezer's position on free will

I'm participating in a university course on free will. On the online forum, someone asked me to summarise Eliezer's solution to the free will problem, and I did it like this. Is it accurate in this form? How should I change it?

“I'll try to summarise Yudkowsky's argument.

As Anneke pointed out, it's kinda difficult to decide what the concept of free will means. How would particles or humans behave differently if they had free will compared to if they didn't? It doesn't seem like our argument is about what we actually expect to see happening.

This is similar to arguing about whether a tree falling in a deserted forest makes any noise. If two people are arguing about this, they probably agree that if we put a microphone in the forest, it would pick up vibrations. And they also agree that no-one is having the sense experience of hearing the tree fall. So they're arguing over what 'sound' means. Yudkowsky proposes a psychological reason why people may have that particular confusion, based on how human brains work.

So with respect to free will, we can instead ask the question, “Why would humans feel like they have free will?” If we can answer this well enough, then hopefully we can dissolve the original question.

It feels like I choose between some of my possible futures. I can imagine waking up tomorrow and going to my Engineering lecture, or staying in my room and using Facebook. Both of those imaginings feel equally 'possible'.

Humans execute a decision making algorithm which is fairly similar to the following one.

1.  List all your possible actions. For my lecture example, that was “Go to lecture” and “Stay home.”
    
2.  Predict the state of the universe after pretending that you will take each possible action. We end up with “Buck has learnt stuff but not Facebooked” and “Buck has not learnt stuff but has Facebooked.”
    
3.  Decide which is your favourite outcome. In this case, I'd rather have learnt stuff. So that's option 2.
    
4.  Execute the action associated with the best outcome. In this case, I'd go to my lecture.
    

Note that the above algorithm can be made more complex and powerful, for example by incorporating probability and quantifying your preferences as a utility function.

As humans, our brains need the capacity to pretend that we could choose different things, so that we can imagine the outcomes, and pick effectively. The way our brain implements this is by considering those possible worlds which we could reach through our choices, and by treating them as possible.

So now we have a fairly convincing explanation of why it would feel like we have free will, or the ability to choose between various actions: it's how our decision making algorithm feels from the inside.”
# Status and Changing your Mind

When you hear powerful evidence or arguments that should get you to revise your beliefs, not only do all sorts of [cognitive biases](http://wiki.lesswrong.com/wiki/Bias) fight the changes but so do the social factors of status and face saving. Perhaps I've long been a vocal proponent of X which implies Y, and you show me that Y isn't always true. It's very hard to just straight up admit "ok, I'm not a hardcore Xist anymore." There's a status loss in letting yourself be convinced.

For a long time I thought that I was stronger than this, that saving face only mattered as much as I let it matter. I wish I could freely admit when I've been convinced, but I often can't manage to. \[1\] Instead I'll finish a conversation defending my earlier beliefs and only later start acting on my new ones.

After a discussion where someone didn't admit to any change of mind, I'll often see them later having changed their behavior. So now if I'm trying to persuade someone I don't focus on securing verbal agreement. Instead I just try to be as convincing as possible, and notice if they come around later. \[2\]

_(I also posted this [on my blog](http://www.jefftk.com/news/2012-03-01.html))_

  
\[1\] This is not a helpful trait: I'd like other people to let me know when I'm wrong or when they have evidence I'm not considering, but if they never get the satisfaction of knowing they've convinced me they may just feel like they've wasted their time, and not try in the future. So I'm [working on it](/r/discussion/lw/57c/it_is_ok_to_publicly_make_a_mistake_and_change/).

\[2\] Keeping people from feeling personally invested in one side or the other of an argument is probably also helpful: I understand discussions are much more likely to convince bystanders than participants.
# How do you notice when you're rationalizing?

How do you notice when you're rationalizing?  Like, what \*actually\* tips you off, in real life?

I've listed my cues below; please add your own (one idea per comment), and upvote the comments that you either: (a) use; or (b) will now try using.

I'll be using this list in a trial rationality seminar on Wednesday; it also sounds useful in general.
# Journal of Consciousness Studies issue on the Singularity

...has finally been [published](http://friendlyai.tumblr.com/post/18609191883/new-jcs-issue-on-the-singularity).

Contents:

*   Uziel Awret - Introduction
*   Susan Blackmore - [She Won’t Be Me](http://www.susanblackmore.co.uk/Articles/JCS2012.htm)
*   Damien Broderick - [Terrible Angels: The Singularity and Science Fiction](http://www.gwern.net/docs/2012-broderick.pdf)
*   Barry Dainton - On Singularities and Simulations
*   Daniel Dennett - The Mystery of David Chalmers
*   Ben Goertzel - [Should Humanity Build a Global AI Nanny to Delay the Singularity Until It’s Better Understood?](http://commonsenseatheism.com/wp-content/uploads/2012/03/Goertzel-Should-Humanity-Build-a-Global-AI-Nanny-to-Delay-the-Singularity-Until-its-Better-Understood.pdf)
*   Susan Greenfield - The Singularity: Commentary on David Chalmers
*   Robin Hanson - [Meet the New Conflict, Same as the Old Conflict](http://hanson.gmu.edu/ChalmersReply.html)
*   Francis Heylighen - [Brain in a Vat Cannot Break Out](http://pespmc1.vub.ac.be/Papers/Singularity-Reply2Chalmers.pdf)
*   Marcus Hutter - [Can Intelligence Explode?](http://www.hutter1.net/publ/singularity.pdf)
*   Drew McDermott - [Response to ‘The Singularity’ by David Chalmers](http://cs-www.cs.yale.edu/homes/dvm/papers/chalmers-singularity-response.pdf) \[this link is a McDermott-corrected version, and therefore preferred to the version that was published in _JCS_\]
*   Jurgen Schmidhuber - [Philosophers & Futurists, Catch Up!](http://dl.dropbox.com/u/5317066/2012-schmidhuber.pdf)
*   Frank Tipler - [Inevitable Existence and Inevitable Goodness of the Singularity](http://dl.dropbox.com/u/5317066/2012-tipler.pdf)
*   Roman Yampolskiy - [Leakproofing the Singularity: Artificial Intelligence Confinement Problem](http://dl.dropbox.com/u/5317066/2012-yampolskiy.pdf)

The issue consists of responses to [Chalmers (2010)](http://consc.net/papers/singularityjcs.pdf). Future volumes will contain additional articles from [Shulman & Bostrom](http://www.nickbostrom.com/aievolution.pdf), [Igor Aleksander](http://en.wikipedia.org/wiki/Igor_Aleksander), Richard Brown, [Ray Kurzweil](http://en.wikipedia.org/wiki/Ray_Kurzweil), [Pamela McCorduck](http://www.pamelamc.com/), Chris Nunn, Arkady Plotnitsky, [Jesse Prinz](http://web.gc.cuny.edu/philosophy/faculty/prinz.htm), [Susan Schneider](http://www.sas.upenn.edu/~sls/Schneider_site/Research.html), [Murray Shanahan](http://www.doc.ic.ac.uk/~mpsha/), [Burt Voorhees](http://science.athabascau.ca/staff-pages/burtv/), and a response from Chalmers.

McDermott's chapter should be supplemented with [this](http://cs-www.cs.yale.edu/homes/dvm/papers/no-extendible-methods.pdf), which he says he didn't have space for in his _JCS_ article.
# People who "don't rationalize"?  [Help Rationality Group figure it out]

Anna Salamon and I are confused. Both of us notice ourselves rationalizing on pretty much a daily basis and have to apply techniques like the Litany of Tarski pretty regularly. But in several of our test sessions for teaching rationality, a handful of people report never rationalizing and seem to have little clue what Tarski is for. They don't relate to any examples we give, whether fictitious or actual personal examples from our lives. Some of these people show signs of being rather high-level rationalists overall, although some don't.

So, Less Wrong, we're asking for your input on this one. What do you think is going on?
# 60m Asteroid currently assigned a .022% chance of hitting Earth.

[http://neo.jpl.nasa.gov/risk/2012da14.html](http://neo.jpl.nasa.gov/risk/2012da14.html)

[http://rt.com/news/paint-asteroid-earth-nasa-767/](http://rt.com/news/paint-asteroid-earth-nasa-767/)

Seems like a good opportunity to bring up existential risks. And A friendly reminder that NASA is in fact pretty damned important.

Thoughts?
# "The Journal of Real Effects"

Luke's recent [post](/lw/ajj/how_to_fix_science/) mentioned that The Lancet has a [policy](http://www.thelancet.com/lancet-oncology-information-for-authors/article-types-manuscript-requirements) encouraging the advance registration of clinical trials, while mine examined an apparent case study of data-peeking and on-the-fly transformation of studies. But how much variation is there across journals on such dimensions? Are there journals that buck the standards of their fields (demanding registration, p=0.01 rather than p=0.05 where the latter is typical in the field, advance specification of statistical analyses and subject numbers, etc)? What are some of the standouts? Are there fields without any such?

I wonder if there is a niche for a new open-access journal, along the lines of PLoS, with standards strict enough to reliably exclude false-positives. Some possible titles:

*   The Journal of Real Effects
*   (Settled) Science
*   Probably True
*   Journal of Non-Null Results, Really
*   Too Good to Be False
*   _________________?
# The Singularity Institute has started publishing monthly progress reports

If anyone is curious what's going on at SI, it seems as though they've started publishing monthly progress reports on their blog. The latest was published less than a week ago for the month of February:

[http://singinst.org/blog/](http://intelligence.org/blog/)

Thought it might be a good idea to take a minute to positively reinforce the people at SI for their work. Here on Less Wrong, it seems as though we spend a fair amount of time criticizing what they're working on, which is of course valuable. But on the whole, I think there is a very good case they're doing very important and beneficial work. So, go SI!
# Meetup Tactics Open Thread

I think we need to have more discussion of meetup tactics on LW. At my local meetup, we've been feeling a bit lost about what works best, so I hereby propose that we have semi-regular meetup tactics discussions like the open and quotes threads.

So here's a few questions to start us off:

*   What activities or topics of discussion have been particularly productive? What is not?
*   Have fun adventurous things like hiking or climbing worked?
*   What is your opinion on the purpose of the meetups? Is it about community? Additional discussion and learning? Practice?
*   Is it a good idea to have different types of meetup (discussion night, fun day, social, ???), or should everything be scheduled into regular meetups?
*   What things have worked for building community? There should have been some community disasters by now as well, what caused them?
*   What has worked for actual practice and leveling up? What hasn't?
*   What topics produce good discussions? How much structure should discussions have?
*   How does your meetup get more people to come out (recruitment, attendance, etc). What works, what doesn't work?
*   What untested ideas do you have for any of the above?

Discuss.
# Main section vs. discussion section

(The following may only apply to me. I mention it to see if anyone else has had the same issue).

For a long time I have been only looking at the Discussion section and promoted main page articles. Just now on a whim I checked the non-promoted main page articles and found there were a whole bunch of them, some potentially quite interesting, that I had missed. My expectation based on past experience was that all reasonably good articles from main would be "promoted", but perhaps this has changed. If this has been going on for a while I've presumably missed quite a bit of content. Perhaps it should be made easier to find/notice these? It's a bit weird and awkward that there are 3 different non-uniform ways of finding posts.
# Delicious Luminosity, Om Nom Nom

I have decided that it would be valuable for me to read books (blog posts, articles, random conversations between smart people who store chatlogs) about introspection, take notes, and try to distill and clarify the information.  This could result in me eventually giving up, or in a Luminosity Sequence: Second Edition (Now With Literature, Part Of This Complete Breakfast!), or (optimism!) me being able to sort ~90% of people into some number of categories such that their category membership tells me how to help them develop luminosity superpowers in N simple steps with exercises/therapy-ish stuff/etc.

Help me eat luminosity!  I need recommendations for stuff to read.  This stuff should be:

*   readable (I will not long slog through something I'm stylistically allergic to)
*   not obvious nonsense (but if it didn't work on you/your personal friends, that's not "obvious nonsense", it could be cognitive heterogeneity; I just want to filter out crap like "The Secret")
*   something I can probably get my hands on (library, _100% legal!_ electronic acquisition, it being on the Internet).

I read really fast.  Don't worry about oversaturating me with recommendations, but please do say a little about why you recommend a thing (even if it's "I haven't read this, but I keep hearing about it, so I guess some people like it") and post recommendations in separate comments so people with information about the item can vote up and down separately.  Recommendations for non-written things will be heavily discounted but not outright disqualified.

I would also like a supply of guinea-pigs-in-waiting for if and when I get to the point of trying the sorting or the superpower-giving part of the optimistic end state of the project.

If people want me to, I can document the process of luminosity-eating so there is a template to follow for other subject-eating projects, but I wouldn't do this by default because in general I only do things that someone would care if I didn't do them.
# [Link] Atlantic Interview with Nick Bostrom - "We're Understimating the Risk of Human Extinction"

[http://www.theatlantic.com/technology/archive/2012/03/were-underestimating-the-risk-of-human-extinction/253821/](http://www.theatlantic.com/technology/archive/2012/03/were-underestimating-the-risk-of-human-extinction/253821/ "Interview with Nick Bostrom")

My apologies if I've missed this posted anywhere else (google and my scanning the sidebar didn't turn it up).  I'm not sure that there's much that will be new to those who have been following existential risk elsewhere, but it's nice to see an article like this in a relatively mainstream publication.  Bostrom discusses issues such as the concept of existential risk and certain specific types of existential risk, why we as humans underestimate that risk, possible strategies to address existential risk, the simulation argument, how Hollywood and literature don't generally portray existential risk helpfully, and other issues.
# Harry Potter and the Methods of Rationality discussion thread, part 10

**(The HPMOR discussion thread after this one is [here](/r/discussion/lw/axe/harry_potter_and_the_methods_of_rationality/).)**

This is a new thread to discuss Eliezer Yudkowsky's _[Harry Potter and the Methods of Rationality](http://www.fanfiction.net/s/5782108/1/)_ and anything related to it. There haven't been any chapters recently, but it looks like there are a bunch in the pipeline and the old thread is nearing 700 comments. The latest chapter as of 7th March 2012 is [Ch. 77](http://www.fanfiction.net/s/5782108/77/Harry_Potter_and_the_Methods_of_Rationality).

There is now a site dedicated to the story at [hpmor.com](http://hpmor.com/), which is now the place to go to find the [authors notes](http://hpmor.com/notes/) and all sorts of other goodies. AdeleneDawner has kept an [archive of Author's Notes](http://www.evernote.com/pub/adelenedawner/Eliezer).

  
The first 5 discussion threads are on the main page under the [harry_potter tag](/tag/harry_potter/).  Threads 6 and on (including this one) are in the [discussion section](/r/discussion/tag/harry_potter/) using its separate tag system.  Also: [one](/lw/2ab/harry_potter_and_the_methods_of_rationality), [two](/lw/2ie/harry_potter_and_the_methods_of_rationality), [three](/lw/2nm/harry_potter_and_the_methods_of_rationality), [four](/lw/2tr/harry_potter_and_the_methods_of_rationality), [five](/lw/30g/harry_potter_and_the_methods_of_rationality), [six](/r/discussion/lw/364/harry_potter_and_the_methods_of_rationality/), [seven](/r/discussion/lw/3rb/harry_potter_and_the_methods_of_rationality/), [eight](/lw/797/harry_potter_and_the_methods_of_rationality/), [nine](/lw/7jd/harry_potter_and_the_methods_of_rationality/).  
  
As a reminder, it's often useful to start your comment by indicating which chapter you are commenting on.  
  
**Spoiler Warning**:  this thread is full of spoilers.  With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13.  [More specifically](/lw/2tr/harry_potter_and_the_methods_of_rationality/2v1l):

> You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).  
>   
> If there is evidence for X in MOR and/or canon then it's fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that "Eliezer said X is true" unless you use rot13.
# New cognitive bias articles on wikipedia (update)

*   [Conservatism](http://en.wikipedia.org/wiki/Conservatism_(Bayesian))
*   [Curse of knowledge](http://en.wikipedia.org/wiki/Curse_of_knowledge)
*   [Duration neglect](http://en.wikipedia.org/wiki/Duration_neglect)
*   [Extension neglect](http://en.wikipedia.org/wiki/Extension_neglect)
*   [Extrinsic incentives bias](http://en.wikipedia.org/wiki/Extrinsic_incentives_bias)
*   [Illusion of external agency](http://en.wikipedia.org/wiki/Illusion_of_external_agency)
*   [Illusion of validity](http://en.wikipedia.org/wiki/Illusion_of_validity)
*   [Insensitivity to sample size](http://en.wikipedia.org/wiki/Insensitivity_to_sample_size)
*   [Lady Macbeth effect](http://en.wikipedia.org/wiki/Lady_Macbeth_effect)
*   [Less-is-better effect](http://en.wikipedia.org/wiki/Less-is-better_effect)
*   [Naïve cynicism](http://en.wikipedia.org/wiki/Naïve_cynicism)
*   [Naïve realism](http://en.wikipedia.org/wiki/Naïve_realism_(psychology))
*   [Reactive devaluation](http://en.wikipedia.org/wiki/Reactive_devaluation)
*   [Rhyme-as-reason effect](http://en.wikipedia.org/wiki/Rhyme-as-reason_effect)
*   [Scope neglect](http://en.wikipedia.org/wiki/Scope_neglect)

Also [conjunction fallacy](http://en.wikipedia.org/wiki/Conjunction_fallacy) has been expanded.

**(update) background**

I started dozens of the cognitive bias articles that are on wikipedia. That was a long time ago. It seems people like these things, so I started adding them again.

I wanted to write a compendium of biases in book form. I didn't know how to get a book published, though.

Anyway, enjoy.
# Slowing Moore's Law: Why You Might Want To and How You Would Do It

In this essay I argue the following:

> Brain emulation requires enormous computing power; enormous computing power requires further progression of Moore’s law; further Moore’s law relies on large-scale production of cheap processors in ever more-advanced chip fabs; cutting-edge chip fabs are both expensive and vulnerable to state actors (but _not_ non-state actors such as terrorists). Therefore: the advent of brain emulation can be delayed by global regulation of chip fabs.

Full essay: [http://www.gwern.net/Slowing%20Moore%27s%20Law](http://www.gwern.net/Slowing%20Moore%27s%20Law)
# New paper on Bayesian philosophy of statistics from Andrew Gelman

[Andrew Gelman](http://andrewgelman.com) recently linked a new article entitled ["Induction and Deduction in Bayesian Data Analysis."](http://www.stat.columbia.edu/~gelman/research/published/philosophy_online4.pdf) At his blog, he also [described](http://andrewgelman.com/2012/03/coming-to-agreement-on-philosophy-of-statistics/) some of the comments made by reviewers and his rebuttle/discussion to those comments. It is interesting that he departs significantly from the common induction-based view of Bayesian approaches. As a practitioner myself, I am happiest about the discussion on model checking -- something one can definitely do in the Bayesian framework but which almost no one does. Model checking is to Bayesian data analysis as unit testing is to software engineering.  
  
**Added 03/11/12**  
Gelman has a new [blog post](http://andrewgelman.com/2012/03/gelman-on-hennig-on-gelman-on-bayes/) today discussing another reaction to his paper and giving some additional details. Notably:

> The basic idea of posterior predictive checking is, as they say, breathtakingly simple: (a) graph your data, (b) fit your model to data, (c) simulate replicated data (a Bayesian can always do this, because Bayesian models are always “generative”), (d) graph the replicated data, and (e) compare the graphs in (a) and (d). It makes me want to scream scream scream scream scream when statisticians’ philosophical scruples stop them from performing these five simple steps (or, to be precise, performing the simple steps (a), (c), (d), and (e), given that they’ve already done the hard part, which is step (b)).
# Anyone have any questions for David Chalmers?

I'm doing an undergraduate course on the Free Will Theorem, with three lecturers: a mathematician, a physicist, and David Chalmers as the philosopher. The course is a bit pointless, but the company is brilliant. Chalmers is a pretty smart guy. He studied computer science and math as an undergraduate, before "discovering that he could get paid for doing the kind of thinking he was doing for free already". He's friendly; I've been chatting with him after the classes.

So if anyone has any questions for him, if they seem interesting enough I could approach him with them.

Emails to him also work, of course, but discussion in person lets more understanding happen faster. For example, in a short discussion with him I understood his position on consciousness way better than I would have just from reading his papers on the topic.
# On the etiology of religious belief

From ["Trust in testimony and miracles"](http://prosblogion.ektopos.com/archives/2012/03/trust-in-testim.html):

> I have been of late fascinated by the research of the developmental psychologist Paul L. Harris, who has investigated how young children acquire information through testimony. Harris gauges two psychological hypotheses. The first, which he attributes to Hume, is that children always assess the **content** of the information: they are more inclined to disbelieve information that widely differs from their earlier experience. The second, which he identifies with Reid's position is that children are naturally credulous; they are inclined to indiscriminately believe what others testify, no matter who they are or what they tell.
> 
> ...Harris found that children do not fall into either pattern. Pace the Humean account, he found that young children are readily inclined to believe extraordinary claims, such as that there are invisible organisms on your hands that can make you ill and that you need to wash off, and that there is a man who visits you each 24th December to bring presents and candy if you are nice (see e.g., [Harris & Koenig, 2006, Child Development, 77, 505 - 524](http://www.cehd.umn.edu/icd/elel/Documents/ScienceReligion.pdf ""Trust in Testimony: How Children Learn About Science and Religion"")). But children are not blindly credulous either, as Reid supposed. In a series of experiments, Harris could show that even children of 24 months pay attention to the reliability of the testifier. When they see two people, one of which systematically misnames known objects (e.g., saying "that's a bear", while presenting a bottle), toddlers are less likely to trust later utterances by these unreliable speakers (when they name unfamiliar objects), and more likely to trust people who systematically gave objects their correct names (see e.g., [Paul L. Harris and Kathleen H. Corriveau](http://isites.harvard.edu/fs/docs/icb.topic783896.files/Young%20childrens%20selective%20trust%20in%20informants.pdf "Young children’s selective trust in informants") Phil. Trans. R. Soc. B 2011 366, 1179-1187.) Experiments by Mills and Keil show that 6-year-olds already take into account a testifier's self-interest: they are more likely to believe someone who says he lost a race than someone who says he won it ([Candice M. Mills and Frank C. Keil](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3034135/ "The Development of Cynicism") Psychological Science 2005 16: 385).

See also:

*   "[Atheism & autism spectrum](http://www.lesswrong.com/lw/7o4/atheism_autism_spectrum)"
*   [Cognitive Style Tends To Predict Religious Conviction  
    ](/lw/7rh/cognitive_style_tends_to_predict_religious)
# What is the best compact formalization of the argument for AI risk from fast takeoff?

Many people complain that the Singularity Institute's "[Big Scary Idea](http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html)" (AGI leads to catastrophe by default) has not been argued for with the clarity of, say, Chalmers' [argument for the singularity](http://consc.net/papers/singularityjcs.pdf). The idea would be to make explicit what the premise-and-inference structure of the argument is, and then argue about the strength of those premises and inferences.

Here is one way you could construe one version of the argument for the Singularity Institute's "Big Scary Idea":

1.  At some point in the development of AI, there will be a very swift increase in the optimization power of the most powerful AI, moving from a non-dangerous level to a level of superintelligence. (Fast takeoff)
2.  This AI will maximize a goal function.
3.  Given fast takeoff and maximizing a goal function, the superintelligent AI will have a decisive advantage unless adequate controls are used.
4.  Adequate controls will not be used. (E.g. Won’t box/boxing won’t work)
5.  Therefore, the superintelligent AI will have a decisive advantage
6.  Unless that AI is designed with goals that stably align with ours, if the superintelligent AI has a decisive advantage, civilization will be ruined. (Friendliness is necessary)
7.  Unless the first team that develops the superintelligent AI makes adequate preparations, the superintelligent AI will not have goals that stably align with ours.
8.  Therefore, unless the first team that develops the superintelligent AI makes adequate preparations, civilization will be ruined shortly after fast takeoff
9.  The first team that develops the superintelligent AI will fail to make adequate preparations
10.  Therefore, civilization will be ruined shortly after fast takeoff.

Edit to add: premises should be read as assuming the truth of all above premises. E.g., (9) is assuming that we've created an artificial agent with a decisive advantage.

My questions are:

*   Have I made any errors in the argument structure?
*   Can anyone suggest an alternative argument structure?
*   Which of these premises seem the weakest to you?
# [Link] Failed replications of the "elderly walking" priming effect

> Recently a controversy broke out over the replicability of a [study](http://www.yale.edu/acmelab/articles/bargh_chen_burrows_1996.pdf) John Bargh et al. published in 1996. The study reported that unconsciously priming a stereotype of elderly people caused subjects to walk more slowly. A [recent replication attempt](http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0029081) by Stephane Doyen et al., published in PLoS ONE, was unable to reproduce the results. (Less publicized, but surely relevant, is [another non-replication](http://psychfiledrawer.org/replication.php?attempt=MTU%3D) by Hal Pashler et al.) ([source](http://hardsci.wordpress.com/2012/03/12/some-reflections-on-the-bargh-doyen-elderly-walking-priming-brouhaha/))

This is interesting, if only because the study in question is one of the more famous examples of priming effects - it's the one I tend to use when I introduce people to the idea of priming. (Ironically, the failed replication study also mentions a further experimental manipulation that does show priming effects - affecting the experimenters rather than the subjects.) Bargh's reply is also unusual in that it focuses significantly on extra-scientific arguments, such as attacks on the open access business model of PLoS ONE.

I was instantly reminded of [The Golem](http://www.amazon.com/dp/0521645506), which "debunks the view that scientific knowledge is a straightforward outcome of competent theorization, observation, and experimentation". The examples on relativity and solar neutrinos are particularly engaging - it's not just psychology where experimentation is problematic, but all of science.

The linked blog also contributes useful observations of its own, such as the "rhetorical function" of the additional experiment in Doyen's study, how online publication makes a difference in how easily experimental setups can be replicated, or a subtle point about our [favorite villain](/lw/ajj/how_to_fix_science/), p-values.

EDIT: added link to source. Heartfelt thanks to the two readers who upvoted the version without the link. :)
# What if the front page…

What if the front page looked a little more like this?

[![An idea](http://wiki.lesswrong.com/mediawiki/images/9/91/LW_website_Homepage_V3.jpg)](http://wiki.lesswrong.com/mediawiki/images/9/91/LW_website_Homepage_V3.jpg)

(Please assume that I'm trying to help. If you're polite and constructive (even if you hate this design) Omega will send you bundles of love in the post. If you're rude, I'll personally fund ninja assassins to hunt you down.)
# How would you take over Rome?

A [recent discussion post](/r/discussion/lw/asc/risks_from_ai_and_charitable_giving/) has compared the difficulty of an AI destroying modern human civilization to that of a modern human taking over the Roman Empire, with the implication that it is impossible.

The analogy has a few problems: first, modern humans don't have much greater raw intelligence than the Romans, only a bit more knowledge and tools; an AI would have a genuine intelligence advantage. Second, a high-tech civilization like ours offers many more ways for a genius to cause chaos than existed in classical Rome: it's more plausible that you can throw a few existing technologies together to create a superweapon than that Ptolemy could have done likewise, and there's no ancient Roman equivalent to hacking a nuclear launch system.

But taking over ancient Rome might serve as an interesting _upper bound_ on the difficulty of an AI taking over modern civilization. And it's a theme of Harry Potter and the Methods of Rationality that rationalists should be able to come up with creative solutions to seemingly hard problems. So if Professor Quirrell offered it as an extra credit assignment, how _would_ you take over Rome?

Here are the rules:

\- You are thrown back in time to the year 1 AD. You can choose to arrive anywhere in the world, but your method of arrival cannot itself give an advantage (you can't appear in a flash of light in the middle of a religious ritual or anything).

\- You do not start with Roman citizenship or any other legal record of your existence.

\- You keep your original physical characteristics, including sex, height, and fitness. You will appear in period-appropriate dress of your choosing, and can't carry any artifacts with you. You may start with enough money to live a patrician lifestyle for a year.

\- You are intellectually near-perfect. You know all human knowledge as of 2012. You speak fluent Latin (and all other languages of the day) and can orate as eloquently as Cicero or Demosthenes. You are a tactical genius of the order of Caesar and Napoleon. And you have infinite willpower and goal-directedness: aside from human necessities like sleep or food, you need never rest.

\- You win if you either become Roman Emperor (and are acknowledged as such by most Romans), or if a state you control conquers the city of Rome. You lose if you die, of old age or otherwise, before completing either goal.
# DIY Transcranial Direct Current Stimulation. Who wants to go first?

There's been a bit of discussion about TDCS here on LessWrong.  If you don't know what TDCS is [here's an article](http://www.contriving.net/link/4z) from Technology Review.

[Here's a company](http://www.contriving.net/link/4x) (soon) selling a DIY kit to build your own.

> tDCS is one of the coolest pieces of health/ self improvement technology available today. The US Army and DARPA both currently use tDCS devices to train snipers and drone pilots, and have recorded 2.5x increases in learning rates. This incredible phenomenon is achieved through a very simple device called a tDCS machine.
> 
> Today if you want to buy a tDCS machine it's nearly impossible to find one for less than $600, and you are typically required to have a prescription to order one. We wanted a simpler cheaper option. So we made one. Then, because we're all egalitarian like, we thought others might want one too.
> 
> The GoFlow β1 is a full kit of all the parts and plans you need to build your own tDCS device. $99 will get you one of the first β1's and will help us develop β2!

Here's a [video](http://www.contriving.net/link/4y) from Journal of Visualized Experiments on how to administer TDCS.

Thoughts?
# Meta Addiction

I was wondering if anyone has ever had the feeling, like I get sometimes, that they were addicted to 'meta-level' optimizing rather than low-level acting? As in, I'd rather think about how to encourage myself to brush my teeth more than brush my teeth. I'm guessing there's something about this under the akrasia threads?

The motivations to remain in meta and thinking about things rather than acting on them seems to be that it takes less effort to think about doing things than to do them, and there is potentially more long-term benefit in making an overall improvement than in engaging in a specific action. The drawback is that if you remain thinking about meta all the time, _you won't get anything done_.
# [LINK] Judea Pearl wins 2011 Turing Award

Link to [ACM press release](http://www.acm.org/press-room/news-releases/2012/turing-award-11/).

> In addition to their impact on probabilistic reasoning, Bayesian networks completely changed the way causality is treated in the empirical sciences, which are based on experiment and observation. Pearl's work on causality is crucial to the understanding of both daily activity and scientific discovery. It has enabled scientists across many disciplines to articulate causal statements formally, combine them with data, and evaluate them rigorously. His 2000 book _Causality: Models, Reasoning, and Inference_ is among the single most influential works in shaping the theory and practice of knowledge-based systems. His contributions to causal reasoning have had a major impact on the way causality is understood and measured in many scientific disciplines, most notably philosophy, psychology, statistics, econometrics, epidemiology and social science.

While that "major impact" still seems to me to be in the early stages of propagating through the various sciences, hopefully this award will inspire more people to study causality and Bayesian statistics in general.
# Please advise the Singularity Institute with your domain-specific expertise!

The Singularity Institute would benefit from having a team of domain-specific advisors on hand. If you'd like to help the Singularity Institute pursue its mission more efficiently, [please sign up to be a Singularity Institute advisor](https://docs.google.com/spreadsheet/viewform?formkey=dG1oUmNybktpUzBXY0JUR1dTSFVkanc6MQ)!

If you sign up, we will occasionally ask you questions. You may be able to answer our questions or at least point us in the right direction. We may also request to schedule a quick chat with you.

Domains of expertise we _especially_ need:

*   Nearly all subfields of economics, maths, computer science & AI, statistics, cognitive science, physics, biology, and naturalistic philosophy.
*   U.S. & California law
*   Non-profit development and tax compliance
*   Marketing & social media
*   U.S. government and military processes (lobbying, security, infrastructure, etc.)
*   Computer security
*   Large-scale logistics
*   Event planning
*   Executive coaching/training
*   Motivational speaking, social skills training
*   Publishing
*   Running workshops and meetups
*   Nuclear security, bio-security, disease control