Harley-ml/lesswrong huggingface dataset:
# Khan Academy: Introduction to programming and computer science

[Khan Academy](http://www.khanacademy.org/) now also features a [Computer Science](http://www.khanacademy.org/#computer-science) category. There are not many lessons yet but about 3 new videos are being added each day. They are going to add CS [exercises](http://www.khanacademy.org/exercisedashboard) soon too.

If you don't want to wait for the exercises, there is always the incredible [Project Euler](http://projecteuler.net/index.php?section=problems) that you can use to hone your math and programming skills.
# Is it cool if I post responses/thoughts as I read through the sequences?

I have poked around here on and off but someone recently led me back to the site. After taking the long break, I am ready to jump back into the sequences but have a favor to ask. Is it okay if I post my thoughts on each as I read them in the comments for that post? I don't know what impact that would have on any feeds, threads or whatnot. Things have changed a lot since I was last active.

To be extremely clear, these aren't going to be inherently _nice_. They won't be inherent un-nice either but it helps me to process something by responding to it critically. This means challenging it; squeezing it; banging my head against it. I will get a lot of it wrong. My questions may not make sense. I could start mucking up the place with untrained thoughts.

So, being [wary of fools](/lw/c1/wellkept_gardens_die_by_pacifism/), I give you a chance to just nip it all in the bud if you guys have moved on from the Old Posts. I left off somewhere around [Fake Morality](/lw/ky/fake_morality/) so... I still have a lot of work ahead of me. If it would help to post a few and _then_ get feedback, that works too.
# Volunteers needed to work on LessWrong's public goods problem

I want to figure out how to solve LessWrong's [public goods](http://en.wikipedia.org/wiki/Public_good) problem. There are lots of projects that lots of people are moderately interested in but not interested enough to do themselves. For example:

*   Further contributions to the [Science of Winning at Life](http://wiki.lesswrong.com/wiki/The_Science_of_Winning_at_Life) sequence: cognitive science of learning, cognitive science of memory, ergonomics, behavioral conditioning, fashion, productivity and no doubt many others
*   Explicit guides to applying LW material to your life, smaller sequences and exercises
*   Introductory material that gets people hooked, [MoR](http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality) is great, but surely there should be other avenues
*   User friendly guides to existing LW material
*   Meetup materials: exercises, slides, games, lectures etc.
*   Book reviews on important topics ([this](http://www.amazon.com/Selfish-Reasons-Have-More-Kids/dp/046501867X) book needs critique summary/review)
*   The [cognitive bias Wikipedia entries](http://en.wikipedia.org/wiki/List_of_cognitive_biases) need serious cleanup 
*   Developing exercises and tests that help understand or improve rationality ([this](http://calibratedprobabilityassessment.org/) website could use much better questions and interface)
*   Financial analyses, like [BrandonReinhart's analysis of SIAI](/lw/5il/siai_an_examination/), of The Cryonics Institute and Alcor 

Right now the only way those projects get done there's one person with enough intrinsic interest in the project that they do it mostly regardless of the benefits to others. We need ways to focus dispersed interest into intense interest in people willing and able to do particular projects ([different ways that might be accomplished](/lw/5zd/making_projects_happen/)). 

I think figuring out how to do that is probably best done by a small team, but I am not sure who else might be interested.

I need two other people enthusiastic about solving LessWrong's public goods problem to be part of that team. What would be involved? We'd meet to discuss what things are most likely to ameliorate the public goods problem and the best way to execute them, do research on what works elsewhere and experiment with ideas. For example, I've been slowly experimenting with contests in the hopes that money, prestige and competition will be good motivators. I would like to do more contests, but I could really use help coming up with good test contest ideas, coming up with a good contest formats, and judging contests. 

Any volunteers?

**Update**: We've started a [mailing list](http://groups.google.com/group/lw-public-goods-team) for this project.
# Gender and Libido

[There was a historical shift in beliefs.](http://noseriouslywhatabouttehmenz.wordpress.com/2011/06/22/on-libido-part-two/)

> But the one we’re concerned with is that women’s libidos went from being considered as powerful or more so than men’s to being essentially erased. Pre-Renaissance examples of horny ladies abound, from the Greeks onward; make your own list, but do include Chaucer. He’s such fun. This change in attitudes appears to have been religiously motivated, and based on the idea that women are more spiritual and sacred than men, meaning less horny. Again, make your own list of contemporary leftovers of this attitude; there are plenty.

> By the 18th century, it was taken as read that a woman who did experience (or at least express) sexual desire was suffering from a disorder. One important 1775 study of the subject linked the problem to “secret pollutions”, i.e. wanking, and (I swear I am not making this up) eating too much chocolate. I guess that’d go a ways toward explaining this advertisement. Women were diagnosed with, treated for, and often operated upon for “nymphomania”, the dread condition that causes a woman to want sex. (Talk to your doctor; you may suffer from it yourself!) And yes, by “operated upon”, I mean clitoridectomy. And yes, that’s fucking appalling.

I find this very odd. How could a major cultural lineage be wrong about something so much a part of ordinary experience?

When I say wrong, I don't necessarily mean that we're right, or the ancients were right, though there's a lot of evidence that the Victorians were wrong.

My favorite theory is that people's amount of desire for sex varies sufficiently that there's enough noise to make it easy to see patterns that aren't there. I leave the possibility open that there was a change (possibly dietary) which affected libido levels differently between men and women.

People are sufficiently punitive about sex that there's going to be lies and misdirection to support the current theory about how people are supposed to be.
# Hanson Debating Yudkowsky, Jun 2011

  

> On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private _Jane Street Capital_ event (crude audio [here](http://hanson.gmu.edu/ppt/JaneStreetDebate2011.WMA), from 4:45; better video [here](http://hanson.gmu.edu/ppt/JaneStreetDebate2011vid.wmv) \[as of July 14\]).
> 
> I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my [similar “win”](http://www.overcomingbias.com/2009/04/efficient-economists-pledge.html) over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians [usually](http://www.overcomingbias.com/2009/04/why-refuse-to-debate.html) “win” such debates.
> 
> Our topic was: _Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world_. He was pro, I was con. We also debated this subject here on _Overcoming Bias_ from [June](http://www.overcomingbias.com/2008/06) to [December](http://www.overcomingbias.com/2008/12) 2008\. Let me now try to summarize my current position.
> 
> \[...\]
> 
> It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

**Link:** [overcomingbias.com/2011/07/debating-yudkowsky.html](http://www.overcomingbias.com/2011/07/debating-yudkowsky.html)
# Has SIAI/FHI considered putting up prizes for contributions to important problems?

Because of my interest in reducing [LessWrong's public goods problem](/r/discussion/lw/6h5/volunteers_needed_to_work_on_lesswrongs_public/), I've been [reading about prizes](http://www.mckinsey.com/clientservice/Social_Sector/our_practices/Philanthropy/Knowledge_highlights/And_the_winner_is.aspx). Prizes seem like they could be very useful for shifting the perception of the scientific and mathematical public on the importance of existential risk.

Prizes have a couple of attractive properties

1.  Shift public perception about what is important and possible
2.  Draw submissions from non-traditional sources 
3.  Mobilize existing resources to work on your problem (e.g. people use grants to try to win prizes)

It seems to me that there are several open problems related to existential-risk that might be amenable to prizes. Three prizes seem immediately attractive:

1.  **Reflective Decision Theory  **\- This is an important unsolved problem, and the criteria seem like they could be fairly objective (a set of problems the decision theory must handle 'well', plus some judging). Of course, perhaps SIAI/FHI do not want reflective decision theory to be public; in that case a prize would be a bad idea. This prize should probably be called The Good Prize after I.J. Good.
2.  **Contributions towards a Theory of Friendliness** \- Shifting public opinion on this issue is very important, so I think a prize is attractive. The criteria for strong contributions are fuzzier, so you would need especially respected and trusted judges.
3.  **Contributions towards understanding existential risks** \- Same as above.

Is my enthusiasm for this approach unwarranted? Are there other related topics amenable to prizes? 

_Disclosure_: I have only recently read up on prizes and I think I am in fanboy mode, so take my excitement with a grain of salt.
# Recent site changes, Mon 4th July

We've pushed out some new changes, including a front page. Promoted articles are now available at [http://lesswrong.com/promoted/](/promoted/) and from the MAIN link in the top nav.

Most of the recent changes were requested [here](/r/discussion/lw/6b1/so_i_guess_the_site_redesign_is_live/) or [here](/r/discussion/lw/6c2/they_changed_it_now_it_sucks/).

This is the place to comment on the new changes… but please remember that:

1.  we're watching this thread, and if we've done anything really horrible, you or someone else will tell us, and we'll fix it; so
2.  be polite, and explain clearly what we need to do to make you happy.

Front page "Featured Articles" are edited [here](http://wiki.lesswrong.com/mediawiki/index.php?title=Template:Lesswrong:FeaturedArticles&action=edit). I humbly suggest that willing featured article editors nominate themselves in the comments, and everyone who doesn't get a lot of support forgets this link and leaves that page alone.

And if spam becomes a problem for the front page, the about page, or the comment markup help, poke [a wiki admin](http://wiki.lesswrong.com/mediawiki/index.php?title=Special:ListUsers&group=sysop) and ask them to "protect" [the special pages](http://wiki.lesswrong.com/mediawiki/index.php?title=Special%3ASearch&redirs=0&search=Lesswrong%3A&fulltext=Search&ns0=1&ns4=1&ns10=1&title=Special%3ASearch&advanced=1&fulltext=Advanced+search).
# Outline of possible Singularity scenarios (that are not completely disastrous)

Suppose we could look into the future of our Everett branch and pick out those sub-branches in which humanity and/or human/moral values have survived past the Singularity in some form. What would we see if we then went backwards in time and look at _how_ that happened? Here's an attempt to answer that question, or in other words to enumerate the not completely disastrous Singularity scenarios that seem to have non-negligible probability. Note that the question I'm asking here is distinct from "In what direction should we try to nudge the future?" (which I think logically ought to come second).

1.  Uploading first
    1.  Become superintelligent (self-modify or build FAI), then take over the world
    2.  Take over the world as a [superorganism](/r/discussion/lw/5jb/link_whole_brain_emulation_and_the_evolution_of/)
        1.  self-modify or build FAI at leisure
        2.  (Added) stasis
    3.  Competitive upload scenario
        1.  (Added) subsequent singleton formation
        2.  (Added) subsequent AGI intelligence explosion
        3.  no singleton
2.  IA (intelligence amplification) first
    1.  Clone a million von Neumanns (probably government project)
    2.  Gradual genetic enhancement of offspring (probably market-based)
    3.  Pharmaceutical
    4.  Direct brain/computer interface
    5.  What happens next? Upload or code?
3.  Code (de novo AI) first
    1.  Scale of project
        1.  International
        2.  National
        3.  Large Corporation
        4.  Small Organization
    2.  Secrecy - spectrum between
        1.  totally open
        2.  totally secret
    3.  Planned Friendliness vs "emergent" non-catastrophe
        1.  If planned, what approach?
            1.  "Normative" - define decision process and utility function manually
            2.  "Meta-ethical" - e.g., CEV
            3.  "Meta-philosophical" - program AI how to do philosophy
        2.  If emergent, why?
            1.  Objective morality
            2.  Convergent evolution of values
            3.  Acausal game theory
            4.  Standard game theory (e.g., Robin's idea that AIs in a competitive scenario will respect human property rights due to standard game theoretic considerations)
    4.  Competitive vs. local FOOM
4.  (Added) Simultaneous/complementary development of IA and AI

Sorry if this is too cryptic or compressed. I'm writing this mostly for my own future reference, but perhaps it could be expanded more if there is interest. And of course I'd welcome any scenarios that may be missing from this list.
# How to annoy misanthropes and bleeding-hearts

Related to [Not for the Sake of Selfishness Alone](/lw/6dz/not_for_the_sake_of_selfishness_alone), [Crime and Punishment](/lw/4x9/crime_and_punishment/), and [Separate morality from free will](/lw/4xu/separate_morality_from_free_will).

Here is a simple method for resolving some arguments about free will.  Not for resolving the _question_, mind you.  Just the arguments.

One group of people doesn't want to give people any credit for anything they do.  All good deeds are ultimately done for "selfish" reasons, where even having a goal of helping other people counts as selfish.  The quote from Lukeprog's recent article is a perfect example:

> No one deserves thanks from another about something he has done for him or goodness he has done. He is either willing to get a reward from God, therefore he wanted to serve himself. Or he wanted to get a reward from people, therefore he has done that to get profit for himself. Or to be mentioned and praised by people, therefore, it is also for himself. Or due to his mercy and tenderheartedness, so he has simply done that goodness to pacify these feelings and treat himself.

\- Mohammed Ibn Al-Jahm Al-Barmaki

Another group of people doesn't want to blame people for anything they do.  Criminals sometimes had criminal parents - crime was in their environment _and_ in their genes.  Or, to take a different variety of this attitude, cultural beliefs that seem horrible to us are always justifiable within their own cultural context.

The funny thing is that these are different groups.  Both assert that people should not be given credit, or else blame, for their actions, beyond the degree of free will that they had.  Yet you rarely find the same person who will not give people credit for their good deeds unwilling to blame them for their bad deeds, or vice-versa.

When you find yourself in an argument that appears to be about free will, but is really about credit or blame, ask the person to agree that the matter applies equally to good deeds and bad deeds - however they define those terms.  This may make them lose interest in the argument - because it no longer does what they want it to do.
# An explanation of Aumann's agreement theorem

I've written up a 2-page explanation and proof of Aumann's agreement theorem.  [Here is a direct link](https://www.dropbox.com/s/u04zqlacpcenqvd/Aumann_agreement_theorem.pdf) to the pdf via Dropbox.

The proof in Aumann's original paper is already very short and accessible.  (Wei Dai gave an exposition closely following Aumann's in [this post](/lw/1il/probability_space_aumann_agreement/).)  My intention here was to make the proof even more accessible by putting it in elementary Bayesian terms, stripping out the talk of meets and joins in partition posets.  (Just to be clear, the proof is just a reformulation of Aumann's and not in any way original.)

I will appreciate any suggestions for improvements.

* * *

**Update**:  I've added an abstract and made one of the conditions in the formal description of "common knowledge" explicit in the informal description.

**Update**:  [Here is a direct link](http://dl.dropbox.com/u/34639481/Aumann_agreement_theorem.pdf) to the pdf via Dropbox (ht to Vladimir Nesov).

**Update**:  [In this comment](/r/discussion/lw/6je/an_explanation_of_aumanns_agreement_theorem/4hc5), I explain why the definition of "common knowledge" in the write-up is the same as Aumann's.

**Update 2020-05-23**: I fixed the Dropbox link and removed the Scribd link.
# Experiment: Knox case debate with Rolf Nelson

Recently, on the main section of the site, [Raw_Power](/user/Raw_Power) posted an [article](/lw/6j1/find_yourself_a_worthy_opponent_a_chavruta/) suggesting that we find "worthy opponents" to help us avoid mistakes.

As you may recall, [Rolf Nelson disagrees with me about Amanda Knox](/lw/1pc/debunking_komponisto_on_amanda_knox_long/) \-\- rather sharply. Of course, the same can be said of lots of other people (if not so much here on Less Wrong). But Rolf isn't your average "guilter". Indeed, considering that he speaks fluent [Bayesian](http://wiki.lesswrong.com/wiki/Bayesian), is [one of the Singularity Institute's largest donors](http://intelligence.org/donors), and is also (as I understand it) signed up for [cryonics](http://wiki.lesswrong.com/wiki/Cryonics), it's hard to imagine an "opponent" more "worthy". The Amanda Knox case may not be in the same category of importance as many other issues where Rolf and I probably agree; but my opinion on it is very confident, and it's the opposite of his. If we're both aspiring rationalists, at least one of us is doing something wrong.  

As it turns out, Rolf is [interested in having a debate with me on the subject](/lw/6g0/making_an_amanda_knox_prediction_market/4gp9), to see if one of us can help to [change the other's mind](http://wiki.lesswrong.com/wiki/How_to_Actually_Change_Your_Mind). I'm setting this post up as an experiment, to see if LW can serve as a suitable venue for such an exercise. I hope it can: Less Wrong is almost unique in the extent to which the social norms governing discussion reflect and coincide with the requirements of personal epistemic rationality. ([For example:](http://yudkowsky.net/rational/virtues) "Do not believe you do others a favor if you accept their arguments; the favor is to you.") But I don't think we've yet tried an organized one-on-one debate -- so we'll see how it goes. If it proves too unwieldy or inappropriate for some other reason, we can always move to another venue.

Although the primary purpose of this post is a one-on-one debate between Rolf Nelson and myself, this is a LW Discussion post like any other, and it goes without saying that others are welcome and encouraged to comment. Just be aware that we, the main protagonists, will try to keep our discussion focused on each other's arguments. (Also, since our subject is an issue where there is already a strong LW consensus, one would prefer to avoid a sort of "gangup effect" where lots of people "pounce" on the person taking the contrarian position.)

With that, [here we go](/r/discussion/lw/6k7/experiment_knox_case_debate_with_rolf_nelson/4hfu)...
# The Sequences in MP3 Format

I can drive and listen, but I can't drive and read!  The same is true for most kinds of exercise.

If you are in my situation - wanting to read the sequences without having enough time - feel free to download [these audio files](http://www.yakitome.com/cgi-bin/output.py?filter=user&usrID=7153797555) for your smart phone or MP3 player.

My vision is to build a podcast feed or downloadable MP3 repository of all the major sequences.  The files I have now are not organized enough to scale out to hundreds of posts, and some of the artifacts of text-to-speech could be reduced with the right pre-processing.  Before I spend more time on this, I want the right tools and process in place. 

Any ideas on how to to proceed?  Would you like to help?  How should I publish these files?
# Thinking without words?

Before language, people must have thought without words.  I often have the impression that I have a thought fully-formed in my head, yet I wait to listen to it unfold in words before moving on to the next thought.  Perhaps I could think much faster if I weren't addicted to words.

Has anyone developed techniques for thinking without words?

This would have a little in common with Buddhist practices of emptying your mind, but wouldn't be the same thing.  For one thing, Buddhists also try to empty their minds of images.  More importantly, they are trying not to think, while I'm trying to think - just not unpack everything into words.
# Community norm question: brief text ad signatures

I've written several posts on this site, many of which were promoted to the front page. Some of my writings have been quite popular. Looking through the Google Analytics stats for this site showed that my writing has had at least 50 000 unique views over time. The full total is probably more, since I only looked at the stats of the 1000 most viewed Less Wrong pages.  
  
As I have been looking for sources of side income, I was wondering whether it'd be deemed acceptable if I started signing my posts to the main section with something brief like this:  
  
_To see the novel I'm writing, [click here](http://xuenay.livejournal.com/tag/city%20of%20light%20and%20fire). If you liked this post, you may also [Flattr me here](https://flattr.com/thing/197452/Xuenay-on-Flattr).  
_  
I'm currently thinking about trying to make a living on writing, and being able to do something like this would make it considerably easier to help build a personal brand. Although I wouldn't dare to claim to write as well as Eliezer or Yvain, say, I would like to think that my posts have been valuable for several people. The fact that sixteen of my articles were among the 1000 most viewed LW pages would support this. Being able to get back some of that value would only seem fair to me.  
  
As an additional bonus, currently my LW writing has gotten sidelined as it hasn't seemed that useful for my personal goals. Being allowed to have such ads would make writing on LW more personally useful for me, incentivizing me to spend more time on writing quality posts here.  
  
I can understand not everyone being fully enthusiastic about this, though. For one, several Internet communities are quite stringent about things that might be considered spam. Also, people might also be worried about the fact that LW is currently mostly operating as a gift economy. Letting people make money off their posts directly, such as with Flattr links, might change the community norms in an undesirable direction. Folks such as matt, who are hard at work at improving the technical side of the site, might rightfully feel that they deserve a cut.  
  
So I figured I'd better ask first. For what it's worth, I [have donated to SIAI](http://intelligence.org/donors) in the past, and do intend to donate to either SIAI or FHI or both in the future. Part of any income I'd make through this site would go to fund my donations. I'm also already spending some of my time writing academically about LW-related issues. You can e.g. expect to see a full-length paper draft based on my [previous conference paper](http://intelligence.org/upload/mostly-harmless.pdf) before the month is over. Extra income sources would also let me spend more time doing such work.
# Other Useful Sites LWers Read

There seems to be a lot of knowledge that LessWrongers have which is fairly awesome, but not particularly well-covered on Less Wrong.

What other sites do people recommend?

I originally came to LW through HN, and Paul Graham was somewhat influential on my early high school thoughts.

*   [http://news.ycombinator.com/](http://news.ycombinator.com/)
*   [http://www.paulgraham.com/articles.html](http://www.paulgraham.com/articles.html)

I also fairly regularly read

*   [http://www.ribbonfarm.com/](http://www.ribbonfarm.com/)
*   [http://www.sebastianmarshall.com/](http://www.sebastianmarshall.com/)

I've poked through these a bit on recommendation from other people, and they seem ridiculously useful.

*   [http://www.bulletproofexec.com/](http://www.bulletproofexec.com/)
*   [http://quantifiedself.com/](http://quantifiedself.com/)

There are a few blogs in the LW cluster, but I'm sure others read more of them than me.  
  
If you want to be super nice, could you link to some of your favorite posts/articles?
# Having Useful Conversations

Holding conversations in person is useful; feedback is quick, and it seems to be much easier to change your behavior as a result of actually talking with people. 

Having effective goal-oriented conversations is somewhat difficult. One source of difficulty is a strong tendency to stray from useful talk into entertaining talk. A typical example is the tendency of many (otherwise potentially productive) conversations between rationalists simply wandering into an extended dialog about the nature of existential risk or some interesting philosophical problem, and then stagnating there (potentially treading interesting new intelligence-demonstrating terrain, but not in point of fact getting anything done or refining beliefs in a meaningful way).

If this is what all participants want out of the conversation, then it's great that we've found a community where people can get their kicks in this particular abstruse way. If this is what some but not all participants want out of the conversation, then perhaps the conversation should divide or conclude. But conversations seem to get derailed--either for significant lengths of time, or indefinitely--even when participants honestly want to get things done, and view conversations with other rationalists as instruments to serve their values. 

In the interest of getting things done, I (and Nick Tarleton and Michael Curzi, with the tiniest bit of testing) suggest that the rationalist community try really hard to adopt the following norm: when someone else is talking, and the conversation would be significantly better served by them stopping, let them know. Either point out that the topic is nice to think about but unhelpful, that the topic should be considered later rather now, or whatever else the speaker seems to have failed to notice. To help make adoption a little easier, it might be help to choose one person in advance who will have some responsibility to arbitrate. 

If a participant disagrees about the relevance of a remark, don't push it--our hope is that such a system could help people who have honestly wandered from the topic pursuing an interesting tangent or happy thought, not to resolve any actual dispute. If a participant doesn't want to adhere closely to any particular notion of usefulness--for example, if someone is having a conversation to simply enjoy themselves and unwind--then the conversing parties should resolve their misunderstanding, or if not possible simply stop talking to each other and save some time. 

Have any LWers considered other lightweight measures to hold more useful conversations? There seems to be low-hanging fruit here, and there seems to be a lot to gain.
# ALCOR finances

Mike Darwin has posted some financial history of Alcor in a post, ["The Armories of the Latter Day Laputas, Part 5"](http://chronopause.com/index.php/2011/07/11/the-armories-of-the-latter-day-laputas-part-5/). While there were some errors (see [the comments](http://chronopause.com/index.php/2011/07/11/the-armories-of-the-latter-day-laputas-part-5/#comments)), Darwin has apparently corrected them.

It makes interesting reading in general. It's not a straight analysis of filings like Brandon's ["SIAI - An Examination"](/lw/5il/siai_an_examination/), but more of a financial history (eg. the [graph of revenues vs expenses](http://i293.photobucket.com/albums/mm55/mikedarwin1967/alcorgraph2.jpg) etc.)
# A funny argument for traditional morality

I just had a long conversation with my brother, a devout Christian. With my help he has outlined the following argument why it might be good for me to follow Christian [deontology](http://en.wikipedia.org/wiki/Deontological_ethics):

1.  Many of my moral values arose from my upbringing, as opposed to biology. This is evidenced by the fact that biologically similar people living in different places and epochs have different ideas of what's right.
2.  Therefore many of my values originally came from the society that raised me.
3.  Society's values were strongly influenced by Christian values, and many of our core moral prohibitions are inherited from Christian tradition.
4.  The world is full of people who may want to edit my values ever-so-slightly while I'm not looking, in order to further their own agenda.
5.  Also my values may drift, and most drift is harmful from the perspective of my current values.
6.  A good recipe for countering this insidious deterioration of values is to consciously pull them back toward their original source, as long as it's something unchanging, like a book.
7.  That means editing my values to more closely match Christianity. QED.

What do you think?
# Do not ask what rationalists should do

Recently there has been a couple of articles in the discussion page asking whether rationalists should do action A. Now such questions are not uninteresting, but by saying "rationalist" they are poorly phrased.

The rational decision at any time is the decision, given a human with a specific utility function B, and information C, should make to maximise B, given their knowledge (and knowledge about their knowledge) of C. It's not a decision a rationalist should make, it's a decision any human should make. If Omega popped into existence and carefully explained why action A is the best thing for this human to do given their function B, and their information C, then said human should agree.

The important question is not what a rationalist should do, but what your utility function and current information is. This is a more difficult question. Humans are often wrong about what they want in the long term, and it's questionable how much we should value happiness now over happiness in the future (in particular, I suspect current and future me might disagree on this point). Quantifying our current information is also rather hard- we are going to make bad probability estimates, if we can make them at all, which lead us into incorrect decisions just because we haven't considered the evidence carefully enough.

Why is this an important semantic difference? Well it's important for the cause of refining rationality that we don't get caught with associating the notion of rationality with certain goals. Some rationalists believe that they want to save the world, and the best way to do it is by creating friendly AI. This is because they have certain utility functions, and certain beliefs about the probabilities of the singularity. Not all rationalists have these utility functions. Some just want to have a happy home life, meet someone nice, and raise a family. These are different goals, and they can be helped by rationality, because rationality IS the art of winning. Being able to clearly state ones goals and work out the best way to acheieve them is useful pretty much no matter what those goals are. (pretty much to prevent silly examples here!)
# "A good volunteer is hard to find"

From the [GiveWell blog](http://blog.givewell.org/), which is often interesting & applicable to our interests, comes [a post on the quality of their volunteers](http://blog.givewell.org/2011/07/13/a-good-volunteer-is-hard-to-find/):

> "In our experience, valuable volunteers are rare. The people who email us about volunteer opportunities generally seem enthusiastic about GiveWell’s mission, and motivated by a shared belief in our goals to give up their free time to help us. Yet, the majority of these people never complete useful work for us.
> 
> We ask new volunteers to first complete a test assignment that takes about 2-4 hours. The assignment involves fixing the formatting of our list of sources on two practice pages and allows us to get a sense of their attention to detail and commitment to volunteer hours. Of the 34 people who emailed us expressing an interest in volunteering between September 2010 (when we started keeping track) and May 2011, only 7 have completed the test assignment and gone on to complete valuable work for us.
> 
> Of the 34, 10 never responded to my email outlining what GiveWell volunteers do and asking them if they’d like me to send the first assignment. 13 responded to this email and I sent them the first assignment, but they didn’t complete it. The final 4 completed the test assignment, but didn’t send back the next (real) assignment I sent.
> 
> It seems rather surprising that almost 80% of people who take the initiative to seek us out and ask for unpaid work fail to complete a single assignment. But maybe this shouldn’t be surprising. Writing an email is quick and exciting; spending a few hours fixing punctuation is not."

(The dropout rate is probably not due to the perceived low utility of the work - GiveWell seems to be up-front that the test assignment is a test.)

I draw a few lessons from this:

*   there is likely low-hanging fruit for volunteers in charities or communities in the area of sustained tedious tasks; collecting anecdotes, reports, links, that sort of thing come to mind as LW examples
*   additional incentives like [jsalvatier's contests](/lw/69p/prize_new_contest_for_spaced_repetition/) may be necessary to draw out community volunteer resources
*   tricking volunteers into work might be a fruitful approach - perhaps asking for explicit pointers to research or other help might not work, but presenting a half-complete version will elicit useful responses one can mine. (A more productive kind of trolling.)
# Cognitive Load from simple computations

[From Ken Burnside, a game designer](http://esr.ibiblio.org/?p=3481#comment-314050)

> Counting is easier than addition.
> 
> Addition is easier than subtraction.
> 
> Subtraction can be done, but combining addition and subtraction into the same game mechanic will slow down play considerably. (“I have +2 for flanking, and -2 for lighting…”) is surprisingly fiddly in play.
> 
>   
> BIG GAP HERE
> 
>   
> Subtraction and addition in the same operation is easier than most forms of multiplication. For example, it’s better to express a critical hit as “+100% damage” rather than “x2″ damage. Not only does it avoid edge cases (where you have two critical hit multipliers that both apply) it’s faster at the gaming table because it’s addition.
> 
> Multiplication by 10, 5, 4 and 2 are light weight enough to be usable. If you’re using a multiplicative operator other than one of those four digits, you need to change something earlier in the process. When in doubt, make it addition.
> 
>   
> BIGGER GAP HERE
> 
>   
> Division is only tolerable when A) you’re dividing an integer and expect an integer outcome, or are rounding to an integer outcome, and B) the divisor is 2 or 10. I specifically built some parts of my games for modulus division to avoid these problem.
> 
>   
> GINORMOUS GAP HERE
> 
>   
> Square roots, cube roots and exponentiation are probably not game friendly to most people. When in doubt, transform your equations to use one of the first two items on this list.

My impression is that Ken's games are geeky, so it's reasonable to assume his players are more adept with arithmetic than most people. I'm posting this because it might be worth knowing for anyone who's trying to explain something that involves numbers.
# I just donated to the SIAI.

My purpose in writing this is twofold.

First (chronologically: I thought of this earlier than the other), I want to discuss some of the pragmatic points in how I got myself to do it.

The most important thing is that I _didn't_ try to force the decision through with willpower. Instead, I slipped it through with doublethink. I _knew_ perfectly well - and have known for months - that giving money to SIAI was the right thing to do. But I didn't do it. I spent money on things like Minecraft instead.

But [somehow](http://intelligence.org/donate/) I found myself at the donation page, and I didn't think about it. Or, rather, I didn't let myself think about the fact that I was thinking about it. I made a series of expected-value guesstimates aimed at working around my own cognitive limitations.

I chose monthly donation over one-time because $20 monthly _sounds like_ about the same amount of money as $20; past experience with recurring donations suggests that I tend to leave automatic recurring donations in place for about a year or two, so that probably gained me about a factor of 20. Similarly, I chose $20 as the largest amount that wouldn't put me in serious risk of chickening out and not donating anything.

In order to pull this off, I had to _avoid thinking_ certain true thoughts. Numbers like "$240 per year" only drifted through my consciousness just long enough to make the expected-value judgment, and were then discarded quickly so as to avoid setting off my rotten-meat hypervisor.

This was not the first time I decided that I should give money to SIAI. It was the first time I _actually did_ give them money. (Except for that one time with the $1 charity-a-day thing, which actually might have helped with dissolving psychological barriers to the general idea.)

I think this is important.

The second fold of my purpose is to reinforce the behavior using the glowy feeling that comes from having other people know what an awesome person I am.^1^ Anyone else who's done anything worthwhile should feel free to post in this thread too.

1\. It's true. Statistically speaking, I probably saved like a jillion people's lives per dollar. And more-than-doubled quality of life for a zillion more. Let me also note that you can get in on this action.

I know that sounds advertisementy, but... well, that's kind of the point. Practice Dark Arts on yourself for fun and profit.
# AGI/FAI Theorist for Hire

I'm nearing the end of my employment at SIAI and looking for my next gig. If all else fails I will likely move back to the Bay area (I am currently in Japan) and take a job as a programmer somewhere. However, I would prefer to focus my attention directly on developing AGI and FAI theory. In addition to my current projects (described below), I can try to answer mathematical, philosophical, or other questions for a bit of cash. For some of my previous work, see my page on the [arXiv](http://arxiv.org/find/cs/1/au:+Blanc_P/0/1/0/all/0/1).

In the past year I've been involved in two major projects at SIAI. Steve Rayhawk and I were asked to review existing AGI literature and produce estimates of development timelines for AGI. My work on this project got rather bogged down and proceeded slowly, although I did learn a lot and I've moved in the direction of predicting AGI soonish (5-20 years). After this I tried to produce an AGI technology demo for Google's AGI-11 conference. I was unable to finish my demo in time for the submission deadline, and shortly afterwards SIAI decided to let me go.

I have several projects that I would like to move forward with, and if I can get adequate funds (about $1000 per month to ensure my survival, or $2000 to live comfortably) I will be able to work on them.

Current project ideas:

1.  Continue development on my incomplete AGI project (optimally, technical details not to be published).
2.  Write a paper on AGI models that can be used as a basis for FAI research (similar to the way AIXI and its ilk are used now, but closer to reality than AIXI).
3.  Figure out how an AI can reason formally about using objects in its environment as tools for performing computations.
4.  I'm also interested in repurposing machine learning algorithms used for finding plausible hypotheses about data distributions into algorithms for finding action policies with high expected utility.

I'm open to suggestions for other topics. I don't consider myself an expert at empiricism, so I prefer to work in domains where I can reason formally. Some thing I'd be up for:

1.  If you have informal questions or concerns, I can try to think of formal mathematical questions that are similar.
2.  Once we're dealing with a mathematical question, I can try to answer it.
3.  If a question looks too hard for me to answer (as will often be the case), I can try to figure out exactly what is hard about it.
4.  I'm also interested in writing problem sets. If you want to learn about some weird domain that no textbook exists for, I'll try to figure out what some introductory problems in that domain would look like.

Prices for any of these services are negotiable. You can contact me here or at [peter@spaceandgames.com](mailto:peter@spaceandgames.com).
# Experiment: Psychoanalyze Me

From Yvain's [latest post](/lw/6p6/the_limits_of_introspection/):

> These studies suggest that people do not have introspective awareness to the processes that generate their behavior. They guess their preferences, justifications, and beliefs by inferring the most plausible rationale for their observed behavior, but are unable to make these guesses qualitatively better than outside observers.

I guess this probably applies to beliefs as well as behavior. That is, the reasons people give for their beliefs are probably not based on real introspection either, which would explain why it's often so hard to find one's [true rejection](/lw/wj/is_that_your_true_rejection/).

If I do not have highly privileged access to my own reasoning and decision making processes, it stands to reason that other people should sometimes be able to tell me things about my goals or beliefs that I myself have missed. But apparently it's not that simple. In the True Rejection post, Eliezer wrote

> However, attempts to directly, publicly psychoanalyze the Other may cause the conversation to degenerate _very_ fast, in my observation.

This seems important enough to gather more data on. How and why do such conversations degenerate? Can we do something to prevent degeneration while still providing useful psychological insights to each other? So, as a first step, I hereby extend an open invitation to LW: tell me, whenever my stated goals and/or reasons do not seem to match up with my actual goals/reasons, what you think they really are.

(Presumably, the degeneration has to do with [status and offense](/lw/13s/the_nature_of_offense/). But perhaps in our community, one can gain status by conspicuously _not_ taking offense to such analyses, and instead taking them [seriously](/lw/2l6/taking_ideas_seriously/)?)
# Recent Less Wrong downtime

Many users may have noticed that Less Wrong experienced about 6 hours of downtime on 16/7/2011.

**CAUSE:** The server was put under an unusual amount of load and started up a new instance to load-balance the traffic.  Unfortunately, there was a bug in the script that starts the new instance that caused it to use an inconsistent mix of old and new code.  The symptom seen by users was that any post with comments was inaccessible.

**RESPONSE**: A hotfix was deployed as soon as the problem was detected, unfortunately it was a Saturday so this reponse time was slower than we would like.  We have since implemented a proper fix for the particular bug that caused this problem.  We are also creating some extra monitoring probes so we'll be notified promptly of any similar problems in the future.

Apologies for the inconvenience.
# Illustrator needed: Intuitive Bayes 2.0

A huge revision of _The Intuitive Explanation of Bayesian Reasoning_ is in progress, aimed at being considerably more accessible and hence with a lot more graphics.  I need someone who can turn out versions of the illustrations that are technically accurate enough to try on beta readers, fast enough and reliably enough that I can ask for revised versions of the illustrations a day later if the beta reader says they didn't understand it.  There is a Google Doc in progress which you would be given permission to edit, containing some hand-drawn attempts on my part to indicate what the illustrations should look like, and a number of finished illustrations from an illustrator who unfortunately cannot put in any further work on this job.

For this job, technical accuracy (i.e., if a ratio is 3:4, it should not look like 1:9), understandability, and speed is much more important than beauty - the idea is to get as quickly as possible to a working version with understandable illustrations that has been verified by the beta readers.

If interested, email me at yudkowsky@gmail.com.
# GiveWell interview with major SIAI donor Jaan Tallinn

GiveWell recently release notes from their interview with Jaan Tallinn, Skype co-founder and a major SIAI donor, about SIAI ([link](http://groups.yahoo.com/group/givewell/message/287)). Holden Karnofsky says 

> \[M\]y key high-level takeaways are that
> 
> 1.  I appreciated Jaan's thoughtfulness and willingness to engage in depth. It was an interesting exchange.
> 2.  I continue to disagree with the way that SIAI is thinking about the "Friendliness" problem. 
> 3.  It seems to me that all the ways in which Jaan and I disagree on this topic have more to do with philosophy (how to quantify uncertainty; how to deal with conjunctions; how to act in consideration of low probabilities) and with social science-type intuitions (how would people likely use a particular sort of AI) than with computer science or programming (what properties has software usually had historically; which of these properties become incoherent/hard to imagine when applied to AGI)