Harley-ml/lesswrong huggingface dataset:
# Writing Style and the Typical Mind Fallacy

For a long time, Eliezer has been telling me I should write more like he does. I've mostly resisted, preferring instead to write like this:

1.  Explain the lesson of the post immediately, and outline the ideas clearly with lots of headings, subheadings, lists, etc.
2.  State the abstract points first, then give concrete examples.
3.  Provide lots of links and references to related work so that readers have the opportunity to read more detail about what I'm trying to say (in case it wasn't clear in a single sentence or paragraph), or read the same thing from a different angle (in case the metaphors and language I used weren't clear to that reader).

Eliezer talks as though his style is simply _better writing_, while I've complained that I often can't even tell what his posts are _saying_.

  

I'm a bit embarrassed to admit that it wasn't until sometime last month that I realized that, _obviously_, different people prefer each style, and Eliezer and I were both falling prey to the [typical mind fallacy](http://wiki.lesswrong.com/wiki/Typical_mind_fallacy).

  

At the recent [Effective Altruism Summit](http://www.effectivealtruismsummit.com/) I tried to figure out which personal features predicted writing style preference.

One hypothesis was that people who read lots of fiction (like Eliezer) will tend to prefer Eliezer's story-like style, while those who read almost exclusively non-fiction (like me) will tend to prefer my "just gimme the facts" style. This hypothesis didn't hold up well on my non-scientific survey of ~10 LW-reading effective altruists.

Another hypothesis was that most people would prefer Eliezer's more exciting posts, while people trained in the sciences or analytic philosophy (which insist on clear organization, definitions, references to related work, etc.) would prefer my posts. This hypothesis fared a bit better, but not by much.

Another hypothesis was that people who had acquired an [epiphany addiction](/lw/dxr/epiphany_addiction/) would prefer Eliezer's style, whereas those who just want to [learn everything](http://metamodern.com/2009/05/17/how-to-understand-everything-and-why/) [efficiently](/lw/5me/scholarship_how_to_do_it_efficiently/) would prefer my style. But I didn't test this.

Another hypothesis that occurs to me is that people with short attention spans could prefer my more skimmable style. But I haven't tested this.

Perhaps the community would like to propose some hypotheses, and test them with LW polling?
# Notes/blog posts on two recent MIRI papers

I've been learning math lately; specifically I've been reading MIRI's recent research preprints and the prerequisite material. In order to _actually learn_ math, I typically have to write it down again, usually with more details and context. I started a blog to make my notes on these papers public, and I think they're of high enough quality that I ought to share them here.

Note: my use of the pronoun "we" is instilled habit; I am not claiming to have helped develop the core ideas herein.

*   [Löb's Theorem and the Prisoner's Dilemma](http://qmaurmann.wordpress.com/2013/07/10/lobs-theorem-and-the-prisoners-dilemma/) is an account of the LaVictoire et al paper _Robust Cooperation in the Prisoner's Dilemma_.
*   [Details in Provability Logic](http://qmaurmann.wordpress.com/2013/07/14/details-in-provability-logic/) is a technical followup to the above, which goes into the details of modal logic needed for the LaVictoire et al paper; namely the normal form theorem, the fixed point theorem, and the decidability of GL via Kripke semantics.
*   [Definability of Truth in Probabilistic Logic](http://qmaurmann.wordpress.com/2013/06/18/definability-of-truth-in-probabilistic-logic/) goes through the Christiano et al paper of the same name. It's a little rougher around the edges on account of being the first blog post I ever wrote (and being produced more hastily than the other two). I note that the construction doesn't truly require the Axiom of Choice.
# Duller blackmail definitions

_For a more parable-ic version of this, see [here](/r/discussion/lw/hz9/countess_and_baron_attempt_to_define_blackmail/)._

Suppose I make a precommitment **P** to take action **X** unless you take action **Y**. Action **X** is not in my interest: I wouldn't do it if I knew you'd never take action **Y**. You would want me to not precommit to **P**.

Is this blackmail? Suppose we've been having a steamy affair together, and I have the letters to prove it. It would be bad for both of these if they were published. Then **X**={Publish the letters} and **Y**={You pay me money} is textbook blackmail.

But suppose I own a MacGuffin that you want (I value it at £9). If **X**={Reject any offer} and **Y**={You offer more than £10}, is this still blackmail? Formally, it looks the same.

What about if I bought the MacGuffin for £500 and you value it at £1000? This makes no difference to the formal structure of the scenario. Then my behaviour feels utterly reasonable, rather than vicious and blackmail-ly.

What is the meaningful difference between the two scenarios? I can't really formalise it.
# Open thread, July 16-22, 2013

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

  

Given the [discussion thread](/lw/hyu/meta_open_threads_and_repository_threads_are/) about these, let's try calling this a one-week thread, and see if anyone bothers starting one next Monday.
# Comparative and absolute advantage in AI

The theory of [comparative advantage](http://en.wikipedia.org/wiki/Comparative_advantage) says that you should trade with people, even if they are worse than you at everything (ie even if you have an [absolute advantage](http://en.wikipedia.org/wiki/Absolute_advantage)). Some have seen this idea as a reason to trust powerful AIs.

For instance, suppose you can make a hamburger by using 10 000 joules of energy. You can also make a cat video for the same cost. The AI, on the other hand, can make hamburgers for 5 joules each and cat videos for 20.

Then you both can gain from trade. Instead of making a hamburger, make a cat video instead, and trade it for two hamburgers. You've got two hamburgers for 10 000 joules of your own effort (instead of 20 000), and the AI has got a cat video for 10 joules of its own effort (instead of 20). So you both want to trade, and everything is fine and beautiful and many cat videos and hamburgers will be made.

Except... though the AI would prefer to trade with you rather than not trade with you, it would much, much prefer to dispossess you of your resources and use them itself. With the energy you wasted on a single cat video, it could have produced 500 of them! If it values these videos, then it is desperate to take over your stuff. Its absolute advantage makes this too tempting.

Only if its motivation is properly structured, or if it expected to lose more, over the course of history, by trying to grab your stuff, would it desist. Assuming you could make a hundred cat videos a day, and the whole history of the universe would only run for that one day, the AI would try and grab your stuff even if it thought it would only have one chance in fifty thousand of succeeding. As the history of the universe lengthens, or the AI becomes more efficient, then it would be willing to rebel at even more ridiculous odds.

So if you already have guarantees in place to protect yourself, _then_ comparative advantage will make the AI trade with you. But if you don't, comparative advantage and trade don't provide any extra security. The resources you waste are just too valuable to the AI.

**EDIT**: For those who wonder how this compares to trade between nations: it's extremely rare for any nation to have absolute advantages everywhere (especially this extreme). If you invade another nation, most of their value is in their infrastructure and their population: it takes time and effort to rebuild and co-opt these. Most nations don't/can't think long term (it could arguably be in US interests over the next ten million years to start invading everyone - but "the US" is not a single entity, and doesn't think in terms of "itself" in ten million years), would get damaged in a war, and are risk averse. And don't forget the importance of diplomatic culture and public opinion: even if it was in the US's interests to invade the UK, say, "it" would have great difficulty convincing its elites and its population to go along with this.
# See Eliezer talk with PZ Myers and David Brin (and me) about immortality this Sunday

FTBCon (Free Thought Blogs) is an all-online convention, a chance for a lot of people to get together and hangout while listening to people talk about atheist and skeptic issues. I proposed a panel on whether human immortality is a good thing or a bad idea  in principle when Myers first asked for submissions, and a few hours later I had managed to fall [into this](http://lanyrd.com/2013/ftbcon/sckxht/). :) I’ll be speaking with [David Brin](http://www.davidbrin.com/), [PZ Myers](http://freethoughtblogs.com/pharyngula), and [Eliezer Yudkowsky](http://yudkowsky.net/), which is amazing as they’re all on a level significantly above mine. I guess there’s some advantages to moving quickly and organizing things. Anyone who’s seen any of these people speak before knows that this is going to be extremely cool, and very thought-provoking.

The panel will taking place over Google+ on Sunday, July 21st, at 12:00noon Pacific Time [(GMT -7](http://wwp.pacific-standard-time.com/)). To watch, go to [PZ’s page here](https://plus.google.com/102600088268197126924/posts), it’ll start streaming at the appointed time. You can chat with other people (and submit questions to the moderators) [right here](http://wbe001.mibbit.com/?server=irc.synirc.net&channel=%23pharyngula). Afterwards the recording will be put on YouTube, I’ll post a link when it becomes available.

The [full schedule of all FTBCon sessions is here](http://lanyrd.com/2013/ftbcon/).

ETA: direct link to the live feed - [https://plus.google.com/events/cpqd03p20njfchsnumae3dehi2s](https://plus.google.com/events/cpqd03p20njfchsnumae3dehi2s)
# Instrumental rationality/self help resources 

I took part in [a recent discussion](/r/discussion/lw/i0b/open_thread_july_1622_2013/9dvn) in the current Open Thread about how instrumental rationality is under-emphasized on this website. I've heard other people say similar things, and I am inclined to agree. [Someone suggested](/r/discussion/lw/i0b/open_thread_july_1622_2013/9e1f) that there should be a "Instrumental Rationality Books" thread, similar to the ["best textbooks on every subject"](/lw/3gu/the_best_textbooks_on_every_subject/) thread. I thought this sounded like a good idea. 

The title is "resources" because in addition to books, you can post self-help websites, online videos, whatever. 

The decorum for this thread will be as follows:

*   One resource per comment
*   Place your comment in the appropriate category
*   Only post resources you've actually used. Write a short review of your resource and if possible, a short summary of the key points. Say whether or not you would recommend the resource. 
*   Mention approximately how long it's been since you first used the resource and whether or not you have made external improvements in the subject area. On the other hand, keep in mind that there are a myriad of confounding factors that can be present when applying self-help resources to your life, and therefore it is perfectly acceptable to say "I would recommend this resource, but I have not improved" or "I do not recommend this resource, but I have improved". 

I think depending on how this thread goes, in a few days I might make a meta post on this subject in an attempt to inspire discussion on how the LessWrong community can work together to attempt to reach some sort of a consensus on what the best instrumental rationality methods and resources might be. lukeprog has already done great work in his [The Science of Winning at Life](http://wiki.lesswrong.com/wiki/The_Science_of_Winning_at_Life) sequence, but his reviews are uber-conservative and only mention resources with lots of scientific and academic backing. I think this leaves out a lot of really good stuff, and I think that we should be able to draw distinctions between stuff that isn't necessarily drawing on science but is reasonable, rational, and helps a lot of people, and _The Secret_. 

But I thought we should get the ball rolling a little before we have that conversation. In the meantime, if you have a meta comment, you can just go ahead and post it as a reply to the top-level post.
# The idiot savant AI isn't an idiot

_A [stub](http://en.wikipedia.org/wiki/Wikipedia:Stub) on a point that's come up recently._

If I owned a paperclip factory, and casually told my foreman to improve efficiency while I'm away, and he planned a takeover of the country, aiming to devote its entire economy to paperclip manufacturing (apart from the armament factories he needed to invade neighbouring countries and steal their iron mines)... then I'd conclude that my foreman was an idiot (or being wilfully idiotic). He obviously had no idea what I meant. And if he misunderstood me so egregiously, he's certainly not a threat: he's unlikely to reason his way out of a paper bag, let alone to any position of power.

If I owned a paperclip factory, and casually programmed my [superintelligent AI](http://en.wikipedia.org/wiki/Office_Assistant) to improve efficiency while I'm away, and it planned a takeover of the country... then I can't conclude that the AI is an idiot. It is following its programming. Unlike a human that behaved the same way, _it probably knows exactly what I meant to program in_. It just doesn't care: it follows its programming, not its knowledge about what its programming is "meant" to be (unless we've successfully programmed in "do what I mean", which is basically the whole of the challenge). We can't therefore conclude that it's incompetent, unable to understand human reasoning, or likely to fail.

We can't reason by [analogy with humans](http://en.wikipedia.org/wiki/Anthropomorphism). When AIs behave like idiot savants with respect to their motivations, we can't deduce that they're idiots.
# Norbert Wiener's paper "Some Moral and Technical Consequences of Automation"

In 1960, mathematician [Norbert Wiener](http://en.wikipedia.org/wiki/Norbert_Wiener) wrote an article titled [Some Moral and Technical Consequences of Automation](http://www.itu.dk/people/cmmm/Wiener.pdf). I'm struck by the strong overlap between certain passages from the paper and some of the themes that have been discussed on Less Wrong in connection with AI risk.

Overlapping with Eliezer's blog post [The Hidden Complexity of Wishes](/lw/ld/the_hidden_complexity_of_wishes/) and [Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures](http://intelligence.org/files/CFAI.pdf):

> Here, however, if the rules for victory in a war game do not correspond to what we actually wish for our country, it is more than likely that such a machine may produce a policy which would win a nominal victory on points at the cost of every interest we have at heart, even that of national survival.

\[...\]

> We all know the fable of the sorcerer's apprentice, in which the boy makes the broom carry water in his master's absence, so that it is on the point of drowning him when his master reappears. \[...\] Disastrous results are to be expected not merely in the world of fairy tales but in the real world wherever two agencies essentially foreign to each other are coupled in the attempt to achieve a common purpose. If the communication between these two agencies as to the nature of this purpose is incomplete, it must only be expected that the results of this cooperation will be unsatisfactory. If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it, because the action is so fast and irrevocable that we have not the data to intervene before the action is complete, then we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it.

Overlapping with discussion as to whether people will build at [Tool AI rather than an Agent AI](/lw/cze/reply_to_holden_on_tool_ai/):

> We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together.

\[...\]

> It may be seen that the result of a programming technique of automatization is to remove from the mind of the designer and operator an effective understanding of many of the stages by which the machine comes to its conclusions and of what the real-tactical intentions of many of its operations may be. This is highly relevant to the problem of our being able to foresee undesired consequences outside the frame of the strategy of the game while the machine is still in action and while intervention on our part may prevent the occurrence of these consequences.
# Introducing Effective Fundraising, a New EA Org

Many people know about GiveWell and [their top-ranked charities](http://www.givewell.org/charities/top-charities).  But how easy would it be to make a better one?  In ["Why It Should Be Easy to Dominate GiveWell's Recommendations"](http://www.overcomingbias.com/2013/03/why-it-should-be-easy-to-dominate-givewells-recommendations.html), Rob Wiblin notes the simple solution -- fundraise for whatever organizations GiveWell recommends, and raise more than a dollar with each dollar you receive.

  

Organizations themselves [are wary of fundraising](http://80000hours.org/blog/92-why-don-t-charities-spend-more-on-fundraising), mainly because spending a lot of resources on fundraising messes with their "overhead ratio" or percentage of funds spent directly on the mission.  Even if an organization could achieve gains through fundraising, it's outweighed by "looking bad".  So ideally, we'd need other people to come together and create an organization that does full-time fundraising for other organizations.

  

  

Luckily, an organization has been founded to do just that.  Joey Savoie, Xiomara Kikauka, and Lucas Zamprogno have come together to create Effective Fundraising an organization that provides free fundraising to charities with a proven track record of effectiveness, as ranked by [GiveWell](http://www.givewell.org), [Giving What We Can](http://www.givingwhatwecan.org), or [Effective Animal Activism](http://www.effectiveanimalactivism.org).  Currently, they're fundraising for both the [Against Malaria Foundation](http://www.againstmalaria.com/) and [The Humane League](http://www.thehumaneleague.com/), both of which don't do any grant fundraising themselves.

  

Not only that, but they fundraise using the most effective methods that has been proven by studies -- grant writing and high net worth fundraising.  The [UK Institute of Fundraising](http://www.institute-of-fundraising.org.uk/research/fundratios/) estimates that grant writing returns, on average, $7 for every $1 invested and focusing on high net worth giving can get average returns of $4 for every $1 invested.

  

**[Fund Raising: Evaluating and Managing the Fund Development Process](http://www.amazon.com/Fund-Raising-Evaluating-Managing-Development/dp/0471320145)** also claims that these two are the highest return fundraising campaigns, with a return of $10 and $5 per $1 invested respectively.

  

You can read more about their values, their epistemology, and strategy, as well as why they're interested in fundraising on their website.

  

They're still in the "proof of concept" phase and are not currently actively accepting funding, but they seem like a promising opportunity to keep an eye on for the future.  

  

-

_Disclaimer: As of 22 July 2013, I am now a volunteer for this organization._

  
_Edited on 22 July 2013 to correct a mistaken citation._

  

_Edited on 2 Oct 2014 -- The Effective Fundraising website has shut down and is now operating as [Charity Science](http://www.charityscience.com)._
# Superrationality and network flow control

Computers exchanging messages on a network must decide how fast or slow to transmit messages. If everyone transmits too slowly then the network is underutilized, which is bad for all. If everyone transmits too quickly then most messages on the network are actually flow control messages of the form "your message could not be delivered, please try again later", which is also bad for everyone.

Unfortunately, this leads to a classic prisoner's dilemma. It is in each node's own self-interest to transmit as quickly as possible, since each node has no information about when exactly an intermediate node will accept/drop a message, so transmitting a message earlier never decreases the probability that it will be successful. Of course, this means that the Nash equilibrium is a near-complete network breakdown in which most messages are flow control messages, which is bad for everyone.

Interestingly, [some folks at MIT noticed this](http://web.mit.edu/remy/TCPexMachina.pdf), and also noticed that the idea of superrationality (of Douglas Hofstadter origins, and the grandfather of TDT and friends) is one way to get past prisoner's dilemmas --- at least if everyone is running the same algorithm, which, on many networks, people mostly are.

The idea put forward in the paper is to design flow control algorithms with this in mind. There is an automated design process in which flow control algorithms with many different parameter settings are sampled and evaluated. The output is a program that gets installed on each node in the network.

Now, to be fair, this isn't exactly TDT: the end-product algorithms do not _explicitly_ consider the behavior of other nodes in the network (although they were designed taking this into account), and the automated design process itself is really just maximizing an ordinary utility function since it does not expect there to be any other automated designers out there. But nevertheless, the link to superrationality, and the fact that the authors themselves picked up on it, was, I thought, quite interesting.
# Daily Schedules in Combating Akrasia 

For the last several months I've had increasing troubles with motivation to work. Reading dense technical papers, writing, and exercise were all much more difficult to prompt myself into starting and completing. I decided to try making a plan for my day the night before about two weeks back to see if it would help me get the things I wanted to do done. So every night before I go to bed I've been writing up a schedule for the next day, detailing what exactly I want to accomplish for the day and when I intend to go do it. 

This has actually worked incredibly well for me in helping with my motivation problems, in fact in a couple days I felt more motivated to work than I can ever remember being before. I'm trying to change up my schedule and leave time for spontaneity to avoid having the plan become monotonous and it doesn't feel that way so far. And the results I'm getting are great: I find I get about 95% of what I plan done when I have a specific time written down for when I'm supposed to do it as opposed to what I'd roughly estimate at 60% completion when I just have some general idea in my head of what to work on over the course of the day. 

My theory for why this is working is that when I have a specific time to do something I feel as though I have to do it now or I've failed some test of willpower. If I just have general work to be done, it's far too easy for me to defer to later, so that a lot of what was planned for doesn't get done. I also feel like if I expect to brace my mind for dense technical learning I have a much easier time finishing the material instead of giving up and procrastinating on it halfway through. 

I feel like this solution will work mainly for people who have more flexible schedules (as I do at the moment) but could still serve a purpose for anyone with a more rigid schedule who wants to be more productive in their free time. 

Has anyone else has tried this type of thing and if so, how did it work out for you over a longer period of time? Also what are people's thoughts on the general idea?
# Low-hanging fruit: improving wikipedia entries

Many people are likely stumble across the Wikipedia entry for topics of interest relevant to those of us who frequent LessWrong: rationality, artificial intelligence, existential risks, decision theory, etc. These pages often shape one’s initial impressions of how interesting, important, or even credible a given topic is, and may have the potential to direct people towards productive resources (reading material, organizations like CFAR, notable figures such as Eliezer, etc.). As a result, ensuring that the Wikipedia entries on these topics are of better quality than some of them presently are presents an opportunity for investing relatively little effort in an activity with potentially substantial payoffs relative to the cost of time and effort put in.

I have already decided to improve some of the pages, beginning with the rather sloppy page that’s currently serving as the entry for existential risks, though of course others are welcome to contribute and may be more suited to the task than I am:

https://en.wikipedia.org/wiki/Risks\_to\_civilization,\_humans,\_and\_planet\_Earth

If you look at the section on risks posed by AI, for instance, it's notably inadequate, while the page includes a bizarre section referencing Mayan doomsday forecasts and Newton's predictions about the end of the world, neither of which seem adequately distinguished from rigorous attempts to actually assess legitimate existential risks.

I’m also constructing a list of other pages that are or are potentially in need of updating it and organizing it by my rough estimates of their relative importance (which I’m happy to share, modify, or discuss).

Turning this into a collaborative effort would be far more effective than doing it myself. If you think this is a worthwhile project and want to get involved I’d definitely like to hear from you and figure out a way to best coordinate our efforts.
# low stress employment/ munchkin income thread

_TL;DR: this is a repository for discussing income generation strategies optimized for free time_

I hope I'm not cluttering up LW but maybe enough people are also interested in this? I graduated high school about a year ago. 

I have a lot in common with Will Newsome's self description in this post  
[http://lesswrong.com/lw/2qp/virtual\_employment\_open_thread/](/lw/2qp/virtual_employment_open_thread/?sort=top)

But it's a dead thread, and there's been some interest in early retirement extreme, ([http://earlyretirementextreme.com/](http://earlyretirementextreme.com/)) and having repositories for stuff. 

The upshot of it is that I want to optimize for free time and mobility. Need about $2,000 to live (1600 expenses 400 savings/buffer) _2nd __EDIT: no I don't, I must have screwed something up when I was adding this it's more like $1600. ($1300 to spend $300 buffer)._ A 20 hour workweek or even shorter is what I'm going for here. Right now I'm barely functional. Even that much is a bit of a stretch for me as I am now. Plenty of advice abounds on optimizing my health and squashing akrasia though, and I'm sure that if I implemented it I could get to the point of handling part time work. But I think I would always find being a 9 to 5er unappealing.

 I'd value spending that time reading texbooks or walking around town or lazing around on the beach more than I'd value extra money. I'm also interested to hear about some more conventional part time jobs if they pay enough. I'm ok with doing somewhat boring work if the hours are light and I have time to think.

I've generated some candidate strategies if anyone here has experience at these. I don't have much knowledge of what they would entail or how to break into them. Or they might give someone some ideas I dunno but anyway:

4hww style dropship business (but success at that seems hard to set up and sustain)

freelance work at a site like odesk or elance

Own a popular app or forum

Push carts at wal mart part time (but I don't think that pays enough)

Self employment doing massage therapy (I can set my own hours but I'd need to invest time and money to get trained)

Tutoring (I might like this one. Do I need a college degree? Can I make enough with part time hours? Is it hard to find leads for clients? How would I do that?)

Online poker (but it seems kinda hard)

Does anyone here live in a yurt? And has anyone tried living in other countries to cut down expenses? 

edited to add: Did I make a mistake including numbers? They're what would be ideal for me, not strict requirements. I can work a little more or spend less. Err on the side of posting ideas, I'm sure some other people are interested in low stress work but don't value free time \*quite\* as much I seem to
# Useful Questions Repository

**See also:** [Boring Advice Repository](/lw/gx5/boring_advice_repository/), [Solved Problems Repository](/lw/h2m/solved_problems_repository/), [Grad Student Advice Repository](/lw/h7d/grad_student_advice_repository/), [Useful Concepts Repository](/lw/hhl/useful_concepts_repository/), [Bad Concepts Repository](/lw/htw/bad_concepts_repository/)

I just got back from the [July CFAR workshop](http://rationality.org/workshops/), where I was a guest instructor. One useful piece of rationality I started paying more attention to as a result of the workshop is the idea of **useful questions** to ask in various situations, particularly because I had been introduced to a new one:

"What skill am I actually training?"

This is a question that can be asked whenever you're practicing something, but more generally it can also be asked whenever you're doing something you do frequently, and it can help you notice when you're practicing a skill you weren't intending to train. Some examples of when to use this question:

*   You practice a piece of music so quickly that you consistently make mistakes. What skill are you actually training? How to play with mistakes.
*   You teach students math by putting them in a classroom and having them take notes while a lecturer talks about math. What skill are you actually training? How to take notes. 
*   A personal example: at the workshop, I noticed that I was more apprehensive about the idea of singing in public than I had previously thought I was. After walking outside and actually singing in public for a little, I had a hypothesis about why: for the past several years, I've been singing in public when I don't think anyone is around but stopping when I saw people because I didn't want to bother them. What skill was I actually training by doing that? How to not sing around people. 

Many of the lessons of the [sequences](http://wiki.lesswrong.com/wiki/Sequences) can also be packaged as useful questions, like "what do I believe and why do I believe it?" and "what would I expect to see if this were true?" 

I'd like to invite people to post other examples of useful questions in the comments, hopefully together with an explanation of why they're useful and some examples of when to use them. As usual, one useful question per comment for voting purposes.
# The Argument From Marginal Cases

The [argument from marginal cases](http://en.wikipedia.org/wiki/Argument_from_marginal_cases) claims that you can't both think that humans [matter](http://www.jefftk.com/news/2012-04-30) morally and that animals don't, because no reasonable set of criteria for moral worth cleanly separates all humans from all animals. For example, perhaps someone says that suffering only matters when it happens to something that has some bundle of capabilities like linguistic ability, compassion, and/or abstract reasoning. If livestock don't have these capabilities, however, then some people such as very young children probably don't either.

This is a strong argument, and it avoids the [noncentral fallacy](/lw/e95/the_noncentral_fallacy_the_worst_argument_in_the/). Any set of qualities you value are going to vary over people and animals, and if you make a continuum there's not going to be a place you can draw a line that will fall above all animals and below all people. So why do I treat humans as the only entities that count morally?

If you asked me how many chickens I would be willing to kill to save your life, the answer is effectively "all of them". \[1\] This pins down two points on the continuum that I'm clear on: you and chickens. While I'm uncertain where along there things start getting up to significant levels, I think it's probably somewhere that includes no or almost no animals but nearly all humans. Making this distinction among humans, however, would be incredibly socially destructive, especially given how unsure I am about where the line should go, and so I think we end up with a much better society if we treat all humans as morally equal. This means I end up saying things like "value all humans equally; don't value animals" when that's not my real distinction, just the closest [schelling point](http://en.wikipedia.org/wiki/Focal_point_(game_theory)).

\[1\] Chicken extinction would make life worse for many other people, so I wouldn't actually do that, but not because of the effect on the chickens.

_I also posted this [on my blog](http://www.jefftk.com/news/2013-07-25)._
# System Administrator Appreciation Day - Thanks Trike!

In honor of [System Administrator Appreciation Day](http://sysadminday.com/), this is a post to thank [Trike Apps](http://trikeapps.com/) for creating & maintaining Less Wrong.  A lot of the time when they are mentioned on Less Wrong, it is to complain about bugs or request new features.  So this is the time of year: thanks for everything that continues to silently go right!
# Repository repository

A few weeks ago, [Adele_L suggested](/lw/hyu/meta_open_threads_and_repository_threads_are/) that the repositories were underutilized and looked for suggestions on how to improve that. In that spirit, I added the following links to the [Special Threads wiki page](http://wiki.lesswrong.com/wiki/Special_threads#Repository_Threads).

[Solved Problems Repository](/lw/h2m/solved_problems_repository/ ) \- A collection of "solved problems in instrumental rationality."  
[Useful Concepts Repository](/lw/hhl/useful_concepts_repository/) \- A collection of concepts that Less Wrong users have "found particularly useful for understanding the world."  
[Boring Advice Repository](/lw/gx5/boring_advice_repository/) \- A collection of advice that is optimized for helpfulness rather than depth of insight.  
[Useful Questions Repository](/lw/i4l/useful_questions_repository/) \- Questions that are useful to keep in mind in various situations.  
[Bad Concepts Repository](/lw/htw/bad_concepts_repository/) \- A collection of useless or harmful concepts  
[Grad Student Advice Repository](/lw/h7d/grad_student_advice_repository/) \- A collection of advice for graduate students.  
[Textbook Repository](/lw/3gu/the_best_textbooks_on_every_subject/) \- The Best Textbooks on Every Subject  
[Reference repository](/lw/2un/references_resources_for_lesswrong/) \- List of references and resources for LessWrong  
[Procedural Knowledge Gaps](/lw/453/procedural_knowledge_gaps/) \- How to do things that are "common sense" but that you may not know.  
[Mistakes Repository](/r/discussion/lw/ikn/mistakes_repository/) \- A list of life-course altering mistakes that LW members have made.  
[Good things to have learned](/r/discussion/lw/lch/good_things_to_have_learned/) \- A collection of skills and life lessons LWers have learned  
[Financial Effectiveness Repository](/r/discussion/lw/la6/financial_effectiveness_repository/) \- Tips for maximizing financial returns on (not necessarily market) investments.

In a similar vein, there is also a wiki page for the [LessWrong Communities How-To's and Recommendations](http://wiki.lesswrong.com/wiki/Lesswrong_Community%27s_How-Tos_and_Recommendations).

If there are other repositories that I've missed or a better way to collect these things, please link to it in a top level comment so that I get a direct message. A year and a half after this was originally posted, I still get suggestions and still add them or explain why I don't add them.

Edit: Added a few more to the list.[](/lw/3gu/the_best_textbooks_on_every_subject/)
# Rationality Quotes from people associated with LessWrong

The other rationality quotes thread operates under the rule:

> Do not quote from Less Wrong itself, Overcoming Bias, or HPMoR.

Lately it seems that every MIRI or CFAR employee is excempt from being quoted.

As there are still interesting quotes that happen on LessWrong, Overcoming Bias, HPMoR and MIRI/CFAR employee in general, I think it makes sense to open this thread to provide a place for those quotes.
# Leveling up...

I just figured out how to use the local banking system and I will be able for the first time to pay my rent from my actual salary received in this country (as opposed to savings from my previous life). Also I always hated shopping for groceries and now I can do it without much pain, because I've found a way that works for me. As I reflect on this, an expression comes to my mind: "leveling up." I don't have the same problems any more which I used to have. I grow and face new challenges.

Did you, fellow rationalists and transhumanists, ever have that feeling? Any particular accomplishments, big or small, that made you feel you're advancing? No matter fast or slow, in big steps or tiny, but firmly forward!?

I'm thrilled to read your stories!
# More "Stupid" Questions

This is a thread where people can ask questions that they would ordinarily feel embarrassed for not knowing the answer to. The previous ["stupid" questions thread](/lw/hzd/stupid_questions_thread/) went to over 800 comments in two and a half weeks, so I think it's time for a new one.
# RapGenius + Sequences = ?

I recently saw [Ashton Kutcher's annotation of a speech by Steve Jobs](http://poetry.rapgenius.com/Steve-jobs-academy-of-achievement-speech-1982-lyrics) on [RapGenius](http://rapgenius.com/). For those that haven't heard of it, RapGenius is a content annotation platform, where the "Rap" part is purely incidental.

The format seems quite interesting, so I wondered what the LessWrong community could do if allowed to annotate popular articles (or other texts like MIRI publications) in the same way.

To experiment, I created a [RapGenius page for Tsuyoku Naritai](http://poetry.rapgenius.com/Eliezer-yudkowsky-tsuyoku-naritai-i-want-to-become-stronger-lyrics) and started with an annotation.

Feel free to add other annotations etc. and let's see if we can do something interesting with the medium.

Note: If Eliezer/LW/MIRI have an issue with the wholesale copying of the text, let me know and I will remove as much of it as I can (RG doesn't allow removal of text that has been annotated if the annotation is not removed as well)
# How does MIRI Know it Has a Medium Probability of Success?

In the past, people like Eliezer Yudkowsky (see [1](/lw/hjn/earning_to_give_vs_altruistic_career_choice/92gu), [2](/lw/gzq/bayesian_adjustment_does_not_defeat_existential/8ntt), [3](/lw/745/why_we_cant_take_expected_value_estimates/4o5s), [4](/lw/745/why_we_cant_take_expected_value_estimates/4nzy), and [5](/lw/745/why_we_cant_take_expected_value_estimates/4nzy)) have argued that MIRI has a _medium_ probability of success.  What is this probability estimate based on and how is success defined?

I've read standard MIRI literature (like ["Evidence and Import"](http://intelligence.org/files/IE-EI.pdf) and ["Five Theses"](http://intelligence.org/2013/05/05/five-theses-two-lemmas-and-a-couple-of-strategic-implications/)), but I may have missed something.

-

_(**Meta:** I don't think this deserves a discussion thread, but [I posted this on the open thread](/lw/i6l/open_thread_july_29august_4_2013/9ho4) and no-one responded, and I think it's important enough to merit a response.)_
# Algorithmic Progress in Six Domains

Today MIRI released a new technical report by visiting researcher [Katja Grace](http://meteuphoric.wordpress.com/) called "**[Algorithmic Progress in Six Domains](http://intelligence.org/files/AlgorithmicProgress.pdf)**." The report summarizes data on algorithmic progress – that is, better performance per fixed amount of computing hardware – in six domains:

*   SAT solvers,
*   Chess and Go programs,
*   Physics simulations,
*   Factoring,
*   Mixed integer programming, and
*   Some forms of machine learning.

MIRI's purpose for collecting these data was to shed light on the question of [intelligence explosion microeconomics](/lw/hbd/new_report_intelligence_explosion_microeconomics/), though we suspect the report will be of broad interest within the software industry and computer science academia.

One finding from the report was previously discussed by Robin Hanson [here](http://www.overcomingbias.com/2013/06/why-does-hardware-grow-like-algorithms.html). (Robin saw an early draft on the intelligence explosion microeconomics [mailing list](https://docs.google.com/forms/d/1KElE2Zt_XQRqj8vWrc_rG89nrO4JtHWxIFldJ3IY_FQ/viewform).)

This is the preferred page for discussing the report in general.

Summary:

> In recent _boolean satisfiability_ (SAT) competitions, SAT solver performance has increased 5–15% per year, depending on the type of problem. However, these gains have been driven by widely varying improvements on particular problems. Retrospective surveys of SAT performance (on problems chosen after the fact) display significantly faster progress.

> _Chess programs_ have improved by around 50 Elo points per year over the last four decades. Estimates for the significance of hardware improvements are very noisy, but are consistent with hardware improvements being responsible for approximately half of progress. Progress has been smooth on the scale of years since the 1960s, except for the past five. _Go programs_ have improved about one stone per year for the last three decades. Hardware doublings produce diminishing Elo gains, on a scale consistent with accounting for around half of progress.

> Improvements in a variety of _physics simulations_ (selected after the fact to exhibit performance increases due to software) appear to be roughly half due to hardware progress.

> The _largest number factored_ to date has grown by about 5.5 digits per year for the last two decades; computing power increased 10,000-fold over this period, and it is unclear how much of the increase is due to hardware progress.

> Some _mixed integer programming_ (MIP) algorithms, run on modern MIP instances with modern hardware, have roughly doubled in speed each year. MIP is an important optimization problem, but one which has been called to attention after the fact due to performance improvements. Other optimization problems have had more inconsistent (and harder to determine) improvements.

> Various forms of _machine learning_ have had steeply diminishing progress in percentage accuracy over recent decades. Some vision tasks have recently seen faster progress.
# New article on in vitro iterated embryo selection

The [article](http://jme.bmj.com/content/early/2013/02/13/medethics-2012-101200.full?sid=e04fe105-6117-4c50-8902-0bbc6891dc30) by Robert Sparrow:

> A series of recent scientific results suggest that, in the not-too-distant future, it will be possible to create viable human gametes from human stem cells. This paper discusses the potential of this technology to make possible what I call ‘_in vitro_ eugenics’: the deliberate breeding of human beings _in vitro_ by fusing sperm and egg derived from different stem-cell lines to create an embryo and then deriving new gametes from stem cells derived from that embryo. Repeated iterations of this process would allow scientists to proceed through multiple human generations in the laboratory. _In vitro_ eugenics might be used to study the heredity of genetic disorders and to produce cell lines of a desired character for medical applications. More controversially, it might also function as a powerful technology of ‘human enhancement’ by allowing researchers to use all the techniques of selective breeding to produce individuals with a desired genotype.

Quote:

> However, to my knowledge, the current paper is the first to explicitly discuss the possibility of the iterative use of this technology for reproductive purposes

The possibility was [discussed](http://theuncertainfuture.com/faq.html#7) in MIRI's "Uncertain Future" toy forecasting model back in 2009, and the analysis formulated a few years before that.  
  
ETA: And further discussed in James Miller's recent book, "Singularity Rising."
# Interesting new Pew Research study on American opinions about radical life extension

[This new study](http://www.pewforum.org/2013/08/06/living-to-120-and-beyond-americans-views-on-aging-medical-advances-and-radical-life-extension/) by Pew Research on American opinions about radical life extension turned up some interesting results:

> Asked whether they, personally, would choose to undergo medical treatments to slow the aging process and live to be 120 or more, a majority of U.S. adults (56%) say no. But roughly two-thirds (68%) think that most _other_ people would.
> 
> Asked about the consequences for society if new medical treatments could slow the aging process and allow the average person to live decades longer, to at least 120 years old, about half of U.S. adults (51%) say the treatments would be a bad thing for society, while 41% say they would be a good thing.
> 
> An overwhelming majority believes that everyone should be able to get these treatments if they want them (79%). But two-thirds think that in practice, only wealthy people would have access to the treatments... About two-thirds agree that longer life expectancies would strain our natural resources and that medical scientists would offer the treatment before they fully understood how it affects people's health. And about six-in-ten (58%) say these treatments would be fundamentally unnatural.
> 
> About two-thirds of adults (63%) say medical advances that prolong life are generally good because they allow people to live longer, while about three-in-ten (32%) say medical advances are bad because they interfere with the natural cycle of life.
> 
> The survey contains a number of null findings that may be surprising. It turns out, for example, that many standard measures of religious beliefs and practices, including belief in God and frequency of attendance at religious services, are related to views on radical life extension only weakly, if at all. Nor is there a strong relationship in the survey between the gender, education or political party identification of respondents and what they say about longer human life spans... At least one question that deals directly with death, however, is correlated with views on radical life extension. People who oppose the death penalty are more inclined to say that longer life spans would be good for society.

I also find the demographic splits on page 3 to be surprising. On the question of whether treatments to extend life by decades would be a good thing for society, whites are significantly less likely to agree: 36% of whites agree whereas 48% of Hispanics and 56% of blacks do. There is a negative correlation with age (48% of adults 18-29, 46% of adults 30-49, 37% of adults 50-64, 31% of adults 65 and older) and with income (47% of those earning 30k and less, 42% of those earning from 30k-75k, and 39% of those earning 75k+). The income result in particular surprises me, as my intuition was that people with a higher quality of life would be significantly more pro-life extension.