Harley-ml/lesswrong huggingface dataset:
# Identity crafting

I spend a LOT of time on what I'll call "identity crafting".  It's probably my most insidious procrastination tactic--far worse than, say, Facebook or Reddit.

What do I mean by "identity crafting"?  Here are some examples:

*   Brainstorming areas of my life where I want to improve (e.g. social skills, sleep habits)
*   Searching for new hobbies to start (e.g. snowboarding, guitar)
*   Making a "character sheet" for myself, complete with a huge list of "badass skills" that I'd want to learn (e.g. martial arts, lock picking)
*   Creating and revising my "four-year plan", i.e. schedule of university courses (at my university I had a lot of flexibility in which courses to take each term)
*   Finding books that I "ought to read" (bonus points if the list includes "The Art of Computer Programming") and movies that I "ought to watch"
*   Looking up variants on "renaissance man" (e.g. "Four Arts of the Chinese Scholar"), and imagining how I could become one

In other words, "identity crafting" is some combination of making lists and daydreaming.  And since the vast majority of the "identities" that I "craft" never become reality, I should really say that "identity crafting" is some combination of making lists and self-aggrandizing delusion.

What's so bad about this?  Besides the obvious waste of time, this gives me a false sense of accomplishment and productivity--I often feel as though the "identity" that I "crafted" were already real, and I often feel as though I've already done enough for the day (week, month, year).  Thus in the short term, this is a great way to ensure that I don't do any "actual work", and in the long term, this is a great way to become a poser with an epically inflated opinion of myself.

So... does anyone else do this?
# Overpaying for happiness?

Happy New Year, everyone!

In the past few months I've been thinking several thoughts that all seem to point in the same direction:

1) People who live in developed Western countries usually make and spend much more money than people in poorer countries, but aren't that much happier. It feels like we're overpaying for happiness, spending too much money to get a single bit of enjoyment.

2) When you get enjoyment from something, the association between "that thing" and "pleasure" in your mind gets stronger, but at the same time it becomes less sensitive and requires more stimulus. For example if you like sweet food, you can get into a cycle of eating more and more food that's sweeter and sweeter. But the guy next door, who's eating much less and periodically fasting to keep the association fresh, is actually getting more pleasure from food than you are! The same thing happens when you learn to deeply appreciate certain kinds of art, and then notice that the folks who enjoy "low" art are visibly having more fun.

3) People sometimes get unrealistic dreams and endlessly chase them, like trying to "make it big" in writing or sports, because they randomly got rewarded for it at an early age. I wrote a [post](/lw/ksy/follow_your_dreams_as_a_case_study_in_incorrect/) about that.

I'm not offering any easy answers here. But it seems like too many people get locked in loops where they spend more and more effort to get less and less happiness. The most obvious examples are drug addiction and video gaming, but also "one-itis" in dating, overeating, being a connoisseur of anything, striving for popular success, all these things follow the same pattern. You're just chasing after some Skinner-box thing that you think you "love", but it doesn't love you back.

Sooo... if you like eating, give yourself a break every once in a while? If you like comfort, maybe get a cold shower sometimes? Might be a good idea to make yourself the kind of person that can get happiness cheaply.

Sorry if this post is not up to LW standards, I typed it really quickly as it came to my mind.
# Understanding Who You Really Are

> Here are 14 ways in which you reveal who you really are. If you’re brave enough, or if you dare, aim to share who you really are, little by little, everyday, with those you trust.
> 
> \- A typical 'Who You Really Are' article on [Lifehack](http://www.lifehack.org/articles/communication/14-ways-that-reveal-who-you-really-are.html)

Take a minute to consider the following questions.

_Who are you?_  
_Who are you, really?  
Who do you really think you are inside?_

It took me a full year to find the answer to these.  The answer was that these questions, when posed as philosophical dilemmas, were bullshit.  This post is not about ‘understanding who you really are’. It's about understanding, 'who you really are'.

“Who are you” is a question that sounds grandiose.  It’s hard to come up with a philosophically solid answer, and this makes it seem interesting.  It is not interesting.  It just lacks context.

What would you say if you were asked “who are you?” by the police?  By a doctor? By a relative? By a potential boss? By a space alien?

You should say different things, because these people would be using the same words to mean different things. 

What they really want is information about you that is of decision relevance to them.   A police cares where you are from. The doctor cares how old you are. A relative cares about who you are related to. A boss cares what skills you have. A space alien cares about your number of eyes and hands.  “Who are you?” really means, “given your understanding of my position, what simple information about yourself do you think is useful to me?”

So when a young philosopher follows up your response with, “no really, who _**are**_ you?”, you should respond with asking, “what in particular would you like to know?”

Some may respond to this saying that there does exist a _true_ self. A _real_ self.  This is what the phrase should really mean, and this is what I personally spent a year pondering.

But first, the very idea of there being a _true_ _self_ is specific to a set of religions and philosophies that you may not believe in.  If you’re a empirical atheist, you shouldn’t.  David Hume fought the notion of an inner self 250 years ago. \[1\] Derek Parfit fought it more concretely in the last 30 years. \[2\]

Second, even if you do ascribe to a belief system where there is some sort of _true_ self, this would not give you a clear way to describe it.  Should you say that you are a Capricorn inside?  Or that a small fraction of your brain believes in Libertarianism?  Or that you possess soul #988334?

Of course not.  The question of “who are you?” is wrongly worded, and the one of “who are you, really?” should be placed on hold until the questioner can figure out what they are actually trying to ask.  

\[1\] [David Hume's view on Personal Identity](https://askaphilosopher.wordpress.com/2013/01/22/david-humes-view-on-personal-identity/), Skinner (2013)

\[2\] Reasons and Persons, Parfit (1986)
# Non-obvious skills with highly measurable progress?

A lot of my significant personal improvement happened as a result of highly measurable progress and tight feedback loops.  For example:

*   Project Euler
*   Go (the game has a very accurate ranking system)
*   Strength training

However, these are somewhat obvious examples, and I feel like it would be a waste not to push such a useful improvement mechanism as far as possible.

  

What are some non-obvious examples of skills with highly measurable progress and tight feedback loops?
# Negative polyamory outcomes?

Related article: [Polyhacking](/lw/79x/polyhacking/)

Note: This article was posted [earlier](/r/discussion/lw/lhn/negative_polyamory_outcomes/) for less than a day but accidentally deleted.

Although polyamory isn't one of the "official" topics of LW interest (human cognition, AI, probability, etc...), this is the only community I'm part of where I expect a sufficiently high number of members to have experience with it to give useful feedback. 

If you go looking for advice or articles about polyamory on the internet, you mostly get stuff written by polyamorists that are happy with their decisions. Is this selection bias? Where are the people whose relationships (or social lives, out anything) got damaged or ruined by experimenting with Consensual Non-Monogamy?

I'm posting this hoping for feedback, negative AND positive, on experiences with polyamory. I considered putting this in an Open Thread, but it occurred to me that many other LW readers might be interested in whether polyamory has drawbacks they need to be aware of. If you have experience with CNM (including first-hand witnessing, which has the added bonus of not requiring you to out yourself while still participating in the dialogue), please comment with your overall impression and as much detail as you would like to include (I am also putting my experiences there rather than in this post). If you've seen multiple poly relationships, multiple comments would make tallying slightly easier. I will try to upvote people who feed me data, a la LW surveys. If there are sufficient comments, I will periodically go through them and post a rough ratio of good to bad experiences at the bottom of this article.

PSA: The Username account is available for use by any who wish to remain anonymous. The password is left as an exercise for the reader. Hat tip... Username.
# Low Hanging fruit for buying a better life

What can I purchase with $100 that will be the best thing I can buy to make my life better?

I've decided to budget some regular money to improving my life each month. I'd like to start with low hanging fruit for obvious reasons - but when I sat down to think of improvements, I found myself thinking of the same old things I'd already been planning to do anyway... and I'd like out of that rut.  
  
**Constraints/more info:**

1.  be concrete. I know - "spend money on experiences" is a good idea - but what experiences are the best option to purchase \*first\*
2.  "better" is deliberately left vague - choose how you would define it, so that I'm not constrained just by ways of "being better" that I'd have thought of myself.
3.  please assume that I have all my basic needs met (eg food, clothing, shelter) and that I have budgeted separately for things like investing for my financial future and for charity.
4.  apart from the above, assume nothing - Especially don't try and tailor solutions to anything you might know and/or guess about me specifically, because I think this would be a useful resource for others who might have just begun.
5.  don't constrain yourself to exactly $100 - I could buy 2-3 things for that, or I could save up over a couple of months and buy something more expensive... I picked $100 because it's a round number and easy to imagine.
6.  it's ok to add "dumb" things - they can help spur great ideas, or just get rid of an elephant in the room.
7.  try thinking of your top-ten before reading any comments, in order not to bias your initial thinking. Then come back and add ten more once you've been inspired by what everyone else came up with.

**Background:**

This is a question I recently posed to my local Less Wrong group and we came up with a few good ideas, so I thought I'd share the discussion with the wider community and see what we can come up with. I'll add the list we came up with later on in the comments...

It'd be great to have a repository of low-hanging fruit for things that can be solved with (relatively affordable) amounts of money. I'd personally like to go through the list - look at candidates that sound like they'd be really useful to me and then make a prioritised list of what to work on first.
# Exams and Overfitting

When I hear something like _"What's going to be on the exam?"_, part of me gets indignant.  WHAT?!?!  You're defeating the whole point of the exam!  You're committing the Deadly Sin of Overfitting!

Let me step back and explain my view of exams.

When I take a class, my goal is to learn the material.  Exams are a way to answer the question, "How well did I learn the material?"\[1\].  But exams are only a few hours long, so it's unfeasible to have questions on all of the material.  To deal with this time constraint, an exam takes a random sample of the material and gives me a "statistical" rather than "perfect" answer to the question, "How well did I learn the material?"

If I know in advance what topics will be covered on the exam, and if I then prepare for the exam by learning only those topics, then _I am screwing up this whole process_.  By doing very well on the exam, I get the information, _"Congratulations!  You learned the material covered on the exam very well."_  But who knows how well I learned the material covered in class as a whole?  This is a textbook case of [overfitting](https://en.wikipedia.org/wiki/Overfitting).

To be clear, I don't necessarily lose respect for someone who asks, _"What's going to be on the exam?"_.  I understand that different people have different priorities\[2\], and that's fine by me.  But if you're taking a class because you truly want to learn the material, in spite of any sacrifices that you might have to make to do so\[3\], then I'd like to encourage you not to "study for the test".  I'd like to encourage you not to overfit.

* * *

\[1\] When I say "learned", I mean in the "[Feynman](http://v.cx/2010/04/feynman-brazil-education)" sense, not in the "[teacher's password](/lw/iq/guessing_the_teachers_password/)" sense.  I believe that a necessary (but not sufficient) condition for an exam to check for this kind of learning is to have problems that I've never seen before.

\[2\] Someone might care much more about getting into medical school than, say, mastering classical mechanics.  I respect that choice, and I acknowledge that someone might be in a system where getting a good grade in physics is required for getting into medical school, even though mastering classical mechanics isn't required for becoming a good doctor.

\[3\] There were a few terms when I felt like I did a really good job of learning the material (conveniently, I also got really good grades during these terms).  But for these terms, one (or both) of the following would happen:

*   I would take a huge hit in social status, because I was taking barely more than the minimum courseload.  At my university, there was a lot of social pressure to always take the maximum courseload (or petition to exceed the maximum courseload), and still participate in lots of extracurricular activities.
*   My girlfriend at the time would break up with me because of all the time I was spending on my coursework (and not with her).
# Programming-like activities?

Programming is quite a remarkable activity:

*   It has an extremely low barrier to entry
    *   You don't need expensive equipment
    *   You don't need to be in a particular location
    *   You don't need special credentials
    *   You can finding information / resources just by opening the internet
    *   You can learn it / do it independently
*   It gives you rapid feedback (which can lead to rapid growth)
*   It gives you frequent rewards (which gives a huge boost in motivation)
*   It's objective and unforgiving (this is a good thing, because it teaches you how to confront reality)
*   It's intellectually stimulating
*   It's useful in the real world
    *   Corollary: you can make money or even build a career out of it
*   It's badass (or are you telling me that [Hackers](https://en.wikipedia.org/wiki/Hackers_(film)) WASN'T your favorite movie of all time?)

**What are some other "programming-like" activities?**

  

I mean this in the sense of "activities that also satisfy the above criteria", but suggestions don't have to satisfy ALL of the criteria.  Here are some of the first ideas that come to mind when I try to answer the question myself:

*   Electronics (but this is basically still programming)
*   Math (lacks "rapid feedback" and "frequent rewards"; "useful in the real world" is also questionable)
*   Go, poker, video games (usually lacks "useful in the real world", sometimes lacks "badass")
*   Juggling, poi (lacks "intellectually stimulating" and "useful in the real world")

However, I've already exhausted my creativity and I'm hoping to go much deeper than this.  Thoughts?
# 2015 Repository Reruns - Boring Advice Repository

This is the first post of the 2015 repository rerun, which [appears to be a good idea](/r/discussion/lw/lht/open_thread_jan_511_2015/btjs). The motivation for this rerun is that while the 12 [repositories](/lw/i64/repository_repository/) (go look them up, they're awesome!) exist and people might look them up, few new comments are posted there. In effect, there might be useful stuff that should go in those repositories, but is never posted due to low expected value and no feedback. With the rerun, attention is shifted to one topic per month. This might allow us to have a lively discussion on the topic at hand and gather new content for the repository.

* * *

The first repository to be rerun is the [Boring Advice Repository](/lw/gx5/boring_advice_repository/), because of... on a whim.

Enter original motivation (by **[Qiaochu_Yuan](/user/Qiaochu_Yuan/)**):

> This is an extension of a comment I made that I can't find and also a request for examples. It seems plausible that, when giving advice, many people optimize for deepness or punchiness of the advice rather than for actual practical value. There may be good reasons to do this - e.g. advice that sounds deep or punchy might be more likely to be listened to - but as a corollary, there could be valuable advice that people generally don't give because it doesn't sound deep or punchy. Let's call this **boring advice**. 
> 
> An example that's been discussed on LW several times is "make [checklists](/lw/8vm/the_rationalists_checklist/)." Checklists are [great](/lw/cnr/share_your_checklists/). We should [totally make checklists](/lw/fc3/checklist_of_rationality_habits/). But "make checklists" is not a deep or punchy thing to say. Other examples include "google things" and "exercise."

The Boring Advice Repository is filled with lots of diverse advice, I've summarized some of it in a comment below.

* * *

So what should go here? To go with **[Qiaochu_Yuan](/user/Qiaochu_Yuan/)** again (adding emphasis):

> \[...\] **Post other examples of boring advice. If you can, provide evidence and/or a plausible argument that your boring advice actually is useful, but** \[...\] **err on the side of boring but not necessarily useful** \[...\]**. **
> 
> **Upvotes on advice posted in this thread should be based on your estimate of the usefulness of the advice;** \[...\] **do not vote up advice just because it sounds deep or punchy.**

I don't know if you should post new advice here or in the original repository. Perhaps search the old repository with ctrl+f (when on windows) and if you don't get results, post it here.
# Existential Risk and Existential Hope: Definitions

I'm pleased to announce [Existential Risk and Existential Hope: Definitions](http://www.fhi.ox.ac.uk/Existential-risk-and-existential-hope.pdf), a short new FHI technical report.

Abstract:

> We look at the strengths and weaknesses of two existing definitions of existential risk, and suggest a new definition based on expected value. This leads to a parallel concept: ‘existential hope’, the chance of something extremely good happening.

  

I think MIRI and CSER may be naturally understood as organisations trying to reduce existential risk and increase existential hope respectively (although if MIRI is aiming to build a safe AI this is also seeking to increase existential hope). What other world states could we aim for that increase existential hope?
# Research Priorities for Artificial Intelligence: An Open Letter

The Future of Life Institute has published their document [Research priorities for robust and beneficial artificial intelligence](http://futureoflife.org/static/data/documents/research_priorities.pdf) and written an [open letter](http://futureoflife.org/misc/open_letter) for people to sign indicating their support.

> Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to research how to maximize these benefits while avoiding potential pitfalls. This document gives numerous examples (which should by no means be construed as an exhaustive list) of such worthwhile research aimed at ensuring that AI remains robust and beneficial.
# [meta] New LW moderator: NancyLebovitz

During the following months my time and attention will be heavily occupied by some personal stuff, so I will be unable to function as a LW moderator. The new LW moderator is... [NancyLebovitz](/user/NancyLebovitz/overview/)!

From today, please direct all your complaints and investigation requests to Nancy. Please _not_ everyone during the first week. That can be a bit frightening for a new moderator.

There are a few old requests I haven't completed yet. I will try to close everything during the following days, but if I don't do it till the end of January, then I will forward the unfinished cases to Nancy, too.

Long live the new moderator!
# I'm the new moderator

Viliam Bur [made the announcement](/lw/lje/meta_new_lw_moderator_nancylebovitz/) in Main, but not everyone checks main, so I'm repeating it here.

> During the following months my time and attention will be heavily occupied by some personal stuff, so I will be unable to function as a LW moderator. The new LW moderator is...[NancyLebovitz](/user/NancyLebovitz/overview/)!
> 
> From today, please direct all your complaints and investigation requests to Nancy. Please _not_ everyone during the first week. That can be a bit frightening for a new moderator.
> 
> There are a few old requests I haven't completed yet. I will try to close everything during the following days, but if I don't do it till the end of January, then I will forward the unfinished cases to Nancy, too.
> 
> Long live the new moderator!
# Elon Musk donates $10M to the Future of Life Institute to keep AI beneficial 

We are delighted to report that technology inventor Elon Musk, creator of Tesla and SpaceX, has decided to donate $10M to the Future of Life Institute to run a global research program aimed at keeping AI beneficial to humanity. 

There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. A long list of leading AI-researchers have signed an [open letter](http://futureoflife.org/misc/open_letter) calling for research aimed at ensuring that AI systems are robust and beneficial, doing what we want them to do. Musk's donation aims to support precisely this type of research: _"Here are all these leading AI researchers saying that AI safety is important"_, says Elon Musk. _"I agree with them, so I'm today committing $10M to support research aimed at keeping AI beneficial for humanity." _

\[...\] The $10M program will be administered by the Future of Life Institute, a non-profit organization whose scientific advisory board includes AI-researchers Stuart Russell and Francesca Rossi. \[...\]

The research supported by the program will be carried out around the globe via an open grants competition, through an application portal at [http://futureoflife.org](http://futureoflife.org) that will open by Thursday January 22. The plan is to award the majority of the grant funds to AI researchers, and the remainder to AI-related research involving other fields such as economics, law, ethics and policy  (a detailed list of examples can be found [here](http://futureoflife.org/static/data/documents/research_priorities.pdf) \[PDF\]). _"Anybody can send in a grant proposal, and the best ideas will win regardless of whether they come from academia, industry or elsewhere"_, says FLI co-founder Viktoriya Krakovna. 

\[...\] Along with research grants, the program will also include meetings and outreach programs aimed at bringing together academic AI researchers, industry AI developers and other key constituents to continue exploring how to maximize the societal benefits of AI; one such meeting was held in Puerto Rico last week with many of the open-letter signatories. 

[Elon Musk donates $10M to keep AI beneficial, Future of Life Institute, Thursday January 15, 2015](http://futureoflife.org/misc/ai)
# An example and discussion of extension neglect

I recently used an automatic tracker to learn how I was spending my time online. I learned that my perceptions were systemically biased: I spend less time than I thought on purely non-productive sites, and far more time on sites that are quasi-productive.  
  
For example, I felt that I was spending too much time reading the news, but I learned that I spend hardly time doing so. I didn't feel that I was spending much time reading Hacker News, but I was spending a huge amount of time there!  
  
Is this a specific case of a more general error?  
  
A general framing: "Paying too much attention to the grouping whose items have the most extreme quality, when the value of focusing on this grouping is eclipsed by the value of focusing on a larger grouping of less extreme items".

So in this case, once I had formed the desire to be more productive, I overestimated how much potential productive time I could gain by focusing on those sites that I felt were maximally non-productive, and underestimated the potential of focusing on marginally more productive sites.

In pseudo-technical terms: We think about items in groups. But then we think of the total value of a group as being closer to _average_value_ than to _average\_value * size\_of_group_.

This falls under the category of [Extension Neglect](http://en.wikipedia.org/wiki/Extension_neglect), which includes errors caused by ignoring the size of a set. Other patterns in this category are:

*   Base rate neglect: Inferring the category of an item as if all categories were the same size.
*   The peak-end rule: Giving the value of the ordered group as a function of _max_value_ and _end_value_.
*   Not knowing how set size interacts with randomness.

For the error given above, some specific examples might be:

*   Health: Focusing too much on eating desert at your favorite restaurant; and not enough on eating pizza three times a week.
*   Love: Fights and romantic moments; daily interaction.
*   Stress: Public speaking; commuting
*   Ethics: Improbable dilemmas; reducing suffering (or doing anything externally visible)
*   Crime: Serial killers; domestic violence
# Slides online from "The Future of AI: Opportunities and Challenges"

In the first weekend of this year, the Future of Life institute hosted a landmark conference in Puerto Rico: "[The Future of AI: Opportunities and Challenges](http://futureoflife.org/misc/ai_conference)". The conference was unusual in that it was not made public until it was over, and the discussions were under [Chatham House rules](https://en.wikipedia.org/wiki/Chatham_House_Rule). The [slides from the conference](http://futureoflife.org/misc/ai_conference) are now available. The [list of attenders](http://futureoflife.org/PDF/attendees.pdf) includes a great many famous names as well as lots of names familiar to those of us on Less Wrong: Elon Musk, Sam Harris, Margaret Boden, Thomas Dietterich, all three DeepMind founders, and many more.

This is shaping up to be another extraordinary year for AI risk concerns going mainstream!
# ... And Everyone Loses Their Minds

Chris Nolan's Joker is a very clever guy, almost Monroesque in his ability to identify hypocrisy and inconsistency. [One of his most interesting scenes](https://www.youtube.com/watch?v=h-gf29nuYYA) in the film has him point out how people estimate horrible things differently depending on whether they're part of what's "normal", what's "expected", rather than on how inherently horrifying they are, or how many people are involved.

Soon [people extrapolated this observation](http://knowyourmeme.com/memes/everyone-loses-their-minds) to other such apparent inconsistencies in human judgment, where a behaviour that once was acceptable, with a simple tweak or change in context, becomes the subject of a much more serious reaction.

I think there's rationalist merit in giving these inconsistencies a serious look. I intuit that there's some sort of underlying pattern to them, something that makes psychological sense, in the roundabout way that most irrational things do. I think that much good could come out of figuring out what that root cause is, and how to predict this effect and manage it.

Phenomena that come to mind, are, for instance, from an Effective Altruism point of view, the expenses incurred in counter-terrorism (including some wars that were very expensive in treasure and lives), and the number of lives said expenses save, compared with the number of lives that could be saved by spending that same amount into improving road safety, increasing public helathcare expense where it would do the most good, building better lightning rods ([in the USA you're four times more likely to be struck by thunder than by terrorists](http://reason.com/archives/2011/09/06/how-scared-of-terrorism-should)), or legalizing drugs.

What do y'all think? Why do people have their priorities all jumbled-up? How can we predict these effects? How can we work around them?
# [Link] An argument on colds 

[Source](https://www.facebook.com/robert.wiblin/posts/666376266505?pnref=story).

> It's illegal to work around food when showing symptoms of contagious diseases. Why not the same for everyone else? Each person who gets a cold infects one other person on average. We could probably cut infection rates and the frequency of colds in half if sick people didn't come in to work.
> 
> And if we want better biosecurity, why not also require people to be able to reschedule flights if a doctor certifies they have a contagious disease?
> 
> Due to the 'externalities', the case seems very compelling.

Moving my commentary to a separate comment, so as to disambiguate votes on my [commentary](/r/discussion/lw/lk5/link_an_argument_on_colds/bvkv) and the original argument.
# Purchasing research effectively open thread

Many of the biggest historical [success stories](http://blog.givewell.org/2012/03/01/philanthropys-success-stories/) in philanthropy have come in the form of funding for academic research.  This suggests that the topic of how to purchase such research well should be of interest to [effective altruists](/lw/3gj/efficient_charity_do_unto_others/).  Less Wrong [survey results](/lw/lhg/2014_survey_results/) indicate that a nontrivial fraction of LW has firsthand experience with the academic research environment.  Inspired by the recent [Elon Musk donation announcement](/r/discussion/lw/ljo/elon_musk_donates_10m_to_the_future_of_life/), this is a thread for discussion of effectively using money to enable important, useful research.  Feel free to [brainstorm your own questions and ideas](/lw/bjm/exploring_the_idea_space_efficiently/) before reading what's written in the thread.
# Bill Gates: problem of strong AI with conflicting goals "very worthy of study and time"

[Steven Levy](https://en.wikipedia.org/wiki/Steven_Levy): Let me ask an unrelated question about the raging debate over whether [artificial intelligence poses a threat to society](http://www.bbc.co.uk/news/technology-30777834), or even the survival of humanity. Where do you stand?

[Bill Gates](https://en.wikipedia.org/wiki/Bill_Gates): I think it’s definitely important to worry about. There are two AI threats that are worth distinguishing. One is that AI does enough labor substitution fast enough to change work policies, or \[affect\] the creation of new jobs that humans are uniquely adapted to — the jobs that give you a sense of purpose and worth. We haven’t run into that yet. I don’t think it’s a dramatic problem in the next ten years but if you take the next 20 to 30 it could be. Then there’s the longer-term problem of so-called strong AI, where it controls resources, so its goals are somehow conflicting with the goals of human systems. Both of those things are very worthy of study and time. I am certainly not in the camp that believes we ought to stop things or slow things down because of that. But you can definitely put me more in the Elon Musk, Bill Joy camp than, let’s say, the Google camp on that one.

"[Bill Gates on Mobile Banking, Connecting the World and AI](https://medium.com/backchannel/bill-gates-on-mobile-banking-connecting-the-world-and-ai-937f35c8a110)", Medium, 2015-01-21
# New, Brief Popular-Level Introduction to AI Risks and Superintelligence

The [very](https://www.facebook.com/waitbutwhy) [popular](http://www.ozy.com/good-sht/wait-but-why-making-nerdiness-viral/3161) blog Wait But Why [has published the first part](http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html) of a two-part explanation/summary of AI risks and superintelligence, and it looks like the second part will be focused on Friendly AI. I found it very clear, reasonably thorough and appropriately urgent without signaling paranoia or fringe-ness. It may be a good article to share with interested friends.

**Update**: Part 2 is now [here](http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html).
# Donate to Keep Charity Science Running

Charity Science is looking for $35,000 to fund our 2015 operations. We fundraise for GiveWell-recommended charities, and over 2014 we moved over $150,000 to them that wouldn’t have been given otherwise: that’s $9 for every $1 we spent. We can’t do this work without your support, so please consider making a donation to us - however small, it will be appreciated. Donate now and you’ll also be matched by Matt Wage.

The donations pages below list other reasons to donate to us, which include:

*   Our costs are extremely low: the $35,000 CAD pays for three to four full-time staff.

*   We experiment with many different forms of fundraising and record detailed information on how these experiments go, so funding us lets the whole EA community learn about their prospects.

*   We carefully track how much money each experiment raises, subtract money which would have been given anyway, and shut down experiments that don’t work.

*   Our fundraising still has many opportunities to continue to scale as we try new ideas we haven’t tested yet.

There’s much more information, including our full budget and what we’d do if we raised over $35,000, in the linked document, and we’d be happy to answer any questions. Thank you in advance for your consideration.

[Donate in American dollars](http://bit.ly/15CmrSv) 

[Donate in British pounds](http://bit.ly/1z2h7SU) 

[Donate in Canadian dollars](http://bit.ly/18kDmuS)
# CFAR fundraiser far from filled; 4 days remaining

We're 4 days from the end of our matching [fundraiser](/lw/lfg/cfar_in_2014_continuing_to_climb_out_of_the/), and still only about [1/3rd](http://rationality.org/donate/) of the way to our target (and to the point where pledged funds would cease being matched).

If you'd like to support the growth of rationality in the world, do please consider donating, or asking me about any questions/etc. you may have.  I'd love to talk.  I suspect funds donated to CFAR between now and Jan 31 are quite high-impact.

As a random bonus, I promise that if we meet the $120k matching challenge, I'll post at least two posts with some never-before-shared (on here) rationality techniques that we've been playing with around CFAR.
# [link] Speed is the New Intelligence

From [Scott Adams Blog](http://blog.dilbert.com/post/109295555116/speed-is-the-new-intelligence)

The article really is about speeding up government, but the key point is speed as a component of smart: 

> A smart friend told me recently that speed is the new intelligence, at least for some types of technology jobs. If you are hiring an interface designer, for example, the one that can generate and test several designs gets you further than the “genius” who takes months to produce the first design to test. When you can easily test alternatives, the ability to quickly generate new things to test is a substitute for intelligence.

This shifts the focus from the ability to grasp and think through very complex topics (includes good working memory and memory recall in general) to the ability new topics quickly (includes quick learning and unlearning, creativity).

> Smart people in the technology world no long believe they can think their way to success. Now the smart folks try whatever plan looks promising, test it, tweak it, and reiterate. In that environment, speed matters more than intelligence because no one has the psychic ability to pick a winner in advance. All you can do is try things that make sense and see what happens. Obviously this is easier to do when your product is software based.

This also changes the type of grit needed. The grit to push through a long topic versus the grit try lots of new things and to learn from failures.
# Is there a rationalist skill tree yet?

A while back I came across a delightful [web developer skill tree](http://www.dungeonsanddevelopers.com/), and I was wondering if technical rationality has gotten to the point where someone could make one of these for an aspiring rationalist.

I think seeing a clear progression from beginning skills to advanced ones laid out graphically helps those starting on the path conceptualize the process.
# How to save (a lot of) money on flying

I was going to wait to post this for _reasons_, but realized that was pretty dumb when the difference of a few weeks could literally save people hundreds, if not thousands of collective dollars.

If you fly regularly (or at all), you may already know about this method of saving money.  The method is quite simple: instead of buying a round-trip ticket from the airline or reseller, you hunt down much cheaper one-way flights with layovers at your destination and/or your point of origin.  [Skiplagged](http://www.skiplagged.com) is a service that will do this automatically for you, and has been in the news recently because the creator was sued by United Airlines and Orbitz.  While Skiplagged will allow you to click-through to purchase the one-way ticket _to_ your destination, they have broken or disabled the functionality of the redirect to the one-way ticket _back_ (possibly in order to raise more funds for their legal defense).  However, finding the return flight manually is fairly easy as the provide all the information to filter for it on other websites (time, airline, etc).  I personally have benefited from this - I am flying to Texas from Southern California soon, and instead of a round-trip ticket which would cost me about $450, I spent ~$180 on two one-way tickets (with the return flight being the "layover" at my point-of-origin).  These are, perhaps, larger than usual savings; I think 20-25% is more common, but even then it's a fairly significant amount of money.

Relevant warnings by gwillen:

> You should be EXTREMELY CAREFUL when using this strategy. It is, at a minimum, against airline policy.
> 
> If you have any kind of airline status or membership, and you do this too often, they will cancel it. If you try to do this on a round-trip ticket, they will cancel your return. If the airlines have any means of making your life difficult available to them, they WILL use it.
> 
> Obviously you also cannot check bags when using this strategy, since they will go to the wrong place (your ostensible, rather than your actual, destination.) This also means that if you have an overhead-sized carryon, and you board late and are forced to check it, your bag will NOT make it to your intended destination; it will go to the final destination marked on your ticket. If you try to argue about this, you run the risk of getting your ticket cancelled altogether, since you're violating airline policies by using a ticket in this way.

Additionally, you should do all of your airline/hotel/etc shopping using whatever private browsing mode your web browser has.  This will often let you purchase the exact same product for a cheaper price.

That is all.
# Intrapersonal comparisons: you might be doing it wrong.

[Cross-posted](http://rulerstothesky.com/2015/02/03/motivation/)

Nothing weighty or profound today, but I noticed a failure mode in myself which other people might plausibly suffer from so I thought I'd share it.

Basically, I noticed that sometimes when I discovered a more effective way of doing something -- say, going from conventional flashcards to Anki -- I found myself getting _discouraged_.

I realized that it was because each time I found such a technique, I automatically compared my current self to a version of me that had had access to the technique the whole time. Realizing that I wasn't as far along as I could've been resulted in a net loss of motivation. 

Now, I deliberately compare two future versions of myself, one armed with the technique I just discovered and one without. Seeing how much farther along I _will_ be results in a net gain of motivation.

A variant of this exercise is taking any handicap you might have and wildly exaggerating it. I suffer from mild Carpal Tunnel (or something masquerading as CT) which makes progress in programming slow. When I feel down about this fact I imagine how hard programming would be without hands.

Sometimes I go as far as to plan out what I might do if I woke up tomorrow with a burning desire to program and nothing past my wrists. Well, I'd probably figure out a way to [code by voice](https://www.youtube.com/watch?v=8SkdfdXWYaI) and then practice [mnemonics](http://www.pmemory.com/) because I wouldn't be able to write anything down. Since these solutions exist I can implement one or both of them the moment my carpal tunnel gets bad enough.

With this realization comes a boost in motivation knowing I can go a different direction if required.
# Questions from an imaginary statistical methods exam

Answers to these questions should be expressed numerically, where possible, but no number should be given without a justification for the specific value.

**1.** Suppose that you have mislaid your house keys, something most people have experienced at one time or another. You look in various places for them: where you remember having them last, places you've been recently, places they should be, places they shouldn't be, places they couldn't be, places you've looked already, and so on. Eventually, you find them and stop looking.

Every time you looked somewhere, you were testing a hypothesis about their location. You may have looked in a hundred places before finding them.

As a piece of scientific research to answer the question "where are my keys?", this procedure has massive methodological flaws. You tested a hundred hypotheses before finding one that the data supported, ignoring every failed hypothesis. You really wanted each of these hypotheses in turn to be true, and made no attempt to avoid bias. You stopped collecting data the moment a hypothesis was confirmed. When you were running out of ideas to test, you frantically thought up some more. You repeated some failed experiments in the hope of getting a different result. Multiple hypotheses, file drawer effect, motivated cognition, motivated stopping, researcher degrees of freedom, remining of old data: there is hardly a methodological sin you have not committed.

**(a) **Should these considerations modify your confidence or anyone else's that you have in fact found your keys? If not, why not, and if so, what correction is required?

**(b)** Should these considerations affect your subsequent decisions (e.g. to go out, locking the door behind you)?

**2.** You have a lottery ticket. (Of course, you are far too sensible to ever buy such a thing, but nevertheless suppose that you have one. Maybe it was an unexpected free gift with your groceries.) The lottery is to be drawn later that day, the results available from a web site whose brief URL is printed on the ticket. You calculate a chance of about 1 in 100 million of a prize worth getting excited about.

**(a)** Once the lottery results are out, do you check your ticket? Why, or why not?

**(b)** Suppose that you do, and it appears that you have won a very large sum of money. But you remember that the prior chance of this happening was 1 in 100 million. How confident are you at this point that you have won? What alternative hypotheses are also raised to your attention by the experience of observing the coincidence of the numbers on your ticket and the numbers on the lottery web site?

**(c)** Suppose that you go through the steps of contacting the lottery organisers to make a claim, having them verify the ticket, collecting the prize, seeing your own bank confirm the deposit, and using the money in whatever way you think best. At what point, if any, do you become confident that you really did win the lottery? If never, what alternative hypotheses are you still seriously entertaining, to the extent of acting differently on account of them?
# Don't estimate your creative intelligence by your critical intelligence

When I criticize, I'm a genius. I can go through a book of highly-referenced scientific articles and find errors in each of them. Boy, I feel smart. How are these famous people so dumb?

But when I write, I suddenly become stupid. I sometimes spend half a day writing something and then realize at the end, or worse, after posting, that what it says simplifies to something trivial, or that I've made several unsupported assumptions, or claimed things I didn't really know were true. Or I post something, then have to go back every ten minutes to fix some point that I realize is not quite right, sometimes to the point where the whole thing falls apart.

If someone writes an article or expresses an idea that you find mistakes in, that doesn't make you smarter than that person. If you create an equally-ambitious article or idea that no one else finds mistakes in, then you can start congratulating yourself.
# MIRI's technical agenda: an annotated bibliography, and other updates

This week, I'm pleased to present [Aligning Superintelligence with Human Interests: An Annotated Bibliography](http://intelligence.org/files/AnnotatedBibliography.pdf). This annotated bibliography complements our [technical agenda](http://intelligence.org/files/TechnicalAgenda.pdf) and the six supporting papers that I've released over the past few months. Once you've read the supporting papers, this annotated bibliography will help you figure out what to read next on any given topic, in order to get to the cutting edge.

This annotated bibliography concludes my series of updates on MIRI's technical agenda. To review, I've presented a series of eight papers which sketch out MIRI's research strategy and overview a number of active research areas. Those papers are:

*   **[Aligning Superintelligence with Human Interests: A Technical Research Agenda](/lw/lfc/miris_technical_research_agenda/)**
*   [Formalizing Two Problems of Realistic World Models](/lw/lkk/formalizing_two_problems_of_realistic_world_models/)
*   [Toward Idealized Decision Theory](/lw/lef/new_paper_from_miri_toward_idealized_decision/)
*   [Questions of Reasoning Under Logical Uncertainty](/lw/lgd/questions_of_reasoning_under_logical_uncertainty/)
*   [Vingean Reflection: Reliable Reasoning for Self-Modifying Agents](/lw/ljp/vingean_reflection_reliable_reasoning_for/)
*   [Corrigibility](/lw/l53/introducing_corrigibility_an_fai_research_subfield/)
*   [The Value Learning Problem](/lw/lmn/the_value_learning_problem/)
*   Aligning Superintelligence with Human Interests: An Annotated Bibliography

The entire suite can be found in one place on our new [technical agenda page](http://intelligence.org/technical-agenda/). You're encouraged to link people to that page if they're wondering what sort of technical research MIRI is up to.

  

I've also updated the [research guide](https://intelligence.org/research-guide/) to synchronize it with the technical agenda. The new annotated bibliography serves a similar purpose to the research guide, but where the research guide is geared more towards people starting from scratch, the annotated bibliography is geared towards people who already have the background knowledge and want to know which papers to read in which order without any fluff.

  

Enjoy!