trentmkelly/LessWrong-43k huggingface dataset:
Decision Theory FAQ
by lukeprog — 2013-02-28T14:15:55.090Z

Co-authored with crazy88. Please let us know when you find mistakes, and we'll fix them. Last updated 03-27-2013.

Contents:

1. What is decision theory?

2. Is the rational decision always the right decision?

3. How can I better understand a decision problem?

4. How can I measure an agent's preferences?

4.1. The concept of utility

4.2. Types of utility

5. What do decision theorists mean by "risk," "ignorance," and "uncertainty"?

6. How should I make decisions under ignorance?

6.1. The dominance principle

6.2. Maximin and leximin

6.3. Maximax and optimism-pessimism

6.4. Other decision principles

7. Can decisions under ignorance be transformed into decisions under uncertainty?

8. How should I make decisions under uncertainty?

8.1. The law of large numbers

8.2. The axiomatic approach

8.3. The Von Neumann-Morgenstern utility theorem

8.4. VNM utility theory and rationality

8.5. Objections to VNM-rationality

8.6. Should we accept the VNM axioms?

9. Does axiomatic decision theory offer any action guidance?

10. How does probability theory play a role in decision theory?

10.1. The basics of probability theory

10.2. Bayes theorem for updating probabilities

10.3. How should probabilities be interpreted?

11. What about "Newcomb's problem" and alternative decision algorithms?

11.1. Newcomblike problems and two decision algorithms

11.2. Benchmark theory (BT)

11.3. Timeless decision theory (TDT)

11.4. Decision theory and “winning”

1. What is decision theory?

Decision theory, also known as rational choice theory, concerns the study of preferences, uncertainties, and other issues related to making "optimal" or "rational" choices. It has been discussed by economists, psychologists, philosophers, mathematicians, statisticians, and computer scientists.

We can divide decision theory into three parts (Grant & Zandt 2009; Baron 2008). Normative decision theory studies what an ideal agent (a perfectly rational agent, with infinite computing power, etc.) would choose. Descriptive decision theory studies how non-ideal agents (e.g. humans) actually choose. Prescriptive decision theory studies how non-ideal agents can improve their decision-making (relative to the normative model) despite their imperfections.

For example, one's normative model might be expected utility theory, which says that a rational agent chooses the action with the highest expected utility....
Seize the Maximal Probability Moment
by diegocaleiro — 2013-02-28T11:22:11.463Z

Try and remember 3 or 4 things that you think would be effective hacks for your life but you have not so far implemented. Really, find three.

Probably that was not so hard.

Now think of at which moment in time did you have a maximal probability of having implemented such hacks. Sometimes you had no idea that was the moment. But sometimes you did, like when a friend tells you "I just read this great paper on how people report cartoons being funnier when their face is shaped in a more smiling fashion." and you thought "Great! I may one day implement the algorithm: if studying, force a smile".

You knew you didn't plan to read the article, you knew you trust that friend, and you knew you'd either forget it later, or in any case that from that moment on, the likelihood of you implementing the algorithm would lower.

So my hack of the day is: If you feel you are likely at the maximal probability moment to start a new policy, start immediately.

My friend was telling me about how he went abroad to research: "...so at this place and people there used very strong lights as cognitive enhancement and yadda yadda yadda... (stopped listening for 40s) yadda yadda yadda.... and I wrote a paper on ..." By that time my room had an extra 110W light working.

Just now0 I thought: It was good I installed that light. Why didn't I do the same when I felt like finding a personalized shirt website where the front would be "I Don't want to talk about: [list]" and the back "Pick your topic: [list]" to once and for all stop the gossip and sports ice-breakers?

I didn't seize the maximal probability moment. That's what happened.

Then I noticed that that1 was the maximal probability moment to install in my mind the maximal probability moment algorithm, I did, and that2 was the maximal probability moment of writing this post.

Now if you'll excuse me, I have3 a shirt to buy.
[SEQ RERUN] An African Folktale
by MinibearRex — 2013-02-28T05:43:37.940Z

Today's post, An African Folktale was originally published on 16 February 2009. A summary (taken from the LW wiki):

A story that seems to point to some major cultural differences.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was An Especially Elegant Evpsych Experiment, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Risk-aversion and investment (for altruists)
by paulfchristiano — 2013-02-28T03:44:13.645Z

(Cross-posted from rationalaltruist)

Suppose I hope to use my money to do good some day, but for now I am investing it and aiming to maximize my returns. I face the question: how much risk should I be willing to bear? Should I pursue safe investments, or riskier investments with higher returns?

My knee-jerk response is to say “An altruist should be risk neutral. If you have twice as much money, you can do twice as much good. Sure, there are some diminishing returns, but my own investment is minuscule compared to an entire world full of philanthropists. So in the regime where I am investing, returns are roughly linear.” (I might revise this picture if I thought that I was a very unusual philanthropist, and that few others would invest in the same charitable causes as me—in that case I alone might represent a significant fraction of charitable investment in my causes of choice, so I should expect to personally run into diminishing returns.)

But on closer inspection there is something fishy about this reasoning. I don’t have great data on the responsiveness of charitable giving to market performance, but at the individual level it seems that the elasticity of charitable giving to income is about 1—if I am 50% richer (in one possible world than another), I tend to give 50% more to charity. So in worlds where markets do well, we should expect charities to have more money. If markets (rather, the average investor) do 10% better, I should expect 10% more money to be available for any particular charitable cause, regardless of how many donors it has.

Of course, there is a difference between the market’s performance and my portfolio’s performance—the funds available to a charity depend little on how my portfolio does, just how the average donor’s portfolio does. So what is the relevance of the above result to my own risk aversion? Intuitively it seems that the performance of the market is correlated with the performance of any particular investor, but how tightly?

There is a simple observation / folk theorem that applies to this situation. Suppose two investors’ portfolios are not perfectly correlated. Then (supposing those investors are earning the same returns) each would prefer to trade 1/2 of their portfolio for 1/2 of the others—averaging two imperfectl...
Need some psychology advice
by Kenoubi — 2013-02-27T17:03:38.642Z

I started going out with a fantastic girl a couple of weeks ago. Everything is great, except that whenever I've sent her a text message or email requesting something and haven't received a response yet, I experience significant dysphoric anxiety, fearing that her response will be not just "no" but "no and I don't want to date you any more". This is due to brain chemistry or personal history, take your pick—either seems like a possible explanation to me. But there's certainly no evidence supporting the idea that this is likely to happen, nor is the anxiety helping me prevent it or helping me in any other way.

Does anyone have evidence-based advice, or pointers to same, on dealing with this kind of issue? It is the only splotch on what have otherwise been the best two weeks of my life.
[VIDEO] Harm reduction, hacker psychology
by NancyLebovitz — 2013-02-27T15:50:12.390Z

Hackers As A High Risk Population-- this is an hour-long video by Violet Blue, and it's about what she's been thinking about rather than something with definitive answers.

It starts with an overview of harm reduction-- an approach to public health which begins by accepting that people are going to do the things they want to do, and continues by finding ways to make the things people want to do less dangerous. The most detail is about harm reduction for drug users, sexual minorities, and homeless people.

Violet Blue got interested in the risks to programmers when a start-up entrepreneur got some bad press and committed suicide, but shifted over to studying hackers-- it turned out that the isolation, risk, long hours, and (in some cases?) responsibility led to psychological problems like those of spies and soldiers.

It turned out that hackers aren't especially likely to have Asperger's, but the intense world model might be appropriate. The idea is that people on the spectrum (?) do notice other people's emotions, but get overwhelmed by them, and one way of dealing with the stress is to focus on intellectual details.

She thinks of hackers as people who are especially driven by curiosity, which is a virtue.
Idea: Self-Improving Task Management Software
by rlp10 — 2013-02-27T13:04:04.409Z

So what the world needs is yet another task management program, right?

My idea is software which automatically implements productivity strategies, measures the effectiveness of those strategies, and analyses which strategies work best for you. Hopefully, using the software would result in a sustained increase in your productivity over time.

By "productivity strategies" I mean things like: the recommendations in the the anti-procrastination algorithm, the pomodoro technique, exercising regularly, pre-commitment, experimenting with sleep patterns, gamifying your tasks and so forth.

In practical terms, what I'm envisioning is an extensible software framework. The core program would be a simple task list manager: add tasks to be done in the future, check off items as done when completed and send notifications to the user.

This core framework would then be extended by plugins, which represented different productivity strategies. For example, the pomodoro plugin might make your first task at 9am each morning to review your task list and choose the most important three tasks (MITs), your second task to set and begin a timer for 30 minutes and your third task to complete that top MIT you chose. After 30 minutes, it would add a new task of taking a five minute relaxation break and send you a notification to let you know. Five minutes later, it would notify you again to finish your relaxation break task, with a fresh task to re-start the timer and then back to your MITs for a further 30 minutes.

The software could independently activate and deactivate the plugins in order to collect sufficient data to suggest which strategies were most effective for you. Over time, more plugins would be written as people made further suggestions. Existing plugins could be potentially improved and automatically reviewed using A/B testing.

When deciding whether a strategy is "effective", I mean that a large number of tasks are completed, that the remaining number of tasks on the list is small and that the age of those tasks is not too great. However, the criteria could be extended to ask for an indication of mood from the user, to allow for low stress optimisation, for example. Perhaps stochastic self sampling would work well here.

If users were willing to opt into providing anonymous data, the software could automate a community ...
Need help with an MLP fanfiction with a transhumanist theme.
by Salivanth — 2013-02-27T11:57:09.574Z

EDIT: I am now taking arguments for alicornism. Alicornism being the placeholder term I've given to the stance that all ponies should be alicorns. Please PM me or post here if you have a good one, or an argument against one of anti-alicornism's strongest points: Overpopulation/over-use of resources, magical abuse/existential risk, or upheaval of the respect ponies have for their rulers due to their alicorn status. I would prefer general arguments for alicornism over counter-arguments if possible. Deathist / anti-alicornist arguments are still fine to post here.
Disclaimer: I'm not sure if this is worthy of a discussion post, but I figured, given the amount of people on LW who like My Little Pony, it would have at least as many potentially interested people as a regional meet-up thread would, so I figured I'd give it a shot. If this is too trivial or frivolous for LW, feel free to tell me and/or downvote, and I'll refrain from such threads in future. A place where I could go to find some help instead of the Discussion section would also be greatly appreciated in such a case.
So I had an idea for a one-shot or small novella, depending on how the plot developed, about an argument between Twilight and Celestia. Twilight finds out she's immortal now that she's an alicorn, and Twilight then decides that, given the standard anti-death concepts that immortality is good, death is bad, and so on, they should turn everyone who wants to be an alicorn into one.

The problem is, I'm having a very difficult time coming up with actual arguments for Celestia.

- Celestia herself is immortal, she's lived for well over a thousand years, and she isn't horrifically depressed, so clearly, immortal life is worth living and there's enough stuff to do with an extended lifespan.

- For the purposes of this fic, it's possible to turn anypony into an alicorn. I'm likely going to go with the idea that the spell can only be used a few times a year, but that's still enough to turn anyone who wants it into an alicorn within a couple of decades via exponentiation: The first targets can all be gifted unicorns who can be easily trained to use the magic.

- In most of the "Immortality sucks" fics I've read, the only real argument that immortality sucks is that you have to watch everyone else grow up and die. If a large majority of the population were turned alicorn, this wouldn't be a problem anym...
Why might the future be good?
by paulfchristiano — 2013-02-27T07:22:09.782Z

(Cross-posted from Rational Altruist. See also recent posts on time-discounting and self-driving cars.)

When talking about the future, I often encounter two (quite different) stories describing why the future might be good:

Decisions will be made by people whose lives are morally valuable and who want the best for themselves. They will bargain amongst each other and create a world that is good to live in. Because my values are roughly aligned with their aggregate preferences, I expect them to create a rich and valuable world (by my lights as well as theirs).

Some people in the future will have altruistic values broadly similar to my own, and will use their influence to create a rich and valuable world (by my lights as well as theirs).

Which of these pictures we take more seriously has implications for what we should do today. I often have object level disagreements which seem to boil down to disagreement about which of these pictures is more important, but rarely do I see serious discussion of that question. (When there is discussion, it seems to turn into a contest of political ideologies rather than facts.)

If we take picture (1) seriously, we may be interested in ensuring that society continues to function smoothly, that people are aware of and pursue what really makes them happy, that governments are effective, markets are efficient, externalities are successfully managed, etc. If we take picture (2) seriously, we are more likely to be concerned with changing what the people of the future value, bolstering the influence of people who share our values, and ensuring that altruists are equipped to embark on their projects successfully.

I'm mostly concerned with the very long run---I am wondering what conditions will prevail for most of the people who live in the future, and I expect most of them to be alive very far from now.

It seems to me that there are two major factors that control the relative importance of pictures (1) and (2): how prominent should we expect altruism to be in the future, and how efficiently are altruistic vs. selfish resources being used to create value? My answer to the second question is mostly vague hand-waving, but I think I have something interesting to say on the first question.

How much altruism do we expect?

I often hear people talking about the future, and the present for that matter, as if we are falling toward...
[SEQ RERUN] An Especially Elegant Evpsych Experiment
by MinibearRex — 2013-02-27T06:33:00.172Z

Today's post, An Especially Elegant Evpsych Experiment was originally published on 13 February 2009. A summary (taken from the LW wiki):

An experiment comparing expected parental grief at the death of a child at different ages, to the reproductive success rate of children at that age in a hunter gatherer tribe.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Evolutionary-Cognitive Boundary, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : West LA Meetup - Nootropics
by  — 2013-02-27T05:06:38.467Z

Discussion article for the meetup : West LA Meetup - Nootropics

WHEN:
27 February 2013 07:00:00PM (-0800)

WHERE:
10850 West Pico Blvd, Los Angeles, CA 90064

When: 7:00pm Wednesday, February 27th.

Where: The Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters. The entrance sign says "Lounge".

Parking is free for 3 hours.

Discussion Topic: This week we will talk about Nootropics and other associated things. This will range from discussion of the biochemistry to evaluating efficacy and safety of such things. The objective isn't to say "chemicals are the best way to make me smart!", but rather "how can I tell if this or that chemical is worth my time and money as a means of fulfilling my goals?"

No foreknowledge or exposure to Less Wrong is necessary; this will be generally accessable and useful to anyone who values thinking for themselves. There will be open general conversation until 7:30, and that's always a lot of good, fun, intelligent discussion!

I will bring a whiteboard with Bayes' Theorem written on it.

Discussion article for the meetup : West LA Meetup - Nootropics
When should you give to multiple charities?
by jefftk — 2013-02-27T02:56:39.785Z

The logic that you should donate only to a single top charity is very strong. But when faced with two ways of making the world better there's this urge deny the choice and do both. Is this urge irrational or is there something there?

At the low end splitting up your giving can definitely be a problem. If you give $5 here and $10 there it's depressing how much of your donations will be eaten up by processing costs:

The most extreme case I've seen, from my days working at a nonprofit, was an elderly man who sent $3 checks to 75 charities. Since it costs more than that to process a donation, this poor guy was spending $225 to take money from his favorite organizations.

By contrast, at the high end you definitely need to divide your giving. If a someone decided to give $1B to the AMF it would definitely do a lot of good. Because charities have limited room for more funding, however, after the first $20M or so there are probably other anti-malaria organizations that could do more with the money. And at some point we beat malaria and so other interventions start having a greater impact for your money.

Most of us, however, are giving enough that our donations are well above the processing-cost level but not enough to satisfy an organization's room for more funding. So what do you do?

If one option is much better than another then you really do need to make the choice. The best ones are enough better than the average ones that you need to buckle down and pick the best.

But what about when you're not sure? Even after going through all the evidence you can find you just can't decide whether it's more effective to take the sure thing and help people now or support the extremely hard to evaluate but potentially crucial work of reducing the risk that our species wipes itself out. The strength of the economic argument for giving only to your top charity is proportional to the difference between it and your next choice. If the difference is small enough and you find it painful to pick only one it's just not worth it: give to both.

(It can also be worth it to give to multiple organizations because of what it indicates to other people. I help fund 80,000 Hours because I think spreading the idea of effective altruism is the most important thing I can do. But it looks kind of sketchy to only give to metacharities, so I divide my giving between them and GiveWell's top pic...
Grace-Hanson Podcast
by KatjaGrace — 2013-02-27T02:26:08.000Z

Cross posted from Overcoming Bias. Comments there.

***

Robin and I have a new podcast on the subject of play (mp3, wav, m4a). Older ones are here.

Don’t be thrown by a bit of silence at the start of the m4a one. We also don’t have the time right now to figure out how to put it in better formats. Sorry about that. If anyone else does, and posts such files, I’ll link to them.