trentmkelly/LessWrong-43k huggingface dataset:
We Don't Have a Utility Function
by  — 2013-04-02T03:49:10.589Z

Related: Pinpointing Utility

If I ever say "my utility function", you could reasonably accuse me of cargo-cult rationality; trying to become more rational by superficially immitating the abstract rationalists we study makes about as much sense as building an air traffic control station out of grass to summon cargo planes.

There are two ways an agent could be said to have a utility function:

It could behave in accordance with the VNM axioms; always choosing in a sane and consistent manner, such that "there exists a U". The agent need not have an explicit representation of U.

It could have an explicit utility function that it tries to expected-maximize. The agent need not perfectly follow the VNM axioms all the time. (Real bounded decision systems will take shortcuts for efficiency and may not achieve perfect rationality, like how real floating point arithmetic isn't associative).

Neither of these is true of humans. Our behaviour and preferences are not consistent and sane enough to be VNM, and we are generally quite confused about what we even want, never mind having reduced it to a utility function. Nevertheless, you still see the occasional reference to "my utility function".

Sometimes "my" refers to "abstract me who has solved moral philosophy and or become perfectly rational", which at least doesn't run afoul of the math, but is probably still wrong about the particulars of what such an abstract idealized self would actually want. But other times it's a more glaring error like using "utility function" as shorthand for "entire self-reflective moral system", which may not even be VNMish.

But this post isn't really about all the ways people misuse terminology, it's about where we're actually at on the whole problem for which a utility function might be the solution.

As above, I don't think any of us have a utility function in either sense; we are not VNM, and we haven't worked out what we want enough to make a convincing attempt at trying. Maybe someone out there has a utility function in the second sense, but I doubt that it actually represents what they would want.

Perhaps then we should speak of what we want in terms of "terminal values"? For example, I might say that it is a terminal value of mine that I should not murder, or that freedom from authority is good.

But what does "terminal value" mean? Usually, it means that the value of ...
The Unintuitive Power Laws of Giving
by jefftk — 2013-04-02T02:10:03.624Z

Why give globally? Why give money? Why health charities? Why single-issue organizations? At first glance these all seem like arbitrary choices: what if I would rather volunteer, or donate to local charities? Why does it matter? It comes down to two distributions: cost-effectiveness and income.

DALYs per $1000

This shows the cost-effectiveness of a large number of health interventions, with taller bars in cases where we can avert more death and suffering per dollar. The shape of this chart is important: while we can do a lot of good if we pick an intervention at random or support a 'horizontal' effort that works on everything, we can do 300 times better by picking one in the top 10%. This is why single-issue charities make sense: you can pick one that focuses on a top intervention.

(Don't let the small bars on the left fool you: nearly every intervention on that chart is worth doing [1], some are just far more valuable than others.)

This only considers health: what about other ways of helping people? Political advocacy, development, literacy, human rights, why not them? The big thing health has going for it is that we can measure impact, which lets us choose only the best options. In other fields where we can't measure we could end up anywhere on the impact curve.

Let's look at another distribution:

So some people have a lot more money that other people, we knew that, right? But have a look at the scale. Someone earning at the poverty line in the US is richer than 90% of people. This is why giving globally is so powerful: small amounts of your money can mean a huge amount to people who have so much less.

Neither of these distributions are intuitive: we don't feel that rich, and charities all seem kind of interchangeable. But understanding them can make the difference between trying to do good and really succeeding.

(I first saw these charts in a talk by Toby Ord of Giving What We Can (GWWC). The data for the first chart, DALYs per $1000, comes from the DCP2. This was a project that, among other things, compiled cost effectiveness estimates for a very wide range of health interventions. I made the chart from the csv version of the data from here, excluding the ~60 interventions (of 171) that didn't have estimates. The second chart is straight from GWWC's website, and you can read the details there by clicking on footnote 4.)

[1] The median intervention there is...
Upcoming meditation workshop in the Bay Area
by KenChen — 2013-04-01T22:44:49.827Z

Hey everybody,

Michael "Valentine" Smith is teaching a Meditation workshop in Alamo, CA on May 3-5. The workshop is aimed at rationalist-type folk who know about the benefits of meditation and would like to be meditating more, but all are welcome to join. Val is a curriculum designer at CFAR and has been developing his family's meditation tradition for his whole life.

http://www.mindfulnessengineering.com/

Note: This is not a CFAR event.
Welcome to Less Wrong! (5th thread, March 2013)
by orthonormal — 2013-04-01T16:19:17.933Z

If you've recently joined the Less Wrong community, please leave a comment here and introduce yourself. We'd love to know who you are, what you're doing, what you value, how you came to identify as a rationalist or how you found us. You can skip right to that if you like; the rest of this post consists of a few things you might find helpful. More can be found at the FAQ.

(This is the fifth incarnation of the welcome thread; once a post gets over 500 comments, it stops showing them all by default, so we make a new one. Besides, a new post is a good perennial way to encourage newcomers and lurkers to introduce themselves.)

A few notes about the site mechanics

Less Wrong comments are threaded for easy following of multiple conversations. To respond to any comment, click the "Reply" link at the bottom of that comment's box. Within the comment box, links and formatting are achieved via Markdown syntax (you can click the "Help" link below the text box to bring up a primer).

You may have noticed that all the posts and comments on this site have buttons to vote them up or down, and all the users have "karma" scores which come from the sum of all their comments and posts. This immediate easy feedback mechanism helps keep arguments from turning into flamewars and helps make the best posts more visible; it's part of what makes discussions on Less Wrong look different from those anywhere else on the Internet.

However, it can feel really irritating to get downvoted, especially if one doesn't know why. It happens to all of us sometimes, and it's perfectly acceptable to ask for an explanation. (Sometimes it's the unwritten LW etiquette; we have different norms than other forums.) Take note when you're downvoted a lot on one topic, as it often means that several members of the community think you're missing an important point or making a mistake in reasoning— not just that they disagree with you! If you have any questions about karma or voting, please feel free to ask here.

Replies to your comments across the site, plus private messages from other users, will show up in your inbox. You can reach it via the little mail icon beneath your karma score on the upper right of most pages. When you have a new reply or message, it glows red. You can also click on any user's name to view all of their comments and posts.

It's definitely worth yo

...
A hypothesis testing video game
by Swimmy — 2013-04-01T05:41:22.221Z

The Blob Family is a simple game made by Leon Arnott. At heart, it's a game about testing hypotheses and getting the right answer with the least amount of evidence you can.

The mechanics work like so: Balls bounce around the screen randomly and you control a character who needs to avoid them. You can aim the mouse anywhere and activate a sonar. On the right side are rules for how various balls will react to this, and your goal is to figure out which ball is which. As you use the sonar more, the balls speed up, so it becomes more difficult to stay alive, thus giving an incentive to test your hypothesis in as few clicks as possible.

It very nicely illustrates the principle that, to test a hypothesis, you must design tests to falisfy your intuitions rather than to confirm them. For example, in one level, when you use the sonar:

1 ball heads toward the center

1 ball heads away from the center

1 ball heads away from the mouse

1 ball heads away from you

I found myself mistakenly clicking in the center of the screen to test hypothesis 1, but this is insufficient. To design the proper tests, you need to keep the mouse out of the center, keep it away from you, and depending on the position of the balls keep it off a straight line from you.

It could also demonstrate the ability of a fast brain to test hypotheses quickly. For many levels, if you could slow time down and set up a very good test, you could solve the problem with a single click. But we humans aren't usually so attentive.

Just thought the LW crowd might enjoy it.
[SEQ RERUN] On Things that are Awesome
by MinibearRex — 2013-04-01T05:23:19.747Z

Today's post, On Things that are Awesome was originally published on 24 March 2009. A summary (taken from the LW wiki):

Seven thoughts: I can list more than one thing that is awesome; when I think of "Douglas Hofstadter" I am really thinking of his all-time greatest work; the greatest work is not the person; when we imagine other people we are imagining their output, so the real Douglas Hofstadter is the source of "Douglas Hofstadter"; I most strongly get the sensation of awesomeness when I see someone outdoing me overwhelmingly, at some task I've actually tried; we tend to admire unique detailed awesome things and overlook common nondetailed awesome things; religion and its bastard child "spirituality" tends to make us overlook human awesomeness.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was You're Calling *Who* A Cult Leader?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Mountain View: Cure Light Aversions
by fiddlemath — 2013-04-01T05:02:55.120Z

Discussion article for the meetup : Mountain View: Cure Light Aversions

WHEN:
02 April 2013 07:30:00PM (-0700)

WHERE:
167 Jasmine Ct, Mountain View, CA

In the time I've spent volunteering with DI, I've learned a few fun improv games. I also know a few other tricks for evoking helpful, unusual social atmospheres. Let's try these:

The Repetition Game - this sounds absurd. It feels absurd, too, but maybe not how you'd expect. You should try it.

Various improv games. I know several, and will be prepared with more. If you have some favorites, let's do those, too.

Comfort-Zone Expansion: a brief, mild round of comfort-zone expansion.

Aaron and Jess will open the doors at 7:00pm; we'll get started at 7:30.

If you're in the San Francisco Bay area and reading this, consider joining the Bay Area Less Wrong mailing list.
Regular meetups in Mountain View and Berkeley are announced and discussed there, as are other events of interest to the local community.

Discussion article for the meetup : Mountain View: Cure Light Aversions
A Rational Altruist Punch in The Stomach
by Neotenic — 2013-04-01T00:42:58.954Z

Robin Hanson wrote, five years ago:

Very distant future times are ridiculously easy to help via investment. A 2% annual return adds up to a googol (10^100) return over 12,000 years, even if there is only a 1/1000 chance they will exist or receive it.

So if you are not incredibly eager to invest this way to help them, how can you claim to care the tiniest bit about them? How can you think anyone on Earth so cares? And if no one cares the tiniest bit, how can you say it is "moral" to care about them, not just somewhat, but almost equally to people now? Surely if you are representing a group, instead of spending your own wealth, you shouldn’t assume they care much.

So why do many people seem to care about policy that effects far future folk? I suspect our paternalistic itch pushes us to control the future, rather than to enrich it. We care that the future celebrates our foresight, not that they are happy.

In the comments some people gave counterarguments. For those in a rush, the best ones are Toby Ord's. But I didn't bite any of the counterarguments to the extent that it would be necessary to counter the 10^100. I have some trouble conceiving of what would beat a consistent argument a googol fold.

Things that changed my behavior significantly over the last few years have not been many, but I think I'm facing one of them. Understanding biological immortality was one, it meant 150 000 non-deaths per day. Understanding the posthuman potential was another. Then came the 10^52 potential lives lost in case of X-risk, or if you are conservative and think only biological stuff can have moral lives on it, 10^31. You can argue about which movie you'll watch, which teacher would be best to have, who should you marry. But (if consequentialist) you can't argue your way out of 10^31 or 10^52. You won't find a counteracting force that exactly matches, or really reduces the value of future stuff by

3 000 000 634 803 867 000 000 000 000 000 000 777 000 000 000 999 fold

Which is way less than 10^52

You may find a fundamental and qualitative counterargument "actually I'd rather future people didn't exist", but you won't find a quantitative one. Thus I spend a lot of time on X-risk related things.

Back to Robin's argument: so unless ...
What are some in depth / meta-analytic, professionally edited wiki's? Examples inside
by someonewrongonthenet — 2013-03-31T07:47:13.910Z

I'm sure most people here are familiar with http://www.scholarpedia.org , a well curated encyclopedia of scientific topics.
I recently came across http://examine.com/ which is a combination of encyclopedia and meta-analysis for supplements and cognitive enhancers. Given the popularity of the "Practical" section on http://www.gwern.net/ I think most of us would be interested in this information, and it's incredibly readable and well presented.
Can anyone suggest websites of similar quality? To be specific, I'm talking about

1) many searchable topics collected in one place

2) Well cited and in-depth reviews of literature and/or thorough meta-analyses
3) Adequate quality control. For example, all examine.com edits must pass two-man review team and Scholarpedia does decentralized peer review.
Topic or focus is not important, as long as there is a wealth of dense, high quality, **well cited** information. Suggestions?
[SEQ RERUN] You're Calling *Who* A Cult Leader?
by MinibearRex — 2013-03-31T04:18:40.926Z

Today's post, You're Calling *Who* A Cult Leader? was originally published on 22 March 2009. A summary (taken from the LW wiki):

Paul Graham gets exactly the same accusations about "cults" and "echo chambers" and "coteries" that I do, in exactly the same tone - e.g. comparing the long hours worked by Y Combinator startup founders to the sleep-deprivation tactic used in cults, or claiming that founders were asked to move to the Bay Area startup hub as a cult tactic of separation from friends and family. This is bizarre, considering our relative surface risk factors. It just seems to be a failure mode of the nonconformist community in general. By far the most cultish-looking behavior on Hacker News is people trying to show off how willing they are to disagree with Paul Graham, which, I can personally testify, feels really bizarre when you're the target. Admiring someone shouldn't be so scary - I don't hold back so much when praising e.g. Douglas Hofstadter; in this world there are people who have pulled off awesome feats and it is okay to admire them highly.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Tolerate Tolerance, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Existential risks open thread
by John_Maxwell — 2013-03-31T00:52:46.589Z

We talk about a wide variety of stuff on LW, but we don't spend much time trying to identify the very highest-utility stuff to discuss and promoting additional discussion of it. This thread is a stab at that. Since it's just comments, you can feel more comfortable bringing up ideas that might be wrong or unoriginal (but nevertheless have relatively high expected value, since existential risks are such an important topic).
[SEQ RERUN] Tolerate Tolerance
by MinibearRex — 2013-03-30T20:14:20.498Z

Today's post, Tolerate Tolerance was originally published on 21 March 2009. A summary (taken from the LW wiki):

One of the likely characteristics of someone who sets out to be a "rationalist" is a lower-than-usual tolerance for flawed thinking. This makes it very important to tolerate other people's tolerance - to avoid rejecting them because they tolerate people you wouldn't - since otherwise we must all have exactly the same standards of tolerance in order to work together, which is unlikely. Even if someone has a nice word to say about complete lunatics and crackpots - so long as they don't literally believe the same ideas themselves - try to be nice to them? Intolerance of tolerance corresponds to punishment of non-punishers, a very dangerous game-theoretic idiom that can lock completely arbitrary systems in place even when they benefit no one at all.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Why Our Kind Can't Cooperate, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Berlin, practical rationality
by blob — 2013-03-30T19:16:14.720Z

Discussion article for the meetup : Berlin, practical rationality

WHEN:
05 April 2013 07:30:00PM (+0200)

WHERE:
S Wuhletal, 12621 Berlin, Germany

As announced on the mailing list:
The next meetup will happen on Friday, April 5th, 19:30 at my house and we'll try something new.
I declare it a 'practical rationality' or 'work' meetup, in which I'd like to establish some differences from the regular social gatherings we've done so far.

Have a plan. Schedule food and activities. Actually follow it.

Moderate discussion more strongly. Each subgroup will have one person responsible for keeping things on track, stop inadvertent topic switches, interrupt excessive anecdotes.

Will probably alternate with social meetups.

Here's what I suggest for Friday:

19:30 - 20:00 - Settling down and food.

Edit: Fixme.

20:00 - 20:30 - What have you learned since we last met? What have you done?

This could be an interesting regular activity where we get a glimpse of what you're doing and which could help seed discussion later. Everyone is prompted to talk, answers should be short (< 3 min).

20:30 - 21:15 - Discussion groups?

I'd like to try breaking into discussion groups. The idea is that there are a several people who've announced beforehand that they'd lead a discussion on some topic and have briefly studied it. At the meetup, they give a mini-presentation (5 minute-ish) and everyone else decides which group to join. At the end of the session, one person from each group summarises for everyone else.

Check the mailing list for topics.

21:15 - end - Chat.

See you there!

Discussion article for the meetup : Berlin, practical rationality
LW wiki spam filtering
by gwern — 2013-03-30T16:13:12.929Z

Long-time readers may have noticed that spam on the wiki has been a very persistent problem for the past 2 years or so; I've been dealing with it so far by hand, but I recently reached a breaking point and asked Trike to resolve it or find a new wiki administrator. (Speaking of which, is anyone interested?)

So Trike has enabled a MediaWiki extension called the edit filter: a small functional programming language which lets you define predicates applied to edits which trigger one of a set of actions, like banning a user, deleting an edit/page, or stopping an edit from going through. I have so far defined one rule: page creation is forbidden for users younger than 24 hours. This so far seems to have worked well; spam pages have fallen from 5-10/day to ~5 over the past 2 weeks. This is much more manageable, and I am hopeful that this new anti-spam measure will be effective longer than the previous additions did (but if it doesn't, I'll look into adding more rules dealing with images and external links, and perhaps also ban users whose names end in a numeric digit as almost all the spam accounts do).

If you've run into this edit filter before by making a page and seeing the submission rejected with an error message, fret not: merely wait 24 hours. (If your account is more than a day old and you're still getting errors, please contact me or Trike.)
Meetup : Shanghai Meetup
by Barry_Cotter — 2013-03-30T13:56:28.062Z

Discussion article for the meetup : Shanghai Meetup

WHEN:
02 April 2013 07:30:00PM (+0800)

WHERE:
Boxing Cat Brewery, 82 Fuxing Xi Lu, near Yongfu Lu, Shanghai, China

I will be here_shanghai)) from 7. I look like this. The food has been decent every time I've been and it's non smoking inside.
If you want, read the Checklist of Rationality Habits before you come and we can discuss that. No compulsion to do so, one of the important things is to have fun. The nearest metro station is Shanghai Library.
82 Fuxing Xi Lu, near Yongfu Lu 复兴路82号, 近永福路

Discussion article for the meetup : Shanghai Meetup