trentmkelly/LessWrong-43k huggingface dataset:
An example and discussion of extension neglect
by emr — 2015-01-16T06:10:06.483Z

I recently used an automatic tracker to learn how I was spending my time online. I learned that my perceptions were systemically biased: I spend less time than I thought on purely non-productive sites, and far more time on sites that are quasi-productive.
For example, I felt that I was spending too much time reading the news, but I learned that I spend hardly time doing so. I didn't feel that I was spending much time reading Hacker News, but I was spending a huge amount of time there!
Is this a specific case of a more general error?
A general framing: "Paying too much attention to the grouping whose items have the most extreme quality, when the value of focusing on this grouping is eclipsed by the value of focusing on a larger grouping of less extreme items".

So in this case, once I had formed the desire to be more productive, I overestimated how much potential productive time I could gain by focusing on those sites that I felt were maximally non-productive, and underestimated the potential of focusing on marginally more productive sites.

In pseudo-technical terms: We think about items in groups. But then we think of the total value of a group as being closer to average_value than to average_value * size_of_group.

This falls under the category of Extension Neglect, which includes errors caused by ignoring the size of a set. Other patterns in this category are:

Base rate neglect: Inferring the category of an item as if all categories were the same size.

The peak-end rule: Giving the value of the ordered group as a function of max_value and end_value.

Not knowing how set size interacts with randomness.

For the error given above, some specific examples might be:

Health: Focusing too much on eating desert at your favorite restaurant; and not enough on eating pizza three times a week.

Love: Fights and romantic moments; daily interaction.

Stress: Public speaking; commuting

Ethics: Improbable dilemmas; reducing suffering (or doing anything externally visible)

Crime: Serial killers; domestic violence
Meetup : West LA—Keep Your Identity Small
by OpenThreadGuy — 2015-01-16T03:51:53.875Z

Discussion article for the meetup : West LA—Keep Your Identity Small

WHEN:
21 January 2015 07:00:00PM (-0800)

WHERE:
11066 Santa Monica Blvd, Los Angeles, CA

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: We often repeat the mantra "Keep your identity small," and for good reason. What is that reason? Well, maybe you should show up to this meetup and find out. Or read the recommended reading. You know, either way. Preferably both

Recommended Reading:

Keep Your Identity Small by Paul Graham

Cached Selves

A Theory of Identity

The Identity Hack, The Curse of Identity

Your Beliefs as Temple

Easier to Deny or Rationalize Behavior Than Evolve Your Own Identity

Leave a Line of Retreat

No prior exposure to Less Wrong is required; this will be generally accessible. And remember, keep your secret identity small.

Discussion article for the meetup : West LA—Keep Your Identity Small
Group Rationality Diary, January 16-31
by therufs — 2015-01-16T01:54:30.945Z

This is the public group rationality diary for January 16-31.

It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.

Previous diary: January 1-15

Next diary: February 1-14

Rationality diaries archive
Vingean Reflection: Reliable Reasoning for Self-Improving Agents
by So8res — 2015-01-15T22:47:55.625Z

I'm pleased to announce a new paper from MIRI: Vingean Reflection: Reliable Reasoning for Self-Improving Agents.

Abstract:

Today, human-level machine intelligence is in the domain of futurism, but there is every reason to expect that it will be developed eventually. Once artificial agents become able to improve themselves further, they may far surpass human intelligence, making it vitally important to ensure that the result of an "intelligence explosion" is aligned with human interests. In this paper, we discuss one aspect of this challenge: ensuring that the initial agent's reasoning about its future versions is reliable, even if these future versions are far more intelligent than the current reasoner. We refer to reasoning of this sort as Vingean Reflection.

A self-improving agent must reason about the behavior of its smarter successors in abstract terms, since if it could predict their actions in detail, it would already be as smart as them. This is called the Vingean principle, and we argue that theoretical work on Vingean reflection should focus on formal models that reflect this principle. However, the framework of expected utility maximization, commonly used to model rational agents, fails to do so. We review a body of work which instead investigates agents that use formal proofs to reason about their successors. While it is unlikely that real-world agents would base their behavior entirely on formal proofs, this appears to be the best currently available formal model of abstract reasoning, and work in this setting may lead to insights applicable to more realistic approaches to Vingean reflection.

This is the fourth in a series of six papers discussing various components of MIRI's technical research agenda. It motivates the field of Vingean reflection, which studies methods by which agents can reason reliably about agents that are more intelligent than themselves. Toy models used to study this problem in the past include the "tiling agent" models that have been discussed on LessWrong in the past. The introduction to the paper runs as follows:

In a 1965 article, I.J. Good introduced the concept of an "intelligence explosion" (Good 1965):

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ul

...
Elon Musk donates $10M to the Future of Life Institute to keep AI beneficial 
by Paul Crowley — 2015-01-15T16:33:48.640Z

We are delighted to report that technology inventor Elon Musk, creator of Tesla and SpaceX, has decided to donate $10M to the Future of Life Institute to run a global research program aimed at keeping AI beneficial to humanity.

There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. A long list of leading AI-researchers have signed an open letter calling for research aimed at ensuring that AI systems are robust and beneficial, doing what we want them to do. Musk's donation aims to support precisely this type of research: "Here are all these leading AI researchers saying that AI safety is important", says Elon Musk. "I agree with them, so I'm today committing $10M to support research aimed at keeping AI beneficial for humanity."

[...] The $10M program will be administered by the Future of Life Institute, a non-profit organization whose scientific advisory board includes AI-researchers Stuart Russell and Francesca Rossi. [...]

The research supported by the program will be carried out around the globe via an open grants competition, through an application portal at http://futureoflife.org that will open by Thursday January 22. The plan is to award the majority of the grant funds to AI researchers, and the remainder to AI-related research involving other fields such as economics, law, ethics and policy (a detailed list of examples can be found here [PDF]). "Anybody can send in a grant proposal, and the best ideas will win regardless of whether they come from academia, industry or elsewhere", says FLI co-founder Viktoriya Krakovna.

[...] Along with research grants, the program will also include meetings and outreach programs aimed at bringing together academic AI researchers, industry AI developers and other key constituents to continue exploring how to maximize the societal benefits of AI; one such meeting was held in Puerto Rico last week with many of the open-letter signatories.

Elon Musk donates $10M to keep AI beneficial, Future of Life Institute, Thursday January 15, 2015
Meetup : San Francisco Meetup: Unconference
by rocurley — 2015-01-15T06:31:51.232Z

Discussion article for the meetup : San Francisco Meetup: Unconference

WHEN:
19 January 2015 06:00:00PM (-0800)

WHERE:
219 6th st. , San Francisco, CA

THE LOCATION MAY HAVE CHANGED SINCE YOU LAST LOOKED AT THIS POST!

We'll be having an unconference! People will have the opportunity to give short presentations on something that interests them. We'll probably devolve to general conversation afterwards.
Call (301-458-0764) if you have any trouble.

Discussion article for the meetup : San Francisco Meetup: Unconference
Meetup : Canberra: the Hedonic Treadmill
by DanielFilan — 2015-01-15T04:02:44.807Z

Discussion article for the meetup : Canberra: the Hedonic Treadmill

WHEN:
23 January 2015 06:00:00PM (+1100)

WHERE:
108 North Road, Acton, ACT, 0200

I will be giving a brief talk about what does and doesn't make us happier in the long run, introducing the idea of the 'hedonic treadmill'. Discussion will (hopefully) ensue afterwards. Vegan snacks will be provided.

General meetup info:

If you use Facebook, please join our group.

Structured meetups are (usually) held on the second Saturday and fourth Friday of each month from 6 pm until late in the CSIT building, room N101.

Discussion article for the meetup : Canberra: the Hedonic Treadmill
Meetup : London Social Meetup, 18/01/2015
by Tenoke — 2015-01-14T17:49:45.075Z

Discussion article for the meetup : London Social Meetup, 18/01/2015

WHEN:
18 January 2015 02:00:00PM (+0000)

WHERE:
Shakespeare's Head, Shakespeare's Head, Africa House, 64-68 Kingsway, London WC2B 6BG, UK-68 Kingsway, London WC2B 6BG, UK

LessWrong London is having another meetup this Sunday (18/01) at 2:00 PM. We are meeting at our usual venue - The Shakespeare's Head by Holborn tube station. There is no fixed topic of discussion nor is there anything planned so be prepared for anything. There will be a sign identifying us and if you have any problems feel free to contact me on 07425168803.

About London LessWrong:

We run this meetup almost every week; these days we tend to get in the region of 5-15 people in attendance. By default, meetups are just unstructured social discussion about whatever strikes our fancy: books we're reading, recent posts on LW/related blogs, logic puzzles, toilet usage statistics....
Sometimes we play The Resistance or other games. We usually finish around 7pm, give or take an hour, but people arrive and leave whenever suits them.

If you want more information about the meetup or anything else come by our google group or alternatively our facebook group.

Discussion article for the meetup : London Social Meetup, 18/01/2015
Selfish preferences and self-modification
by Manfred — 2015-01-14T08:42:51.489Z

One question I've had recently is "Are agents acting on selfish preferences doomed to having conflicts with other versions of themselves?" A major motivation of TDT and UDT was the ability to just do the right thing without having to be tied up with precommitments made by your past self - and to trust that your future self would just do the right thing, without you having to tie them up with precommitments. Is this an impossible dream in anthropic problems?

In my recent post, I talked about preferences where "if you are one of two copies and I give the other copy a candy bar, your selfish desires for eating candy are unfulfilled." If you would buy a candy bar for a dollar but not buy your copy a candy bar, this is exactly a case of strategy ranking depending on indexical information.

This dependence on indexical information is inequivalent with UDT, and thus incompatible with peace and harmony.

To be thorough, consider an experiment where I am forked into two copies, A and B. Both have a button in front of them, and 10 candies in their account. If A presses the button, it deducts 1 candy from A. But if B presses the button, it removes 1 candy from B and gives 5 candies to A.

Before the experiment begins, I want my descendants to press the button 10 times (assuming candies come in units such that my utility is linear). In fact, after the copies wake up but before they know which is which, they want to press the button!

The model of selfish preferences that is not UDT-compatible looks like this: once A and B know who is who, A wants B to press the button but B doesn't want to do it. And so earlier, I should try and make precommitments to force B to press the button.

But suppose that we simply decided to use a different model. A model of peace and harmony and, like, free love, where I just maximize the average (or total, if we specify an arbitrary zero point) amount of utility that myselves have. And so B just presses the button.

(It's like non-UDT selfish copies can make all Pareto improvements, but not all average improvements)

Is the peace-and-love model still a selfish preference? It sure seems different from the every-copy-for-themself algorithm. But on the other hand, I'm doing it for myself, in a sense.

And at least this way I don't have to waste time with precomittment. In fact, self-modifying to this form of preferences is such an effective ac...
Meetup : Raleigh, NC (RTLW) Discussion Meetup
by therufs — 2015-01-14T03:47:32.618Z

Discussion article for the meetup : Raleigh, NC (RTLW) Discussion Meetup

WHEN:
07 May 2022 07:30:00PM (-0400)

WHERE:
Durham NC 27701

The Triangle area LessWrong meetup is held the 1st, 3rd, and 5th Thursdays of the month, usually in either Durham or Raleigh. For dates, topics, and locations, please join our mailing list for the most up to date information, or email Ruthan.

Our next meeting will be 5/5, probably at a place, and we'll do a thing. If you'd like to influence the specificity of "place" and "thing", our scheduling document, Shotgun Rules for RTLW Meetings, can be found here.

Hope to see you there!

Discussion article for the meetup : Raleigh, NC (RTLW) Discussion Meetup
Quantum cat-stencil interference projection? What is this?
by pre — 2015-01-14T00:06:56.290Z

Sorry I don't hang around here much. I keep meaning to. You're still the ones I come to when I have no clue at all what a quantum-physics article I come across means though.
http://io9.com/heres-a-photo-of-something-that-cant-be-photographed-1678918200
So. Um. What?
They have some kind of double-slit experiment that gets double-slitted again then passed through a stencil before being recombined and recombined again to give a stencil-shaped interference pattern?
Is that even right?

Can someone many-worlds-interpretation describe that at me, even if it turns out its just a thought-experiment with a graphics mock-up?
I'm the new moderator
by NancyLebovitz — 2015-01-13T23:21:55.848Z

Viliam Bur made the announcement in Main, but not everyone checks main, so I'm repeating it here.

During the following months my time and attention will be heavily occupied by some personal stuff, so I will be unable to function as a LW moderator. The new LW moderator is... NancyLebovitz!

From today, please direct all your complaints and investigation requests to Nancy. Please not everyone during the first week. That can be a bit frightening for a new moderator.

There are a few old requests I haven't completed yet. I will try to close everything during the following days, but if I don't do it till the end of January, then I will forward the unfinished cases to Nancy, too.

Long live the new moderator!
[meta] New LW moderator: NancyLebovitz
by Viliam_Bur — 2015-01-13T21:41:04.301Z

During the following months my time and attention will be heavily occupied by some personal stuff, so I will be unable to function as a LW moderator. The new LW moderator is... NancyLebovitz!

From today, please direct all your complaints and investigation requests to Nancy. Please not everyone during the first week. That can be a bit frightening for a new moderator.

There are a few old requests I haven't completed yet. I will try to close everything during the following days, but if I don't do it till the end of January, then I will forward the unfinished cases to Nancy, too.

Long live the new moderator!
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-01-13T21:12:19.339Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
18 January 2015 03:00:00PM (-0500)

WHERE:
National Portrait Gallery

We will be meeting in the Kogod Courtyard of the National Portrait Gallery (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance) to hang out, play games, and engage in fun conversation.

Note: Feel free to send an email to the Google Group - lesswrong-dc - if you have a game you wish to recruit people to play in advance. This is especially useful for games that take 3+ hours, as you will want to start playing by 3:30 to be sure of finishing by 7:00.

Edit: WMATA has posted the weekend service adjustment advisory - trains will be running less often on all lines. The Verizon Center has no events scheduled for the 18th.

Upcoming meetups:

Jan. 25: Fermi Estimates (calculating approximate answers to random questons)

Feb. 1: Performance of Masculinity discussion (read and discuss a pair of articles about male gender roles in U.S.)

Feb. 8: Fun & Games (bring games, play games, converse, socialize, or any combination thereof)

Feb. 15: Book Swap (lend books, borrow books, talk about books)

Discussion article for the meetup : Washington, D.C.: Fun & Games
Why you should consider buying Bitcoin right now (Jan 2015) if you have high risk tolerance
by Ander — 2015-01-13T20:02:20.732Z

LessWrong is where I learned about Bitcoin, several years ago, and my greatest regret is that I did not investigate it more as soon as possible, that people here did not yell at me louder that it was important, and to go take a look at it. In that spirit, I will do so now.

First of all, several caveats:

* You should not go blindly buying anything that you do not understand. If you don't know about Bitcoin, you should start by reading about its history, read Satoshi's whitepaper, etc. I will assume that hte rest of the readers who continue reading this have a decent idea of what Bitcoin is.

* Under absolutely no circumstances should you invest money into Bitcoin that you cannot afford to lose. "Risk money" only! That means that if you were to lose 100% of you money, it would not particularly damage your life. Do not spend money that you will need within the next several years, or ever. You might in fact want to mentally write off the entire thing as a 100% loss from the start, if that helps.

* Even more strongly, under absolutely no circumstances whatsoever will you borrow money in order to buy Bitcoins, such as using margin, credit card loans, using your student loan, etc. This is very much similar to taking out a loan, going to a casino and betting it all on black on the roulette wheel. You would either get very lucky or potentially ruin your life. Its not worth it, this is reality, and there are no laws of the universe preventing you from losing.

* This post is not "investment advice".

* I own Bitcoins, which makes me biased. You should update to reflect that I am going to present a pro-Bitcoin case.

So why is this potentially a time to buy Bitcoins? One could think of markets like a pendulum, where price swings from one extreme to another over time, with a very high price corresponding to over-enthusiasm, and a very low price corresponding to despair. As Warren Buffett said, Mr. Market is like a manic depressive. One day he walks into your office and is exuberant, and offers to buy your stocks at a high price. Another day he is depressed and will sell them for a fraction of that.

The root cause of this phenomenon is confirmation bias. When things are going well, and the fundamentals of a stock or commodity are strong, the price is driven up, and this ...
Less exploitable value-updating agent
by Stuart_Armstrong — 2015-01-13T17:19:27.909Z

My indifferent value learning agent design is in some ways too good. The agent transfer perfectly from u maximisers to v maximisers - but this makes them exploitable, as Benja has pointed out.

For instance, if u values paperclips and v values staples, and everyone knows that the agent will soon transfer from a u-maximiser to a v-maximiser, then an enterprising trader can sell the agent paperclips in exchange for staples, then wait for the utility change, and sell the agent back staples for paperclips, pocketing a profit each time. More prosaically, they could "borrow" £1,000,000 from the agent, promising to pay back £2,000,000 tomorrow if the agent is still a u-maximiser. And the currently u-maximising agent will accept, even though everyone knows it will change to a v-maximiser before tomorrow.

One could argue that exploitability is inevitable, given the change in utility functions. And I haven't yet found any principled way of avoiding exploitability which preserves the indifference. But here is a tantalising quasi-example.

As before, u values paperclips and v values staples. Both are defined in terms of extra paperclips/staples over those existing in the world (and negatively in terms of destruction of existing/staples), with their zero being at the current situation. Let's put some diminishing returns on both utilities: for each paperclips/stables created/destroyed up to the first five, u/v will gain/lose one utilon. For each subsequent paperclip/staple destroyed above five, they will gain/lose one half utilon.

We now construct our world and our agent. The world lasts two days, and has a machine that can create or destroy paperclips and staples for the cost of £1 apiece. Assume there is a tiny ε chance that the machine stops working at any given time. This ε will be ignored in all calculations; it's there only to make the agent act sooner rather than later when the choices are equivalent (a discount rate could serve the same purpose).

The agent owns £10 and has utility function u+Xv. The value of X is unknown to the agent: it is either +1 or -1, with 50% probability, and this will be revealed at the end of the first day (you can imagine X is the output of some slow computation, or is written on the underside of a rock that will be lifted).

So what will the agent do? It's easy to see that it can never get more than 1...