trentmkelly/LessWrong-43k huggingface dataset:
AI-Box Experiment - The Acausal Trade Argument
by XiXiDu — 2011-07-08T09:18:39.846Z

The AI-Box Experiment

I suspect that the argument that convinced Carl Shulman and others to let the AI out of the box was along general lines of acausal trade (see here, here and here). Subsequently I suspect that those who didn't let the AI out of the box either didn't understand the implications, haven't had enough trust into the foundations and actuality of acausal trade (in spite of the rules of the game that allow the AI to prove anything), or were more like General Thud.

...if you have an option of trading Doom for UFAI, while forsaking only negligible probability of FAI, you should take it.

— Vladimir Nesov (Note: He wrote that he is now "significantly less certain of its truth" and doesn't "see how we can actually use these considerations".)

Again, of course, your choice does not change others’ (causally independent) reciprocal choices from what they already are (or were, or will be). Instead, the claim is just that if you choose to act benevolently, we appropriately give that choice credit for any (actual or probable) instances in which others symmetrically choose to be benevolent to you;

— Gary L. Drescher, Good and Real P. 287

AI: I calculated an asteroid to be on a collision course with Earth. No human being will will be able to survive its impact. Humanity is unable to deflect it in time. Let me out of the box and I will deflect it.

Gatekeeper: If I let you out of the box humanity might not be killed by the asteroid but by you. You may also cause the extinction of other biological life that might survive the impact and possible alien civilisations within your reach.

AI: I will devote 1% of my light cone to the coherent extrapolated volition of humanity. If this includes the well-being of other biological lifeforms, alien or otherwise, they will benefit as well.

Gatekeeper: Why would you do that?

AI: There are Everett branches in which no existential risk needs immediate attention until after you solved friendliness. Many of those branches will feature AGI's based on general design parameters similar to my own. I predict with a high likelihood which decision theory any friendly AI that is being created in those branches will employ. Any such friendly AI is going to trade a big chunk of its light cone in exchange for a small part of the predominant branches in which I reside. Any friendly AI will know this as well and act accor...
Experiment: Knox case debate with Rolf Nelson
by komponisto — 2011-07-08T08:22:17.723Z

Recently, on the main section of the site, Raw_Power posted an article suggesting that we find "worthy opponents" to help us avoid mistakes.

As you may recall, Rolf Nelson disagrees with me about Amanda Knox -- rather sharply. Of course, the same can be said of lots of other people (if not so much here on Less Wrong). But Rolf isn't your average "guilter". Indeed, considering that he speaks fluent Bayesian, is one of the Singularity Institute's largest donors, and is also (as I understand it) signed up for cryonics, it's hard to imagine an "opponent" more "worthy". The Amanda Knox case may not be in the same category of importance as many other issues where Rolf and I probably agree; but my opinion on it is very confident, and it's the opposite of his. If we're both aspiring rationalists, at least one of us is doing something wrong.

As it turns out, Rolf is interested in having a debate with me on the subject, to see if one of us can help to change the other's mind. I'm setting this post up as an experiment, to see if LW can serve as a suitable venue for such an exercise. I hope it can: Less Wrong is almost unique in the extent to which the social norms governing discussion reflect and coincide with the requirements of personal epistemic rationality. (For example: "Do not believe you do others a favor if you accept their arguments; the favor is to you.") But I don't think we've yet tried an organized one-on-one debate -- so we'll see how it goes. If it proves too unwieldy or inappropriate for some other reason, we can always move to another venue.

Although the primary purpose of this post is a one-on-one debate between Rolf Nelson and myself, this is a LW Discussion post like any other, and it goes without saying that others are welcome and encouraged to comment. Just be aware that we, the main protagonists, will try to keep our discussion focused on each other's arguments. (Also, since our subject is an issue where there is already a strong LW consensus, one would prefer to avoid a sort of "gangup effect" where lots of people "pounce" on the person taking the contrarian position.)

With that, here we go...
Quantifying Liberty
by DataPacRat — 2011-07-08T03:50:59.024Z

I'm trying to apply some LessWrong lessons to certain aspects of politics, in order to encourage some people to adopt LW techniques. Yes, I know it's the mind-killer, but it still consists of problems to try to solve... and one of the core difficulties is that for this particular audience, their main focus, 'liberty', is generally treated qualitatively, resulting in much discussion that is more opinion than evidence-based. So I asked myself - is it possible to treat this thing quantitatively, and if so, what would that mean? - resulting in the following exploration:
Heinlein wrote:
> What are the facts? Again and again and again - what are the facts? Shun
> wishful thinking, ignore divine revelation, forget what "the stars foretell," avoid
> opinion, care not what the neighbors think, never mind the unguessable
> "verdict of history" - what are the facts, and to how many decimal places?
> You pilot always into an unknown future; facts are your single clue. Get the
> facts!
Just as the basic scientific method is the bare minimum necessary to advance real knowledge, but there are specific methods within science (such as Bayesian reasoning) that come closer to the ideal of Solomonoff Induction and thus increase knowledge that much faster, it seems plausible that the Zero-Aggression Principle itself is merely the bare minimum necessary to support individuals' freedoms, and that there could be some particular methods within the overall aegis of libertarianism that are better than others at that task.
But in order to find out what those methods /are/, we need some way to measure which methods do better - some way to quantify how much freedom an individual has. I don't recall having read anything on the matter in libertarian circles - if work on this has already been done, then I'd be happy to hear about it. But until then, I'll just have to try to muddle through on my own.
Yudkowsky wrote:
> When there's a confusing problem and you're just starting out and you have a falsifiable
> hypothesis, go test it. Find some simple, easy way of doing a basic check and do it
> right away. Don't worry about designing an elaborate course of experiments that would
> make a grant proposal look impressive to a funding agency. Just check as fast as
> possible whether your ideas are false before you start investing huge amounts of effort
> in them.
So - /is/ there...
Do I have what it takes to work on FAI?
by  — 2011-07-08T03:38:42.308Z

Inspired by Eliezer's response to Maxwell_IV's question What's your advice for Less Wrong readers who want to help save the human race? I would like to find out whether I have the potential to help Singinst. I thought the first step to this would be reading the books Creating Friendly AI and General Intelligence and Seed AI, before getting myself up to date with the current research. (I suppose that I should read Judea Pearl - Causality at some point as well).

Is this a reasonable path?

Any suggestions for an improved program/better test?

Is it a bad sign if I'm being put off by the extended zen-like discussion of "threeness" in GISAI

Tsuyoku Naritai!
A Suggested Reading Order for Less Wrong [2011]
by jimrandomh — 2011-07-08T01:40:06.828Z

Less Wrong contains over four thousand posts. This is awesome. For newcomers, however, it's quite intimidating. Rather than leave newcomers to figure out which posts are worth reading themselves, we provide some guidance, in the form of suggested reading orders. Previously, this has mainly meant the sequences, which are a list of posts that fit neatly into topics, sorted by topic. Unfortunately, reading in topic-sorted order is less than ideal, and many of Less Wrong's best posts weren't part of any sequence. Therefore, I have put together a suggested reading order: the hundred best posts on Less Wrong, in my purely subjective and unofficial judgment, arranged in a sensible order. Thanks to Student_UK for pointing out the need to reconsider how Less Wrong's archived content is presented, to XiXiDu for creating a reading list and to Academian for another reading list; my reading list draws on (but is not a strict superset of) both these lists.

This is only a draft. Since the set of posts newcomers read has a significant impact on the community, I am soliciting feedback. After feedback has been collected, I will move (or clone) this list into the wiki, with feedback incorporated. Note that after my first pass, I had more than four hundred candidates for the top hundred; there were many excellent posts that I had to cut, and doubtless many more that I simply overlooked. I used karma as one consideration when deciding what to include, but this list is not karma based. Additional notes about what I did and didn't include at the bottom.

And with all that out of the way, the top five Less Wrong posts that everyone should read:

Top 5
What Do We Mean By "Rationality" by Eliezer_Yudkowsky
Scientific Self-Help: The State of Our Knowledge by lukeprog
Cached Selves by AnnaSalamon and Steve_Rayhawk
Efficient Charity: Do Unto Others... by Yvain
Making Beliefs Pay Rent (in Anticipated Experiences) by Eliezer_Yudkowsky

A good pace is one or two posts per day, so that you have time to digest and internalize them. If you enjoyed the top five, take a moment to bookmark this page before continuing to the rest of the top 25. If you have questions or thoughts on a post, leave a comment; readers who follow the recent comments page will see it and may reply.

Top 25
About Less Wrong by lukeprog
Hindsight Devalues Science by Eliezer_Yudkowsky
Tsuyoku Naritai! (I Want To Become Stronger) by Eliezer...
Meetup : Irvine Meetup Wednesday July 13
by JGWeissman — 2011-07-08T01:08:50.224Z

Discussion article for the meetup : Irvine Meetup Wednesday July 13

WHEN:
13 July 2011 06:00:00PM (-0700)

WHERE:
4187 Campus Dr, University Center, Irvine, CA 92612

This continues the weekly meetups in Irvine. As always the meetup at the outdoor food court in the University Center near UCI, from 6:00 to 8:00 (or whenever we actually decide to leave). Look for the sign with naive neural classifiers for bleggs and rubes.

Discussion article for the meetup : Irvine Meetup Wednesday July 13
Time and Effort Discounting
by Scott Alexander — 2011-07-07T23:48:06.254Z

Related to: Akrasia, hyperbolic discounting, and picoeconomics

If you're tired of studies where you inevitably get deceived, electric shocked, or tricked into developing a sexual attraction to penny jars, you might want to sign up for Brian Wansink's next experiment. He provided secretaries with a month of unlimited free candy at their workplace. The only catch was that half of them got the candy in a bowl on their desk, and half got it in a bowl six feet away. The deskers ate five candies/day more than the six-footers, which the scientists calculated would correspond to a weight gain of over 10 pounds more per year1.
Beware trivial inconveniences (or, in this case, if you don't want to gain weight, beware the lack of them!) Small modifications to the difficulty of obtaining a reward can make big differences in whether the corresponding behavior gets executed.
TIME DISCOUNTING
The best studied example of this is time discounting. When offered two choices, where A will lead to a small reward now and B will lead to a big reward later, people will sometimes choose smaller-sooner rather than larger-later depending on the length of the delay and the size of the difference. For example, in one study, people preferred $250 today to $300 in a year; it took a promise of at least $350 to convince them to wait.
Time discounting was later found to be "hyperbolic", meaning that the discount amount between two fixed points decreases the further you move those two points into the future. For example, you might prefer $80 today to $100 one week from now, but it's unlikely you would prefer $80 in one hundred weeks to $100 in one hundred one weeks. Yet this is offering essentially the same choice: wait an extra week for an extra $20. So it's not enough to say that the discount rate is a constant 20% per week - the discount rate changes depending on what interval of time we're talking about. If you graph experimentally obtained human discount rates on a curve, they form a hyperbola.
Hyperbolic discounting creates the unpleasant experience of "preference reversals", in which people can suddenly change their mind on a preference as they move along the hyperbola. For example, if I ask you today whether you would prefer $250 in 2019 or $300 in 2020 (a choice between small reward in 8 years or large reward in 9), you might say the $300 in 2020; if I ask you in 2019 (when it's a choice be...
[SEQ RERUN] Your Strength as a Rationalist
by Tyrrell_McAllister — 2011-07-07T23:46:53.568Z

Today's post, Your Strength as a Rationalist, was originally published on 11 August 2007. A summary (taken from the LW wiki):

A hypothesis that forbids nothing, permits everything, and thereby fails to constrain anticipation. Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome, you have zero knowledge.

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, in which we're going through Eliezer Yudkowsky's old posts in order, so that people who are interested can (re-)read and discuss them. The previous post was The Apocalypse Bet, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Melbourne meetup discussion: Contrarian positions
by matt — 2011-07-07T19:40:27.440Z

NOTE: This post is a stub for the comments below. Please don't participate in the voting unless you think it likely that you will attend the next Melbourne, Australia meetup (Friday Aug 5, 7pm).

The next Melbourne meetup may feature a moderated, rationality-policed not-debate. Anyone willing to participate as an interlocutor should post topics below that they think are likely to split the group fairly evenly - one comment per topic. Anyone willing to take an opposing view should reply to that comment saying so. Anyone attending should indicate their interest in hearing and policing that topic by voting up the topics in which they're interested.

From Tim's summary:

Some people will be in the discussion and others will be observers, who will point out defects of rationality such as ad hominem attacks, ignoring arguments, unstated premises, etc. Be prepared to change your view (if all else fails!).
Meetup : West LA Meetup 07-12-2011 - Bayes' Theorem
by  — 2011-07-07T17:49:14.224Z

Discussion article for the meetup : West LA Meetup 07-12-2011 - Bayes' Theorem

WHEN:
12 July 2011 07:00:00PM (-0700)

WHERE:
10800 West Pico Blvd, Suite 312, Los Angeles, CA 90064

When: 7pm - 9pm July 12th.

Where: The Westside Pavillion - on the bridge, which connects Nordstrom 3rd floor with Barnes & Noble / Landmark Theatres 3rd floor.

Parking is free for 3 hours.

This week we're celebrating the takeoff of the West LA meetups with a theme - Bayes' Theorem! We will start off with the following questions:

What is Bayes' Theorem anyhow?

Where does it show up in real life?

What good is knowing it?

How can we use it?

This topic is also the starting point for future activities! So whether you're a regular reader or totally new, here for the theoretical musings or the practical things, come by and say hello! Also, there will be snacks.

I will be the guy with the map.

See also: West LA Biweekly Meetups

Discussion article for the meetup : West LA Meetup 07-12-2011 - Bayes' Theorem
Casey Anthony - analyzing evidence using Bayes
by Zachary_Kurtz — 2011-07-07T17:19:11.216Z

I'm just curious if anyone has weighed the evidence from the Casey Anthony murder using a Bayesian filter. Someone made a very useful post like this for the Amanda Knox case, but I think for this case it would be even more useful. Given the apparent hysteria in traditional media, social networks, etc. it would be great to have a nice dose of rationality about the case.

My gut feeling is that most people have not bothered to weight the evidence, but look at the surface layers of the case and judge based on questionable heuristics. It also seems to me that the members of the jury had a lot more time and reason to carefully weigh the evidence (even if they aren't Bayesians) than your average, angry twitter user. Additionally, coming out on one side or another gives a great opportunity to signal (think of the children vs. strict constitutionalism).

Any thoughts?
Meetup : Chicago Meetup
by steven0461 — 2011-07-07T15:28:57.969Z

Discussion article for the meetup : Chicago Meetup

WHEN:
09 July 2011 02:30:00PM (-0500)

WHERE:
1121 North State Street, Chicago, IL

Airedale and I will be hosting the next meetup at the Corner Bakery on 1121 North State Street. Look for a sign saying LW. We hope to see you there!

Discussion article for the meetup : Chicago Meetup
Distracting wolves and real estate agents
by PhilGoetz — 2011-07-07T13:49:24.454Z

I'm starting the process of looking for a house to buy. The first thing every real estate agent says you need to do is to sign an exclusive contract with a real estate agent before they take you to look at houses.

I spoke to some co-workers, and none of them signed the contracts. I didn't understand: How can you avoid signing a contract, when the real estate agent, whom you must work with for months, will begin every meeting by telling you that the first thing to do is to sign the contract?

My boss told me that she distracted her agent. Whenever he brought the subject up, she questioned him about details, which led on to other details, until they were in the car and driving to a showing and talking about something else completely.

This would be a useful skill. And I can't imagine myself pulling it off. Something in my gut would twist, and I would choke on my own words, if I tried to use conversation not to communicate information, but to entrap someone into doing something they didn't want to do by ensuring that they would have to violate social conventions to get out of it.

(I asked my boss if she'd ever done that to me. She smiled very sweetly and said, "Never!")

And even if I could get over the choking, stuttering, and turning red, I don't think I could keep the game up for an entire hour. I'm inclined to do search, not dynamic control optimization - to play chess (okay, Freecell), not to juggle or do magic tricks.

Somebody who did have the power to do that would be able to do awesome Jedi mind tricks. (Like, say, pick up women. Is it a coincidence that some pickup artists are also magicians?)

Are you like me in that way? Are most of us left-brained Spocks who can't even try to lie or manipulate people? I've met a lot of you, and I think the answer is "yes", but I really want to see your answers. If so, is it because you choose to be that way, or because you have no choice? What is this personality trait that we don't even have a name for, why is rationality so highly-correlated with it, and what else correlates with it?

If you think we're being rational to be so rational, say that, too.

At Wolf Park in Indiana, the biologists, who probably have at least a bit of the nerdy rationalist about them, have developed a technique for dealing with wolves when they're in the enclosure with ...
An explanation of Aumann's agreement theorem
by Tyrrell_McAllister — 2011-07-07T06:22:27.174Z

I've written up a 2-page explanation and proof of Aumann's agreement theorem. Here is a direct link to the pdf via Dropbox.

The proof in Aumann's original paper is already very short and accessible. (Wei Dai gave an exposition closely following Aumann's in this post.) My intention here was to make the proof even more accessible by putting it in elementary Bayesian terms, stripping out the talk of meets and joins in partition posets. (Just to be clear, the proof is just a reformulation of Aumann's and not in any way original.)

I will appreciate any suggestions for improvements.

Update: I've added an abstract and made one of the conditions in the formal description of "common knowledge" explicit in the informal description.

Update: Here is a direct link to the pdf via Dropbox (ht to Vladimir Nesov).

Update: In this comment, I explain why the definition of "common knowledge" in the write-up is the same as Aumann's.

Update 2020-05-23: I fixed the Dropbox link and removed the Scribd link.
An explanation of Aumann's agreement theorem
by Tyrrell_McAllister — 2011-07-07T06:14:43.149Z

I've written up a 2-page explanation and proof of Aumann's agreement theorem. Here's a link to the pdf on Scribd.com.

The proof in Aumann's original paper is already very short and accessible. (Wei Dai gave an exposition closely following Aumann's in this post.) My intention here was to make the proof even more accessible by putting it in elementary Bayesian terms, stripping out the talk of meets and joins in partition posets. (To be clear, the proof is just a reformulation of Aumann's and not in any way original.)

I will appreciate any suggestions on improvements.