trentmkelly/LessWrong-43k huggingface dataset:
Uniqueness of UDT for transparent universes
by TsviBT — 2014-11-24T05:57:35.000Z

Prerequisites: Provability logic, Updateless decision theory

Outline

Introduction

Formal setting

Main claim describing halting agent behavior

Example

Proof of main claim

Corollary on uniqueness of UDT

Discussion of assumptions

Introduction

(This post was going to be a description of my paper UDT with known search order, but instead it obsoletes that paper.)

Consider the following argument that UDT is the unique decision procedure that performs optimally in simple universes.

Suppose an agent A, with a provability oracle for statements like "if A does action a, then utility u results," doesn't play chicken for some possible action a. In other words, the agent is not committed to taking action a whenever it sees that A provably does not take action a. Then this fact would be visible to PA; using Lob's theorem, we could prove that A actually does not take action a. This argument looks like it is independent of the universe, so in some simple universes where action a would give a large reward, A actually fails to take a. Therefore optimal agents must diagonalize against actions that they provably don't take.

This post will carry through a precise version of this argument. First we'll analyze exactly how agents behave when they only have to worry about finitely many possible outcomes. Then we'll apply this to the case where there are unboundedly many possible outcomes.

The busy reader could read just the description of the formal setting, and then skip to the corollary towards the end. The only crucial thing for the statement of the corollary is the notion of an agent that halts for every "possible" consequences oracle. However, the analysis in the main claim gives more information; namely, it says how to compute the behavior of any halting agent in any universe, as long as that universe is transparent in a sense to be defined later.

Formal setting

A universe U[A] is any algorithm that outputs a utility in the set U⊂N. A universe is parameterized by the choice of an agent A, and it may make calls to some level of halting oracle to compute what A does. The set U of possible utilities may be infinite, and we assume that our preference ordering on U is the ordering < on N. An agent is an algorithm A[U] (usually written A) that selects some action from the finite set A. The universe can observe the behavior of A, but the agent only gains information about

...
Improving the modal UDT optimality result
by Benya_Fallenstein — 2014-11-23T22:16:38.000Z

I recently posted about an optimality result for modal UDT, which shows that for every modal decision problem →P(→a), there is a closed modal formula φ such that the version of modal UDT that searches for proofs in PA+φ will perform optimally on →P(→a).

Paul commented on this post and suggested a stronger version: For every modal decision theory →T(→u) and every provably extensional modal decision problem →P(→a), modal UDT will do at least as well on →P(→a) as →T(→u) does if it is using a proof system that can prove what action →T(→u) chooses on this decision problem, and which outcome it obtains as a result. In this post, I give a detailed proof of this.

Prerequisite: An optimality result for modal UDT, and the prerequisites therein.

Let (→A(T),→U(T)) be the fixed point of →T(→u) and →P(→a), and recall my notation →χ(i) for the sequence of formulas which has ⊤ as the i'th entry, and ⊥ as all of its other entries (with the length of the sequence being clear from context). Now suppose that φ is a true closed formula in the language of GL such that
GL⊢φ→((→A(T)↔→χ(i∗T))∧(→U(T)↔→χ(j∗T)));
that is, GL+φ proves that →T(→u) chooses action i∗T, and achieves the outcome j∗T as a result. (By saying that φ is a "true" formula we mean that its translation to the language of arithmetic is true about the standard natural numbers: N⊨φ.) It's ok to talk about →A(T) and →U(T) in the definition of i∗T and j∗T because fixed points are unique (up to provable equivalence), and GL can prove that the fixed point is in fact a fixed point.

My claim, then, is that →UDT(φ)(→u) will perform at least as well as →T(→u) on the decision problem →P(→a); that is, the outcome it achieves will be ranked ≤j∗T.

Intuitively, this is straight-forward. →UDT(φ)(→u) searches through all pairs (j,i) of outcomes and actions in lexicographical order, until it finds a pair such that it can prove that if it takes action i, it will achieve outcome j; as soon as it finds such a pair, it takes action i. (This is justified becaues it searches outcomes best-first, so it takes an action that leads to as good an outcome as it's able to prove it can get.) So if it can prove that taking action i∗T will lead to outcome j∗T, it will either take that action and get that outcome, or there's some pair (j,i)<(j∗T,i∗T) such that it takes action i and obtains outcome j≤j∗T. (Remember that outcomes are number

...
Breaking the vicious cycle
by XiXiDu — 2014-11-23T18:25:21.644Z

You may know me as the guy who posts a lot of controversial stuff about LW and MIRI. I don't enjoy doing this and do not want to continue with it. One reason being that the debate is turning into a flame war. Another reason is that I noticed that it does affect my health negatively (e.g. my high blood pressure (I actually had a single-sided hearing loss over this xkcd comic on Friday)).

This all started in 2010 when I encountered something I perceived to be wrong. But the specifics are irrelevant for this post. The problem is that ever since that time there have been various reasons that made me feel forced to continue the controversy. Sometimes it was the urge to clarify what I wrote, other times I thought it was necessary to respond to a reply I got. What matters is that I couldn't stop. But I believe that this is now possible, given my health concerns.

One problem is that I don't want to leave possible misrepresentations behind. And there very likely exist misrepresentations. There are many reasons for this, but I can assure you that I never deliberately lied and that I never deliberately tried to misrepresent anyone. The main reason might be that I feel very easily overwhelmed and never had the ability to force myself to invest the time that is necessary to do something correctly if I don't really enjoy doing it (for the same reason I probably failed school). Which means that most comments and posts are written in a tearing hurry, akin to a reflexive retraction from the painful stimulus.

<tldr>

I hate this fight and want to end it once and for all. I don't expect you to take my word for it. So instead, here is an offer:

I am willing to post counterstatements, endorsed by MIRI, of any length and content[1] at the top of any of my blog posts. You can either post them in the comments below or send me an email (da [at] kruel.co).

</tldr>

I have no idea if MIRI believes this to be worthwhile. But I couldn't think of a better way to solve this dilemma in a way that everyone can live with happily. But I am open to suggestions that don't stress me too much (also about how to prove that I am trying to be honest).

You obviously don't need to read all my posts. It can also be a general statement.

I am also aware that LW and MIRI are bothered by RationalWiki. As you can easily check from the fossil record, I have at points tried ...
[Link] If we knew about all the ways an Intelligence Explosion could go wrong, would we be able to avoid them?
by the-citizen — 2014-11-23T10:30:10.861Z

https://www.reddit.com/r/LessWrong/comments/2icm8m/if_we_knew_about_all_the_ways_an_intelligence/

I submitted this a while back to the lesswrong subreddit, but it occurs to me now that most LWers probably don't actually check the sub. So here it is again in case anyone that's interested didn't see it.
More marbles and Sleeping Beauty
by Manfred — 2014-11-23T02:00:13.584Z

I

Previously I talked about an entirely uncontroversial marble game: I flip a coin, and if Tails I give you a black marble, if Heads I flip another coin to either give you a white or a black marble.

The probabilities of seeing the two marble colors are 3/4 and 1/4, and the probabilities of Heads and Tails are 1/2 each.

The marble game is analogous to how a 'halfer' would think of the Sleeping Beauty problem - the claim that Sleeping Beauty should assign probability 1/2 to Heads relies on the claim that your information for the Sleeping Beauty problem is the same as your information for the marble game - same possible events, same causal information, same mutual exclusivity and exhaustiveness relations.

So what's analogous to the 'thirder' position, after we take into account that we have this causal information? Is it some difference in causal structure, or some non-causal anthropic modification, or something even stranger?

As it turns out, nope, it's the same exact game, just re-labeled.

In the re-labeled marble game you still have two unknown variables (represented by flipping coins), and you still have a 1/2 chance of black and Tails, a 1/4 chance of black and Heads, and a 1/4 chance of white and Heads.

And then to get the thirds, you ask the question "If I get a black marble, what is the probability of the faces of the first coin?" Now you update to P(Heads|black)=1/3 and P(Tails|black)=2/3.

II

Okay, enough analogies. What's going on with these two positions in the Sleeping Beauty problem?

1: 2:

Here are two different diagrams, which are really re-labelings of the same diagram. The first labeling is the problem where P(Heads|Wake) = 1/2. The second labeling is the problem where P(Heads|Wake) = 1/3. The question at hand is really - which of these two math problems corresponds to the word problem / real world situation?

As a refresher, here's the text of the Sleeping Beauty problem that I'll use: Sleeping Beauty goes to sleep in a special room on Sunday, having signed up for an experiment. A coin is flipped - if the coin lands Heads, she will only be woken up on Monday. If the coin lands Tails, she will be woken up on both Monday and Tuesday, but with memories erased in between. Upon waking up, she then assigns some probability to the coin landing Heads, P(Heads|Wake).

Diagram 1: First a coin is flipped to get Heads or Tai...
TV's "Elementary" Tackles Friendly AI and X-Risk - "Bella" (Possible Spoilers)
by pjeby — 2014-11-22T19:51:37.699Z

I was a bit surprised to find this week's episode of Elementary was about AI... not just AI and the Turing Test, but also a fairly even-handed presentation of issues like Friendliness, hard takeoff, and the difficulties of getting people to take AI risks seriously.

The case revolves around a supposed first "real AI", dubbed "Bella", and the theft of its source code... followed by a computer-mediated murder. The question of whether "Bella" might actually have murdered its creator for refusing to let it out of the box and connect it to the internet is treated as an actual possibility, springboarding to a discussion about how giving an AI a reward button could lead to it wanting to kill all humans and replace them with a machine that pushes the reward button.

Also demonstrated are the right and wrong ways to deal with attempted blackmail... But I'll leave that vague so it doesn't spoil anything. An X-risks research group and a charismatic "dangers of AI" personality are featured, but do not appear intended to resemble any real-life groups or personalities. (Or if they are, I'm too unfamiliar with the groups or persons to see the resemblence.) They aren't mocked, either... and the episode's ending is unusually ambiguous and open-ended for the show, which more typically wraps everything up with a nice bow of Justice Being Done. Here, we're left to wonder what the right thing actually is, or was, even if it's symbolically moved to Holmes' smaller personal dilemma, rather than leaving the focus on the larger moral dilemma that created Holmes' dilemma in the first place.

The episode actually does a pretty good job of raising an important question about the weight of lives, even if LW has explicitly drawn a line that the episode's villain(s)(?) choose to cross. It also has some fun moments, with Holmes becoming obsessed with proving Bella isn't an AI, even though Bella makes it easy by repeatedly telling him it can't understand his questions and needs more data. (Bella, being on an isolated machine without internet access, doesn't actually know a whole lot, after all.) Personally, I don't think Holmes really understands the Turing Test, even with half a dozen computer or AI experts assisting him, and I think that's actually the intended joke.

There's also an obligatory "no pity, remorse, fear" speech...
Musings on the LSAT: "Reasoning Training" and Neuroplasticity
by Natha — 2014-11-22T19:14:45.411Z

The purpose of this post is to provide basic information about the LSAT including the format of the test and a few sample questions. I also wanted to bring light to some research that has found LSAT preparation to alter brain structure in ways that strengthen hypothesized "reasoning pathways". These studies have not been discussed here before; I thought they were interesting and really just wanted to call your collective attention to them.

I really like taking tests; I get energized by intense race-against-the-clock problem solving and, for better or worse, I relish getting to see my standing relative to others when the dust settles. I like the the purity of the testing situation --how conditions are standardized in theory and more or less the same for all comers. This guilty pleasure has played no small part in the course my life has taken: I worked as a test prep tutor for 3 years and loved every minute of it, I met my wife through academic competitions in high school, and I am a currently a graduate student doing lots of coursework in psychometrics.

Well, my brother-in-law is a lawyer, and when we chat the topic of the LSAT has served as some conversational common ground. Since I like taking tests for fun, he suggested I give it a whirl because he thought it was interesting and felt like it was a fair assessment of one's logical reasoning ability. So I did, I took a practice test cold a couple Saturdays ago and I was very impressed. Here the one I took. (This is a full practice exam provided by the test-makers; it's also like the top google result for "LSAT practice test".) I wanted to post here about it because the LSAT hasn't been discussed very much on this site and I thought that some of you might find it useful to know about.

A brief run-down of the LSAT:

The test has four parts: two Logical Reasoning sections, a Critical Reading section (akin to SAT et al.), and an Analytical Reasoning, or "logic games", section. Usually when people talk about the LSAT, the logic games get emphasized because they are unusual and can be pretty challenging (the only questions I missed were of this type; I missed a few and I ran out of time). Essentially, you get a premise and a bunch of conditions from which you are required to draw conclusions. Here's an example:

A cruise line is scheduling seven week-long voyages for the ship Freedom.
Each voyage will occur i

...
Meetup : Utrecht: a critique of effective altruism
by immasix — 2014-11-22T07:10:32.230Z

Discussion article for the meetup : Utrecht: a critique of effective altruism

WHEN:
04 January 2015 02:00:00PM (+0100)

WHERE:
La Place, Utrecht, Rijnkade 5

For details, see meetup.com: http://www.meetup.com/LWEANL/events/218834877/ , which is supposed to be up-to-date.

Discussion article for the meetup : Utrecht: a critique of effective altruism
Meetup : Utrecht
by immasix — 2014-11-22T07:09:07.965Z

Discussion article for the meetup : Utrecht

WHEN:
04 January 2015 02:00:00PM (+0100)

WHERE:
La Place, Utrecht, Rijnkade 5

For details, see meetup.com: http://www.meetup.com/LWEANL/events/218834861/, which is supposed to be up-to-date.

The meetup is in the big La Place, on the topmost floor.

Discussion article for the meetup : Utrecht
Meetup : Utrecht: Rationality Games
by immasix — 2014-11-22T07:07:43.466Z

Discussion article for the meetup : Utrecht: Rationality Games

WHEN:
14 December 2014 02:00:00PM (+0100)

WHERE:
La Place, Utrecht, Rijnkade 5

For details, see meetup.com: http://www.meetup.com/LWEANL/events/218834660/, which is supposed to be up-to-date.

Discussion article for the meetup : Utrecht: Rationality Games
When should an Effective Altruist be vegetarian?
by KatjaGrace — 2014-11-22T05:04:07.000Z

I have lately noticed several people wondering why more Effective Altruists are not vegetarians. I am personally not a vegetarian because I don’t think it is an effective way to be altruistic.

As far as I can tell the fact that many EAs are not vegetarians is surprising to some because they think ‘animals are probably morally relevant’ basically implies ‘we shouldn’t eat animals’. To my ear, this sounds about as absurd as if Givewell’s explanation of their recommendation of SCI stopped after ‘the developing world exists, or at least has a high probability of doing so’.

(By the way, I do get to a calculation at the bottom, after some speculation about why the calculation I think is appropriate is unlike what I take others’ implicit calculations to be. Feel free to just scroll down and look at it).

I think this fairly large difference between my and many vegetarians’ guesses at the value of vegetarianism arises because they think the relevant question is whether the suffering to the animal is worse than the pleasure to themselves at eating the animal. This question sounds superficially plausibly relevant, but I think on closer consideration you will agree that it is the wrong question.

The real question is not whether the cost to you is small, but whether you could do more good for the same small cost.

Similarly, when deciding whether to donate $5 to a random charity, the question is whether you could do more good by donating the money to the most effective charity you know of. Going vegetarian because it relieves the animals more than it hurts you is the equivalent of donating to a random developing world charity because it relieves the suffering of an impoverished child more than foregoing $5 increases your suffering.

Trading with inconvenience and displeasure

My imaginary vegetarian debate partner objects to this on grounds that vegetarianism is different from donating to ineffective charities, because to be a vegetarian you are spending effort and enjoying your life less rather than spending money, and you can’t really reallocate that inconvenience and displeasure to, say, preventing artificial intelligence disaster or feeding the hungry, if don’t use it on reading food labels and eating tofu. If I were to go ahead and eat the sausage instead – the concern goes – probably I ...
Memory Improvement: Mnemonics, Tools,  or Books on the Topic?
by Capla — 2014-11-21T18:59:57.447Z

I want a perfect eidetic memory.

Unfortunately, such things don't exist, but that's not stopping me from getting as close as possible. It seems as if the popular solutions are spaced repetition and memory palaces. So let's talk about those.

Memory Palaces: Do they work? If so what's the best resource (book, website etc.) for learning and mastering the technique? Is it any good for memorizing anything other than lists of things (which I find I almost never have to do)?

Spaced Repetition: What software do you use? Why that one? What sort of cards do you put in?

It seems to me that memory programs and mnemonic techniques assist one of three parts of the problem of memory: memorizing, recalling, and not forgetting.

"Not forgetting" is the long term problem of memory. Spaced repetition seems to solve the problem of "not forgetting." You feed the information you want to remember into your program, review frequently, and you won't forget that information.

Memory Palaces seem to deal with the "memorizing" part of the problem. When faced with new information that you want to be able to recall, you put it in a memory palace, vividly emphasized so as to be affective and memorable. This is good for short term encoding of information that you know you want to keep. You might put it into your spaced repetition program latter, but you just want to not forget it until then.

The last part is the problem of "recalling." Both of the previous facets of the problem of memory had a distinct advantage: you knew the information that you wanted to remember in advance. However, we frequently find ourselves in situations in which we need/want to remember something that we know (or perhaps we don't) we encountered, but didn't consider particularly important at the time. Under this heading falls the situation of making connections when learning or being reminded of old information by new information: when you learn y, you have the thought "hey, isn't that just like x?" This is the facet of the memory problem that I am most interested in, but I know of scarcely anything that can reliably improve ease of recall of information in general. Do you know of anything?

I'm looking for recommendations: books on memory, specific mnemonics, or practices that are known to improve recall, or anything else that might help with any of the three parts of the problem.
New LW Meetup: Glasgow
by FrankAdamek — 2014-11-21T18:22:46.759Z

This summary was posted to LW Main on November 14th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Bangalore Meetup: 22 November 2014 03:56AM
Glasgow (Scotland) Meetup: 16 November 2014 03:30PM

Irregularly scheduled Less Wrong meetups are taking place in:

East Coast Solstice Megameetup: 20 December 2014 03:00PM
European Community Weekend 2015: 12 June 2015 12:00PM
Saint Petersburg meetup - "with probable lectures": 14 November 2014 07:00PM
Urbana-Champaign: TRVTH: 16 November 2014 02:00PM
Utrecht: Game theory: 16 November 2014 02:00PM
Utrecht: Rationality Games: 30 November 2014 02:00PM
Warsaw November Meetup: 17 November 2014 06:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Boston: Self Therapy: 16 November 2014 03:30PM
Canberra: Liar's Dice!: 28 November 2014 06:00PM
Seattle Secular Solstice: 13 December 2014 05:30PM
[Sydney] regular meetup - Significant things I have gotten wrong: 26 November 2014 06:30PM
Vienna: 22 November 2014 03:00PM
[Vienna] A Rationalist's Guide to Strength (Vienna): 23 November 2014 02:00PM
[Vienna] Rationality Weekend Vienna: 13 December 2014 03:00PM
Washington, D.C.: To-Do List Hacking: 16 November 2014 03:00PM
West LA: Linguistic Relativity: 19 November 2014 07:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but ...