trentmkelly/LessWrong-43k huggingface dataset:
The fairness of the Sleeping Beauty
by MrMind — 2015-07-07T08:25:56.863Z

This post will attempt a (yet another) analysis of the problem of the Sleeping Beauty, in terms of Jaynes' framework "probability as extended logic" (aka objective Bayesianism).

TL,DR: The problem of the sleeping beauty reduces to interpreting the sentence “a fair coin is tossed”: it can mean either that no results of the toss is favourite, or that the coin toss is not influenced by anthropic information, but not both at the same time. Fairness is a property in the mind of the observer that must be further clarified: the two meanings cannot be confused.

What I hope to show is that the two standard solutions, 1/3 and 1/2 (the 'thirder' and the 'halfer' solutions), are both consistent and correct, and the confusion lies only in the incorrect specification of the sentence "a fair coin is tossed".

The setup is given both in the Lesswrong's wiki and in Wikipedia, so I will not repeat it here.

I'm going to symbolize the events in the following way:

- It's Monday = Mon
- It's Tuesday = Tue
- The coin landed head = H
- The coin landed tail = T
- statement "A and B" = A & B
- statement "not A" = ~A

The problem setup leads to an uncontroversial attributions of logical structure:

1) H = ~T (the coin can land only on head or tail)

2) Mon = ~Tue (if it's Tuesday, it cannot be Monday, and viceversa)

And of probability:

3) P(Mon|H) = 1 (upon learning that the coin landed head, the sleeping beauty knows that it’s Monday)

4) P(T|Tue) = 1 (upon learning that it’s Tuesday, the sleeping beauty knows that the coin landed tail)

Using the indifference principle, we can also derive another equation.

Let's say that the Sleeping Beauty is awaken and told that the coin landed tail, but nothing else. Since she has no information useful to distinguish between Monday and Tuesday, she should assign both events equal probability. That is:

5) P(Mon|T) = P(Tue|T)

Which gives

6) P(Mon & T) = P(Mon|T)P(T) = P(Tue|T)P(T) = P(Tue & T)

It's here that the analysis between "thirder" and "halfer" starts to diverge.

The wikipedia article says "Guided by the objective chance of heads landing being equal to the chance of tails landing, it should therefore hold that". We know however that there's no such thing as 'the objective chance'.

Th...
Meetup : The first meetup in Kyiv, Ukraine
by Artem Batogovsky (forux) — 2015-07-07T08:18:05.481Z

Discussion article for the meetup : The first meetup in Kyiv, Ukraine

WHEN:
02 August 2015 06:30:00PM (+0300)

WHERE:
Kyiv, Ukraine

Hello =)

I glad to announce the first meetup in Kyiv.

Our google group: https://groups.google.com/forum/?#!forum/lesswrong-kyiv

Please join it =)

We will meet in cafe "Жар Піца" underground at metro station "Дружби Народів"

at 08-02-2015 16:30:00

p.s. also you can found additional information about meeting in our google group.

Discussion article for the meetup : The first meetup in Kyiv, Ukraine
Asymptotic Logical Uncertainty: Connection to Random Logical Extensions.
by Scott Garrabrant — 2015-07-06T22:55:42.000Z

This post is part of the Asymptotic Logical Uncertainty series. Here, I will explain how to talk about other parts of logical uncertainty in the framework of Asymptotic Logical Uncertainty (ALU).

Much work has been done in Logical Uncertainty on the question on how to choose a random complete logical extension of a given incomplete theory. This is equivalent to the question of how to choose a coherent probability assignment as described in section 3 here.

One of the most important properties of a coherent probability assignment is that the probabilities be computably approximable. We can define computably approximable in terms of the machines used in ALU.

Let P be a coherent probability assignment. Let M be a Turing machine which on input N runs quickly and outputs a probability M(N), which represents the probability assigned to ϕN. For each sentence ψ, let {ψn} be defined by ψ0=ψ and ψn+1=¬¬ψn.

If for every ψ, we have limn→∞M(sn)=P(ψ), then we say M computably approximates P and P is computably approximable.

Now, given your favorite computably approximable coherent probability assignment P, we can ask the following question:

Is it possible for a machine M to simultaneously get the correct limiting probability for all irreducible patterns, and converge to probability P(ψ) on {ψn}?

Note that BenfordLearner gets the correct probability whenever ψ is provable or disprovable, because then {ψn} is itself an irreducible pattern with probability 1 or 0. However, when ψ is independent, it is not clear whether or not the probabilities assigned to {ψn} even converge.

Getting the correct probability on irreducible patterns is only affected by the probabilities assigned to provable or disprovable sentences, while further getting the correct probabilities on {ψn} is only affected by the probabilities assigned to independent sentences. It is therefore feasible to satisfy both conditions at once, but we cannot easily combine them, because there is no way to determine which sentences are independent.

Using the ALU framework, we can also generalize notions from the theory of random logical extensions. Just to give an example, given a machine M and a simple increasing infinite sequence {sn}, such that ϕsn↔ϕsn+1 for all n, one can ask whether or not M(sn) always converges. This is a generalization of requirement that for coherent P, P(ϕ)=P(ψ) whenever ϕ↔ψ.

There are some choices to be m

...
Meetup : Paris Meetup: Saturday, July 11 
by Emile — 2015-07-06T21:42:56.543Z

Discussion article for the meetup : Paris Meetup: Saturday, July 11

WHEN:
11 July 2015 02:00:00PM (+0200)

WHERE:
51 Rue de Turbigo, 75003 Paris, France

The irregular-and-last-minute-schedule Paris Meetup! (as usual, we discuss it on the mailing list first, lesswrong-paris@googlegroups.com)
So meet us in front of the Arts & Metiers this Saturday!

Discussion article for the meetup : Paris Meetup: Saturday, July 11
European Community Weekend 2015 - Followup
by Lachouette — 2015-07-06T21:20:29.983Z

Seven months ago, the Berlin LW community announced the second European LessWrong Community Weekend. We wrote:

From June 12th to 14th awesome people from all across Europe are coming to Berlin to meet, exchange ideas and start projects. The focus is on forming new and strengthening existing ties between our local communities. In addition to being a vibrant social event, it’s also about sharing your world-improvement projects as well as about teaching and learning valuable skills.

After the success of the 2014 Community Weekend, we were aiming for an even better and larger 2015 event. Three weeks ago, 74 participants from 16 countries met up in Berlin for an extended weekend of talking, learning, playing and connecting.

Compared to the first event, we gave participants a lot more dedicated space and time for spontaneous activities or conversations by offering content in parallel sessions. This way, everyone could create their own weekend schedule based on the existing content offers or extend it by choosing to run activities or organizing something spontaneously. This was used by many participants and the schedule quickly filled with workout sessions, swimming in the nearby lake, construction of a large blanket fort, massages, telescope-aided stargazing and many more. Since this flexibility was received well, next year’s event will be planned similarly.

The main highlights of the event as planned were the keynote by Michael Valentine Smith, a four-hour workshop focused on CFAR-style debugging by Kaj Sotala and the premiere of the second of Daniel Böttger’s Seven Secular Sermons. This sparked the idea to record Sermon excerpts in song, e.g. to the melody of GoT’s “Rains of Castamere”. Additionally, professional facilitator Alisa Leube lead a session of relating games in which participants were guided through connection exercises to create a setting for mutual trust. Many participants reported that the games gave them a chance to be more open throughout the weekend and did their part in creating a warm, welcoming atmosphere.

To provide diverse topics as discussion starters and get as many people on stage as possible we reserved five hours for lightning talks, providing both reserved spots as well as spontaneous talks. This worked amazingly well and we got overwhelmingly positive feedback. The topics ranged from a case for...
Meetup : LW Cologne meetup
by EGI — 2015-07-06T20:49:11.428Z

Discussion article for the meetup : LW Cologne meetup

WHEN:
11 July 2015 05:00:00PM (+0200)

WHERE:
Marienweg 43, 50858 Köln

Hi everyone,

time for lw-cologne again. My place (Marienweg 43 50858 Köln) 5pm on July 11th. Food and content (discussion topics, games, whatever) appreciated but not necessary. Please pm me one day in advance if you want to come.

Discussion article for the meetup : LW Cologne meetup
Presidents, asteroids, natural categories, and reduced impact
by Stuart_Armstrong — 2015-07-06T17:44:37.268Z

A putative new idea for AI control; index here.

EDIT: I feel this post is unclear, and will need to be redone again soon.

This post attempts to use the ideas developed about natural categories in order to get high impact from reduced impact AIs.

Extending niceness/reduced impact

I recently presented the problem of extending AI "niceness" given some fact X, to niceness given ¬X, choosing X to be something pretty significant but not overwhelmingly so - the death of a president. By assumption we had a successfully programmed niceness, but no good definition (this was meant to be "reduced impact" in a slight disguise).

This problem turned out to be much harder than expected. It seems that the only way to do so is to require the AI to define values dependent on a set of various (boolean) random variables Zj that did not include X/¬X. Then as long as the random variables represented natural categories, given X, the niceness should extend.

What did we mean by natural categories? Informally, it means that X should not appear in the definitions of these random variables. For instance, nuclear war is a natural category; "nuclear war XOR X" is not. Actually defining this was quite subtle; diverting through the grue and bleen problem, it seems that we had to define how we update X and the Zj given the evidence we expected to find. This was put in equation as picking Zj's that minimize

Variance{log[ P(X∧Z|E)*P(¬X∧¬Z|E) / P(X∧¬Z|E)*P(¬X∧Z|E) ]}

where E is the random variable denoting the evidence we expected to find. Note that if we interchange X and ¬X, the ratio inverts, the log changes sign - but this makes no difference to the variance. So we can equally well talk about extending niceness given X to ¬X, or niceness given ¬X to X.

Perfect and imperfect extensions

The above definition would work for an "perfectly nice AI". That could be an AI that would be nice, given any combination of estimates of X and Zj. In practice, because we can't consider every edge case, we would only have an "expectedly nice AI". That means that the AI can fail to be nice in certain unusual and unlikely edge cases, in certain strange set of values of Zj that almost never come up...

...or at least, that almost never come up, given X. Since the "expected niceness" was calibrated given X, the such an expectedly nice AI ...
Grue, Bleen, and natural categories
by Stuart_Armstrong — 2015-07-06T13:47:26.487Z

A putative new idea for AI control; index here.

In a previous post, I looked at unnatural concepts such as grue (green if X was true, blue if it was false) and bleen. This was to enable one to construct the natural categories that extend AI behaviour, something that seemed surprisingly difficult to do.

The basic idea discussed in the grue post was that the naturalness of grue and bleen seemed dependent on features of our universe - mostly, that it was easy to tell whether an object was "currently green" without knowing what time it was, but we could not know whether the object was "currently grue" without knowing the time.

So the naturalness of the category depended on the type of evidence we expected to find. Furthermore, it seemed easier to discuss whether a category is natural "given X", rather than whether that category is natural in general. However, we know the relevant X in the AI problems considered so far, so this is not a problem.

Natural category, probability flows

Fix a boolean random variable X, and assume we want to check whether the boolean random variable Z is a natural category, given X.

If Z is natural (for instance, it could be the colour of an object, while X might be the brightness), then we expect to uncover two types of evidence:

those that change our estimate of X; this causes probability to "flow" as follows (or in the opposite directions):

...and those that change our estimate of Z:

Or we might discover something that changes our estimates of X and Z simultaneously. If the probability flows to X and and Z in the same proportions, we might get:

What is an example of an unnatural category? Well, if Z is some sort of grue/bleen-like object given X, then we can have Z = X XOR Z', for Z' actually a natural category. This sets up the following probability flows, which we would not want to see:

More generally, Z might be constructed so that X∧Z, X∧¬Z, ¬X∧Z and ¬X∧¬Z are completely distinct categories; in that case, there are more forbidden probability flows:

and

In fact, there are only really three "linearly independent" probability flows, as we shall see.

Less pictures, more math

Let's represent the four possible state of affairs by four weights (not probabilities):

Since everything is easier when it's linear, let's set w11 = log(P(X∧Z)) and similarly for the other weights (we neglect cases where some events have zero probability). Weights ...
Zooming your mind in and out
by John_Maxwell — 2015-07-06T12:30:58.509Z

I recently noticed I had two mental processes opposing one another in an interesting way.

The first mental process was instilled by reading Daniel Kahneman on the focusing illusion and Paul Graham on procrastination. This process encourages me to "zoom out" when engaging in low-value activities so I can see they don't deliver much value in the grand scheme of things.

The second mental process was instilled by reading about the importance of just trying things. (These articles could be seen as steelmanning Mark Friedenbach's recent Less Wrong critique.) This mental process encourages me to "zoom in" and get my hands dirty through experimentation.

Both these processes seem useful. Instead of spending long stretches of time in either the "zoomed in" or "zoomed out" state, I think I'd do better flip-flopping between them. For example, if I'm wandering down internet rabbit holes, I'm spending too much time zoomed in. Asking "why" repeatedly could help me realize I'm doing something low value. If I'm daydreaming or planning lots with little doing, I'm spending too much time zoomed out. Asking "how" repeatedly could help me identify a first step.

This fits in with construal level theory, aka "near/far theory" as discussed by Robin Hanson. (I recommend the reviews Hanson links to; they gave me a different view of the concept than his standard presentation.) To be more effective, maybe one should increase cross communication between the "near" and "far" modes, so the parts work together harmoniously instead of being at odds.

If Hanson's view is right, maybe the reason people become uncomfortable when they realize they are procrastinating (or not Just Trying It) is that this maps to getting caught red-handed in an act of hypocrisy in the ancestral environment. You're pursuing near interests (watching Youtube videos) instead of working towards far ideals (doing your homework)? For shame!

(Possible cure: Tell yourself that there's nothing to be ashamed of if you get stuck zoomed in; it happens to everyone. Just zoom out.)

Part of me is reluctant to make this post, because I just had this idea and it feels like I should test it out more before writing about it. So here are my excuses:

1. If I wait until I develop expertise in everything, it may be too late to pass it on.

2. In order to see if...
Green Emeralds, Grue Diamonds
by Stuart_Armstrong — 2015-07-06T11:27:07.710Z

A putative new idea for AI control; index here.

When posing his "New Riddle of Induction", Goodman introduced the concepts of "grue" and "bleen" to show some of the problems with the conventional understanding of induction.

I've somewhat modified those concepts. Let T be a set of intervals in time, and we'll use the boolean X to designate the fact that the current time t belongs to T (with ¬X equivalent to t∉T). We'll define an object to be:

Grue if it is green given X (ie whenever t∈T), and blue given ¬X (ie whenever t∈T).

Bleen if it is blue given X, and green given ¬X.

At this point, people are tempted to point out the ridiculousness of the concepts, dismissing them because of their strange disjunctive definitions. However, this doesn't really solve the problem; if we take grue and bleen as fundamental concepts, then we have the disjunctively defined green and blue; an object is:

Green if it is grue given X, and bleen given ¬X.

Blue if it is bleen given X, and grue given ¬X.

Still, the categories green and blue are clearly more fundamental than grue and bleen. There must be something we can whack them with to get this - maybe Kolmogorov complexity or stuff like that? Sure someone on Earth could make a grue or bleen object (a screen with a timer, maybe?), but it would be completely artificial. Note that though grue and bleen are unnatural, "currently grue" (colour=green XOR ¬X) or "currently bleen" (colour=blue XOR ¬X) make perfect sense (though they require knowing X, an important point for later on).

But before that... are we so sure the grue and bleen categories are unnatural? Relative to what?

Welcome to Chiron Beta Prime

Chiron Beta Prime, apart from having its own issues with low-intelligence AIs, is noted for having many suns: one large sun that glows mainly in the blue spectrum, and multiple smaller ones glowing mainly in the green spectrum. They all emit in the totality of the spectrum, but they are stronger in those colours.

Because of the way the orbits are locked to each other, the green suns are always visible from everywhere. The blue sun rises and sets on a regular schedule; define T to be time when the blue sun is risen (so X="Blue sun visible, some green suns visible" and ¬X="Blue sun not visible, some green suns visi...
A question and a tail
by  — 2015-07-06T09:34:33.963Z

This is a rambling post, and I will appreciate your criticism to help dry it or delete it altogether.

It seems that however little a question I research by reviewing [botanical] literature, there is always a much more complex, and rather difficult to rigorously put, question that I have to ask for the first one to be meaningful. The second answer (or tier of answers) doesn't add much to the information I will build upon, but it might - just might! - add uncertainty to the result or allow predictions in advance. How do we use it in advance? We don't apply formal reasoning, usually, and yet somehow we use it!
1.
Consider: a certain invasive plant has a host of adaptations beneficial to its success. (They probably wouldn't be sufficient if there were some actual effort to manage manmade ecosystems, but duh.) A trait many IP share is the ability to increase their ploidy - from 2 to 3, 4, 6, 8 or even 10 sets of homologous chromosomes, etc. (Polyploidization sometimes happens even in single cells in somatic (= non-reproductive) tissues, so it's really a heavily used shortcut.)
Now, suppose I want to see how a different specific property of the species behaves abroad. I will have to check the ploidy level, of course! Quick, what does the literature say, how many chromosomes can it have?
...but wait. Make no mistake, I do have to count them; but what if there is a continent-wide study showing that it generally has 4n in Eastern Europe?.. That would allow me to at least expect 4n, or whatever amount they found, and see if there is any research specifically dealing with this situation within its native range.
...but wait. Of course, those findings will be useful in discussion if I find 4n, but if I don't, they will be just a point in the overall space of possibilities. Still relevant, but not worth putting much explanatory weight on.
Something in my brain evaluated the usefulness of a piece of data other people have found, which I myself have yet to look up, of whose exact composition I have no idea - perhaps there are simply no other reports! - and placed it in context of what I really expect to do.
2.
Okay, if I can think so about other people's writings without even reading them, then maybe I can compile a dummy set of data I expect right now and compare them to those I will find in the literature. And later, to actual data. Here's a simplified problem that doesn't approac...
Open Thread, Jul. 6 - Jul. 12, 2015
by MrMind — 2015-07-06T07:31:51.802Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Asymptotic Logical Uncertainty: Passing the Benford Test
by Scott Garrabrant — 2015-07-06T03:34:36.000Z

This post is part of the Asymptotic Logical Uncertainty series. Here, we give the proof that BenfordLearner passes the Benford test.

We start with 2 Lemmas.

**Lemma 1: Let S be an irreducible pattern with probability p, and let Z be a Turing machine such that UTM(Z,N) accepts in time T(N) if and only if N∈S. There exists a constant C such that if N∈S, then there exists a P∈JN such that maxY∈TM(N)BN(Z,Y,P)<C.
**

Proof: Let P=⌊pN⌋N. From the definition of irreducible pattern, we have that there exists c such that for all Y,

|FN(Z,Y)−p|<cK(Y)√loglogQN(Z,Y)√QN(Z,Y).
Clearly,
|P−p|≤1N≤1QN(Z,Y)≤1√QN(Z,Y)≤K(Z)K(Y)√loglogQN(Z,Y)√QN(Z,Y).
Setting C=K(Z)+c, we get
|FN(Z,Y)−P|≤|FN(Z,Y)−p|+|P−p|<CK(Y)√loglogQN(Z,Y)√QN(Z,Y),
so
|FN(Z,Y)−P|√QN(Z,Y)K(Y)√loglogQN(Z,Y)<C.

Clearly, K(Z)<C, so BN(Z,Y,P)>C for all Y. Therefore, maxY∈TM(N)BN(Z,Y,P)<C.

□

Lemma 2: Let S be an irreducible pattern with probability p, and let Z be a Turing machine such that UTM(Z,N) accepts in time T(N) if and only if N∈S. For all C, for all ε>0, for all N sufficiently large, for all P∈JN, if N∈S, and minX∈TM(N)BN(X,Z,P)<C, then |P−p|<ε.

Proof: Fix a C and a ε>0. It suffices to show that for all N sufficiently large, if N∈S and |P−p|≥ε, then for all X∈TM(N), we have BN(X,Z,P)≥C.

Observe that since BN(X,Z,P)≥K(X), this claim trivially holds when K(X)≥C. Therefore we only have to check the claim for the finitely many Turing machines expressible in fewer than C bits.

Fix an arbitrary X. Since S is an irreducible pattern, there exists a c such that
|FN(X,Z)−p|<cK(Z)√loglogQN(X,Z)√QN(X,Z).
We may assume that S′(X,Z) is infinite, since otherwise if we take N∈S large enough, X∉TM(N). Thus, by taking N sufficiently large, we can get QN(X,Y) sufficiently large, and in particular satisfy √QN(X,Z)K(Z)√loglogQN(X,Z)ε≥C+c.
Take N∈S large enough that this holds for each X∈TM(N) with K(X)≥C, and assume |P−p|≥ε. By the triangle inequality, we have
|FN(X,Z)−P|≥|P−p|−|FN(X,Z)−p|≥ε−cK(Z)√loglogQN(X,Z)√QN(X,Z). Therefore BN(X,Z,P)≥(ε−cK(Z)√loglogQN(X,Z)√QN(X,Z))√QN(X,Z)K(Z)√loglogQN(X,Z)=√QN(X,Z)K(Z)√loglogQN(X,Z)ε−c≥C, which proves the claim.

□

Main Theorem: Let S be an irreducible pattern with probability p. Then limN→∞N∈SBenfordLearner(N)=p.

Proof: Let Z be a Turing machine such that UTM(Z,N) accepts in time T(N) if and only if N∈S.

By considering the case when X=Z, Lemma 1 implies that t

...