trentmkelly/LessWrong-43k huggingface dataset:
AI, cure this fake person's fake cancer!
by Stuart_Armstrong — 2015-08-24T16:42:16.545Z

A putative new idea for AI control; index here.

An idea for how an we might successfully get useful work out of a powerful AI.

The ultimate box

Assume that we have an extremely detailed model of a sealed room, with a human in it and enough food, drink, air, entertainment, energy, etc... for the human to survive for a month. We have some medical equipment in the room - maybe a programmable set of surgical tools, some equipment for mixing chemicals, a loud-speaker for communication, and anything else we think might be necessary. All these objects are specified within the model.

We also have some defined input channels into this abstract room, and output channels from this room.

The AI's preferences will be defined entirely with respect to what happens in this abstract room. In a sense, this is the ultimate AI box: instead of taking a physical box and attempting to cut it out from the rest of the universe via hardware or motivational restrictions, we define an abstract box where there is no "rest of the universe" at all.

Cure cancer! Now! And again!

What can we do with such a setup? Well, one thing we could do is to define the human in such a way that they have some from of advanced cancer. We define what "alive and not having cancer" counts as, as well as we can (the definition need not be fully rigorous). Then the AI is motivated to output some series of commands to the abstract room that results in the abstract human inside not having cancer. And, as a secondary part of its goal, it outputs the results of its process.

Initially, the AI is likely to simply kill the human, using some hole in the definition of "alive and not having cancer" that we haven't seen. We can then tighten up the definition, and possibly give the AI more tools inside the abstract room. After a few iterations of this, the AI is likely to be able to "cure cancer" in a way that meets our approximate requirements.

And then, it suffices for us to analyse the process used, and its outcome, to establish how the AI went about its task. If the room was defined in sufficient detail, it is likely that that the process can be adapted to curing cancer in the real world. If not, we can vary the input - defining agents with different types of cancer, comparing the AI's approaches, maybe requiring the AI's solution to work across the different situations. Thus it seems that we have extract...
Meetup : San Francisco Meetup: When to trust experts
by rocurley — 2015-08-24T15:45:21.571Z

Discussion article for the meetup : San Francisco Meetup: When to trust experts

WHEN:
24 August 2015 06:15:00PM (-0700)

WHERE:
1061 Market St #4, San Francisco, CA 94103

We'll be meeting to talk about when to trust what experts say, and what to do when there are no experts. There's been some discussion about this on the internet recently: Topher and Scott for example.

I can be reached at 301-458-0764 if you need help getting in. As always, feel free to show up late.

Discussion article for the meetup : San Francisco Meetup: When to trust experts
Meetup : Urbana-Champaign: Quorum for discourse
by Manfred — 2015-08-24T11:36:21.602Z

Discussion article for the meetup : Urbana-Champaign: Quorum for discourse

WHEN:
06 September 2015 02:00:00PM (-0500)

WHERE:
Altgeld Hall, W. Green Street, Urbana, IL, 61801

Another year, another chance to come to a LW meetup. Find us at the scenic north entrance of Altgeld Hall. I'll bring delicious food.
Depending on what people want, discussion may vary in technical level. Math is good, but sometimes fun digressions are fine too.

Discussion article for the meetup : Urbana-Champaign: Quorum for discourse
Meetup : Less Wrong Melbourne September Dojo: Vagueness
by Patrick — 2015-08-24T10:45:01.050Z

Discussion article for the meetup : Less Wrong Melbourne September Dojo: Vagueness

WHEN:
06 September 2015 08:44:00PM (+1000)

WHERE:
Ross House Association, 247 Flinders Lane Melbourne

This month's dojo will be split into two parts: Theory and practice. The theory part will consist of a lecture, and selected readings. The practice part will consist of various rationality drills.
The lecture will cover vagueness: why it's bad, and how to get around it.
Groups will read one of the following excerpts
1. "The Virtue of Narrowness"
2. "Mysterious Answers to Mysterious Questions."
3. "Making beliefs pay rent (in anticipating experiences)"
Schedule below:
3:30pm Arrive
4:00pm: Welcome, intro to less wrong
4:05pm: Lecture on Vagueness
4:30pm: Q&A
4:35: group reading from Rationality: AI to Zombies
4:50: break
5:00 group presents what they've learned
5:15pm: Lightning Talk: Willpower
5:25pm: Meditation
5:30pm: Break
5:40pm: Paired Debugging
5:55pm: Lightning Talk: Trigger Action Planning
6:00pm: Trigger Action Planning (DIY)
6:05pm: Present to paired debugging partner(s)
6:15pm: Being Strategic
6:25pm: Feedback
6:30pm: Support Lottery / Announcements / Wrap up
6:35pm: Dinner

Discussion article for the meetup : Less Wrong Melbourne September Dojo: Vagueness
Open Thread - Aug 24 - Aug 30
by Elo — 2015-08-24T08:14:31.129Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
List of common human goals
by Elo — 2015-08-24T07:58:50.692Z

List of common goal areas:

This list is meant to be in the area of goal-space. It is non-exhaustive and the descriptions are including but not limited to - some hints to help you understand where in the idea-space these goals land. When constructing this list I try to imagine a large venn diagram where sometimes they overlap. The areas mentioned are areas that have an exclusive part to them; i.e. where sometimes knowledge overlaps with self-awareness there are parts of each that do not overlap; so both are mentioned. If you prefer a more "focussing" or feeling base description; Imagine each of these goals is a hammer, designed with a specific weight to hit a certain note on a xylophone. Often one hammer can produce the note that is meant for that key and several other keys as well. But sometimes they can't quite make them sound perfect. What is needed is the right hammer for that block to hit the right note and make the right sound. Each of these "hammers" has some note that cannot be produced through the use of other hammers.

This list has several purposes:

For someone with some completed goals who is looking to move forward to new horizons; help you consider which common goal-pursuits you have not explored and if you want to try to strive for something in one of these directions.

For someone without clear goals who is looking to create them and does not know where to start.

For someone with too many specific goals who is looking to consider the essences of those goals and what they are really striving for.

For someone who doesn't really understand goals or why we go after them to get a better feel for "what" potential goals could be.

What to do with this list?

0. Agree to invest 30 minutes of effort into a goal confirmation exercise as follows.

Go through this list (copy paste to your own document) and cross out the things you probably don't care about. Some of these have overlapping solutions of projects that you can do that fulfils multiple goal-space concepts. (5mins)

For the remaining goals; rank them either "1 to n", in "tiers" of high to low priority or generally order them in some way that is coherent to you. (For serious quantification; consider giving them points - i.e. 100 points for achieving a self-awareness and understanding goal but a pleasure/creativity goal might be only wo

...
Asymptotic Logical Uncertainty: Iterated Resource Bounded Solomonoff Induction
by Scott Garrabrant — 2015-08-24T04:39:57.000Z

This post is part of the Asymptotic Logical Uncertainty series. In this post, I present a possible fix to the strategy of using resource bounded Solomonoff induction for asymptotic logical uncertainty. This algorithm came out of conversations with Benja Fallenstein when I visited MIRI this week.

Warning: I do not have any positive results about this algorithm yet. I just suspect that it has a nontrivial chance of having some good properties, so I wanted to share it in case anyone else wants to think about it.

This algorithm takes in a Turing machine E which outputs an infinite string of bits, an integer n. The algorithm must "quickly" output a bit, and the probability that it outputs 1 represents the "probability" that the nth bit output by E is a 1.

IRBSI(E,n)
0: w=""
1: Run E for n steps to get the first k bits
2: for i = 1 to n
3: sample a randorm Turing machine M
4: for j = 1 to k
5: run M for 2^(n+j-i) steps on input the first j-1 bits output by E
6: if M does not output the jth bit output by E
7: break and goto 3
8: run M for time 2^n on input w
10: if M outputs a 1
11: concatinate a 1 to the end of w
12: else
13: concatinate a 0 to the end of w
14: return the last bit of w

The runtime of this algorithm is in O(n2n), if E is sufficiently slow.

The rough justification behind this algorithm is that the Solomonoff induction inspired approach is good at getting good probabilities for bits conditioned on all the the previous bits. The problems come from the algorithm applying those conditional probabilities to past bits that were necessarily computed with fewer resources.

This algorithm only uses Solomonoff induction only to get conditional probabilities. It uses these conditional probabilities in such a way that they are only combined with other conditional probabilities computed with the exact same 2n amount of resources. Therefore it does not run into problems from trying to stay consistent with its past dumber self.

I do not know how good this algorithm will end up being, but I suspect that it gets simple things like this correct.

It may be that to get good results for this algorithm you have to use tricks like we used with the original Solomonoff approach by sampling the Turing machines in a way that is further biased towards simple machines, or by allowing the machines

...
Asymptotic Logical Uncertainty: Introduction
by Scott Garrabrant — 2015-08-24T04:38:06.000Z

In this post, I will introduce a new way of thinking about logical uncertainty. The main goal of logical uncertainty is to learn how to assign probabilities to logical sentences which have not yet been proven true or false.

One common approach is to change the question, assume logical omniscience and only try to assign probabilities to the sentences that are independent of your axioms (in hopes that this gives insight to the other problem). Another approach is to limit yourself to a finite set of sentences or deductive rules, and assume logical omniscience on them. Yet another approach is to try to define and understand logical counterfactuals, so you can try to assign probabilities to inconsistent counterfactual worlds.

One thing all three of these approaches have in common is they try to allow (a limited form of) logical omniscience. This makes a lot of sense. We want a system that not only assigns decent probabilities, but which we can formally prove has decent behavior. By giving the system a type of logical omniscience, you make it predictable, which allows you to prove things about it.

However, there is another way to make it possible to prove things about a logical uncertainty system. We can take a program which assigns probabilities to sentences, and let it run forever. We can then ask about whether or not the system eventually gives good probabilities.

At first, it seems like this approach cannot work for logical uncertainty. Any machine which searches through all possible proofs will eventually give a good probability (1 or 0) to any provable or disprovable sentence. To counter this, as we give the machine more and more time to think, we have to ask it harder and harder questions.

We therefore have to analyze the machine's behavior not on individual sentences, but on infinite sequences of sentences. For example, instead of asking whether or not the machine quickly assigns 110 to the probability that the 3↑↑↑↑3rd digit of π is a 5 we look at the sequence:

an:= the probability the machine assigns at timestep 2n to the n↑↑↑↑nth digit of π being 5,

and ask whether or not this sequence converges to 110.

Here is the list of posts in this sequence:

Introduction

The Benford Test

Solomonoff Induction Inspired Approach

Irreducible Patterns

A Benford Learner

Passing the Berford Test

Connection to Random Logical Extensions

Concrete Failure of the Solomonoff Approach

...
Asymptotic Logical Uncertainty: Self Reference
by Scott Garrabrant — 2015-08-24T03:22:22.000Z

Part of the Asymptotic Logical Uncertainty series. Here, I will start discussing the connection between asymptotic logical uncertainty and self reference.

For this post, I will switch back to my old model of asymptotical logical uncertainty. Let E be a probabilistic Turing machine which outputs an infinite string of bits. We want to construct a Turing machine M which when input with the code to T must run quickly, (e.g. must output its nth bit by time n^2) and is trying to match the output of E.

We want M to satisfy properties like the Benford Test, where if E outputs a simple subsequence that looks like it is (or actually is) coming from a p-coin, we would like the probability that M outputs a 1 for bits in that subsequence to converge to p.

We have seen one failed approach to this inspired by Solomonoff induction, and one partially successful approach, the Benford learner. (The Benford learner is in a different context, but can be easily ported over.) I will present some other programs in this context soon, including some possible fixes to the Solomonoff approach. However, right now I want to present another desired property for M.

Let E be the Turing machine which quines to run M on E, and negates the output. The the probability that the nth bit output by M on input E is a 1 should converge to 12.

The reason this is a desired property is that the probability of getting each bit correct is maximized when that probability is 12.

The Benford learner very likely gets this problem wrong. That is because it is not trying to maximize its accuracy, but instead has a pattern recognition algorithm hard coded in. An algorithm that has this desired property probably looks a lot more like Solomonoff induction. The failed Solomonoff induction inspired approach might actually gets this specific problem correct, in spite of failing the Benford Test. I have not checked.

There is another more deterministic version of the problem. Let E be the Turing machine which quines to compute the probability that M on input E outputs a 1, and outputs a 1 if and only if that probability is less than 12. The the probability that the nth bit output by M on input E is a 1 should converge to 12.

The purpose of the deterministic version is to make it work when E represents a list of logical sentences.

There are many ways we can generalize both of these desired properties. For example, we can add noi

...
Asymptotic Logical Uncertainty: A Benford Learner
by Scott Garrabrant — 2015-08-24T02:14:48.000Z

This post is part of the Asymptotic Logical Uncertainty series. In this post, I will give an algorithm, BenfordLearner, that passes the Benford test. This algorithm further satisfies limN→∞,N∈SBenfordLearner(N)=p whenever S is an irreducible pattern with probability p.

In the following algorithm, we define logq to be 1 whenever q≤2.

BenfordLearner(N)
1: P=0
2: M=N
3: for j=0 to N
4: M_Y=0
5: for Y a TM expressible in K_Y <log N bits
6: M_X=N
7: for Y a TM expressible in K_X <log N bits
8: if UTM(X,N) and UTM(Y,N) both accept in time T(N)
9: A=0
10: R=0
11: i=1
12: while i<=N
13: if UTM(X,i) and UTM(Y,i) both accept in time T(i)
14: if UTM(L,i) accepts in time T(N)
15: A=A+1
16: else if UTM(L,i) rejects in time T(N)
17: R=R+1
18: else
19: i=N
20: i=i+1
21: F=A/(A+R)
22: Q=A+R
23: if max(K_X,|F-j/N|*sqrt(Q)/K_Y/sqrt(log log Q))<M_X
24: M_X=max(K_X,|F-j/N|*sqrt(Q)/K_Y/sqrt(log log Q))
25: if M_X>M_Y
26: M_Y=M_X
27: if M_Y<M
28: M=M_Y
29: P=j/N
30: return P

Let TM(N) be the set of all Turing machines X expressible in at most logN bits such that UTM(X,N) accepts in time at most T(N). The encoding of Turing machines must be prefix-free, which in particular means that no Turing machine is encoded in 0 bits. Let JN denote the set of rational numbers of the form jN with j=0,…,N.

For X and Y Turing machines, let K(X) be the number of bits necessary to encode X. Let S′(X,Y) be the subset of natural numbers i which are accepted by both UTM(X,i) and UTM(Y,i) in time at most T(i). Let QN(X,Y) be the greatest number less than or equal to N such that for every s in the first QN(X,Y) elements of S′, UTM(L,s) halts in time T(N). Let FN(X,Y) be the proportion of the first QN(X,Y) elements of S′ which L accepts. Let BN(X,Y,P)=max(K(X),|FN(X,Y)−P|√QN(X,Y)K(Y)√loglogQN(X,Y)).

Lemma: The output of BenfordLearner(N) is in
arg minP∈JNmaxY∈TM(N)minX∈TM(N)BN(X,Y,P).

Proof: The algorithm has three for loops, the outer ranging over j=0,…N and the inner two ranging over Y and X respectively, both restricted to Turing machines expressible in logN bits. The condition on line 8 means that X and Y effectively range over all Turing machines in TM(N), and P=jN ranges

...
The Sleeping Beauty problem and transformation invariances
by aspera — 2015-08-23T20:57:48.676Z

I recently read this blog post by Allen Downey in response to a reddit post in response to Julia Galef's video about the Sleeping Beauty problem. Downey's resolution boils down to a conjecture that optimal bets on lotteries should be based on one's expected state of prior information just before the bet's resolution, as opposed to one's state of prior information at the time the bet is made.

I suspect that these two distributions are always identical. In fact, I think I remember reading in one of Jaynes' papers about a requirement that any prior be invariant under the acquisition of new information. That is to say, the prior should be the weighted average of possible posteriors, where the weights are the likelihood that each posterior would be acheived after some measurement. But now I can't find this reference anywhere, and I'm starting to doubt that I understood it correctly when I read it.

So I have two questions:

1) Is there such a thing as this invariance requirement? Does anyone have a reference? It seems intuitive that the prior should be equivalent to the weighted average of posteriors, since it must contain all of our prior knowledge about a system. What is this property actually called?

2) If it exists, is it a corollary that our prior distribution must remain unchanged unless we acquire new information?
Asymptotic Logical Uncertainty: Uniform Coherence 2
by Scott Garrabrant — 2015-08-23T20:50:54.000Z

Part of the Asymptotic Logical Uncertainty series. A couple weeks ago, I proposed a definition of uniform coherence. I decided to change the definition so that it applies to a more general context. This post is a self contained definition of uniform coherence designed so that people can start working on the problem with no background other than this post.

Every Turing machine discussed in this post will have two infinite blank tapes to work with in addition to an infinite write only output tape. There will be a special character # for the output tape which will separate the output tape into chunks.

A sentence enumerator is a machine which outputs an infinite number of # symbols, where between each pair of adjacent # symbols is a binary string which encodes a logical sentence ϕ. If M is a sentence enumerator, we let M(t) denote the sentence written between the two most recent # symbols at time t. If M has not yet written two # symbols by time t, we say that M(t)=⊥.

Similarly, a logical predictor is a machine which outputs an infinite number of # symbols, where between each pair of adjacent # symbols is a binary string which encodes a finite sequence of ordered pairs, with each ordered pair containing a logical sentence ϕ and a rational number 0≤p≤1. If L is a logical predictor, we let L(ϕ,t) denote the rational number p such that when L outputs its tth # symbol, (ϕ,p) has been output between more recently than any other pair of the form (ϕ,p′). If L has not yet output any pair of the form (ϕ,p′) by time it outputs t # symbols, we say that L(ϕ,t)=12.

We say that a logical predictor L is uniformly coherent if the following three axioms hold:

limt→∞L(⊥,t)=0.

If M is a sentence enumerator such that for all n the sentence M(n)→M(n+1) is provable, then limt→∞L(M(t),t) exists.

If M1, M2, and M3 are three sentence enumerators such that for all n sufficiently large, the sentence "Exactly one of M1(n), M2(n), and M3(n) is true" is provable, then limt→∞L(M1(t),t)+L(M2(t),t)+L(M3(t),t)=1.

Open Question: Do there exist any uniformly coherent logical predictors?

[EDIT: This problem is no longer open]

If there are uniformly coherent logical predictors, how frequently can we make a uniformly coherent logical predictor output a # symbol?

Given a logical predictor L, observe that for any sentence ϕ, axiom 2 implies that there must be some real number PL(ϕ) such that limt→∞L(ϕ

...
Asymptotic Logical Uncertainty: The Modified Demski Prior is Uniformly Coherent
by Scott Garrabrant — 2015-08-23T20:48:21.000Z

Part of the Asymptotic Logical Uncertainty series. This post is especially dependent on the previous two posts. Here, we show that the Modified Demski Prior is Uniformly Coherent. This result came out of discussion with Benja Fallenstein, Jessica Taylor, and myself, and is primarily due to Benja.

Theorem: The Modified Demski Prior is Uniformly Coherent.

Proof: We will start with property 3 of uniform coherence. Fix a triple M1, M2, and M3 which meet the conditions for property 3. Consider the Turing machine M′ which outputs all sentences of the form "Exactly one of M1(t), M2(t), and M3(t) is true" in order. (M can generate these sentences by simulating the three Turing machines, or if we assume we have PA in our starting theory, it could just make these statements about the Turing machines.) Note that every sentence output by M4 is provable in the base theory.

As t goes to infinity, the probability that M′ is sampled by M(t) goes to 1. Further, the probability that it is sampled and accepted goes to 1. This is because the only way it can be rejected is if there exists a contradiction in the sentences output by the Turing machines sampled earlier. For every list of Turing machines output earlier, there is a fixed value t after which M(t) will accept M′ if that list is the list of machines sampled before M′. For any ε we can take t large enough that this is true for the list of sentences sampled with probability 1−ε.

Therefore, as t goes to infinity, the probability that M(t) outputs the sentence "Exactly one of M1(t), M2(t), and M3(t) is true" goes to 1. The probability that M(t) later individually outputs the six Turing machines which give the single sentences M1(t), ¬M1(t), M2(t), ¬M2(t), M3(t), and ¬M3(t), consecutively in that order also goes to 1 as t goes to infinity, since the complexity of the Turing machine which outputs those sentences is only a constant more than the complexity it takes to output the integer t, which is on the order of logt, and we sample 2t Turing machines. Since K notices all propositional contradictions and must accept either Mi(t) or ¬Mi(t), the sum of the probabilities that M accepts M1(t), M2(t), and M3(t) must go to one. Therefore, limt→∞L(M1(t),t)+L(M2(t),t)+L(M3(t),t)=1.

The fact that the algorithm satisfies property 1 is trivial, so all that remains is to show that it satisfies property 2. Consider a Turing machine M′ which s

...