trentmkelly/LessWrong-43k huggingface dataset:
Finding reflective oracle distributions using a Kakutani map
by jessicata — 2017-05-02T02:12:06.000Z

(follow-up to: A correlated analogue to reflective oracles)

Motivation

Suppose players are playing in a correlated equilibrium using a reflective oracle distribution. How does the equilibrium they play in vary as a function of the parameters of the game, or of the players' policies? It turns out that the set of equilibria is a Kakutani map of the parameters to the game. This is a lot like it being a continuous map.

This might make it possible for players to reason about the effects of their policy on the equilibrium that they play (since the equilibrium is now a Kakutani map of the players' policies).

Definitions

Let k be some natural number. We will consider reflective oracle distributions whose queries are parameterized on some vector in θ∈Rk.

To do this, let the Turing machines M, instead of outputting a raw query, output a continuous function from θ∈Rk to the query. (The details of representing continuous functions don't seem that important). The reflectivity condition on oracle distributions is now relative to θ (since the queries depend on θ).

Define the map

ParamsToDistrs(θ):={D∈D|D is reflective relative to θ }

which maps the parameters θ to the set of reflective oracle distributions for θ.

Theorem: ParamsToDistrs is a Kakutani map.

Proof:

From the previous post, we have:

For each θ, ParamsToDistrs(θ) is nonempty (Theorem 1)

For each θ, ParamsToDistrs(θ) is convex (Theorem 2)

So it is sufficient to show that ParamsToDistrs has a closed graph.

Let θ1,θ2,... be an infinite sequence of Rk values converging to θ∞. Let D1,D2,... be an infinite sequence of oracle distributions in D converging to D∞. Assume that for each natural n, Dn is reflective relative to θn. We will show that D∞ is reflective relative to θ∞; this is sufficient to show that ParamsToDistrs has a closed graph.

Let M∈M. Let a be such that D∞(O(M)=a)>0. Then D∞(O(M)=a)>ϵ for some ϵ>0. By convergence, there is some N such that for all n≥N, Dn(O(M)=a)>ϵ. By reflectivity of each Dn relative to θi, we have, for each n≥N,

a∈Eval(M)(θn)(Condition(Dn,M,a))

Let qθ:=Eval(M)(θ). Rewriting the above statement:

a∈argmaxi∈{1,...,l(q)}EDn[qθni(O)]

Consider the set {(θ,D)|a∈argmaxi∈{1,...,l(q)}ED[qθi(O)]}. This is the intersection of a finite number of sets of the form {(θ,D)|ED[qθa(O)−qθi(O)]≥0}. Each of these sets is closed because (θ,D)↦ED[qθa(O

...
Experiments, iterations and the scientific method
by Elo — 2017-05-02T00:59:01.552Z

Original post: http://bearlamp.com.au/experiments-iterations-and-the-scientific-method

Today, an ordinary day. I woke up at 6am. It was still dark out. I did a quick self-check.

"Did I wake up with energy?" No not really... note to self. But it is quite early.

Rewind one day.

Yesterday I woke up to a phone call, and did a quick self test...

"Did I wake up with energy?" Yes! Like two cups of coffee and the light of a thousand suns.

"Did I have energy 5 minutes after waking up?" no.

"Did I have vivid dreams?" Yes

"are my fingers and toes cold?" Yes, damn.

(waking up next to me can be jarring and uncomfortable for a sleepy person)

Steps in the right direction.

I got up and did what has become my usual stack. Weighing out 5g creatine, 3g citrulline malate, 10-25 g of soylent v1.5, 60g protein. And to this I removed the Vitamin C and added magnesium.

Even my notes have notes. Confounding factors yesterday include:

High intensity exercise (run in the morning)

garlic (for dinner)

An Iron tablet taken to solve the cold hands/feet problem (status: null)

lots of carbs that I had as part of dinner with a friend.

experimenting in the real world is hard and experimenting on sleep is particularly slow. at the rate of 1 trial a day, that's many trials before you can confirm or deny a result.

Experiments

I recently rediscovered the Scientific method, by which I mean, I realised I wasn't applying it and I needed to figure out how to apply it to my life (haha... Not that I "Rediscovered the scientific method" and am awaiting my Nobel prize).

By ArchonMagnus - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=42164616

A few main features here. The ones that make it possible to apply these steps in real life.

There are two loops, the "run experiements" loop, and the, "make theories in preparation for experiments" loop.

The process goes on forever.

You need to have some knowledge of the world around you before you can generate hypotheses worth testing.

Hypotheses need to be run until you can confirm your models. And being able to disprove them would help.

you might need to rely on wrong-but-useful models on the way to finding the better models.

There are 7 parts to that circle. They are hard to remember. I looked for ways of combining ...
Open thread, May. 1 - May. 7, 2017
by Thomas — 2017-05-01T15:55:32.361Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
May 2017 Media Thread
by ArisKatsaris — 2017-05-01T10:26:48.963Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Bad intent is a disposition, not a feeling
by Benquo — 2017-05-01T01:28:58.345Z

It’s common to think that someone else is arguing in bad faith. In a recent blog post, Nate Soares claims that this intuition is both wrong and harmful:

I believe that the ability to expect that conversation partners are well-intentioned by default is a public good. An extremely valuable public good. When criticism turns to attacking the intentions of others, I perceive that to be burning the commons. Communities often have to deal with actors that in fact have ill intentions, and in that case it's often worth the damage to prevent an even greater exploitation by malicious actors. But damage is damage in either case, and I suspect that young communities are prone to destroying this particular commons based on false premises.

To be clear, I am not claiming that well-intentioned actions tend to have good consequences. The road to hell is paved with good intentions. Whether or not someone's actions have good consequences is an entirely separate issue. I am only claiming that, in the particular case of small high-trust communities, I believe almost everyone is almost always attempting to do good by their own lights. I believe that propagating doubt about that fact is nearly always a bad idea.

It would be surprising, if bad intent were so rare in the relevant sense, that people would be so quick to jump to the conclusion that it is present. Why would that be adaptive?

What reason do we have to believe that we’re systematically overestimating this? If we’re systematically overestimating it, why should we believe that it’s adaptive to suppress this?

There are plenty of reasons why we might make systematic errors on things that are too infrequent or too inconsequential to yield a lot of relevant-feeling training data or matter much for reproductive fitness, but social intuitions are a central case of the sort of things I would expect humans to get right by default. I think the burden of evidence is on the side disagreeing with the intuitions behind this extremely common defensive response, to explain what bad actors are, why we are on such a hair-trigger against them, and why we should relax this.

Nate continues:

My models of human psychology allow for people to possess good intentions while executing adaptations that increase their status, influence, or popularity. My models also don’t deem people poor allies merely on account of t

...
There is No Akrasia
by  — 2017-04-30T15:33:39.710Z

I don’t think akrasia exists.

This is a fairly strong claim. I’m also not going to try and argue it.

What I’m really here to argue are the two weaker claims that:

a) Akrasia is often treated as a “thing” by people in the rationality community, and this can lead to problems, even though akrasia a sorta-coherent concept.

b) If we want to move forward and solve the problems that fall under the akrasia-umbrella, it’s better to taboo the term akrasia altogether and instead employ a more reductionist approach that favors specificity

But that’s a lot less catchy, and I think we can 80/20 it with the statement that “akrasia doesn’t exist”, hence the title and the opening sentence.

First off, I do think that akrasia is a term that resonates with a lot of people. When I’ve described this concept to friends (n = 3), they’ve all had varying degrees of reactions along the lines of “Aha! This term perfectly encapsulates something I feel!” On LW, it seems to have garnered acceptance as a concept, evidenced by the posts / wiki on it.

It does seem, then, that this concept of “want-want vs want” or “being unable to do what you ‘want’ to do” seems to point at a phenomenologically real group of things in the world.

However, I think that this is actually bad.

Once people learn the term akrasia and what it represents, they can now pattern-match it to their own associated experiences. I think that, once you’ve reified akrasia, i.e. turned it into a “thing” inside your ontology, problems occur:

First off, treating akrasia as a real thing gives it additional weight and power over you:

Once you start to notice the patterns, it’s harder to see things again as mere apparent chaos. In the case of akrasia, I think this means that people may try less hard because they suddenly realize they’re in the grip of this terrible monster called akrasia.

I think this sort of worldview ends up reinforcing some unhelpful attitudes towards solving the problems akrasia represents. As an example, here are two paraphrased things I’ve overheard about akrasia which I think illustrate this. (Happy to remove these if you would prefer not to be mentioned.)

“Akrasia has mutant healing powers…Thus you can’t fight ...
Nate Soares' Replacing Guilt Series compiled in epub Format
by  — 2017-04-30T06:36:16.025Z

Hey everyone,

I really liked Nate Soares' Replacing Guilt series, which has had a major positive impact on growing my intrinsic motivation.

Recently, I compiled all the posts into an ePUB for my own reading, and I thought it might be good to share it here if anyone would like to download it for their e-readers / on-the-go-reading. (I got Nate's permission first, so it's all good.)

Google Drive link here.
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2017-04-29T13:27:21.668Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
30 April 2017 03:30:00PM (-0400)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

This week, we will be hanging out, playing games, and engaging in fun conversation.

There is an event in the courtyard in the early afternoon - for simplicity, we'll be meeting in the Luce center, 3rd floor, F street side.

Upcoming meetups:
* May 7: Regression to the Mean
* May 14: AI Risk/Safety
Sorry for the delay in posting. - RobinZ

Discussion article for the meetup : Washington, D.C.: Fun & Games
New meet up in Las Vegas!
by Adam Zerner — 2017-04-28T23:57:21.098Z

Hey guys, I'd just like to announce that I'm starting a new meet up in Las Vegas!

WHEN: First and third Sunday of the month. 7pm-9pm.

WHERE: The Market (downtown on Fremont Street).

See http://lesswrong.com/meetups/1xg.
Meetup : Baltimore / UMBC
by l1n — 2017-04-28T18:50:51.081Z

Discussion article for the meetup : Baltimore / UMBC

WHEN:
30 April 2017 08:00:00PM (-0400)

WHERE:
Performing Arts and Humanities Building, 1000 Hilltop Cir, Baltimore, MD 21250, United States

Casual discussion group! Suggested Reading: [http://lesswrong.com/lw/ouc/project_hufflepuff_planting_the_flag/]. Meetup is on the 4th floor by the English department. Follow the voices. For help getting into the building, please text 619-357-6633.

Format: We meet and start hanging out at 8:00, but don’t officially start doing the meetup topic until 8:15-8:30 to accommodate stragglers.

Parking is available anywhere on campus because its a weekend. I suggest Lot 8.

Discussion article for the meetup : Baltimore / UMBC
Change utility, reduce extortion
by Stuart_Armstrong — 2017-04-28T14:05:56.911Z

Crossposted at the Intelligent Agents Forum.

EDIT: This method is not intended to solve extortion, just to remove the likelihood of extremely terrible outcomes (and slightly reduce the vulnerability to extortion).

A full solution to the extortion problem is sorely elusive. However, there are crude hacks that we can use to mitigate the downside.

Suppose we figured out that a friendly AI should be maximising an unbounded utility function U. The extortion risk is that another AI could threaten a FAI with unbounded disutility if it didn't go along with its plans. This gives the extorting AI - the EAI - a lot of leverage, and things could end up badly if the EAI ends up acting on its threat.

To combat this, we first have to figure out a level z of utility that is a lower bound on what U could ever reach naturally and realistically.

By "naturally" we mean that U going below z would require not just incompetence or indifference, but some AI actively and deliberately arranging the lowering of U. And "realistically" just means that we're confident that getting U lower than z by chance, or having a U-minimising AI, are exceedingly low.

Then what we can do is to cut off U at the z level, replacing U with U'=max(U,U(z)). See z indicated by the red line on this graph of U' versus U:

What's the consequence of this? First of all, it ensures that no EAI would threaten to reduce U (the utility we really care about) below z, because that is not a threat to the FAI. This reduces the leverage of the EAI, and reduces the impact of it acting on its threat.

Since levels of U below z are exceedingly unlikely to happen by chance, the fact the FAI has the wrong utility below z shouldn't affect it's performance much. And, even in that zone, the AI is still motivated to climb U above z.

But we may still feel unhappy about the flatness of that curve, and want it to still prefer higher U to exceedingly low values. If so, we can replace U with U'' as follows (the blue line is at z-1):

In this case, the EAI will not seek to reduce U below z-1 (in fact, it will specifically target that value), while the FAI has the correct ordering of lower values of U. The utility is weird around z, granted, but this is a place where the FAI would not want to be and would almost certainly not reach by accident.

Though this method does not eliminate the threat of extortion, it does seem to reduce its impact.
Change utility, reduce extortion
by Stuart_Armstrong — 2017-04-28T13:58:12.000Z

A full solution to the extortion problem is sorely elusive. However, there are crude hacks that we can use to mitigate the downside.

Suppose we figured out that a friendly AI should be maximising an unbounded utility function U. The extortion risk is that another AI could threaten a FAI with unbounded disutility if it didn't go along with its plans. This gives the extorting AI -- the EAI -- a lot of leverage, and things could end up badly if the EAI ends up acting on its threat.

To combat this, we first have to figure out a level z of utility that is a lower bound on what U could ever reach naturally and realistically.

By "naturally" we mean that U going below z would require not just incompetence or indifference, but some AI actively and deliberately arranging the lowering of U. And "realistically" just means that we're confident that getting U lower than z by chance, or having a U-minimising AI, are exceedingly low.

Then what we can do is to cut off U at the z level, replacing U with U′=max(U,U(z)). See z indicated by the red line on this graph of U′ versus U:

What's the consequence of this? First of all, it ensures that no EAI would threaten to reduce U (the utility we really care about) below z, because that is not a threat to the FAI. This reduces the leverage of the EAI, and reduces the impact of it acting on its threat.

Since levels of U below z are exceedingly unlikely to happen by chance, the fact the FAI has the wrong utility below z shouldn't affect it's performance much. And, even in that zone, the AI is still motivated to climb U above z.

But we may still feel unhappy about the flatness of that curve, and want it to still prefer higher U to exceedingly low values. If so, we can replace U with U′′ as follows:

In this case, the EAI will not seek to reduce U below z−1 (in fact, it will specifically target that value), while the FAI has the correct ordering of lower values of U. The utility is weird around z, granted, but this is a place where the FAI would not want to be and would almost certainly not reach by accident.

Though this method does not eliminate the threat of extortion, it does seem to reduce its impact.
Meetup : Superintelligence chapter 5
by pepe_prime — 2017-04-28T13:17:21.411Z

Discussion article for the meetup : Superintelligence chapter 5

WHEN:
12 May 2017 05:45:00PM (+0200)

WHERE:
Lindstedtsvägen 3, room 1537, SE-114 28 Stockholm, Sverige

Decisive strategic advantage

Will the frontrunner get a decisive advantage?

How large will the successful project be?

Monitoring

International collaboration

From decisive strategic advantage to singleton

We'll have a presentation on the above topics, with opportunities for discussion. We'll repeat this with later chapters every 2 weeks. You don't have to have read the book, though it will probably help to read chapters 1-4.

Format:

We meet and start hanging out at 5:45, but don't officially start doing the meetup topic until 6:00 to accommodate stragglers. We often go out for dinner after the meetup.

How to find us:

The meetup is at a KTH academic building and the room is on the 5th floor, two stairs up.

Influence future meetups:

Times - http://www.when2meet.com/?5723551-cJBhD

Topics - https://druthe.rs/dockets/-KcCvpn97vUhg3tQRrKn

Discussion article for the meetup : Superintelligence chapter 5
Meetup : Stockholm Slate Star Codex Meetup
by pepe_prime — 2017-04-28T13:15:11.502Z

Discussion article for the meetup : Stockholm Slate Star Codex Meetup

WHEN:
06 May 2017 01:30:00PM (+0200)

WHERE:
café Dox, Stora Nygatan 31, 111 27 Stockholm, Sweden

This is a meetup where we will discuss topics related to the blogg Slate Star Codex, for example rationality, science and effective altruism. Having read the blog is preferable but not required. Feel free to invite friends who might be interested.

(added May 3) The theme of discussion this time will be "Meditations on Moloch". http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

Discussion article for the meetup : Stockholm Slate Star Codex Meetup
Learning incomplete models using dominant markets
by Vanessa Kosoy — 2017-04-28T09:57:16.000Z

% operators that are separated from the operand by a space

% operators that require brackets

% operators that require parentheses

% Paper specific

This post is formal treatment of the idea outlined here.

Given a countable set of incomplete models, we define a forecasting function that converges in the Kantorovich-Rubinstein metric with probability 1 to every one of the models which is satisfied by the true environment. This is analogous to Blackwell-Dubins merging of opinions for complete models, except that Kantorovich-Rubinstein convergence is weaker than convergence in total variation. The forecasting function is a dominant stochastic market for a suitably constructed set of traders.

Appendix A contains the proofs. Appendix B restates the theorems about dominant stochastic markets for ease of reference. Appendix C states a variant of the Michael selection theorem due to Yannelis and Prabhakar which is used in Appendix A.

##Notation

Given X a metric space, x∈X and r∈R, Br(x):={y∈X∣d(x,y)<r}.

Given X a topological space:

P(X) is the space of Borel probability measures on X equipped with the weak* topology.

C(X) is the Banach space of continuous functions X→R with uniform norm.

T(X):=C(P(X)×X)

B(X) is the Borel σ-algebra on X.

U(X) is the σ-algebra of universally measurable sets on X.

Given μ∈P(X), suppμ denotes the support of μ.

Given X and Y measurable spaces, K:Xmk−→ Y is a Markov kernel from X to Y. For any x∈X, we have K(x)∈P(Y). Given μ∈P(X), μ⋉K∈P(X×Y) is the semidirect product of μ and K and K∗μ∈P(Y) is the pushforward of μ by K.

Given X, Y Polish spaces, π:X→Y Borel measurable and μ∈P(X), we denote μ∣π the set of Markov kernels K:Ymk−→X s.t. π∗μ⋉K is supported on the graph of π and K∗π∗μ=μ. By the disintegration theorem, μ∣π is always non-empty and any two kernels in μ∣π coincide π∗μ-almost everywhere.

##Results

Consider X any compact Polish metric space and d:X×X→R its metric. We denote Lip(X) the Banach space of Lipschitz continuous functions X→R with the norm

∥f∥Lip:=maxx|f(x)|+supx≠y|f(x)−f(y)|d(x,y)

P(X) (as before, with the weak topology) can be regarded as a compact subset of Lip(X)′ (with the strong topology), yielding a metrization of P(X) which we will call the Kantorovich-Rubinstein metric dKR:

dKR(μ,ν):=sup∥f∥Lip≤1|Eμ[f]−Eν[f]|

In fact, the above differs from the standard definition of the

...