trentmkelly/LessWrong-43k huggingface dataset:
Subagent perfect minimax
by Vanessa Kosoy — 2017-01-06T13:47:12.000Z

% operators that are separated from the operand by a space

% operators that require brackets

% operators that require parentheses

% Paper specific

This post continues the study of minimax forecasting.

The minimax decision rule has the pathology that, when events are sufficiently "optimistic", behavior can become highly suboptimal. This is analogous to off-policy irrational behavior in Nash equilibria of games in extensive form. In order to remedy the problem, we introduce a refinement called "subagent perfect minimax," somewhat analogous to subgame perfect equilibria and other related solution concepts. It is possible to prove existence and, when the model factorizes, dynamic consistency.

The proofs are omitted, but we can easily provide them if necessary.

##Motivation

Consider the following class of environments: during the first step, the agent gains 1$ or action b which leads to gaining $$0$.

The minimax payoff of this class is 0$ in the first step, choose b if you gained 1,whichisthesameastheworst−case(p=0$) payoff of π∗.

Note that the minimax environment has p=0 and π only differs from π∗ on histories which are impossible in that environment. Moreover, if we considered the class p∈[ϵ,1] for some ϵ>0, the minimax payoff would be $$(1+\epsilon)$, and \(\pi^*\) would be the sole minimax policy. Therefore, in order to eliminate the pathology, we need to formulate a stability condition that ensures any admissible history is treated as occurring with at least infinitesimal probability.

##Results

Now consider the forecasting setting, with action set A, observation set O, time discount function γ:N→R≥0 and reward function r:(A×O)∗→R. As before, γ and r define the utility function u:(O∗→A)×Oω→R.

Consider Φ∈PC(Oω) and denote

O+Φ:={x∈O+∣ ∃μ∈Φ:μ(xOω)>0}

#Definition 1

Consider X⊆O+Φ finite. π∗∈P(O∗→A) is called an X-stable minimax policy for Φ when there are sequences {ϵ(n):X→(0,1)}n∈N and {π(n)∈P(O∗→A)}n∈N s.t. ϵ(n)→0, π(n)→π∗ and π(n) is a minimax policy for

Φ(n):={μ∈Φ∣∀x∈O∗,o∈O:xo∈X⟹μ(xoOω)≥ϵ(n)(xo)μ(xOω)}

In particular, the definition assumes Φ(n) is non-empty.

#Proposition 1

For any X⊆O+Φ finite, there exists π∗∈P(O∗→A) which is an X-stable minimax policy for Φ.

#Proposition 2

For any X⊆O+Φ finite and π∗∈P(O∗→A) an X-stable minimax policy for Φ, π∗ is in particular a minimax policy for Φ.

#Proposition 3

Consider X⊆O+Φ finite, π∗∈

...
Actually Practicing Rationality and the 5-Second Level
by  — 2017-01-06T06:50:06.232Z

[I first posted this as a link to my blog post, but I'm reposting as a focused article here that trims some fat of the original post, which was less accessible]

I think a lot about heuristics and biases, and I admit that many of my ideas on rationality and debiasing get lost in the sea of my own thoughts. They’re accessible, if I’m specifically thinking about rationality-esque things, but often invisible otherwise.

That seems highly sub-optimal, considering that the whole point of having usable mental models isn’t to write fancy posts about them, but to, you know, actually use them.

To that end, I’ve been thinking about finding some sort of systematic way to integrate all of these ideas into my actual life.

(If you’re curious, here’s the actual picture of what my internal “concept-verse” (w/ associated LW and CFAR memes) looks like)

Open Image In New Tab for all the details

So I have all of these ideas, all of which look really great on paper and in thought experiments. Some of them even have some sort of experimental backing. Given this, how do I put them together into a kind of coherent notion?

Equivalently, what does it look like if I successfully implement these mental models? What sorts of changes might I expect to see? Then, knowing the end product, what kind of process can get me there?

One way of looking it would to say that if I implemented techniques well, then I’d be better able to tackle my goals and get things done. Maybe my productivity would go up. That sort of makes sense. But this tells us nothing about how I’d actually be going about, using such skills.

We want to know how to implement these skills and then actually utilize them.

Yudkowsky gives a highly useful abstraction when he talks about the five-second level. He gives some great tips on breaking down mental techniques into their component mental motions. It’s a step-by-step approach that really goes into the details of what it feels like to undergo one of the LessWrong epistemological techniques. We’d like our mental techniques to be actual heuristics that we can use in the moment, so having an in-depth breakdown makes sense.

Here’s my attempt at a 5-second-level breakdown for Go

...
Triple or nothing paradox
by Stabilizer — 2017-01-05T21:10:44.132Z

You are at a casino. You have $1. A table offers you a game: you have to bet all your money; a fair coin will be tossed; if it lands heads, you triple your money; if it lands tails, you lose everything.

In the first round, it is rational to take the bet since the expected value of winning is $1.50, which is greater than what you started out with.

If you win the first round, you'll have $3. In the next round, it is rational to take the bet again, since the expected value is $4.50 which is larger than $3.

If you win the second round, you'll have $9. In the next round, it is rational to take the bet again, since the expected value is $13.50 which is larger than $9.

You get the idea. At every round, if you won the previous round, it is rational to take the next bet.

But if you follow this strategy, it is guaranteed that you will eventually lose everything. You will go home with nothing. And that seems irrational.

Intuitively, it feels that the rational thing to do is to quit while you are ahead, but how do you get that prediction out of the maximization of expected utility? Or does the above analysis only feel irrational because humans are loss-averse? Or is loss-aversion somehow optimal here?

Anyway, please dissolve my confusion.
Meetup : Moscow: TDT, paranoid calibration, prediction party
by Slava Matyukhin — 2017-01-05T19:22:10.587Z

Discussion article for the meetup : Moscow: TDT, paranoid calibration, prediction party

WHEN:
08 January 2017 02:00:00PM (+0300)

WHERE:
Москва, ул. Большая Дорогомиловская, д.5к2

Note: most our members join meetups through other channels. Still, the correlation between "found out about Moscow meetups via lesswrong.com" and "is a great fit for our community" is very high. So we're posting just a short link to the hackpad document with the schedule here instead of the full translation of the announcement into English.

Pad with the details about 08.01.2017 meetup.

We're meeting at the "Kocherga" anticafe, as usual.

Discussion article for the meetup : Moscow: TDT, paranoid calibration, prediction party
Meetup : Baltimore Area / UMBC Weekly Meetup
by iarwain1 — 2017-01-05T13:23:25.957Z

Discussion article for the meetup : Baltimore Area / UMBC Weekly Meetup

WHEN:
08 January 2017 08:00:00PM (-0500)

WHERE:
Performing Arts and Humanities Bldg Room 456, 1000 Hilltop Cir, Baltimore, MD 21250

Meeting is on 4th floor of the Performing Arts and Humanities Building. Permit parking designations do not apply on weekends, so park pretty much wherever you want.

Discussion article for the meetup : Baltimore Area / UMBC Weekly Meetup
Why read old philosophy?
by KatjaGrace — 2017-01-05T07:59:16.000Z

I’m going to try to explain a mystery that puzzled me for years. This answer finally dawned on me in the middle of one of those occasional conversations in which non-perplexed friends patiently try to explain the issue to me. So I am not sure if mine is a novel explanation, or merely the explanation that my friends were trying to tell me, in which case my contribution is explaining it in a way that is at all comprehensible to a person like me. If it is novel, apparently some other people disagree with it and have an almost entirely satisfactory alternative, which has the one downside that it is impossible to explain to me.

The puzzle is this:

Why do people read old philosophers to learn about philosophy?

We read old physicists if we want to do original research on the history of physics. Or maybe if we are studying an aspect of physics so obscure that nobody has covered it in hundreds of years. If we want to learn physics we read a physics textbook. As far as I know, the story is similar in math, chemistry, engineering, economics, and business (though maybe some other subjects that I know less about are more like philosophy).

Yet go to philosophy grad school, and you will read original papers and books by historical philosophers. Research projects explore in great detail what it is that Aristotle actually said, thought, and meant. Scholars will learn the languages that the relevant texts were written in, because none of the translations can do the texts the necessary justice. The courses and books will be named after people like ‘Hume’ as often as they are named after topics of inquiry like ‘Causality’ and larger subject areas will be organized by the spatiotemporal location of the philosopher, rather than by the subject matter: Ancient Philosophy, Early Modern Philosophy, Chinese Philosophy, Continental Philosophy.

The physics situation makes a lot more sense to me. Hypothetically, who would I rather read an explanation of ‘The Alice Effect’ by? —Alice, the effect’s seventeenth century discoverer, or Bob, a modern day physics professor authoring a textbook?

Some salient considerations, neutrality not guaranteed:

Alice’s understanding of the Alice effect is probably the most confused understanding of it in all of history, being the first ‘understanding of the Alice effect’ to set itself apart from ‘confusion and ignorance about the Alice effect’.

In the bi

...
The time you have
by Elo — 2017-01-05T02:13:54.772Z

Original post: http://bearlamp.com.au/the-time-you-have/

Part 1: Exploration-Exploitation

Part 2: Bargaining Trade-offs to your brain.

Part 2a: Empirical time management

Part 3: The time that you have

There is a process called The Immunity to Change by Robert Kegan. The process is designed to deal with personal problems that are stubborn. The first step in the process is to make a list of all the things that you are doing or not doing that does not contribute to the goal. As you go through the process you analyse why you do these things based on what it feels like to do them.

The process is meant to be done with structure but can be done simply by asking. Yesterday I asked someone who said he ate sugar, ate carbs, and didn't exercise. Knowing this alone doesn't solve the problem but it helps.

The ITC process was generated by observing patients and therapists for thousands of hours and thousands of cases. Kegan observed what seems to be effective to bring about change, in people and generated this process to assist in doing so. The ITC hits on a fundamental universal. If you read my brief guide on Empirical time management, as well as part 1 - exploration-exploitation of this series it speaks to this universal. Namely what we are doing with our time is everything we are choosing not to do with our time. It's a trade off between our values and it's counter-commitments in ITC that's often discovering the hidden counter commitments to the goals.

The interesting thing about what you end up doing with your time is that these are the things that form your revealed preferences. Revealed preference theory is an economic theory that differentiates between people's stated preferences and their actual actions and behaviours. It's all good and well to say that your preferences are one thing, but if you never end up doing that; your revealed preferences are in fact something entirely different.

For example - if you say you want to be a healthy person, and yet you never find yourself doing the things that you say you want to do in order to be healthy; your revealed preferences suggest that you are in fact not revealing the actions of a healthy person. If you live to the ...
Meetup : Washington, D.C.: Intro to Effective Altruism
by RobinZ — 2017-01-05T02:03:11.769Z

Discussion article for the meetup : Washington, D.C.: Intro to Effective Altruism

WHEN:
08 January 2017 03:30:00PM (-0500)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

We will be meeting in the courtyard to discuss the basics of the effective altruism movement.

Upcoming meetups:

Jan. 15: Half-Assing

Jan. 22: Fun & Games

Discussion article for the meetup : Washington, D.C.: Intro to Effective Altruism
Meetup : SF Meetup: Revealed New Year's Resolutions
by maia — 2017-01-04T21:46:54.589Z

Discussion article for the meetup : SF Meetup: Revealed New Year's Resolutions

WHEN:
09 January 2017 06:15:22PM (-0800)

WHERE:
1769 15th St., SF

It's a yearly tradition!

Instead of talking about what we've resolved to do in the New Year, we'll talk about, from looking back at what you actually did in the past year, what someone would think your New Year's resolutions were. Like revealed preferences, we're reverse-engineering what you were trying to do by looking at what you actually did.

This is an interesting exercise, and sometimes helps inform our goals for the next year and years to come.

For help getting into the building, please call (or text, with a likely-somewhat-slower response rate): 301-458-0764.

Format:

We meet and start hanging out at 6:15, but don’t officially start doing the meetup topic until 6:45-7 to accommodate stragglers. Usually there is a food order that goes out before we start the meetup topic.

About these meetups:

The mission of the SF LessWrong meetup is to provide a fun, low-key social space with some structured interaction, where new and non-new community members can mingle and have interesting conversations. Everyone is welcome.

We explicitly encourage people to split off from the main conversation or diverge from the topic if that would be more fun for them (moving side conversations into a separate part of the space if appropriate). Meetup topics are here as a tool to facilitate fun interaction, and we certainly don’t want them to inhibit it.

Discussion article for the meetup : SF Meetup: Revealed New Year's Resolutions
Lost purposes - Doing what's easy or what's important
by Elo — 2017-01-04T01:26:18.935Z

Original post: http://bearlamp.com.au/lost-purposes-doing-whats-easy-or-whats-important/

A friend came to me with a startup as her main goal. I am keen on my coaching skills so I would from time to time ask her what the top 3 most important things she could be doing today would be. She would proceed to come back with a list. A few days later I would check back and ask her what she got done. She would very excitedly tell me all about the other things she was doing that weren't the top 3 things.

I watched this behaviour for a while before commenting. Eventually I asked her about it and she conceded she was doing tasks that seem easier than the important things because they feel like progress, say by doing four small 15 minute tasks you got four things done right? Whereas if you work on an important task for an hour you only got one thing done.

Our brains like to use Rule of thumb type judgements to know what pathways to follow. In this case, Am I making progress towards my goal was replaced by, have I completed things which was easily gamified by, How do I do the most little things I can.

The same thing happens when people pay attention to their health. Get healthy turns into, Lose weight, which turns into unhealthy body images and great confusion. Although this is probably more related to us not being clear and specific on what the health goal was in the first place.

So what say you have a lost purpose. You want to do X but you find yourself doing the remarkably similar Y. How do you fix that?

1. Be specific

"Would you tell me, please, which way I ought to go from here?"
"That depends a good deal on where you want to get to," said the Cat.
"I don’t much care where–" said Alice.
"Then it doesn’t matter which way you go," said the Cat.

It's not going to work 100% of the time, but it does help.

Imagine you told a monkey to go collect coconuts, every coconut the monkey brings back you will give him one reward. You mean the large useful ones that you might find in a supermarket. Knowing how important coconuts are but not really understanding the task, while collecting large coconuts he finds some smaller ones, realising he can carry two half-size coconuts instead of one large one, he goes to find more smaller cocon...
Buckets and memetic immune disorders
by Tyrrell_McAllister — 2017-01-03T23:51:50.442Z

AnnaSalamon's recent post on "flinching" and "buckets" nicely complements PhilGoetz's 2009 post Reason as memetic immune disorder. (I'll be assuming that readers have read Anna's post, but not necessarily Phil's.) Using Anna's terminology, I take Phil to be talking about the dangers of merging buckets that started out as separate. Anna, on the other hand, is talking about how to deal with one bucket that should actually be several.

Phil argued (paraphrasing) that rationality can be dangerous because it leads to beliefs of the form "P implies Q". If you convince yourself of that implication, and you believe P, then you are compelled to believe Q. This is dangerous because your thinking about P might be infected by a bad meme. Now rationality has opened the way for this bad meme to infect your thinking about Q, too.

It's even worse if you reason yourself all the way to believing "P if and only if Q". Now any corruption in your thinking about either one of P and Q will corrupt your thinking about the other. In terms of buckets: If you put "Yes" in the P bucket, you must put "Yes" in the Q bucket, and vice versa. In other words, the P bucket and the Q bucket are now effectively one and the same.

In this sense, Phil was pointing out that rationality merges buckets. (More precisely, rationality creates dependencies among buckets. In the extreme case, buckets become effectively identical). This can be bad for the reasons that Anna gives. Phil argues that some people resist rationality because their "memetic immune system" realizes that rational thinking might merge buckets inappropriately. To avoid this danger, people often operate on the principle that it's suspect even to consider merging buckets from different domains (e.g., religious scripture and personal life).

This suggests a way in which Anna's post works at the meta-level, too.

Phil's argument is that people resist rationality because, in effect, they've identified the two buckets "Think rationally" and "Spread memetic infections". They fear that saying "Yes" to "Think rationally" forces them to say "Yes" to the dangers inherent to merged buckets.

But Anna gives techniques for "de-merging" buckets in general if it turns out that some buckets were inappropriately merged, or if one bucket should have been several in the first place.

In other words, Anna's post essentially de-merges the two particular buckets "Think...
Progress and Prizes in AI Alignment
by Jacob Falkovich — 2017-01-03T22:15:54.484Z

Edit: In case it's not obvious, I have done limited research on AI alignment organizations and the goal of my post is to ask questions from the point of view of someone who wants to contribute and is unsure how. Read down to the comments for some great info on the topic.

I was introduced to the topic of AI alignment when I joined this very forum in 2014. Two years and one "Superintelligence" later, I decided that I should donate some money to the effort. I knew about MIRI, and I looked forward to reading some research comparing their work to the other organizations working in this space. The only problem is... there really aren't any.

MIRI recently announced a new research agenda focused on "agent foundations". Yet even the Open Philanthropy Project, made up of people who at least share MIRI's broad worldview, can't decide whether that research direction is promising or useless. The Berkeley Center for Human-Compatible AI doesn't seem to have a specific research agenda beyond Stuart Russell. The AI100 Center at Stanford is just kicking off. That's it.

I think that there are two problems here:

There's no way to tell which current organization is going to make the most progress towards solving AI alignment.

These organizations are likely to be very similar to each other, not least because they practically share a zipcode. I don't think that MIRI and the academic centers will do the exact same research, but in the huge space of potential approaches to AI alignment they will likely end up pretty close together. Where's the group of evo-psych savvy philosophers who don't know anything about computer science but are working to spell out an approximation of universal human moral intuitions?

It seems like there's a meta-question that needs to be addressed, even before any work is actually done on AI alignment itself:

How to evaluate progress in AI alignment?

Any answer to that question, even if not perfectly comprehensive or objective, will enable two things. First of all, it will allow us to direct money (and the best people) to the existing organizations where they'll make the most progress.

More importantly, it will enable us to open up the problem of AI alignment to the world and crowdsource it.

For example, the XPrize Foundation is a remarkable organization that creates competitions around achieving goals benefic...
Meetup : Rationality Vienna Meetup
by AnnaLeptikon — 2017-01-03T08:25:21.918Z

Discussion article for the meetup : Rationality Vienna Meetup

WHEN:
28 January 2017 03:00:00PM (+0100)

WHERE:
Kaisermühlenstraße 24, 1220 Vienna

15:00 to 15:30: arrival & extended tea/coffee break
5:30 to ~15:45: round of introductions
!!!talk/discussion - not yet defined!!!

Facebooklink: https://www.facebook.com/events/817077245101402
(join "Rationality Vienna" first)

Discussion article for the meetup : Rationality Vienna Meetup