trentmkelly/LessWrong-43k huggingface dataset:
Versions of AIXI can be arbitrarily stupid
by Stuart_Armstrong — 2015-08-10T13:23:34.541Z

Many people (including me) had the impression that AIXI was ideally smart. Sure, it was uncomputable, and there might be "up to finite constant" issues (as with anything involving Kolmogorov complexity), but it was, informally at least, "the best intelligent agent out there". This was reinforced by Pareto-optimality results, namely that there was no computable policy that performed at least as well as AIXI in all environments, and strictly better in at least one.

However, Jan Leike and Marcus Hutter have proved that AIXI can be, in some sense, arbitrarily bad. The problem is that AIXI is not fully specified, because the universal prior is not fully specified. It depends on a choice of a initial computing language (or, equivalently, of an initial Turing machine).

For the universal prior, this will only affect it up to a constant (though this constant could be arbitrarily large). However, for the agent AIXI, it could force it into continually bad behaviour that never ends.

For illustration, imagine that there are two possible environments:

The first one is Hell, which will give ε reward if the AIXI outputs "0", but, the first time it outputs "1", the environment will give no reward for ever and ever after that.

The second is Heaven, which gives ε reward for outputting "0" and 1 reward for outputting "1", and is otherwise memoryless.

Now simply choose a language/Turing machine such that the ratio P(Hell)/P(Heaven) is higher than the ratio 1/ε. In that case, for any discount rate, the AIXI will always output "0", and thus will never learn whether its in Hell or not (because its too risky to do so). It will observe the environment giving reward ε after receiving "0", behaviour which is compatible with both Heaven and Hell. Thus keeping P(Hell)/P(Heaven) constant, and ensuring the AIXI never does anything else.

In fact, it's worse than this. If you use the prior to measure intelligence, then an AIXI that follows one prior can be arbitrarily stupid with respect to another.
Meetup : LW Cologne meetup
by EGI — 2015-08-10T10:27:33.455Z

Discussion article for the meetup : LW Cologne meetup

WHEN:
15 August 2015 05:00:00PM (+0200)

WHERE:
Marienweg 43, 50858 Köln

Please let
me know if you will attend until Wendesday. If more than 3 people come, I want to try a more structured meetup than the last ones:
1) One game of skill (dextrity focused) (20 to 40 min)
2) Slightly formalised discussion of one topic (30 to 120 min)
3) Dinner (60 min)
4) HPMOR inspired (Hufflepuff bones) creativity focused game (60 to 120min)
5) Slightly formalised discussion of second topic (30 to 120 min)
6) Feedback
7) Open discussion

Otherwise just Open discussion and dinner.

If you know / have a game which fits point 1 please suggest / bring
along, otherwise I will provide one. Discussion topics will be selected
at the beginning of the meetup, suggestions welcome - the earlier the
better. I would like to order dinner at Stückwerk. The creativity game
will be provided by me.

Discussion article for the meetup : LW Cologne meetup
Open thread, Aug. 10 - Aug. 16, 2015
by MrMind — 2015-08-10T07:29:10.355Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Modal SAT: Self Cooperation
by Scott Garrabrant — 2015-08-10T05:48:33.000Z

Post 2 in in Modal SAT series. In this post, we show that SC Modal SAT is equivalent to Modal SAT.

For this, we need just need to prove the following theorem:

Theorem: If there exists a modal agent M such that Ci cooperates with M for each Ci∈C and such that M defects against Di for each Di∈D, then there exists an M′ which satisfies the above properties and cooperates with itself.

Proof: Let n be greater than the total number of boxes in agents in C and D. Consider the agents Xn and Xn+1 defined by Xi(B)=C↔¬□i⊥.

We define M′ by

M′(B)=C if ¬□n+1⊥ and □(¬□n⊥→B(CooperateBot)=C) and ¬□(¬□n−1⊥→B(CooperateBot)=C)

M′(B)=D if ¬□n+2⊥ and □(¬□n+1⊥→B(CooperateBot)=C) and ¬□(¬□n⊥→B(CooperateBot)=C)

M′(B)=C if ¬□n+3⊥ and □(¬□n+1⊥→B(Xn)=C) and □(¬□n+2⊥→B(Xn+1)=D)

M′(B)=M(B) otherwise

Rule 1 says that M′ cooperates with Xn, rule 2 says that M′ defects against Xn+1, and rule 3 says that M′ cooperates with anyone who (provably assuming ¬□n+1⊥) cooperates with Xn and (provably assuming ¬□n+2⊥)defects against Xn+1. Thus, M′ cooperates with itself.

For any bot B with fewer than n boxes, the conditions of 1, 2, and 3 are all false. For 1 and 2, this is because the actions of such bots against CoopearteBot stabilize by the time you assume ¬□n−1⊥. For 3, this is because these bots cannot distinguish between Xn and Xn+1.

Therefore M′ behaves the same as M on all inputs with fewer than n boxes, so M′ cooperates with every bot in C and defects against every bot in D.

□

Note that I was lazy here, and took way more longer to cooperate with myself than I had to. In principle, if there are only k bots that I need to consider (including bots I need to consider because they are referenced by bots I care about), then regardless of how many boxes are in each bot, It should be possible to achieve self cooperation within 2log2k worlds of the Kripke frame. That is, log2k worlds to identify a single bot that is distinguishable from all other bots, and another log2k worlds to ensure that the actions of M′ differ on that bot from all other bots, so that M′ can identify itself without changing its behavior against any other bot.

EDIT: Actually, I think logk + a small constant should suffice, but it does not matter much.
Meetup : West LA: Problem Solving
by abramdemski — 2015-08-09T23:20:24.708Z

Discussion article for the meetup : West LA: Problem Solving

WHEN:
12 August 2015 07:00:00PM (-0700)

WHERE:
11066 santa monica blvd

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: Our group has been doing something we call "rationality moments", sharing success or failure in skills of rationality since we last saw each other. Naturally, this leads to some discussion of how to solve problems which we may be having. It occurs to me that group problem-solving is a skill which can go wrong, and that there is knowledge out there about how to avoid pitfalls and how to get good results.

Recommended Reading:

It's encouraged that you think about the problem and look for good resources yourself, but here is what I found:

The wikipedia article on groupthink (which as I understand it says that we don't know very much at all about what causes groupthink or what may prevent it, in the empirical findings section)

Wikipedia on problem solving has interesting points.

Here is a short document about effective problem-solving for groups.

Hold Off On Proposing Solutions

Don't get caught up solving the wrong problem.

Polya on problem solving

A short article on avoiding bias in decision making, which has the same format of many such articles (IE, not chosen because it especially sticks out).

No prior exposure to Less Wrong is required; this will be generally accessible.

Discussion article for the meetup : West LA: Problem Solving
Some concepts are like Newton's Gravity, others are like... Luminiferous Aether?
by Locaha — 2015-08-09T15:10:17.895Z

Let's compare two theories. One is Newton's gravity, the other Luminiferous Aether. When Einstein's theory of relativity arrived, Newton's Gravity turned to be a subset of it, an approximation that works under specific conditions.

On the other hand, Luminiferous Aether is just plain wrong.

Now, imagine that a scientist in the era before Theory of Relativity built a Strong AI (just roll with me here :-) ) and tasked it with finding out why Newton's Gravity doesn't work quite right around Mercury. The AI derived the Theory of Relativity.

Now, imagine this scientist asking the AI what Luminiferous Aether is made from. The AI is going to throw an OutOfLuminiferousAether exception (don't ask me why the AI is written in Java).

Humorous prelude aside, I am wondering which concepts we have today are only slightly wrong, and which are completely wrong? I am asking mostly about the concepts that are discussed on this forum.

Obviously, the more abstract is the concept, the more risk there that it will turn out to be bunkum.

Personally, I don't trust the concept of values. It's already so complex and fragile, I'm afraid it doesn't actually exist.
Vegetarianism Ideological Turing Test!
by Raelifin — 2015-08-09T14:39:48.951Z

I'm running an Ideological Turing Test (or Caplan Test) in my local rationality group on the topic of vegetarianism. (Based on a survey, it's one that splits my community pretty evenly.) If anyone here is interested, you're welcome to participate! I'll be posting the responses I get on LW for judging, and I'm hoping to get responses from a couple people here that I could use for my local group. After I get responses and the community judges them, I'll post here to share the statistics.

You can PM me or rot13 your entry if you're concerned about information leakage, but I'll also accept plaintext comments. The (soft) deadline for submission is the evening of the 15th. If I don't have enough responses by then (for LW or my local group) I'll extend it.

If you're interested in participating, please read on:

First, please write a paragraph or two about what your general position is on vegetarianism. Please make it clear which way you lean for the purposes of answering/judging. This text will be public knowledge (used for the reveal) so include your name if you want to be known, and keep it anonymous if you don't.

Once you've described yourself, please write a paragraph (or two) to briefly answer each of the following questions. If you do not identify as an omnivore, answer the omnivore questions by pretending to be an (aspiring rationalist) omnivore. If you are not a vegetarian, answer the vegetarian questions by pretending to be an (aspiring rationality) vegetarian. When writing responses to the prompts, do not include information that makes it clear who you are (for example: I shouldn't say "Because I am very tall....")

For Omnivores:

* Do you think the level of meat consumption in America is healthy for individuals? Do you think it's healthy for the planet?

* How do you feel about factory farming? Would you pay twice as much money for meat raised in a less efficient (but "more natural") way?

* Are there any animals you would (without significantly changing your mind) never say it was okay to hunt/farm and eat? If so, what distinguishes these animals from the animals which are currently being hunted/farmed?

* If all your friends were vegetarians, and you had to go out of your way to find meat in a similar way to how vegans must go out of their way right now, do you think you'd still be an omnivore?

For Vegetarians:

* If there was a way to...
Not yet gods
by So8res — 2015-08-09T03:00:00.000Z
Link: https://mindingourway.com/not-yet-gods/

You probably don't feel guilty for failing to snap your fingers in just such a way as to produce a cure for Alzheimer's disease.

Yet, many people do feel guilty for failing to work until they drop every single day (which is a psychological impossibility). They feel guilty for failing to magically abandon behavioral patterns they dislike, without practice or retraining (which is a cognitive impossibility). What gives?

The difference, I think, is that people think they "couldn't have" snapped their fingers and cured Alzheimer's, but they think they "could have" used better cognitive patterns. This is where a lot of the damage lies, I think:

Most people's "coulds" are broken.

People think that they "could have" avoided anxiety at that one party. They think they "could have" stopped playing Civilization at a reasonable hour and gone to bed. They think they "could have" stopped watching House of Cards between episodes. I'm not making a point about the illusion of free will, here — I think there is a sense in which we "could" do certain things that we do not in fact do. Rather, my point is that most people have a miscalibrated idea of what they could or couldn't do.

People berate themselves whenever their brain fails to be engraved with the cognitive patterns that they wish it was engraved with, as if they had complete dominion over their own thoughts, over the patterns laid down in their heads. As if they weren't a network of neurons. As if they could choose their preferred choice in spite of their cognitive patterns, rather than recognizing that choice is a cognitive pattern. As if they were supposed to choose their mind, rather than being their mind.

As if they were already gods.

We aren't gods.

Not yet.

We're still monkeys.

Almost everybody is a total mess internally, as best as I can tell. Almost everybody struggles to act as they wish to act. Almost everybody is psychologically fragile, and can be put into situations where they do things that they regret — overeat, overspend, get angry, get scared, get anxious. We're monkeys, and we're fairly fragile monkeys at that.

So you don't need to beat yourself up when you miss your targets. You don't need to berate yourself when you fail to act exactly as you wish to act. Acting as you wish doesn't happen for free, it only happens after tweaking the environment and training your brain. You're still a monkey!

Don't berate the mon...
Genosets
by  — 2015-08-09T01:37:40.163Z

Since risk from individual SNP's 'should' not be aggregated to indicate an individual's risk based on multiple sources of evidence, how are the magnitudes for genosets determined?. Can bayes or another method be used to interpret a promethease report?

Even genetic epidemiology textbooks seem pessimistic: about the usefulness of the genetic research underpinning precision medicine:

‘...for the repeated failure to replicate positive findings in genetic epidemiology (102; 103) and remains the subejct of an important ongoing debate (101-105)’ -pg. 26 on chapter 1. An Introduction to Genetic Epidemiology

The references in question are about the impact of population stratification on genetic association studies. That doesn’t seem to substantiate such a broad stroke about the non-replicability of genetic epidemiology. I don't know what to make of these findings.

Here is a link to a screenshot of those references

It suprises me that entrepreneurial machine learning analysts don’t beg for genetic research to identify how combinatorial patterns of genes to be able to characterise individual risk. It seems like if/once they can get hold of that information, the sequence from genetic science to consumer actionable health information is bridged. So where are the 'lean gene learning machine' startups? I certainly don’t have the lean gene to do it myself. I don’t know machine learning.

Regulatory issues seems like the biggest hurdle. To the best of my google-fu, 23andme doesn't even disclose what it's 'Established Research' genes are. So, once regulatory hurdles are surmounted, lots of useful research will flood out.
Modal SAT: Introduction
by Scott Garrabrant — 2015-08-08T20:37:33.000Z

Let A1,…,An be a list of modal agents as described in Robust Cooperation in the Prisoner’s Dilemma:
Program Equilibrium via Provability Logic. If a modal agent M is going to play against each of these agents exactly once in the prisoners dilemma, what is the greatest possible score that M can achieve? How can one find a modal agent which achieves that score? This is the first in a series of posts I will make investigating how to answer that question.

We start by turning this into a decision problem. Given two lists of modal agents as input, C1,…,Cn, and D1,…,Dm, we ask whether or not there exists an M which defects against D1,…,Dm and gets C1,…,Cn to cooperate with it. We call this decision problem Modal SAT.

In the special case where n=m=1 and C1=D1, we call this decision problem Exploit.

We will also look at two variations of Modal SAT. In SC Modal SAT, we additionally require that M cooperates with itself. In NE Modal SAT, we require that the agent M is non exploitable.

Note that in these decision problems, we do not require actually constructing the agent M, but all of the algorithms which we plan to discuss will be constructive.

Further, note that the original optimization problem does not reduce obviously in polynomial time to Modal SAT. This is because for a given list of agents and a given score, there could be exponentially many ways to cooperate and defect with the agents to achieve that score. We do however have an exponential time reduction, and we might be able to construct a polynomial time reduction.

As we will soon see, Modal SAT is NP-Hard. However, we believe that we can construct a program that can solve practical instances of Modal SAT quickly. Mihaly Barasz is working on writing up such a program, while I write up some theoretical results about the problem. I suspect that I will be able to prove that all for decision problems are NP-Complete (and therefore in NP), but have not gone through all the details. I further expect to show that Modal SAT and SC Modal SAT are actually the same problem. (no reduction necessary)

I am uncertain what order I will make posts in this series, and I am further uncertain whether or not I will get to all the posts that I plan to. This series is not very high priority for me. As I make new posts, I will post links here.

Introduction

Self Cooperation
Peer-to-peer "knowledge exchanges"
by snarles — 2015-08-08T15:33:54.319Z

I wonder if anyone has thought about setting up an online community dedicated to peer-to-peer tutoring. The idea is that if I want to learn "Differential Geometry" and know "Python programming", and you want to learn "Python programming" and know "Differential geometry," then we can agree to tutor each other online. The features of the community would be to support peer-to-peer tutoring by:

Facilitating matchups between compatible tutors

Allowing for more than two people to participate in a tutoring arrangement

Providing reputation-based incentives to honor tutoring agreements and putting effort into tutoring

Allowing other members to "sit in" on tutoring sessions, if they are made public

Allowing the option to record tutoring sessions

Providing members with access to such recorded sessions and "course materials"

Providing a forum to arrange other events

With such functions, the community would have some overlap with other online learning platforms, but the focus of the community would be to provide free, quality personalized teaching.

The LessWrong community could build the first version of this peer tutoring system. It has people with broad interests, high intellectual standards, and many engineers who could help develop some of the infrastructure. The first iteration of the community would be small, and many of the above features (e.g. a reputation system, and tools for facilitating matchups) would not be needed. The first problems we would need to solve are:

Where should we host the community? (e.g. Google groups?)

What are some basic ground rules to ensure the integrity of the community and ensure safety?

Where can we provide a place for people to list which subjects they want to learn and which subjects they can teach?

Which software should we use for tutoring?

How can people publicize their tutoring schedule in case others want to "sit in"?

How can people record their tutoring sessions if they wish, and how can they make these available?

How should the community be administrated? Who should be put in charge of organizing the development of the community?

How should we recruit new members?
Tell me what hat to buy
by DataPacRat — 2015-08-08T15:10:19.562Z

For years, I've taken being a classic nerd, geek, and hacker as a point of pride - eg, consciously trying to judge people by the code they produce, or whatever else they write, as opposed to judging them on their appearance, to the point that I prefer /not/ to know what my favourite authors look like. I've tried to make what strengths I can out of the resulting weaknesses, such as reducing decision fatigue by keeping a single hair-style for many years, wearing whichever t-shirt is on top of the clean shirt pile, and so on.

I'm no longer satisfied with this. I want to become stronger.

Last month, I bought a dozen button-up, collared shirts... and have noticed slight, but consistent changes in my workflow when I wear them. I want to leverage whatever other clothing-based self-improvements are within my budget.

For some years, I've worn a floppy boonie hat to shade my delicate eyes from the burning rays of the sun. (I've even been seen wearing it with a photographer's vest instead of a daypack while tromping around my hometown.) I'm thinking of trying out the 'Crasche' safety inserts mentioned in the recent Open Thread while hiking far from medical help, which would require a baseball-like cap with a sweatband. Given my proclivity for taking something that works and sticking with it for years, I might be wearing that cap for a very long time.

Thus, a multi-layered question: Which hat should I buy? Which factors should I take into account... and which shouldn't I? Are there any subreddits, forums, or other online discussion groups whose members would be willing to take this question seriously and with only a minimum of mockery?
Meetup : Rationality Reading Group (71-75)
by SeekingEternity — 2015-08-08T08:45:04.124Z

Discussion article for the meetup : Rationality Reading Group (71-75)

WHEN:
10 August 2015 06:30:00PM (-0700)

WHERE:
Paul G. Allen Center (185 Stevens Way, Seattle, WA) Room 503

Reading group for Yudkowsky's "Rationality: AI to Zombies", which is basically an organized and updated version of the Sequences from LW (see http://wiki.lesswrong.com/wiki/Sequences).

The group meets to discuss the topics in the book, how to apply and benefit from them, and related topics in areas like cognitive biases, applied rationality, and effective altruism. You can get a copy of the book here: https://intelligence.org/rationality-ai-zombies/

The reading list for this week is five topics from the "Against Rationalization" section of Book II, "How To Actually Change Your Mind". They are (actually 71-75, LW's auto-formatting is screwing it up):

What Evidence Filtered Evidence?

Rationalization

A Rational Argument

Avoiding Your Belief’s Real Weak Points

Motivated Stopping and Motivated Continuation

We previously covered the "Map and territory" sequence a few months ago, but please don't feel a need to have read everything up to this point to participate in the group.

Event is also on Facebook:
https://www.facebook.com/events/1008817309162312/

We're meeting on the 5th floor. If you show up and the door into the room is locked, knock and look around for us elsewhere on the fifth floor if nobody answers. If the doors to the building are locked, try the other ones and see if you can tailgate in. If the doors are, in fact, locked, we'll try to have somebody to let people in.

There's usually snacks at the meetup, though feel free to bring something. We usually get dinner afterward, around 9PM or so.

Discussion article for the meetup : Rationality Reading Group (71-75)