trentmkelly/LessWrong-43k huggingface dataset:
Learning and Internalizing the Lessons from the Sequences
by Nick5a1 — 2016-09-14T14:40:28.424Z

I'm just beginning to go through Rationality: From AI to Zombies. I want to make the most of the lessons contained in the sequences. Usually when I read a book I simply take notes on what seems useful at the time, and a lot of it is forgotten a year later. Any thoughts on how best to internalize the lessons from the sequences?
Learning values versus learning knowledge
by Stuart_Armstrong — 2016-09-14T13:42:25.000Z

I just thought I'd clarify the difference between learning values and learning knowledge. There are some more complex posts about the specific problems with learning values, but here I'll just clarify why there is a problem with learning values in the first place.

Consider the term "chocolate bar". Defining that concept crisply would be extremely difficult. But nevertheless it's a useful concept. An AI that interacted with humanity would probably learn that concept to a sufficient degree of detail. Sufficient to know what we meant when we asked it for "chocolate bars". Learning knowledge tends to be accurate.

Contrast this with the situation where the AI is programmed to "create chocolate bars", but with the definition of "chocolate bar" left underspecified, for it to learn. Now it is motivated by something else than accuracy. Before, knowing exactly what a "chocolate bar" was would have been solely to its advantage. But now it must act on its definition, so it has cause to modify the definition, to make these "chocolate bars" easier to create. This is basically the same as Goodhart's law - by making a definition part of a target, it will no longer remain an impartial definition.

What will likely happen is that the AI will have a concept of "chocolate bar", that it created itself, especially for ease of accomplishing its goals ("a chocolate bar is any collection of more than one atom, in any combinations"), and a second concept, "Schocolate bar" that it will use to internally designate genuine chocolate bars (which will still be useful for it to do). When we programmed it to "create chocolate bars, here's an incomplete definition D", what we really did was program it to find the easiest thing to create that is compatible with D, and designate them "chocolate bars".

This is the general counter to arguments like "if the AI is so smart, why would it do stuff we didn't mean?" and "why don't we just make it understand natural language and give it instructions in English?"
Learning values versus learning knowledge
by Stuart_Armstrong — 2016-09-14T13:42:06.734Z

I just thought I'd clarify the difference between learning values and learning knowledge. There are some more complex posts about the specific problems with learning values, but here I'll just clarify why there is a problem with learning values in the first place.

Consider the term "chocolate bar". Defining that concept crisply would be extremely difficult. But nevertheless it's a useful concept. An AI that interacted with humanity would probably learn that concept to a sufficient degree of detail. Sufficient to know what we meant when we asked it for "chocolate bars". Learning knowledge tends to be accurate.

Contrast this with the situation where the AI is programmed to "create chocolate bars", but with the definition of "chocolate bar" left underspecified, for it to learn. Now it is motivated by something else than accuracy. Before, knowing exactly what a "chocolate bar" was would have been solely to its advantage. But now it must act on its definition, so it has cause to modify the definition, to make these "chocolate bars" easier to create. This is basically the same as Goodhart's law - by making a definition part of a target, it will no longer remain an impartial definition.

What will likely happen is that the AI will have a concept of "chocolate bar", that it created itself, especially for ease of accomplishing its goals ("a chocolate bar is any collection of more than one atom, in any combinations"), and a second concept, "Schocolate bar" that it will use to internally designate genuine chocolate bars (which will still be useful for it to do). When we programmed it to "create chocolate bars, here's an incomplete definition D", what we really did was program it to find the easiest thing to create that is compatible with D, and designate them "chocolate bars".

This is the general counter to arguments like "if the AI is so smart, why would it do stuff we didn't mean?" and "why don't we just make it understand natural language and give it instructions in English?"
Universal Inductors
by Scott Garrabrant — 2016-09-14T00:09:42.000Z

Now that the Logical Induction paper is out, I am directing my attention towards decision theory. The approach I currently think will be most fruitful is attempting to make a logically updateless version of Wei Dai's Updateless Decision Theory. Abram Demski has posted on here about this, but I think Logical Induction provides a new angle with which we can attack the problem. This post will present an alternate way of viewing Logical Induction which I think will be especially helpful for building a logical UDT. (The Logical Induction paper is a prerequisite for this post.)

This post will make several informal claims without proof. Many of the claims are analogous to things proven in the Logical Induction paper, and the math is not substantially difficult. I might extend the paper to have a section talking about the following concepts more formally.

A Universal Inductor is an infinite sequence {Pn} of probability distributions on infinite bit strings. Satisfying the following two properties:

The function (n,σ)↦Pn(σ) is computable, where σ is a finite prefix, and Pn(σ) is the probability that an infinite bit string begins with the prefix σ.

Consider the propositional theory whose nth atom is identified with the event "the nth bit in the infinite bit string is a 1." Then the probability distribution Pn induces an function from sentences in the language of this theory to probabilities. The sequence of probability assignments must form a Logical Inductor over the empty deductive process.

Note that a Universal Inductor corresponds to a Logical Inductor, but the associated Logical Inductor will not have finite support, and so will be different from the one constructed in the paper. Never the less, Universal Inductors can be shown to exist using a similar construction. Note that the second condition technically implies the first, but I separate them for emphasis.

Here are some facts about Universal Inductors:

The probability distributions converge to a limiting distribution on infinite bit strings, P∞.

This limiting distribution dominates the universal semi measure. In particular, every finite prefix is given nonzero probability.

Given any consistent deductive process, we can construct a logical inductor over that deductive process simply by associating atoms of the theory with bits in the bit string (in an efficient computable manner), conditioning Pn on th

...
IRL is hard
by Vanessa Kosoy — 2016-09-13T14:55:26.000Z

We show that assuming the existence of public-key cryptography, there is an environment in which Inverse Reinforcement Learning is computationally intractable, even though the "teacher" agent, the environment and the utility functions are computable in polynomial-time and there is only 1 bit of information to learn.

Construction

We call "Teacher" the role-model agent which optimizes the "true" utility function (in the context of AI safety this is typically a human) and "Student" the agent performing IRL (the AI). The environment we construct consists of two entities which we call the Gamemaster and the Adversary. The Adversary is an optimizer whose utility is minus the Teacher's utility. To reduce the setup to a single agent environment, we can interpret the Adversary as a specific optimization algorithm which either has access to the Teacher's source code or is a reinforcement learning algorithm that learns from preceding episodes. At any rate, the Adversary is only of secondary importance, as explained in the Discussion. The entire setup depends on a parameter n∈N w.r.t. which the complexities of the computations are measured.

The utility function of the Teacher depends on a bit α∈{0,1} which is unknown to the Student (i.e. the Student's utility function is defined according to a uniform prior on α) but known to the Gamemaster. Learning α is the purpose of IRL in this setting. The Student is assumed to passively observe the Teacher interacting with the environment for q(n) epsiodes for some polynomial q, after which it is required to assume the Teacher's role. For the environment we construct, no polynomial-time Student cannot achieve significantly better performance than the Lazy Student who ignores the Teacher and acts according to its prior.

We fix a public-key encryption scheme which encrypts any message of size n to a message of size n (a trapdoor permutation). Given a public (encryption) key b and a message x∈{0,1}n, we denote its encryption e(x,b). Given a private (decryption) key v and an encrypted message y∈{0,1}n, we denote its decryption d(y,v). Thus, if (b,v) is a matching key pair then for all x∈{0,1}n, d(e(x,b),v)=x.

Below is a step by step description of a single episode of interaction between the Teacher and the environment. Each entity has access to random bits hidden from the other enti...
Meetup : Melbourne Social Meetup: September
by Maelin — 2016-09-13T12:37:51.337Z

Discussion article for the meetup : Melbourne Social Meetup: September

WHEN:
16 September 2016 06:30:00PM (+1000)

WHERE:
Alchemist's Refuge, 328 Little Lonsdale St, Melbourne

Our social meetup for September is on, as usual this Friday!

FB event page: https://www.facebook.com/events/550507195145048/

Social Meetups are casual get-togethers held on the third Friday of each month. They are informal events where we sit around and chat over a few drinks and a meal. Sometimes we will play games as well.

Where? Alchemist's Refuge, 328 Little Lonsdale St, Melbourne

When? Friday 16th September, starting from 6:30 pm. Start time is loose so show up whenever. We are usually there until about 11pm.

Dinner? There are some take away places nearby and the Refuge don't mind people bringing in food. A few of us usually go for a late dinner - possibly at Father's Office which offers a late night menu at half price.

Contact? Any issues or questions, contact Richard on 0421231789

Hope to see you there!

Discussion article for the meetup : Melbourne Social Meetup: September
Meetup : Washington, D.C.: Steelmanning
by RobinZ — 2016-09-12T23:36:29.162Z

Discussion article for the meetup : Washington, D.C.: Steelmanning

WHEN:
18 September 2016 03:30:00PM (-0400)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

We will be meeting in the courtyard to talk about steelmanning - creating a stronger version of an argument against your position to respond to. The Less Wrong Wiki has a page defining the term with a link to the blog post that originated it, and Ozymandias wrote a blog post outlining a couple potential failure modes.

Upcoming meetups:

Sep. 25: Outdoor Fun & Games

Oct. 2: Math Discussion

Oct. 9: Games Discussion

Discussion article for the meetup : Washington, D.C.: Steelmanning
Neutralizing Physical Annoyances
by SquirrelInHell — 2016-09-12T16:36:41.379Z
Link: http://squirrelinhell.blogspot.com/2016/09/neutralizing-physical-annoyances.html

I have moved this post to my blog: http://squirrelinhell.blogspot.com/2016/09/neutralizing-physical-annoyances.html
It’s Bayes All The Way Up
by Scott Alexander — 2016-09-12T13:35:46.000Z

[Epistemic status: Very speculative. I am not a neuroscientist and apologize for any misinterpretation of the papers involved. Thanks to the people who posted these papers in r/slatestarcodex. See also Mysticism and Pattern-Matching and Bayes For Schizophrenics]

Bayes’ Theorem is an equation for calculating certain kinds of conditional probabilities. For something so obscure, it’s attracted a surprisingly wide fanbase, including doctors, environmental scientists, economists, bodybuilders, fen-dwellers, and international smugglers. Eventually the hype reached the point where there was both a Bayesian cabaret and a Bayesian choir, popular books using Bayes’ Theorem to prove both the existence and the nonexistence of God, and even Bayesian dating advice. Eventually everyone agreed to dial down their exuberance a little, and accept that Bayes’ Theorem might not literally explain absolutely everything.

So – did you know that the neurotransmitters in the brain might represent different terms in Bayes’ Theorem?

First things first: Bayes’ Theorem is a mathematical framework for integrating new evidence with prior beliefs. For example, suppose you’re sitting in your quiet suburban home and you hear something that sounds like a lion roaring. You have some prior beliefs that lions are unlikely to be near your house, so you figure that it’s probably not a lion. Probably it’s some weird machine of your neighbor’s that just happens to sound like a lion, or some kids pranking you by playing lion noises, or something. You end up believing that there’s probably no lion nearby, but you do have a slightly higher probability of there being a lion nearby than you had before you heard the roaring noise. Bayes’ Theorem is just this kind of reasoning converted to math. You can find the long version here.

This is what the brain does too: integrate new evidence with prior beliefs. Here are some examples I’ve used on this blog before:

All three of these are examples of top-down processing. Bottom-up processing is when you build perceptions into a model of the the world. Top-down processing is when you let your models of the world influence your perceptions. In the first image, you view the center letter of the the first word as an H and the second as an A, even though they’re the the same character; your model of the world tells you that THE CAT is more likely than TAE CHT. In the second i...
Open thread, Sep. 12 - Sep. 18, 2016
by MrMind — 2016-09-12T06:49:47.987Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
The progressive case for replacing the welfare state with basic income
by morganism — 2016-09-11T22:41:34.545Z

I really like this argument for UBI, it looks at the real problems on the front lines of current problems with existing programs.

Something has to be done about this, none of those manufacturing, or managerial jobs are coming back, and the minimum wage increase movement is going to really hammer youth and the un-skilled. Those are the easiest to radicalize, and once they get really angry, things could get ugly very quickly.

"because they aren’t publicly acknowledging just how poorly our present means-tested programs are targeted by virtue of their applied conditions, and just how unequal one dollar can be to one dollar, however counterintuitive that may seem."

"The fact is that cash welfare, as it exists today, is not given to the overwhelming majority of those living in poverty who need it."

https://techcrunch.com/2016/09/09/the-progressive-case-for-replacing-the-welfare-state-with-basic-income/
Meetup : Rationality Meetup Vienna
by Viliam — 2016-09-11T20:57:16.910Z

Discussion article for the meetup : Rationality Meetup Vienna

WHEN:
24 September 2016 03:00:00PM (+0200)

WHERE:
Kaisermühlenstraße 24

Agenda:
15:00 - 15:30 arrival and social time
15:30 official start with an introduction round
16:00 defining the topic(s) of the day (might be one big presentation or open microphone which means everyone can offer short talks or topics for discussion)
at around 18:00 the "official" part of the meetup ends and we clean the meetup room
afterwards we'll have dinner and then probably stroll through the city
https://www.facebook.com/events/1607525092878284/

Discussion article for the meetup : Rationality Meetup Vienna
Meetup : SF Meetup: Projects
by maia — 2016-09-11T20:36:48.767Z

Discussion article for the meetup : SF Meetup: Projects

WHEN:
12 September 2016 06:15:36PM (-0700)

WHERE:
1597 Howard St., SF

We’ll be meeting to work on projects!

Near the beginning, we’ll go around and talk about what we’ll be working on, then do a couple of pomodoros quietly. At some point we’ll break into general conversations and socializing.

For help getting into the building, please call (or text, with a likely-somewhat-slower response rate): three zero one, three five six, five four two four.

Format:

We meet and start hanging out at 6:15, but don’t officially start doing the meetup topic until 6:45-7 to accommodate stragglers. Usually there is a food order that goes out before we start the meetup topic.

About these meetups:

The mission of the SF LessWrong meetup is to provide a fun, low-key social space with some structured interaction, where new and non-new community members can mingle and have interesting conversations. Everyone is welcome.

We explicitly encourage people to split off from the main conversation or diverge from the topic if that would be more fun for them (moving side conversations into a separate part of the space if appropriate). Meetup topics are here as a tool to facilitate fun interaction, and we certainly don’t want them to inhibit it.

Discussion article for the meetup : SF Meetup: Projects
The Extraordinary Link Between Deep Neural Networks and the Nature of the Universe
by morganism — 2016-09-10T19:13:14.182Z

"The answer is that the universe is governed by a tiny subset of all possible functions. In other words, when the laws of physics are written down mathematically, they can all be described by functions that have a remarkable set of simple properties."

“For reasons that are still not fully understood, our universe can be accurately described by polynomial Hamiltonians of low order.” These properties mean that neural networks do not need to approximate an infinitude of possible mathematical functions but only a tiny subset of the simplest ones."

Interesting article, and just diving into the paper now, but it looks like this is a big boost to the simulation argument. If the universe is built like a game engine, with stacked sets like Mandelbrots, then the simplicity itself becomes a driver in a fabricated reality.

https://www.technologyreview.com/s/602344/the-extraordinary-link-between-deep-neural-networks-and-the-nature-of-the-universe/

Why does deep and cheap learning work so well?

http://arxiv.org/abs/1608.08225
2016 LessWrong Diaspora Survey Analysis: Part Four (Politics, Calibration & Probability, Futurology, Charity & Effective Altruism)
by namespace — 2016-09-10T03:51:44.051Z

Politics

The LessWrong survey has a very involved section dedicated to politics. In previous analysis the benefits of this weren't fully realized. In the 2016 analysis we can look at not just the political affiliation of a respondent, but what beliefs are associated with a certain affiliation. The charts below summarize most of the results.

Political Opinions By Political Affiliation

Miscellaneous Politics

There were also some other questions in this section which aren't covered by the above charts.

PoliticalInterest

On a scale from 1 (not interested at all) to 5 (extremely interested), how would you describe your level of interest in politics?

1: 67 (2.182%)

2: 257 (8.371%)

3: 461 (15.016%)

4: 595 (19.381%)

5: 312 (10.163%)

Voting

Did you vote in your country's last major national election? (LW Turnout Versus General Election Turnout By Country)

Group	Turnout

LessWrong
68.9%

Austrailia
91%

Brazil
78.90%

Britain
66.4%

Canada
68.3%

Finland
70.1%

France
79.48%

Germany
71.5%

India
66.3%

Israel
72%

New Zealand
77.90%

Russia
65.25%

United States
54.9%

Numbers taken from Wikipedia, accurate as of the last general election in each country listed at time of writing.

AmericanParties

If you are an American, what party are you registered with?

Democratic Party: 358 (24.5%)

Republican Party: 72 (4.9%)

Libertarian Party: 26 (1.8%)

Other third party: 16 (1.1%)

Not registered for a party: 451 (30.8%)

(option for non-Americans who want an option): 541 (37.0%)

Calibration And Probability Questions

Calibration Questions

I just couldn't analyze these, sorry guys. I put many hours into trying to get them into a decent format I could even read and that sucked up an incredible amount of time. It's why this part of the survey took so long to get out. Thankfully another LessWrong user, Houshalter, has kindly done their own analysis.

All my calibration questions were meant to satisfy a few essential properties:

They should be 'self contained'. I.E, something you can reasonably answer or at least try to answer with a 5th grade science education and normal life experience.

They should, at least to a certain extent, be Fermi Estimable.

They should progressively scale in difficulty so you can see whether somebody understands basic probability or not. (eg. In an 'or' question do they put a pro

...
2016 LessWrong Diaspora Survey Analysis: Part Four (Politics, Calibration & Probability, Futurology, Charity & Effective Altruism)
by namespace — 2016-09-10T03:38:26.394Z

Politics

The LessWrong survey has a very involved section dedicated to politics. In previous analysis the benefits of this weren't fully realized. In the 2016 analysis we can look at not just the political affiliation of a respondent, but what beliefs are associated with a certain affiliation. The charts below summarize most of the results.

Political Opinions By Political Affiliation

Miscellaneous Politics

There were also some other questions in this section which aren't covered by the above charts.

PoliticalInterest

On a scale from 1 (not interested at all) to 5 (extremely interested), how would you describe your level of interest in politics?

1: 67 (2.182%)

2: 257 (8.371%)

3: 461 (15.016%)

4: 595 (19.381%)

5: 312 (10.163%)

Voting

Did you vote in your country's last major national election? (LW Turnout Versus General Election Turnout By Country)

Group	Turnout

LessWrong
68.9%

Austrailia
91%

Brazil
78.90%

Britain
66.4%

Canada
68.3%

Finland
70.1%

France
79.48%

Germany
71.5%

India
66.3%

Israel
72%

New Zealand
77.90%

Russia
65.25%

United States
54.9%

Numbers taken from Wikipedia, accurate as of the last general election in each country listed at time of writing.

AmericanParties

If you are an American, what party are you registered with?

Democratic Party: 358 (24.5%)

Republican Party: 72 (4.9%)

Libertarian Party: 26 (1.8%)

Other third party: 16 (1.1%)

Not registered for a party: 451 (30.8%)

(option for non-Americans who want an option): 541 (37.0%)

Calibration And Probability Questions

Calibration Questions

I just couldn't analyze these, sorry guys. I put many hours into trying to get them into a decent format I could even read and that sucked up an incredible amount of time. It's why this part of the survey took so long to get out. Thankfully another LessWrong user, Houshalter, has kindly done their own analysis.

All my calibration questions were meant to satisfy a few essential properties:

They should be 'self contained'. I.E, something you can reasonably answer or at least try to answer with a 5th grade science education and normal life experience.

They should, at least to a certain extent, be Fermi Estimable.

They should progressively scale in difficulty so you can see whether somebody understands basic probability or not. (eg. In an 'or' question do they put a pro

...