trentmkelly/LessWrong-43k huggingface dataset:
Destroying the Utility Monster—An Alternative Formation of Utility
by DragonGod — 2017-06-08T12:37:47.371Z

NOTE: This post contains LaTeX; it is recommended that you install “TeX the World” (for chromium users), “TeX All the Things” or other TeX/LaTeX extensions to view the post properly.

Destroying the Utility Monster—An Alternative Formation of Utility

I am a rational egoist, but that is only because there is no existing political system/social construct I identify with. If there was one I identified with, I would be strongly utilitarian. In all moral thought experiments, I err on the side of utilitarianism, and I’m faithful in my devotion to its tenets. There are some criticisms against utilitarianism, and one of the most common—and most powerful—is the utility monster which allegedly proves “utilitarianism is not egalitarian’’. [1]

For those who may not understand the terms, I shall define them below:

Utilitarianism is an ethical theory that states that the best action is the one that maximizes utility. "Utility" is defined in various ways, usually in terms of the well-being of sentient entities. Jeremy Bentham, the founder of utilitarianism, described utility as the sum of all pleasure that results from an action, minus the suffering of anyone involved in the action. Utilitarianism is a version of consequentialism, which states that the consequences of any action are the only standard of right and wrong. Unlike other forms of consequentialism, such as egoism, utilitarianism considers all interests equally.

[2]

The utility monster is a thought experiment in the study of ethics created by philosopher Robert Nozick in 1974 as a criticism of utilitarianism
A hypothetical being, which Nozick calls the utility monster, receives much more utility from each unit of a resource they consume than anyone else does. For instance, eating a cookie might bring only one unit of pleasure to an ordinary person but could bring 100 units of pleasure to a utility monster. If the utility monster can get so much pleasure from each unit of resources, it follows from utilitarianism that the distribution of resources should acknowledge this. If the utility monster existed, it would justify the mistreatment and perhaps annihilation of everyone else, according to the mandates of utilitarianism, because, for the utility monster, the pleasure they receive outweighs the suffering they may cause.[1] Noz

...
Bet or update: fixing the will-to-wager assumption
by cousin_it — 2017-06-07T15:03:23.923Z

(Warning: completely obvious reasoning that I'm only posting because I haven't seen it spelled out anywhere.)

Some people say, expanding on an idea of de Finetti, that Bayesian rational agents should offer two-sided bets based on their beliefs. For example, if you think a coin is fair, you should be willing to offer anyone a 50/50 bet on heads (or tails) for a penny. Jack called it the "will-to-wager assumption" here and I don't know a better name.

In its simplest form the assumption is false, even for perfectly rational agents in a perfectly simple world. For example, I can give you my favorite fair coin so you can flip it and take a peek at the result. Then, even though I still believe the coin is fair, I'd be a fool to offer both sides of the wager to you, because you'd just take whichever side benefits you (since you've seen the result and I haven't). That objection is not just academic: using your sincere beliefs to bet money against better informed people is a bad idea in real world markets as well.

Then the question arises, how can we fix the assumption so it still says something sensible about rationality? I think the right fix should go something like this. If you flip a coin and peek at the result, then offer me a bet at 90:10 odds that the coin came up heads, I must either accept the bet or update toward believing that the coin indeed came up heads, with at least these odds. I don't get to keep my 50:50 beliefs about the coin and refuse the bet at the same time. More generally, a Bayesian rational agent offered a bet (by another agent who might have more information) must either accept the bet or update their beliefs so the bet becomes unprofitable. The old obligation about offering two-sided bets on all your beliefs is obsolete, use this one from now on. It should also come in handy in living room Bayesian scuffles, throwing some money on the table and saying "bet or update!" has a nice ring to it.

What do you think?
New circumstances, new values?
by Stuart_Armstrong — 2017-06-06T08:20:33.344Z

Crossposted at the Intelligent Agents Forum.

Quick, is there anything wrong with a ten minute pleasant low-intensity conversation with someone we happen to disagree with?

Our moral intuitions say no, as do our legal system and most philosophical or political ideals since the enlightenment.

Quick, is there anything wrong with brainwashing people into perfectly obedient sheep, willing and eager to take any orders and betray all their previous ideals?

There’s a bit more disagreement there, but that generally is seen as a bad thing.

But what happens when the low-intensity conversation and the brainwashing are the same thing? At the moment, no human can overwhelm most other humans in the course of ten minutes talking, and rewrite their goals into anything else. But an AI may well be capable of doing so - people have certainly fallen in love within less than ten minutes, and we don’t know how “hard” this is to pull off, in some absolute sense.

This is a warning that relying on revealed and stated preferences or meta-preferences won’t be enough. Our revealed and (most) stated preferences are that the ten minute conversation is probably ok. But disentangling how much of that “ok” relies on our understanding the consequences will be a challenge.
New circumstances, new values?
by Stuart_Armstrong — 2017-06-06T08:18:39.000Z

A putative new idea for AI control; index here.

Quick, is there anything wrong with a ten minute pleasant low-intensity conversation with someone we happen to disagree with?

Our moral intuitions say no, as do our legal system and most philosophical or political ideals since the enlightenment.

Quick, is there anything wrong with brainwashing people into perfectly obedient sheep, willing and eager to take any orders and betray all their previous ideals?

There's a bit more disagreement there, but that generally is seen as a bad thing.

But what happens when the low-intensity conversation and the brainwashing are the same thing? At the moment, no human can overwhelm most other humans in the course of ten minutes talking, and rewrite their goals into anything else. But an AI may well be capable of doing so - people have certainly fallen in love within less than ten minutes, and we don't know how "hard" this is to pull off, in some absolute sense.

This is a warning that relying on revealed and stated preferences or meta-preferences won't be enough. Our revealed and (most) stated preferences are that the ten minute conversation is probably ok. But disentangling how much of that "ok" relies on our understanding the consequences will be a challenge.
Becoming a Better Community
by Sable — 2017-06-06T07:11:42.367Z

So I've been following Project Hufflepuff, the efforts of the rationalist community to become, rather than better rationalists (per se), but a better community. I recently read the summary of the recent Project Hufflepuff Unconference, and I had a thought.

The Problem

LessWrong And Guardedness

I can only speak to my own experiences in joining the community, but I have always felt that the rationalist community holds its members to a very high standard. This isn't a bad thing but it creates, at least in me, a sense of guardedness. I don't want to be the rationalist who sounds stupid or the one who contributes less to the conversation.

Every post I've made here on LessWrong (not that there have been many), has been reviewed and edited with the same kind of diligence that I normally reserve for graded essays or business documentation. Other online communities I'm a part of (and meatspace communities) require far less diligence from me as a contributor. (Note: This isn't a value judgement, rather a description of my experience.)

However, my best experiences in communities and friendships have generally occurred in very unguarded atmospheres. Not that my friends and I aren't smart or can't be smart, but most of the fun I've had with them happens when we're playing card or board or video games, or just hanging out and talking. Doing things like going out to eat, playing ping-pong, and talking about bad TV shows have led to some of the strongest relationships in my life.

So Where Is The Fun?

So - where is this in the Rationalist Community? Now, it is very possible that the fun is there and I'm simply missing it. I haven't been to any meetups, I don't live in the bay area, and I don't even know any rationalists in meatspace. But if it is, aside from the occasional meetup, I don't see any evidence of it.

I tried to do some research on how friendships and communities are formed, and there seemed to be little consensus in the field. A New York Times article on making friendships as an adult mentions three factors:

As external conditions change, it becomes tougher to meet the three conditions that sociologists since the 1950s have considered crucial to making close friends: proximity; repeated, unplanned interactions; and a setting that encourages people to let their guar

...
Argument From Infinity
by DragonGod — 2017-06-05T21:33:25.054Z

NOTE: This post contains LaTeX; it is recommended that you install “TeX the World” (for chromium users), “TeX All the Things” or other TeX/LaTeX extensions to view the post properly.

The Argument From Infinity

If you live forever then you will definitely encounter a completely terrible scenario like being trapped in a black hole or something.

I have noticed a tendency, for people to conclude that an infinite set implies that the set contains some potential element $Y$.

Say for example, that you live forever, this means that your existence is an infinite set. Let’s denote your existence as $E$.

$E = {x_1, x_2, x_3, …}$
Where each $x_i$ is some event that can potentially happen to you.
The fallacy of infinity is positing that because $E$ is infinite, $E$ contains $x_j$.

However, this is simply wrong. Before I prove that the infinity fallacy is in fact a logical fallacy, I will posit a hypothesis as to the underlying cause of the fallacy of infinity.

I suspect it is because people have a poor understanding of the nature of infinity. They assume, that because $E$ is infinite, $E$ contains all potential $x_i$. If $E$ did not contain any potential $x_i$, then $E$ would not be infinite, and since the premise is that $E$ is infinite, then $E$ contains $x_j$.
&nsbp;

Counter Argument.

I shall offer an algorithm that would demonstrate how to generate an infinite number of infinite subsets from an infinite set.

Pick an element $i$ in $N$. Exclude $i$ from $N$. You have generated an infinite subset of $N$.
There are $\aleph_0$ possible such infinite subsets.
Pick any two elements from $n$ and exclude them. You have generated another infinite subset of $N$. There are $\aleph_0$ \choose $2$ possible infinite subsets.
In general, we can generate an infinite subset by excluding $k$ elements from $N$. The number of such infinite subsets generated is $\aleph_0$ \choose $k$.

To find out the total number of infinite subsets that can be generated, take
$$\sum_{k=1}{\aleph_0} {\aleph_0 \choose k}$$

However, these are only the infinite subsets of finite complements. To get infinite subsets of infinite complements, we can pick any (finite) subset of $\aleph_1$, and find the product of that set. Take only all multiples of that set, or exclude all multiples of that set. That gives yo

...
Mode Collapse and the Norm One Principle
by tristanm — 2017-06-05T21:30:58.027Z

[Epistemic status: I assign a 70% chance that this model proves to be useful, 30% chance it describes things we are already trying to do to a large degree, and won't cause us to update much.]

I'm going to talk about something that's a little weird, because it uses some results from some very recent ML theory to make a metaphor about something seemingly entirely unrelated - norms surrounding discourse.

I'm also going to reach some conclusions that surprised me when I finally obtained them, because it caused me to update on a few things that I had previously been fairly confident about. This argument basically concludes that we should adopt fairly strict speech norms, and that there could be great benefit to moderating our discourse well.

I argue that in fact, discourse can be considered an optimization process and can be thought of in the same way that we think of optimizing a large function. As I will argue, thinking of it in this way will allow us to make a very specific set of norms that are easy to think about and easy to enforce. It is partly a proposal for how to solve the problem of dealing with speech that is considered hostile, low-quality, or otherwise harmful. But most importantly, it is a proposal for how to ensure that the discussion always moves in the right direction: Towards better solutions and more accurate models.

It will also help us avoid something I'm referring to as "mode collapse" (where new ideas generated are non-diverse and are typically characterized by adding more and more details to ideas that have already been tested extensively). It's also highly related to the concepts discussed in the Death Spirals and the Cult Attractor portion of the Sequences. Ideally, we'd like to be able to make sure that we're exploring as much of the hypothesis space as possible, and there's good reason to believe we're probably not doing this very well.

The challenge: Making sure we're searching for the global optimum in model-space sometimes requires reaching out blindly into the frontiers, the not well-explored regions, which runs the risk of ending up somewhere very low-quality or dangerous. There are also sometimes large gaps between very different regions of model-space where the quality of the model is very low in-between, but very high on each side of the gap. This requires traversing through potentially dangerous...
The Simple World Hypothesis
by DragonGod — 2017-06-05T19:34:06.488Z

Part of a Series in the Making: "If I Were God".

Introduction

The current universe is the simplest possible universe with the same degree of functionality.

This hypothesis posits that the current universe is the simplest universe possible which can do all that our universe can do. Here, simplicity refers to the laws which make up the universe. It may be apt to mention the Multiverse Axioms at this juncture:
Axiom 1 (axiom of consistency):

Any possible universe is logically consistent and strictly adheres to well-defined laws.

Axiom 2 (axiom of inclusivity):

Whatever can happen (without violating 1) happens—and in every way possible (without violating 1).

Axiom 3 (axiom of simplicity):
The underlying laws governing the Multiverse are as simple as possible (while permitting 1 and 2).

The simple world hypothesis posits that our universe has the fewest laws which can enable the same degree of functionality that it currently possesses. I’ll explain the concept of “degree of functionality”. Take two universes: U_i and U_j with degrees of functionality d_i and d_j. Then the below three statements are true:
d_i > d_j implies that U_i can simulate U_j.
d_j < d_i implies that U_j cannot simulate U_i. d_i = d_j implies that U_i can simulate U_j, and U_j can in turn simulate U_i.

Let’s consider a universe like Conway’s Game of Life. It is far simpler than our universe and possesses only four laws. The simple world hypothesis argues that Conway’s Game of Life (U_c) cannot simulate our universe (U_0). The degree of functionality of Conway’s Game of Life (d_c) < the degree of functionality of our universe d_0. An advance prediction of the simple world hypothesis regarding U_c is the below:

Human level intelligence cannot emerge in U_c.

The above implicitly assumes that Conway’s Game of Life is simpler than our universe—is that really true?

Simplicity

It is only prudent that I clarify what it is I mean by simplicity. For any two Universes U_i and U_j, let their simplicity be denoted S_i and S_j respectively. The simplicity of a universe is the Kolmogorov complexity of the set of laws which make up that universe.
For U_c, those laws are:
1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
2. Any live cell with two or three live neighbour...
Open thread, June 5 - June 11, 2017
by Elo — 2017-06-05T04:23:51.697Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting
Birth of a Stereotype
by DragonGod — 2017-06-05T03:29:41.294Z

I imagine that many ‘enlightened’ people disbelieve in stereotypes, thinking them useless, and something below them; only the ignorant masses rely on stereotypical thinking and generalisations. Stereotypes are improbable, and making such generalisations from anecdotal evidence at best is faulty. I used to think like that—and I probably still do—nevertheless, someone raised an argument that stereotypes aren’t useless; they exist for a reason. There was a reason why the stereotype was born. As such, you would expect that a stereotype conveys at least some information and is better than nothing.

I think a few factors contribute to the formation of a stereotype. I will take a minute to explain a stereotype. A stereotype is a relation of the form X => Y. It maps a class of people/individuals/what have you to a property X. For example, people who wear glasses are smart. Occasionally, some individuals may conceive the relation as Y <=> X. E.g. Smart people wear glasses. I suspect this is due to reasons unrelated to the stereotype (e.g. inability to distinguish between ’=>’ and ’<=>’). I hope this is not common among the general population—the average human can’t be that irrational, right? I shall give a charitable interpretation of the masses, and discuss only the relation 'X => Y’. I would stick to two particularly conspicuous stereotypes and try to hypothesise how they originated. For one, anecdotal evidence, combined with the availability heuristic make people overrate the occurrence of certain relations. I shall pick the “people who wear glasses are smart” example: Some smart people use glasses. Due to the availability of smart people who use glasses the relation 'glasses’ => 'smart’ gets reinforced. However, this hides an implicit assumption; that the average IQ/intelligence (perceived or otherwise)/'smartness’ of people who wear glasses. Is this true? Normally, my common sense tells me that this is false, however there is correlation between height and intelligence: NCBI, Wikipedia, medicalxpress. I have not yet read any of them in sufficient detail—or any detail for that matter—as at the time of writing this, but I have bookmarked the links for future study. I cannot ascertain whether glasses actually...
A Comment on Expected Utility Theory
by DragonGod — 2017-06-05T03:26:23.688Z

A Comment on Expected Utility Theory

Expected utility theory/expected value decision making—as the case may be—is quite interesting I guess. In times, past (for a few months at longest as I am a neophyte to rationality) I habitually trusted the answers expected utility theory provided without bothering to test them, or ponder for myself why they would be advisable. I mean, I first learned the concept of expected value in statistics and when we studied it in Operations Research—as part of an introduction to decision theory—it just seemed to make sense. However, after the recent experiment I did (the decision problem between a guaranteed \$250,000 and a 10\% chance to get \$10,000,000) I began to start doubting the appropriateness of Expected Utility Theory. Over 85\% of subjects chose the first option, despite the latter having an expected value 4 times higher than the former. I myself realised that the only scenario in which I would choose the \$10,000,000 was one in which \$250,000 was an amount I could pass up. Now I am fully cognisant of expected utility theory, and my decision to pick the first option did not seem to be prey to any bias, so a suspicion on the efficacy of expected utility theory began to develop in my mind. I took my experiment to http://www.reddit.com/r/lesswrong; a community who I expected would be more rational decision makers—they only confirmed the decision making of the first group. I realised then, that something was wrong; my map didn’t reflect the territory. If expected utility theory was truly so sound, then a community of rationalists should have adhered to its dictates. I filed this information at the back of my mind. My brain began working on it, and today while I was reading “Thinking, Fast and Slow” by Daniel Kahneman my brain delivered an answer to me.

I do not consider myself a slave to rationality; it is naught but a tool for me to achieve my goals. A tool to help me “win”, and to do so consistently. If any ritual of cognition causes me to lose, then I abandon it. There is no sentimentality on the road to victory, and above all I endeavour to be efficient—ruthlessly so if needed. As such, I am willing to abandon any theory of decision making, when I determine it would cause me to lose. Nevertheless, as a rationalist I had to wonder; if ex

...
Rationality as A Value Decider
by DragonGod — 2017-06-05T03:21:50.322Z

Rationality As a Value Decider

A Different Concept of Instrumental Rationality

Eliezer Yudkowsky defines instrumental rationality as “systematically achieving your values” and goes on to say: “Instrumental rationality, on the other hand, is about steering reality—sending the future where you want it to go. It’s the art of choosing actions that lead to outcomes ranked higher in your preferences. I sometimes call this ‘winning.’” [1]

I agree with Yudkowsky’s concept of rationality as a method for systematised winning. It is why I decided to pursue rationality—that I may win. However, I personally disagree with the notion of “systematically achieving your values” simply because I think it is too vague. What are my values? Happiness and personal satisfaction? I find that you can maximise this by joining a religious organisation, in fact I think I was happiest in a time before I discovered the Way. But this isn’t the most relevant, maximising your values isn’t specific enough for my taste, it’s too vague for me.

“Likewise, decision theory defines what action I should take based on my beliefs. For any consistent set of beliefs and preferences I could have about Bob, there is a decision-theoretic answer to how I should then act in order to satisfy my preferences.” [2]

This implies that instrumental rationality is specific; from the above statement, I infer:
“For any decision problem to any rational agent with a specified psyche, there is only one correct choice to make.”

However, if we only seek to systematically achieve our values, I believe that instrumental rationality fails to be specific—it is possible that there’s more than one solution to a problem in which we merely seek to maximise or values. I cherish the specificity of rationality; there is a certain comfort, in knowing that there is a single correct solution to any problem, a right decision to make for any game—one merely need find it. As such, I sought a definition of rationality that I personally agree with; one that satisfies my criteria for specificity; one that satisfies my criteria for winning. The answer I arrived at was: “Rationality is systematically achieving your goals.”

I love the above definition; it is specific—g

...