trentmkelly/LessWrong-43k huggingface dataset:
Coalition Dynamics as Morality
by abramdemski — 2017-06-23T18:00:00.000Z

At risk of re-hashing some things which have already been covered a lot, I wanted to outline some of my current thinking on ethics/morality/meta-ethics. I haven't yet found a distinction between these things which feels to me like more than a distinction of the moment. What I'm talking about for the purposes of this post is game-theoretic reasoning which has a tendency to promote cooperation and get good results. I'll call this "game-theoretic morality" here.

I suspect there isn't something like an objectively correct game-theoretic morality. The pragmatically best approach depends too much on what universe you're in. Players can enforce weird equilibria in iterated Prisoner's Dilemma. If you find yourself in a strange playing field, all sorts of irrational-looking strategies may be optimal. That being said, we can try to capture working principles for the situations we find we tend to get into, and hope they don't generalize too badly.

Coalition Dynamics

I think of game-theoretic morality largely in terms of coalition dynamics. In some sense, the ideal outcome is for everyone to be on the same team, maximizing a combined utility function. Unfortunately, that's not always possible. A pure altruist, who values everyone equally, is exploitable; unqualified altruism isn't a winning strategy (or rather, isn't always a winning strategy), even from the perspective of global coordination. A more pragmatic strategy is to give consideration to others in a way which incentivises joining your coalition. This often allows you to convince selfish agents to "grow the circle of empathy", creating overall better outcomes through coordination.

This line of thinking leads to things like Nash bargaining and Shapley value. Everyone in the coalition coordinates to provide maximum value to the coalition as a whole, but not treating everyone equally. Members of the coalition are valued based on their contribution to the coalition. If you want the setup to be more egalitarian, that's a matter of your values (which your coalition partners should take into account), but it's not part of the ideal game-theoretic morality.

This point is similar to Friendship is Utilitarian. Even if you have egalitarian altruistic goals, and even if your potential allies don't, it can be overall better to form alliances in w

...
Pair Debug to Understand, not Fix
by Vaniver — 2017-06-21T23:25:40.480Z

I'm an adjunct instructor at CFAR, but this is my opinion, not CFAR's.

At CFAR, one of the exercises is 'pair debugging'; one person is the protagonist exploring one of their problems, and the other person is the helper, helping them understand and solve the problem. (Like many things at CFAR, this is a deliberate and distilled version of something that already happens frequently in normal life.)

This used to have a different frame; we used to talk about the debugger and the debuggee, the person solving the problem and the person who had it. This predictably led to problems, because the frame mismatched what it actually meant to do the task well. This post is an attempt to point at the difference between the two, and why I think it's important to lean heavily towards the 'understand' side. There seem to be two broad clusters of reasons, which I'm going to label "model-based" and "social."

image taken from here

"Root cause analysis" is the term we'd use in industry, or "five whys." The point is that when you explore an issue in the right way with sufficient depth, you come up with a better solution; not to the immediate situation in front of you, but the entire class of situations that are caused by the same root cause. One of the pieces of advice that Duncan, CFAR's curriculum director, gives is that the worst success at a pair debug is you solving their problem for them. The best is that by the end of the debug, the two of you aren't the sort of person for whom that type of problem could happen anymore.

That is, someone else having a problem isn't just a task to be done and forgotten about; it's an opportunity to learn about how their mind works, and how your mind works. I think one of the things that's helped CFAR instructors 'level up' is both getting rich models of how other people think, but also exposing lots of their own models and how they think. ("Oh, in this situation I would do X, how do I explain X to someone else?")

It's also often the case that the original frame for a problem is not one where the right solution is readily apparent. (If that were the case, generally the solution would have been implemented already!) Exploring a problem and uncovering other frames and perspectives can help discover the ontology in whic...
Epistemic Spot Check: The Demon Under the Microscope (Thomas Hager)
by Elizabeth — 2017-06-21T16:00:01.016Z

Description

How much would it suck to be the guy who invented sulfa drugs? You dedicate your life to preventing a repeat of the horrors you saw in the war, succeed in that and so much more, and then 10 years later some idiot leaves a petri dish open and completely replaces you as the father of man’s triumph against bacteria. Actually he left the lid off before you found your thing, but ignored the result until you hit it big because everyone knew you couldn’t fight disease with chemicals, until you proved you could. It’s the ultimate silver medal. The Demon Under the Microscope is the tale of that guy.

It’s by the same author (Thomas Hagen) as The Alchemy of Air. It’s also set in the same corporation, and about field that was transforming from science to industry. The writing style is similar. I originally didn’t intend to fact check this book very hard because I already knew what to expect from the author (a little too invested in the subject but basically accurate), but the habit is too ingrained at this point and I couldn’t keep reading until I’d checked out the first few chapters.

Evaluation

Claim: “Domagk [the researcher] had the ability to see. He watched everything, noted slight variations, quietly filed it all away.” (p. 18).

The wounds themselves he accepted as the results of war. But the infections that followed—surely science could do something to stop those. He focused on the bacteria, his personal demons, “these terrible enemies of man that murder him maliciously and treacherously without giving him a chance.” “I swore before God and myself,” he later wrote, “to counter this destructive madness.” (p. 20).

Who knows but it’s pretty. Someone in the same position as thousands of others (in this case a WW1 medic), caring more , and going to fix it (via sulfa drugs) is my moral aesthetic. Of course there could be another surgeon in the same place with just as much care and potential who got blow up or gassed. The Alchemy of Air prioritized poetry over provability, so I don’t entirely trust this, but I like it.

Claim: Cholera was a big problem for German soldiers.

This would be a weird thing to make up, but I’m a little confused. There had been a cholera vaccine for over 20 years by that point.

Claim: Gas gangrene is bad.

True.

Claim: Sir Almroth Wright create...
Priors Are Useless
by DragonGod — 2017-06-21T11:42:31.214Z

NOTE.

This post contains Latex. Please install Tex the World for Chromium or other similar Tex typesetting extensions to view this post properly.

Priors are Useless.

Priors are irrelevant. Given two different prior probabilities , and for some hypothesis .
Let their respective posterior probabilities be and .
After sufficient number of experiments, the posterior probability .
Or More formally:
.
Where is the number of experiments.
Therefore, priors are useless.
The above is true, because as we carry out subsequent experiments, the posterior probability gets closer and closer to the true probability of the hypothesis . The same holds true for . As such, if you have access to a sufficient number of experiments the initial prior hypothesis you assigned the experiment is irrelevant.

To demonstrate.
http://i.prntscr.com/hj56iDxlQSW2x9Jpt4Sxhg.png
This is the graph of the above table:
http://i.prntscr.com/pcXHKqDAS\_C2aInqzqblnA.png

In the example above, the true probability of Hypothesis is and as we see, after sufficient number of trials, the different s get closer to .

To generalize from my above argument:

If you have enough information, your initial beliefs are irrelevant—you will arrive at the same final beliefs.

Because I can’t resist, a corollary to Aumann’s agreement theorem.
Given sufficient information, two rationalists will always arrive at the same final beliefs irrespective of their initial beliefs.

The above can be generalized to what I call the “Universal Agreement Theorem”:

Given sufficient evidence, all rationalists will arrive at the same set of beliefs regarding a phenomenon irrespective of their initial set of beliefs regarding said phenomenon.

Exercise For the Reader

Prove .
The Battle for Psychology
by Jacob Falkovich — 2017-06-21T03:51:31.000Z

Epistemic status for the first part of this post:

Epistemic status for the second part:

Part 1 – The opposite of power is bullshit

I recently met a young lady, and we were discussing statistical power. No, I don’t see why that’s weird. Anyway, I mentioned that an underpowered experiment is useless no matter if it gets a result or not. For example, the infamous power posing experiment had a power of merely 13%, so it can’t tell us whether power posing actually does something or not.

The young lady replied: For the power pose study, if the power was .13 (and beta was .87), then isn’t it impressive that they caught the effect despite the chance being only 13%?

This question reflects a very basic confusion about what statistical power is. I wish I could explain what it is with a link, but I had trouble finding a clear explanation of that anywhere online, including in my previous post. Wikipedia isn’t very helpful, and neither are the first 5 Google results. So, let me take a step back and try to explain as best I can what power is and what it does.

Statistical power is defined as the chance of correctly identifying a true positive result (if it exists) in an experiment. It is a function of the effect size, the measurement error, and the desired rate of false positives.

The effect size is what you’re actually after, but you can estimate it ahead of time and it has to be part of the hypothesis you’re trying to prove. You don’t care about power If your study is exploratory, but then you’re not setting to confirm any hypotheses either. The bigger the effect size, the higher the statistical power.

Measurement error is a function of your measurement accuracy (which you can’t really change) and the sample (which you can). The bigger the sample, the smaller the error, the higher the power.

The rate of false positives is how often you’ll think you have found an effect when none exists. This number is up to you, and it’s traded off against power: if you decrease the chance of confusing a false effect for a true one, you increase the chance of missing out on a true effect. For reasons of historical conspiracy, this is usually set at 5%.

The same historical conspiracy also decided that “identifying a true positive” means “rejecting the null hypothesis”. This is dumb (why do you car...
Distinctions of the Moment
by abramdemski — 2017-06-20T14:38:21.293Z

I could use a better name for this (any suggestions?), but I'll use "distinctions of the moment" for the moment.

A distinction of the moment is a distinction made between two synonyms or near-synonyms (or sometimes antonyms), for the sake of pointing out a difference which is important to the discussion at hand. For example, I might contrast "knowledge" and "understanding" for the sake of a discussion about memorization-and-regurgitation style education vs focusing more on reasons. In a different conversation, I may well use "knowledge" vs "understanding" to talk about knowing-how vs knowing-what (and I might well call either of those two "understanding" and the other "knowledge" -- it doesn't matter, it's just a distinction of the moment).

The important thing about distinctions of the moment is that they just have to be made clear for the purpose of the conversation, and can be thrown away afterwards. This fails if someone thinks that they need to argue the definition of a word as if it were a sacred duty. Perhaps I make a distinction between "liberal" and "progressive" for the purpose of conversation, and someone takes issue with it because they've got a favorite definition of one or both of those terms which they think is importantly correct. This is not useful to the conversation.

A pet peeve of mine is that a lot of people seem to carry around distinctions-of-the-moment forever, as if they were true subtle differences in the meanings of words. I feel this way about distinguishing between "sympathy" and "empathy" -- I'm not sure quite why, but it seems to me that the distinction people tend to draw between the two is something which is just not generally useful enough to be more than a distinction-of-the-moment. The two words have different usage patterns, yes, but I don't think those usage patterns are really well-explained by "feeling emotion in response to others vs putting yourself in their shoes".
Angst, Ennui, and Guilt in Effective Altruism
by Gordon Seidoh Worley — 2017-06-19T20:38:02.001Z
Link: https://mapandterritory.org/angst-ennui-and-guilt-in-effective-altruism-73cf62935dfd

There is danger in learning about existential risk. I don’t mean that as some kind of bad joke, either: some people experience personal, psychological harm after learning that the world might end in preventable ways. They may become depressed, anxious, or nihilistic over not just their own death but the potential death of everything that means anything to them, and they may wallow in that despair. But, if they can wade their way through such feelings — and many people do! — they might seek to take action against existential risks. One such group of action-takers is effective altruists.

There are many existential risks, and so there are many ways effective altruists try to address them. Some tackle fairly prosaic risks and so can use prosaic methods, like applying activism to address risks from climate change and not raising money for emergency response. Others look at more exotic risks from nanotechnology and superintelligence and so may need more exotic mitigation strategies, like performing original research that would otherwise not happen or teaching people to be more rational. And it’s among those who look at exotic risks and find only need of exotic interventions that a particular sort of angst, ennui, and guilt can arise.

Now I don’t much think of myself as an effective altruist, but I quack like one, so you might as well count me among their ranks, and many of my friends identify as effective altruists, so I speak from a place of gnosis when I say effective altruists suffer in these specific ways. And it seems to happen because people follow reasoning along these lines:

We face existential risk X.
The best option to mitigate X is action Y.
I should do Y.

Sometimes this works out great and we get fabulous folks I’m glad to know working at places like CSER, FHI, FRI, MIRI, and others doing vital work on addressing existential risks. Other times it doesn’t, not for lack of will or clarity of thought, but for lack of fit to perform the action Y. But, if you’ve reasoned thusly, you’re now stuck in a spot where you know the world is at grave risk, you know something specific to do about it, and yet you don’t do it. If you feel frustrated or disappointed or to have caused harmed, then you may feel the existential angst, ennui, or guilt, respectively, from not addressi...
Open thread, June. 19 - June. 25, 2017
by Elo — 2017-06-18T23:10:29.214Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting
Instrumental Rationality 1: Starting Advice
by  — 2017-06-18T18:43:23.759Z

Starting Advice

[This is the first post in the Instrumental Rationality Sequence. It's a collection of four concepts that I think are central to instrumental rationality—caring about the obvious, looking for practical things, practicing in pieces, and realistic expectations.

Note that these essays are derivative of things I've written here before, so there may not be much new content in this post. (But I wanted to get something out as it'd been about a month since my last update.)

My main goal with this collection was to polish / crystallize past points I've made. If things here are worded poorly, unclear, or don't seem useful, I'd really appreciate feedback to try and improve.]

In Defense of the Obvious:

[As advertised.]

A lot of the things I’m going to go over in this sequence are sometimes going to sound obvious, boring, redundant, or downright tautological. This essay is here to convince you that you should try to listen to the advice anyway, even if it sounds stupidly obvious.

First off, our brains don’t always see all the connections at once. Thus, even if some given advice is apparentlyobvious, you still might be learning things.

For example, say someone tells you, “If you want to exercise more, then you should probably exercise more. Once you do that, you’ll become the type of person who exercises more, and then you’ll likely exercise more.”

The above advice might sound pretty silly, but it may still be useful. Often, our mental categories for “exercise” and “personal identity” are in different places. Sure, it’s tautologically true that someone who exercises becomes a person who exercises more. But if you’re not explicitly thinking about how your actions change who you are, then there’s likely still something new to think about.

Humans are often weirdly inconsistent with our mental buckets—things that logically seem like they “should” be lumped together often aren't. By paying attention to even tautological advice like this, you’re able to form new connections in your brain and link new mental categories together, perhaps discovering new insights that you “already knew”.

Secondly, obvious advice tends to be low-hanging fruit. If your brain is pattern-matching something as “boring advice” or

...
Welcome to Lesswrong 2.0
by habryka — 2017-06-18T17:23:02.272Z

Lesswrong 2.0 is a project by Oliver Habryka, Ben Pace, and Matthew Graves with the aim of revitalizing the Lesswrong discussion platform. Oliver and Ben are currently working on the project full-time and Matthew Graves is providing part-time support and oversight from MIRI.

Our main goals are to move Lesswrong to a modern codebase, add an effective moderation system, and integrate the cultural shifts that the rationality community has made over the last eight years. We also think that many of the distinct qualities of the Lesswrong community (e.g. propensity for long-form arguments, reasoned debate, and a culture of building on one another's conceptual progress) suggest a set of features unique to the new Lesswrong that will greatly benefit the community.

We are planning to be improving and maintaining the site for many years to come, but whether the site will be successful ultimately depends on whether the community will find it useful. As such, it is important to get your feedback and guidance on how the site should develop and how we should prioritize our resources. Over the coming months we want to experiment with many different content-types and page designs, while actively integrating your feedback, in an attempt to find a structure for Lesswrong that is best suited at facilitating rational discourse.

What follows is a rough summary of how we are currently thinking about the development of Lesswrong 2.0, and what we see as the major pillars of the Lesswrong 2.0 project. We would love to get your thoughts and critiques on these.

Table of Contents:

I. Modern Codebase

II. Effective Moderation

III. Discourse Norms

IV. New Features

V. Beta Feedback Period

I. Modern Codebase

The old lesswrong is one of the only successful forks of the reddit codebase (forked circa 2009). While reddit's code served as a stable platform while our community was in its initial stages, it has become hard to develop and extend because of its age, complexity and monolithic design.

Lesswrong 2.0, on the other hand, is based on modern web technologies designed to make rapid development much easier (to be precise React, GraphQL, Slate.js, Vulcan.js and Meteor). The old codebase was a pain to work with, and almost every developer who tried to contribute gave up after trying their hands at it. The new Lesswrong codebase on the other hand is built with tools that are well-documented ...
LessWrong 2.0 Feature Roadmap & Feature Suggestions
by habryka — 2017-06-17T22:18:41.162Z

This post will serve as a place to discuss what features the new LessWrong 2.0 should have, and I will try to keep this post updated with our feature roadmap plans.

Here is roughly the set of features we are planning to develop over the next few weeks:

UPDATED: August 27th, 2017

Basic quality of life improvements:

Improve rendering speed on posts with many comments
(A lot of improvements made, a lot more to come)
Improve usability on mobile
(After the major rework this is somewhat broken again, will fix it soon)
Add Katex support for comments and posts
Allow merging with old LessWrong 1.0 accounts
Fix old LessWrong 1.0 links DONE!
Create unique links for each comment: DONE!
Make comments collapsible
Highlight new comments since last visit: DONE!
Improve automatic spam-detection
Add RSS feed links with adjustable karma thresholds
Create better documentation for the page, with tooltips and onboarding processes
Better search, including comment search and user search: DONE!

Improved Moderation Tools:

New Karma system that weighs your votes based on your Karma
Give moderators ability to suspend comment threads for a limited amount of time
Give trusted post-authors moderation ability on their own posts (deleting comments, temporarily suspending users from posts, etc.)
Add reporting feature to comments
Give moderators and admins access to a database query interface to identify negative vote patterns

New Content Types:

Add sequences as a top-level content-type with UI for navigating sequences in order, metadata on a sequence, and keeping track of which parts you've read DONE!
Add Arbital-style predictions as a content block in posts (maybe also as a top-level content type)
Add 'Wait-But-Why?' style footnotes to the editor
Discussion page that structures discussions more than just a tree format (here is a mockup I designed while working for Arbital, that I am style excited to implement)
...and we have many more crazy ideas we would like to experiment with

I will also create a comment for each of these under the post, so you can help us prioritize all of these. Also feel free to leave your own feature suggestions and site improvements in the comments.