trentmkelly/LessWrong-43k huggingface dataset:
Bi-Weekly Rational Feed
by sapphire — 2017-07-09T19:11:21.767Z

===Highly Recommended Articles:

Just Saying What You Mean Is Impossible by Zvi Moshowitz - "Humans are automatically doing lightning fast implicit probabilistic analysis on social information in the background of every moment of their lives." This implies there is no way to divorce the content of your communication from its myriad probabilistic social implications. Different phrasings will just send different implications.

In Defense Of Individualist Culture by Sarah Constantin (Otium) - A description of individualist culture. Criticisms of individualist culture: Lacking sympathy, few good defaults. Defenses: Its very hard to change people (psychology research review). A defense of naive personal identity. Traditional culture is fragile. Building a community project is hard in the modern world, prepare for the failure modes. Modernity has big upsides, some people will make better choices than the traditional rules allow.

My Current Thoughts On Miris Highly Reliable by Daniel Dewey (EA forum) - Report by the Open Phil AI safety lead. A basic description of and case for the MIRI program. Conclusion: 10% credence in MIRI's work being highly useful. Reasons: Hard to apply to early agents, few researchers are excited, other approaches seem more promising.

Conversation With Dario Amodei by Jeff Kaufman - "The research that's most valuable from an AI safety perspective also has substantial value from the perspective of solving problems today". Prioritize work on goals. Transparency and adversarial examples are also important.

Cfar Week 1 by mindlevelup - What is working at CFAF actually like. Less rationality research than anticipated. Communication costs scale quadratically. Organization efficiency and group rationality.

The Ladder Of Interventions by mindlevelup - "This is a hierarchy of techniques to use for in-the-moment situations where you need to “convince” yourself to do something." The author uses these methods in practice.

On Dragon Army by Zvi Moshowitz - Long response to many quotes from "Dragon Army Barracks". Duncan't attitude to criticism. Tyler Durden shouldn't appeal to Duncan. Authoritarian group houses haven't been tried. Rationalists undervalue exploration. Loneliness and doing big things. The pendulum model of social progress. Sticking to commitments even when its painful. Saving face when you scre...
Mini map of s-risks
by turchin — 2017-07-08T12:33:53.327Z

S-risks are risks of future global infinite sufferings. Foundational research institute suggested them as the most serious class of existential risks, even more serious than painless human extinction. So it is time to explore types of s-risks and what to do about them.

Possible causes and types of s-risks:

"Normal Level" - some forms of extreme global suffering exist now, but we ignore them:

1. Aging, loss of loved ones, moral illness, infinite sufferings, dying, death and non-existence - for almost everyone, because humans are mortal

2. Nature as a place of suffering, where animals constantly eat each other. Evolution as superintelligence, which created suffering and using it for its own advance.

Colossal level:

1. Quantum immortality creates bad immortality - I survived as old, but always dying person, because of weird observation selection.

2. AI goes wrong. 2.1 Rocobasilisk 2.2. Error in programming 2.3. Hacker's joke 2.4 Indexical blackmail.

3. Two AIs go in war with each other, and one of them is benevolent to human, so another AI tortures humans to get bargain position in the future deal.

4. X-risks, which includes infinite suffering for everyone - natural pandemic, cancer epidemic etc

5. Possible worlds (in Lewis terms) with infinite sufferings qualia in them. For any human a possible world with his infinite sufferings exist. Modal realism makes them real.

Ways to fight s-risks:

1. Ignore them by boxing personal identity inside today

2. Benevolent AI fights "measure war" to create infinitely more copies of happy beings, as well as trajectories in the space of the possible minds from sufferings to happiness

Types of most intensive sufferings:

Qualia based, listed from bad to worse:

1. Eternal, but bearable in each moment suffering (Anhedonia)

2. Unbearable sufferings - sufferings, to which death is the preferable outcome (cancer, death in fire, death by hanging). However, as said Mark Aurelius: “Unbearable pain kills. If it not kills, it is bearable"

3. Infinite suffering - qualia of the infinite pain, so the duration doesn’t matter (not known if it exists)

4. Infinitely growing eternal sufferings, created by constant upgrade of the suffering’s subject (hypothetical type of sufferings created by malevolent superintelligence)

Value based s-risks:

1. Most violent action against one’s main values: like "b

...
New Relationship Energy as an Emotional Bias
by squidious — 2017-07-08T07:00:00.000Z

Some call it The Honeymoon Phase or puppy love. Others mix it in with infatuation or limerence. Views on new relationship energy (NRE) vary, from seeing it as immature to completely normal to a little unhealthy. Regardless, NRE distorts your perception of the person you've just started dating. This can be thrilling, though at times a little scary. It feels like living in an illusion that you know is going to be shattered sooner or later.
The Baader-Meinhof phenomenon (also called the Frequency Illusion) is what happens when you learn about something new, then start to see it everywhere. There is a lot of similarity between what happens here and in NRE, to the extent where NRE can effect your emotions in the same way that Baader-Meinhof effects your mind. It is the feeling of every love song reminding you of your new beau, or every book, movie, and band you like seeming like something to recommend to them. This is also partly because NRE is often when you learn the most about a person. You find out that they like sushi, and suddenly every sushi restaurant seems like a good date location. Often, you associate the person with happiness, and feel compelled to share other things you associate with happiness with them.
The new feelings that come up during NRE function much like the new idea in Baader-Meinhof. When you notice things your new partner likes or would like everywhere, you don't often think of it as "wow, the things this person likes are everywhere", but instead it feels like you can't stop thinking about them. And of course, if you can't stop thinking about a person, this is evidence that you must like them a lot.
The things you learn about a person's traits play into this effect as well. If your paramour is particularly smart, a lot of the things they say or write might draw attention to this. Especially if you were friends with the person for a while before the relationship started, you may feel like you didn't realize how funny, intelligent, or attractive the person was. The emotions are so new that it feels like the person is much more amazing than you ever gave them credit for, and you are only now starting to notice. Part of this is also due to fundamental attribution error, which is what's happening when we assume a person is helping their friend move because they are kind, without considering that their friend may have bullied them into it, or the

...
Against lone wolf self-improvement
by cousin_it — 2017-07-07T15:31:46.908Z

LW has a problem. Openly or covertly, many posts here promote the idea that a rational person ought to be able to self-improve on their own. Some of it comes from Eliezer's refusal to attend college (and Luke dropping out of his bachelors, etc). Some of it comes from our concept of rationality, that all agents can be approximated as perfect utility maximizers with a bunch of nonessential bugs. Some of it is due to our psychological makeup and introversion. Some of it comes from trying to tackle hard problems that aren't well understood anywhere else. And some of it is just the plain old meme of heroism and forging your own way.

I'm not saying all these things are 100% harmful. But the end result is a mindset of lone wolf self-improvement, which I believe has harmed LWers more than any other part of our belief system.

Any time you force yourself to do X alone in your room, or blame yourself for not doing X, or feel isolated while doing X, or surf the web to feel some human contact instead of doing X, or wonder if X might improve your life but can't bring yourself to start... your problem comes from believing that lone wolf self-improvement is fundamentally the right approach. That belief is comforting in many ways, but noticing it is enough to break the spell. The fault wasn't with the operator all along. Lone wolf self-improvement doesn't work.

Doesn't work compared to what? Joining a class. With a fixed schedule, a group of students, a teacher, and an exam at the end. Compared to any "anti-akrasia technique" ever proposed on LW or adjacent self-help blogs, joining a class works ridiculously well. You don't need constant willpower: just show up on time and you'll be carried along. You don't get lonely: other students are there and you can't help but interact. You don't wonder if you're doing it right: just ask the teacher.

Can't find a class? Find a club, a meetup, a group of people sharing your interest, any environment where social momentum will work in your favor. Even an online community for X that will reward your progress with upvotes is much better than going X completely alone. But any regular meeting you can attend in person, which doesn't depend on your enthusiasm to keep going, is exponentially more powerful.

Avoiding lone wolf self-improvement seems like embarrassingly obvious advice. But somehow I see people trying to learn X alone

...
Call to action
by Elo — 2017-07-07T09:10:01.574Z

Core knowledge: List of common human goals
Part 1: Exploration-Exploitation
Part 1a: The application of the secretary problem to real life dating
Part 1b: adding and removing complexity from models
Part 2: Bargaining Trade-offs to your brain.
Part 2a.1: A strategy against the call of the void.
Part 2a.2: The call of the void
Part 2b.1: Empirical time management
Part 2b.2: Memory and notepads
Part 3: The time that you have
Part 3a: A purpose finding exercise
Part 3b: Schelling points, trajectories and iteration cycles
Part 4: What does that look like in practice?
Part 4a: Lost purposes – Doing what’s easy or what’s important
Part 4b.1: In support of yak shaving
Part 4b.2: Yak shaving 2
Part 4c: Filter on the way in, Filter on the way out…
Part 4d.1: Scientific method
Part 4d.2: Quantified self
Part 5: Skin in the game
Part 6: Call to action

A note about the contents list; you can find the list in the main parts, the a,b,c parts are linked to from the main posts. If you understand them in the context they are mentioned you can probably skip them, but if you need the explanation, click through.

If you understand exploration and exploitation, you realise that sometimes you need to stop exploring and take advantage of what you know based on the value of the information that you have. At other times you will find your exploitations are giving you diminishing returns, you are stagnating and you need to dive into the currents again, take some risks. If you are accurately calibrated, you will know what to do, whether to sharpen the saw, educate yourself more or cut down the tree right now.

If you are not calibrated yet and you want to start, you might want to empirically assess your time. You might like to ask yourself in light of the information of your time use all on one page – Am I exploring and exploiting enough? Remembering you probably make the most measurable and ongoing returns in the Exploitation phase, however the exploration might be seem more fun (to find exciting and new knowledge), and the place where you grow, but are you sure that’s what you want to be doing in regard to the value return by exploiting?

Why were you not already exploring and exploiting in the rig...
Bayesian probability theory as extended logic -- a new result
by ksvanhorn — 2017-07-06T19:14:32.163Z

I have a new paper that strengthens the case for strong Bayesianism, a.k.a. One Magisterium Bayes. The paper is entitled "From propositional logic to plausible reasoning: a uniqueness theorem." (The preceding link will be good for a few weeks, after which only the preprint version will be available for free. I couldn't come up with the $2500 that Elsevier makes you pay to make your paper open-access.)

Some background: E. T. Jaynes took the position that (Bayesian) probability theory is an extension of propositional logic to handle degrees of certainty -- and appealed to Cox's Theorem to argue that probability theory is the only viable such extension, "the unique consistent rules for conducting inference (i.e. plausible reasoning) of any kind." This position is sometimes called strong Bayesianism. In a nutshell, frequentist statistics is fine for reasoning about frequencies of repeated events, but that's a very narrow class of questions; most of the time when researchers appeal to statistics, they want to know what they can conclude with what degree of certainty, and that is an epistemic question for which Bayesian statistics is the right tool, according to Cox's Theorem.

You can find a "guided tour" of Cox's Theorem here (see "Constructing a logic of plausible inference"). Here's a very brief summary. We write A | X for "the reasonable credibility" (plausibility) of proposition A when X is known to be true. Here X represents whatever information we have available. We are not at this point assuming that A | X is any sort of probability. A system of plausible reasoning is a set of rules for evaluating A | X. Cox proposed a handful of intuitively-appealing, qualitative requirements for any system of plausible reasoning, and showed that these requirements imply that any such system is just probability theory in disguise. That is, there necessarily exists an order-preserving isomorphism between plausibilities and probabilities such that A | X, after mapping from plausibilities to probabilities, respects the laws of probability.

Here is one (simplified and not 100% accurate) version of the assumptions required to obtain Cox's result:

A | X is a real number.

(A | X) = (B | X) whenever A and B are logically equivalent; furthermore, (A | X) ≤ (B | X) if B is a tautology (an express

...
Steelmanning the Chinese Room Argument
by cousin_it — 2017-07-06T09:37:06.760Z

(This post grew out of an old conversation with Wei Dai.)

Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.

Can you, by observing the input-output behavior of the system, distinguish it from a person who doesn't know the secret, or knows some other secret instead?

Clearly the only reasonable answer is "no, not in general".

Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?

Again, clearly, the only reasonable answer is "not in general".

Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?

A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
The Unreasonable Effectiveness of Certain Questions
by ig0r — 2017-07-04T03:37:38.560Z

Cross-posted on my blog: http://garybasin.com/the-unreasonable-effectiveness-of-certain-questions/

About a year ago I was sitting around trying to grok the concept of Evil — where does it come from and how does it work? After a few hours of spinning in circles, I experienced a sudden shift. My mind conjured up the question: “Is this a thing out in the world or just a projection?” (Map vs Territory). Immediately, a part of my mind replied with “Well, this may not be anything other than a story we tell about the behavior of people we dislike”. Let’s ignore the truth value for today and notice the process. I’m interested in this mechanism of how a simple query — checking if I’m looking at a confusion of map with the territory — was able to instantly reframe a problem in a way that allowed me to effortlessly make a mental leap. What’s fascinating is that you don’t even need someone else’s brain to come up with these questions (although that often helps) — you can try to explain your problem to a rubber duck which creates a conversation with yourself and generates queries, or just go through a list of things to ask yourself when stuck.

There are a few different categories of these types of queries and many examples of each. For instance, when thinking about plans we can ask ourselves to perform prehindsight/inner simulator or reference class forecasting/outside view. When introspecting on our own behavior, we can perform sentence completion to check for limiting beliefs, ask questions like “Why aren’t I done yet?” or “What can I do to 10x my results?”. When thinking about problems or situations, we can ask ourselves to invert, reframe into something falsifiable, and taboo your words or perform paradjitsu. Or consider the miracle question: Imagine you wake up and the problem is entirely solved — what do you see, as concretely as possible, such that you know this is true?

So “we know more than we can tell” — somewhere in our head often lies the answer, if only we could get to it. In some sense, parts of our brain are not speaking to each other (do they even share the same ontologies?) except through our language processor, and only then if

...
Lesswrong Sydney Rationality Dojo on zen koans
by Elo — 2017-07-04T00:10:06.039Z

Link: http://bearlamp.com.au/zen-koans/

Short post here.

Lesswrong Sydney runs a rationality dojo once a month. We usually cover 1-2 topics for an hour or less each. Our regular attendance is 10-20 people.

This month's topics were:

Captain awkward advice

Goal factoring (CFAR)

Understanding zen koans

I only recorded the section on zen koans. Feedback welcome.
Subtle Forms of Confirmation Bias
by abramdemski — 2017-07-03T23:00:00.000Z

There are at least two types of confirmation bias.

The first is selective attention: a tendency to pay attention to, or recall, that which confirms the hypothesis you are thinking about rather than that which speaks against it.

The second is selective experimentation: a tendency to do experiments which will confirm, rather than falsify, the hypothesis.

The standard advice for both cases seems to be "explicitly look for things which would falsify the hypothesis". I think this advice is helpful, but it is subtly wrong, especially for the selective-experimentation type of confirmation bias. Selective attention is relatively straightforward, but selective experimentation is much more complex than it initially sounds.

Looking for Falsification

What the standard (Popperian) advice tells you to do is try as hard as you can to falsify your hypothesis. You should think up experiments where your beloved hypothesis really could fail.

What this advice definitely does do is guard against the mistake of making experiments which could not falsify your hypothesis. Such a test is either violating conservation of expected evidence (by claiming to provide evidence one way without having any possibility of providing evidence the other way), or providing only very weak evidence for your claim (by looking much the same whether your claim is true or false). Looking for tests which can falsify your result steers you towards tests which would provide strong evidence, and helps you avoid violating the law of expected evidence.

However, there are more subtle ways in which confirmation bias can act.

Predicting Results in Advance

You can propose a test which would indeed fit your hypothesis if it came out one way, and which would disconfirm your hypothesis if it came out the other way -- but where you can predict the outcome in advance. It's easy to not realize you are doing this. You'll appear to provide significant evidence for your hypothesis, but actually you've cherry-picked your evidence before even looking at it; you knew enough about the world to know where to look to see what you wanted to see.

Suppose Dr. Y studies a rare disease, Swernish syndrome. Many scientists have formed an intuition that Swernish syndrome has something to do with a chemical G-complex. Dr. Y is thinking on this one night, when the intuition crystallizes into G-complex theory, which would ...
Developmental Psychology in The Age of Ems
by Gordon Seidoh Worley — 2017-07-03T18:53:44.316Z
Link: https://mapandterritory.org/developmental-psychology-in-the-age-of-ems-6f159f701354

Back in December Robin Hanson asked people if they would trade engagement with him on neglected topics. I couldn’t trade with him because I had to make my ideas clearer first, and I still have a ways to go before I can reasonably ask many others to engage much with my ideas. But he’s written an entire book on the implications of brain emulations, or ems, so it seems reasonable that I may try to engage his neglected topic of ems through my neglected topic of phenomenological complexity, and in doing so may make clearer my own ideas while more widely spreading his. If I’m lucky, I may even get some engagement back. Let’s see what happens.

Robin focuses on a possible future where brain emulation — the ability to run a human brain on artificially constructed hardware like computers — develops before other similarly disruptive technologies like superintelligence. Based on the application of the strongest results in physics, engineering, economics, psychology, and sociology, he works out the implications of such an em scenario. You can read his book, The Age of Em, for all the details, but he gives a short overview in this TED talk.

To summarize, we can imagine em cities coexisting with human cities, only em cities are much denser than human cities in order to balance communication, cooling, and energy needs. Rather than experiencing life directly in these cities, ems will live in a virtual reality running orders of magnitude faster than human subjective time so that years in em subjective time might pass in weeks or days. Although there may be millions or billions of unique ems, most ems will be copies of the few hundred most successful ems, and further still most of those copies will be spurs that are deleted after completing a specific task. Ems will live near subsistence levels and need to constantly work to support the resources they run on or retire to slower speeds.

Importantly, since ems are emulations of human brains with little modification, ems can be understood psychologically as humans living in a very different environment, comparable in difference from our modern world to the difference between humans living in forager and farmer environments. Thus much of the analysis of em behavior comes down to taking humans who evolved for forager life, spent several millennia adapting to farmer ...
Open thread, Jul. 03 - Jul. 09, 2017
by MrMind — 2017-07-03T07:20:07.113Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting