trentmkelly/LessWrong-43k huggingface dataset:
A simpler way to think about positive test bias
by cousin_it — 2018-01-22T09:38:03.535Z

Eliezer described "positive bias" (which I'll rename "positive test bias" for reasons explained in Unnamed's comment below) in an LW post and an HPMOR chapter. According to him, dealing with that bias requires a kind of mental gymnastics that doesn't come naturally to most people: "twisty negative thinking" or "flinching toward zero". You're supposed to devise tests that come out false if your hypothesis is true. It's a bit confusing.

I think there's a simpler way to think about it. Positive test bias is just our bias toward strong hypotheses. You can deal with it by asking yourself, how can I test if my hypothesis is too strong?

The LW post about the bias mentions the Wason 2-4-6 task. If you're told that the number sequence 2-4-6 has property X, it's tempting to guess that property X means "ascending arithmetic progression". To test if that hypothesis is too strong, you need to try some other sequences that are arithmetic progressions but not ascending, and some that are ascending but not arithmetic progressions. Easy!
The HPMOR chapter describes Hermione spilling some soda on her robe, and a moment later it mysteriously becomes clean again. Hermione comes up with a hypothesis that her robe is magically self-cleaning. To test if that hypothesis is too strong, she can try spilling something else on her robe. There's no need for any counterintuitive thinking.

That technique is useful in other areas as well. For example, I often get carried away when writing posts, and end up with a draft full of wide-reaching conclusions. But then I ask myself, doesn't that sound a bit too strong? So I look for counterexamples, and the point I'm trying to make either dissolves or becomes much more robust.

Our bias toward strong hypotheses is especially harmful in politics. We will defend a hypothesis like "group X is to blame for everything" for its explanatory power, never noticing the real problem—that it's too strong. We'd all benefit from recognizing when it happens, and weakening our hypotheses until they match reality.
Innovanity
by alkjash — 2018-01-22T05:05:02.941Z
Link: https://radimentary.wordpress.com/2018/01/21/innovanity/

This post describes the shape of my vanity and compels it to pay rent. As a step towards the integration of vanity into the Self, it is an example of The Solitaire Principle in action.

My particular brand of vanity, which I lovingly name innovanity, is characterized by an attachment to feelings of ownership and to my own creations. It is accompanied by a gut refusal to fix my life in ways that other people come up with.

I describe a short-term strategy called Derive and Reverse to leverage vanity as a productive force, rather than an unhealthy self-attachment and resistance to change. Derive by rebranding and making modifications to other people's ideas to acquire a sense of ownership of and attachment to them. Reverse by understanding how you might become equally vain about the reversal of any given trait you are attached to.

The long term goal of this project is to entirely transmute capricious innovanity into a stable meta-confidence, a generalized confidence in one's meta-processes no longer attached to particular object-level traits.

Derive and Reverse

In The Solitaire Principle, I wrote that much of self-improvement might better be understood as the alignment of distinct sub-personalities rather than the optimization of some monolothic whole. One of the key assumptions there is that a mental sub-process like Babble or vanity is more like an independent sub-personality, with its own values, beliefs, and perspectives, than merely an algorithm to be slotted into a hole at will.

The Solitaire Principle impels us to treat our sub-personalities like individual human beings of intrinsic value, and to engage in communication and empathy with them. Derive and Reverse are two exercises I came up with to build and understanding of and mastery over the vanity sub-personality.

Derive

Derive is an algorithm for resisting the inertia of innovanity. It is inspired by the following anecdote from Sunset at Noon:

"Rationalists obviously don't *actually* take ideas seriously. Like, take the Gratitude Journal. This is the one peer-reviewed intervention that *actually increases your subjective well being*, and costs barely anything. And no one I know has even seriously tried it. Do literally *none* of these people care about their own happiness?"
"Huh. Do *you* keep a gratitude journal?"
"Lol. No, obviously."
- Some Guy at the Effective Altr...
Taking it Private: Short Circuiting Demon Threads (working example)
by Raemon — 2018-01-22T00:37:09.135Z

This post is intended as a working example of how I think Demon Threads should be resolved. The gist of my suggestion is:

Step 1. Make it easy and common to take a conversation private if someone is feeling annoyed/threatened/angry/etc (if it seems like the conversation is actually important. Meanwhile, also make it easier to tap out if the conversation doesn't seem like the best use of your time)

Step 2. In private chat, two people do their best to communicate honestly, to notice when they are defensive, to productively find the truth as best they can. (I think this is much easier 1-on-1 than in public)

Step 3. Someone writes a short summary of whatever progress they were able to make (and any major outstanding disagreements that remain), focusing primarily on what they learned and rather than "who's right."

The summary should be something both parties endorse. Ideally they'd both sign off on it. If that trivial inconvenience would prevent you from actually writing the post, and you both generally trust each other, I think it's fine to make a good-faith effort to summarize and then correct each other if they missed some points.

Writing such a summary needs to get you as much kudos / feel-good as winning an argument does.

Step 4. The public conversation continues, with the benefit of whatever progress they made in private.

Ideally, this means the public conversation gets to progress, without being as emotionally fraught, and every time something comes up that does feel fraught, you recurse to steps 1-3 again.

Qiaochu had a criticism of the Demon Thread article. I had said:

Demon Threads are explosive, frustrating, many-tentacled conversations that feel important but aren’t.

He responded:

I want to object to this framing, particularly the "but aren't." It's far from clear to me that demon threads are unimportant. It may seem like nothing much happened afterwards, but that could be due to everyone in the thread successfully canceling out everyone else's damage. If that's true it means that no one side can unilaterally back down in a demon thread without the thing they're protecting potentially getting damaged, even while the actual observed outcome of demon threads is that nobody apparently benefited.

I initially responded publicly. (I think the details are important in their own right, <linked here>, ...
The Tallest Pygmy Effect
by Elizabeth — 2018-01-21T23:00:00.601Z

Status: I thought this was a common economics term, but when I google it I get either unrelated or references using it the way I expect but not defining it. It’s a really useful term, so I’m going to attempt to make it a thing.

“Tallest Pygmy Effect” is when you benefit not from absolute skill or value at a thing, but by being better at it than anyone else. For example, the US dollar is not that great a currency and the US economy is not that great an economy. However, the dollar is more stable than other currencies, so it becomes the currency of choice when you want stability. This high volume makes USD more stable and is in general good for the US economy (because e.g. US companies don’t have to take on currency risk when they borrow money).

Tallest pygmy effects are fragile, especially when they are reliant on self-fulfilling prophecies or network effects. If everyone suddenly thought the Euro was the most stable currency, the resulting switch would destabilize the dollar and hurt both its value and the US economy as a whole.
Book Review: The Secrets of Alchemy
by namespace — 2018-01-21T19:01:16.223Z

Like most decisions that seem bizarre to others, my choice to read Lawrence Principe's book took months to germinate. It started when I saw a post on Hacker News describing Jung's work on alchemical diagrams, noting that premodern people thought very differently and the 'rabbit hole goes deep'. My interest was piqued again months later when another post mentioned a woo-less history of alchemy. Having increasingly become interested in the 'postrationalist' cluster of ideas, I decided maybe a scholarly history of alchemy would help me understand some of the obscure esoteric and occult traditions they seem to draw inspiration from. Over time, I had developed a suspicion that I didn't really understand the ways in which our ancestors thought. Take for example the testimony of John Maynard Keynes, who purchased half of Newton's unpublished papers in 1936, and presumably read them quite closely:

In the eighteenth century and since, Newton came to be thought of as the first and greatest of the modern age of scientists, a rationalist, one who taught us to think on the lines of cold and untinctured reason. I do not see him in this light. I do not think that any one who has pored over the contents of that box which he packed up when he finally left Cambridge in 1696 and which, though partly dispersed, have come down to us, can see him like that. Newton was not the first of the age of reason. He was the last of the magicians, the last of the Babylonians and Sumerians, the last great mind which looked out on the visible and intellectual world with the same eyes as those who began to build our intellectual inheritance rather less than 10,000 years ago. Isaac Newton, a posthumous child born with no father on Christmas Day, 1642, was the last wonderchild to whom the Magi could do sincere and appropriate homage.

I think we all have some notion that our predecessors thought in a different way, our entire modern perspective takes this as a fundamental axiom. But do you really understand the nuances of how our thinking is different, what changed between our eyes and the magicians? Realizing my vision was clouded, I hoped that reading this book would provide clarity. First and foremost I can happily report it did.

At this point you may be thinking that my bizarre decisions are my own, but that doesn't make them a suitable topic for Le...
How the LW2.0 front page could be better at incentivizing good content
by cousin_it — 2018-01-21T16:11:17.092Z

(This should probably be in meta, but I'm not sure how to post to meta. Feel free to move if needed.)

Imagine tomorrow someone writes the best ever LW2.0 post and everyone upvotes it. How can front page visitors read that best and newest post? They must scroll down (no posts are visible above the fold) and then click (no posts are expanded by default).

Attention is a key scarce resource for incentivizing good content creation. Our current front page is an attention firehose pointed away from the best newest stuff. Where does it point instead? 1) Eliezer's sequences, which are finished, so we incentivize nothing. 2) SSC, which already gets a ton of visitors and where only Scott can post anyway, so we incentivize nothing.

Compare with how other websites do it. Reddit and HN put the best newest stuff at the top, though you have to click. Old LW used to put the best newest stuff at the top and expanded by default, so you could check the site first thing in the morning and get the awesomeness. SSC does the same.

Even worse, we have /daily which is more readable than the front page (you don't have to scroll). And getting a post into /daily doesn't require upvotes. The incentive gradient for writers is nowhere to be seen! On old LW, getting my posts promoted to Main felt special. On LW2.0, several of my posts were front paged and curated and it didn't feel special at all.

The best use of the attention firehose is incentivizing good content creation. We should make the front page work like a blog, with the best newest entries shown at the top and expanded. Move the sequences and SSC into a sidebar, like every other website does for good reason. Then we might finally enjoy some actual competition for the front page, which would be a good thing.
Myopia and Comfort Zones
by alkjash — 2018-01-20T23:11:59.249Z
Link: https://radimentary.wordpress.com/2018/01/20/myopia-and-comfort-zones/

As my first CFAR approaches, I meditate on the Kabbalistic interpretations of the name.

The first part is about the relationship between nearsightedness and comfort zones: the curious adjustments that I made to combat myopia, the corresponding shrinking of comfort zones, and the reason it's unfair that Methods of Rationality demeans the wonderful game of Quidditch.

The second part is about a high-level idea for the expansion of comfort zones: the development of a secure home base. I emphasize the paramount importance of close personal relationships as a base from which one can explore the world.

The Seeker's Power

The brightly decorated stands rise like four great siege towers around the Quidditch field, but even they seem to shudder and tremble under the weight and energy of the cheering crowds. The two teams whiz through the air in their shiny Nimbuses, hurling Quaffles, deflecting Bludgers, crowding like gnats around the point-scoring hoops. The boy who will decide this whole commotion feels so tiny on his broomstick.

What is the Seeker's power? He is neither as quick and nimble as the Chasers and Keepers, nor as dangerous and single-minded as the Beaters. The Seeker's power is not physical strength or finesse, but rather his preternatural calm and eagle eye. Even as he shoots and twists through the game, through the cheering Gryffindors and booing Slytherins, through the whizzing of Quaffles and Bludgers, all this falls away like so much white noise. The Seeker draws within himself into a deep meditative state, watching hawklike for that tiny glint of gold across a hundred meter field. The whole game seems to hold its breath around him. Everything rides on his capacity to see that tiny glint of gold.

I fear Quidditch gets a bad rap in HPMOR. As an actual game to be played, it is broken to the point of absurdity for one player to win or lose the game independently of the rest of the team.

I learned this, however, from Jordan Peterson: as a psychological metaphor Quidditch is absolutely sound. The game of Quidditch hinges on a single player's powers of vision and attention. The world needs heroes who can quiet their minds and see far into the distance, past all the noise of crypto-currencies and overblown catastrophes, for that ephemeral glint of gold. Surely the writer of Methods of Rationality, of all people, should agree?

Smartphones a

...
The Desired Response
by squidious — 2018-01-20T22:01:53.591Z

A friend once told me a story of how an interaction with her mother changed her perspective on communication. She said that she had been going through a break up at the time, and was venting to her mother, when her mother responded with "Do you want my advice, or my sympathy?"

Often times, when we have something important to talk about, we consider how we expect the other person to respond, and talk to people who will respond the kind of way we want. We may choose to speak to someone who we know can keep a secret, or who gives good advice, or who will say nothing at all, only listen. Sometimes we may turn to someone who won't want to talk about it at all, and will instead distract us from our problems. This can be great if you have a number of close friends who reliably respond in different ways, and you are able to predict this and make use of it. However, not everyone is in this situation, and so instead, we end up with interactions where people are not getting their desired responses.

Take an interaction between example-humans Alice and Bob. Alice's goal in the conversation may be to be heard, whereas Bob may want to feel validated for what he says. If Alice is talking, Bob may point out something or make a clever remark. Instead of validating Bob, Alice feels like she hasn't been heard, and is quiet in response. Now Bob has not been validated either, and they are both sad.

Sometimes, a person's desired response can be inferred. This is like how generally, if someone tells a joke, their desired response is for you to laugh, or at least acknowledge that the joke was funny. However, this isn't always easy to tell. Some of us may know the best thing for our best friend in a time of stress, but we don't always know what's right for someone else, even if we are close. Often times we will assume that what works for us will work for them, but then there is a risk of emotional damage. Trying to talk about a situation that someone wants to avoid thinking about may exacerbate the problem or cause tension between two people. Usually, it is better to ask a person what they need -- like how my friend's mother asked her.

Going more deeply, some people may tend to have common desired responses, in general interactions. Think of the person constantly telling bad jokes, excited to hear people groan and laugh, or someone who l...
Safe AIs through engineering principles
by  — 2018-01-20T17:31:58.326Z

When nuclear engineers design reactors, they go to elaborate lengths to make sure that under no circumstances (even a meltdown) is enough fissionable material able to accumulate to get a nuclear explosion. You could in fact design a nuclear reactor to be always on the edge of a criticality incident. It would make the reactor much lighter and simpler and higher performance.

When demolition workers place the charges, they are very careful to keep the wiring insulated and to make sure that the initiator is securely locked up until everyone is at a safe distance.

When you install electrical wiring, you always need to have fuses or breakers with trip points lower than what the wiring can handle, and you isolate circuits and make sure under the worst case scenario the insulation won't melt - then add a safety margin on that.

If we ever have the technical means to develop nanotechnology that can self replicate, as long as we store the blueprint data files in macroscale computers and do not give any "nanorobots" we build enough internal memory to hold their own blueprints, we can rest assured that they never can replicate out of our control.

AIs can be made the same way, with similar weak links and deliberate bottlenecks installed in their cognitive paths. First of all, you don't need executive decision making functionality to make an AI to automate most tasks. In most cases, the core of the intelligence can be a small section of code where possible actions are evaluated versus's the agents terminal values, and then the min() or max() from that action array is picked and sent to the output.

Today, that's going to just be some ARM system on a chip running Linux or QNX, and it'll just be a Python script, optimized to a faster language if the project has the budget. It's slow as a snail. The system can't evolve out of control, it's ultimately just a loop picking the lowest/highest number from a list.

There may be methods where the subsystems of this AI are designed by other AIs, but they are still designed to optimize for specific parameters picked by humans, and the outputs of those subsystems are still run through simple optimizers. Similarly, the AI doing the subsystem design is just another idiot savant. It just generates candidate neural architectures, scores them through some numerical...
Kenshō
by Valentine — 2018-01-20T00:12:01.879Z

Follow-up to:

Gears in Understanding

,

Fake Frameworks

This last September, I experienced enlightenment.

I mean to share this as a simple fact to set context. I don’t claim I am enlightened, as though I have some amazing property that makes me better than people who don’t have it. I mean simply that there’s something vaguely like a state that our culture calls “enlightenment” that I’ve been in and have returned to a few times over the last four months. In

Rinzai Zen

one would say that I had a

kenshō

: a moment of understanding that makes the path clear but is not yet full attainment.

Over the last several months I’ve tried to share what I now see so clearly. And this has mostly just failed. People who’ve had a kenshō follow what I’m saying just fine, but most people just get really confused. It feels a bit like being one of the only people around who understand scientific thinking: most people can see that

the behavior of a gyroscope

is

weird

when you show them, but most can’t really see its behavior through the lens of scientific epistemology. They just keep translating what you’re saying into e.g.

isolated facts

.

This is particularly vexing in the case of kenshō because enlightenment isn’t an insight. I claim it’s not a matter of

inferential distance

. It’s more like bothering to notice what you already know. When the moment of seeing struck me, I fell over laughing and basically didn’t stop laughing for two days, because it was so incredibly stunningly obvious. There isn’t something to learn: it’s already always here.

And what is “it”, you might ask? Well, I would honestly love to be able to tell you. But apparently my saying it doesn’t convey it hardly at all, unless you’ve already seen it for yourself.

(And yes, there’s most definitely an “it”. This isn’t just brains getting flooded with feeling-of-profundity without an object. And it totally makes sense that some people think that. Just… from this vantage point, those objections come across a bit like people arguing that science is just another religion. Or more to the point, it’s like trying to convince me that I have no subjective experience: no matter how cunning and logical and well-researched the argument, I’m still here listening to you.)

With all that said, I think I can share something one meta-level up. I think the reason it’s hard to convey enlightenment in words can itself be conveyed with words. And ...