trentmkelly/LessWrong-43k huggingface dataset:
Low Enough To See Your Shadow
by alkjash — 2018-01-19T23:50:01.991Z
Link: https://radimentary.wordpress.com/2018/01/19/low-enough-to-see-your-shadow/

I wrote last time that Jung's approach to humility deserves its own post:

Modern men cannot find God because they will not look low enough.

Here's the first piece of that post.

Look low enough by confronting the darkness in your personal hell.

Humility demands that you fix the problems in your own mind and life before trying to save the world. It is necessary to inspect the lens before you examine the territory. It is necessary to develop competence and purity of heart in humbler realms before seeking world-changing power. The world will fight you with everything it has - a single crack in your own defenses is mission-critical.

All of these are good reasons to conquer your own shadow. Would you trust someone without a shadow?

Here are two more essential reasons to take care of yourself and conquer your shadow before, or at least concurrently with, embarking on a heroic mission.

The first is that you will learn the power of attention by staring straight into the abyss. There is a handicap attached to this: every danger you leave unattended gets worse. Like gasses, shadows grow to fill the space they're in.

The second is another rule of Jung, that the Golden Rule is symmetric. Treat yourself with the respect a human being deserves.

Shadow is Gaseous

This post from Jessica Taylor brought this exquisite detail from Christopher Alexander to my attention:

A public space without a middle is quite likely to stay empty.
…
if there is a reasonable area in the middle, intended for public use, it will be wasted unless there are trees, monuments, seats, fountains—a place where people can protect their backs, as easily as they can around the edge.

The implication is that public spaces must be designed around the fundamental psychological drive to protect your back. Anything behind your back, you cannot watch. Its threat level shoots through the roof. For all you know, every tree, boulder, and passerby behind your back is tiptoeing towards you, knife in hand, waiting for you to let down your guard.

[Necessary trigger warning] SCP-173 is the first horror in the game SCP: Containment Breach. SCP stands for Secure, Contain, Protect.

Special Containment Procedures: Item SCP-173 is to be kept in a locked container at all times. When personnel must enter SCP-173's container, no fewer than 3 may enter at any time and the door is to be relocked behind them. At all times, two p

...
Fashionable or Fundamental Thought in Communities
by lionhearted (Sebastian Marshall) — 2018-01-19T09:03:48.109Z

There's a couple interesting things about the 2011 LessWrong post, "Approving reinforces low-effort behaviors."

The post itself is fascinating to re-read — there's some valuable and profitable ideas in there.

But two sentences also stood out to me on more of a meta-level, and prompted some interesting thought —

>"Unlike the first class of behaviors, we expect to experience akrasia when dealing with this latter sort."

>"This process is especially important to transhumanists."

Reading that post made me remember what LessWrong was like in 2011 — discussions on akrasia ("acting against one's better judgment") and transhumanism were much more common.

Both of these concepts have, to some extent, cycled out of the common discussions on rationality. They still happen, sure, but much less frequently.

On the other hand, certain lines of thought and concepts seem to have become part of the fundamentals of LessWrong and the broader rationalist community. When I mean by "fundamentals" here is more like in sports than in religion — dribbling is a fundamental skill in basketball, and gets practiced by all basketball players.

It's interesting to reflect on why this might happen with a given set of concepts in a community. Akrasia certainly seems like it could become a fundamental area of constant study for people in this space, and yet it did not to the extent that, say, cognitive biases did.

Cognitive biases became a fundamental piece of the rationalist community and has been stably so, whereas it seems like akrasia was more of an intellectual fashion for a time.

I started thinking on why. Here's a few reasons I came up with —

"Solved Problems" are more likely to become fundamentals than areas with disputed or inconsistent solutions. This seems obvious. The Conjunction Fallacy is a cognitive bias that's very common. With some (very basic) statistics and probability theory, it's very clearly proven why one's natural intution in that area is mistaken. After learning it, you make those mistakes less often going forwards. Compare to akrasia which has been around thousands of years — we're using the Ancient Greek word for it, after all — and which hasn't been anywhere close to fully solved.
Ideas that are at the "bottom of tech trees" are more likely to become fun

...
Singularity Mindset
by alkjash — 2018-01-19T00:32:00.839Z
Link: https://radimentary.wordpress.com/2018/01/18/singularity-mindset/

In a fixed mindset, people believe their basic qualities, like their intelligence or talent, are simply fixed traits. In a growth mindset, people believe that their basic qualities can be modified gradually via dedication and incremental progress. Scott Alexander has cast some doubt on the benefits of growth mindset, but I think it still has merit, if only because it is closer to the truth.

Growth mindset is a good thing that doesn't do enough. The situation calls for More Dakka. I present: Singularity Mindset.

In a Singularity Mindset, people believe they are self-modifying intelligences well past the singularity threshold, that their basic qualities can be multiplied by large constants by installing the right algorithms and bug fixes, and that even the best optimized people are nowhere near hardware limitations. People who apply Singularity Mindset come to life with startling rapidity.

The seed of this post was planted in my head by Mind vs. Machine, which I read as a call-to-arms to be radically better humans, or at least better conversationalists. The seed sprouted when I noticed the other day that I've already blogged more words in 2018 (today is January 18) than in 2017.

Apparently, Kurzweil beat me to the name with Singularity University and Exponential Mindset, but (a) that's geared towards businesses and technology instead of individuals, and (b) I'm agnostic about the exact shape of the foom, so I'll stick to Singularity Mindset.

AI Worries

A tiny note of confusion noticed but unresolved can, over the course of years, fester into an all-consuming possession. One such question possessing me has come to a head:

Why is Eliezer Yudkowsky so much more worried about AI risk than I am?

I came up with a standard response rapidly, which goes something like: after years of steadfastly correcting his own biases, acquiring information, and thinking clearly about the world's biggest problems, he came to the right conclusion.

It's a good explanation which provokes in me at least a pretense of updating models, but today I will entertain another narrative.

I think the difference between us is that Eliezer has the lived experience of Singularity Mindset, of deliberately self-modifying to the point of becoming unrecognizably intelligent and productive, and the simultaneous lived experience of seeing his own values drift and require extraord...
A model I use when making plans to reduce AI x-risk
by Ben Pace — 2018-01-19T00:21:45.460Z

I've been thinking about what implicit model of the world I use to make plans that reduce x-risk from AI. I list four main gears below (with quotes to illustrate), and then discuss concrete heuristics I take from it.

A model of AI x-risk in four parts

1. Alignment is hard.

Quoting "Security Mindset and the Logistic Success Curve" (

link

)

Coral: YES. Given that this is a novel project entering new territory, expect it to take at least two years more time, or 50% more development time—whichever is less—compared to a security-incautious project that otherwise has identical tools, insights, people, and resources. And that is a very, very optimistic lower bound.

Amber: This story seems to be heading in a worrying direction.

Coral: Well, I'm sorry, but creating robust systems takes longer than creating non-robust systems even in cases where it would be really, extraordinarily bad if creating robust systems took longer than creating non-robust systems.

2. Getting alignment right accounts for most of the variance in whether an AGI system will be positive for humanity.

Quoting "The Hidden Complexity of Wishes" (

link

)

There are three kinds of genies: Genies to whom you can safely say "I wish for you to do what I should wish for"; genies for which no wish is safe; and

genies that aren't very powerful or intelligent

.

[...]

There is no safe wish smaller than an entire human morality. There are too many possible paths through Time. You can't visualize all the roads that lead to the destination you give the genie... any more than you can program a chess-playing machine by hardcoding a move for every possible board position.

And real life is far more complicated than chess. You cannot predict, in advance, which of your values will be needed to judge the path through time that the genie takes. Especially if you wish for something longer-term or wider-range than rescuing your mother from a burning building.

3. Our current epistemic state regarding AGI timelines will continue until we're close (<2 years from) to having AGI.

Quoting "There is No Fire Alarm for AGI" (

link

)

It's not that whenever somebody says "fifty years" the thing always happens in two years. It's that this confident prediction of things being far away corresponds to an epistemic state about the technology that feels the same way i

...
Dynamic Karma & Static Karma
by SquirrelInHell — 2018-01-18T20:23:44.647Z

The current karma system tracks one number for each user. This number is used both as an indicator of "is this user's writing worth paying attention to" and as a voting weight (using the new powers-of-five system). I think this is hugely broken and splitting these two apart could make the karma system more effective and also simpler in some sense.

One important way in which the current system is broken is that it forces a painful tradeoff between wanting to easily delegate moderation power (which should ideally have a more local scope, and be used up rather than taken away), but also wanting to conservatively signal the importance of contributions (which should include both positive and negative votes over long time horizons).

I expect the idea to not be taken seriously enough, but meh sometimes I just feel like trying anyway. Here go some (tentative) rules of how such as system could work:

"voting" is a transaction that takes some positive amount X of dynamic karma from one user, and to another user gives:

upvote: X amount of static karma, as well as αX (for some α∈(0,1)) of dynamic karma
downvote: -X amount of static karma, no change in dynamic karma

every user's dynamic and static karma is the lifetime sum of what they received by being voted on, starting from zero

however, moderators receive a constant stream of dynamic karma for free (e.g. 50 per day)

static karma is publicly visible, and dynamic karma is private
how much dynamic karma to use for a single vote is left to decide by the voter

the default (one click) could be 1% of currently owned dynamic karma (or 1 if an user has less than 100 karma)
it should probably be capped at some fixed value to reduce abuse (e.g. max 10 points per post and 5 per comment, coming from the same account)

if a moderator is not using their dynamic karma, then above some value (e.g. 500) their "free" dynamic karma should instead be distributed to the community as "dividends" paid proportionally to (positive) stakes of static karma - this is to prevent stagnation from having not enough karma in circulation

A quick example for α=0.5:

M is moderator, A and B are two new users on the site
M upvotes A's post (+5):

now A has +5 static (publicly visible karma) and +2.5 dynamic karma in their private account

A likes B's comment, and upvotes it twice:

now A has +5 static and +0.5 dynamic
B

... ,,
Book Review: Life 3.0: Being Human in the Age of Artificial Intelligence
by J Thomas Moros — 2018-01-18T17:18:20.151Z
Link: http://ferocioustruth.com/2018/review-life-3.0/

My review of Max Tegmark's latest book Life 3.0: Being Human in the Age of Artificial Intelligence about issues of AI and superintelligence. Discusses the strengths and weaknesses section by section while giving a summary of the topics covered.
Beware of black boxes in AI alignment research
by cousin_it — 2018-01-18T15:07:08.461Z

Over the course of the AI Alignment Prize I've sent out lots of feedback emails. Some of the threads were really exciting and taught me a lot. But mostly it was me saying pretty much the same thing over and over with small variations. I've gotten pretty good at saying that thing, so it makes sense to post it here.

Working on AI alignment is like building a bridge across a river. Our building blocks are ideas like mathematics, physics or computer science. We understand how they work and how they can be used to build things.

Meanwhile on the far side of the river, we can glimpse other building blocks that we imagine we understand. Desire, empathy, comprehension, respect... Unfortunately we don't know how these work inside, so from the distance they look like black boxes. They would be very useful for building the bridge, but to reach them we must build the bridge first, starting from our side of the river.

What about machine learning? Perhaps the math of neural networks could free us from the need to understand the building blocks we use? If we could create a behavioral imitation of that black box over there (like "human values"), then building the bridge would be easy! Unfortunately, ideas like adversarial examples, treacherous turns or nonsentient uploads show that we shouldn't bet our future on something that imitates a particular black box, even if the imitation passes many tests. We need to understand how the black box works inside, to make sure our version's behavior is not only similar but based on the right reasons.

(Eliezer made the same point more eloquently a decade ago in Artificial Mysterious Intelligence. Still, with the second round of our prize now open, I feel it's worth saying again.)
Russian Cynicism
by alkjash — 2018-01-18T02:06:45.083Z
Link: https://radimentary.wordpress.com/2018/01/17/russian-cynicism/

I.

"My dear Kostya, you've finally returned to me!"

Kitty comes trotting down the steps of the train station, rushing into Levin's waiting arms.

"It's been but four days, my dear."

Levin's wraps his arms around her slim waist with a passion that belies his nonchalance. Gently resting in each others' embrace, Levin's half-open eyes lazily trace the lines of the orange lace ribbon snaking through Kitty's braids.

"But four days! A whole lifetime without my Kostya."

Tears glint in the corner of her eyes.

"Let me look at you, let me see you."

Kitty cranes her head to see him, with a slight furrow in her brow.

"Did they mistreat you in Moscow?"

"Not at all, love. Everyone was wonderful in Moscow."

Kitty prattles on, deaf to his reassurances. She stands up on tip-toe to caress the new felt hat he's wearing.

"The hat's new! The color suits you - you look dashing in it. But there's something else, something different! Kostya, what have they done to you? Is it that awful man Koznyshev?"

"Nothing's changed, my love, I'm that same Kostya you fell in love with."

The furrow over Kitty's eyes grows deeper, and she pushes him away with trembling hands.

"Oh no no no no no. Not you too, Kostya! It's that damned Russian cynicism, isn't it? Don't let those awful men take my Kostya away."

At that, the lines around Levin's smile stiffen and droop, dragging his face into an unseemly frown. Kitty presses her pale hands against his cheeks plaintively.

"Look at me, Kostya! It was an error, a mistake, an awful awful awful mistake to go to Moscow! Forget it all. Return to me my dear Kostya, my pure farmer boy. Let's go back to the farm at once. At once! Petra is dying to see you."

Kitty yanks Levin's arm to leave, as if to drag him as far away from the bustling apathy of the Matushka station crowds as possible, clinging to him with the ferocity of a mother grizzly trying to save her drowning cub, as if the fresh country air and the grinding of barrow wheels through muddy footpaths and the chubby little cheeks of their freshly fed six year old son might restore to her the youthful heart of this man she once knew. Levin stands as still and unmoving as a cast-iron furnace.

"What I saw in Moscow, I can never ...
Complainy and Explainy voice
by Elo — 2018-01-18T01:47:09.374Z

Original post: http://bearlamp.com.au/complainy-and-explainy-voice/

To my dear friend,

I value you more than you know. I want to share this because it’s so very important to me that you grow. I want you to become a better person. I don’t see the bad. I see the infinite awesome future. What you become. I’m not good at communicating. I can only hope that you hear this as a kindness and not just a criticism. Not an attack but an opportunity to grow. I can only hope that your filter receives this kindly.

There really isn’t much more I can say. I honestly wish you would take this as well as I intend it. Not as an insult. Saying this at all, especially before we established safety of conversation (1 in the link) is something very vulnerable (16 in the link). It’s really scary to look at someone I consider a potential new friend then Dare Greatly. At the risk of failing to communicate in a way that might push you away… I was inspired by Mako when they said:

I realised if I ever wanted the world around me to be halfway acceptable to me, I would have to learn to change people.
People say “you can’t change people”, but they’re just telling themselves that to absolve themselves of this very heavy responsibility we all have to take an active role in each others’ growth.

I would regret not trying to take this vulnerable leap. To give myself an active role in your growth, and the growth of all the people around me.

I want what I have said so far to not come across as “about me”. I want this to be about you and how I want to help you.

You have two particular voices. The Explainy Voice and the Complainy voice. They present at the same time or in rapid succession. I am not sure that you have noticed that there are two, because they can be used so close together and interchangeably.

Explainy Voice

Internally your behaviour would feel like, “I just found something you don’t know. Let me teach you all about it”. The trailing verbiage will be some kind of educational material that you deemed valuable and relevant for sharing. I love this. It’s an honest gift of knowledge. It’s a chance to learn directly from yourself who feels like they are a master. Training and guiding in “the way of things”.

Often your Explainy voice comes from a miscommunication, a lack of understanding or clarity, or a need for learning. The sentiment behind Explainy voice can be understood a...
An Apology is a Surrender
by Zachary Jacobi — 2018-01-18T00:06:32.814Z

[Epistemic Status: This post is aimed at people who've hurt loved ones by trying to fast talk their way out of apologies; remember the law of equal and opposite advice. A different version of this post, with examples drawn from responses to #MeToo, previously appeared on my blog.]

When news of Meltdown and Spectre broke, Intel released one of the worst non-apologies I've ever seen.

They lead with "Intel believes these exploits do not have the potential to corrupt, modify or delete data", which is only a consolation if you don't care about any of your passwords being stolen by hackers. They follow that one up with 'Recent reports that these exploits are caused by a “bug” or a “flaw” and are unique to Intel products are incorrect', which is baffling, until you realize that they're actually using conditional logic in a press release.

They end the train wreck with "Intel believes its products are the most secure in the world", which makes me worried that Intel management has lost touch with reality.

I might be old fashioned, but when a hardware manufacturer ships a security flaw that's this big, with a fix that might severely impact performance, I expect an apology, not a rearguard.

What I mean is: as long as you're defending yourself, you aren't internalizing the consequences of your actions. For as long as you keep fighting, you get to keep believing that maybe consequences won't materialize. Maybe you'll say the right thing; maybe the consequences will disappear.

An apology accepts consequences.

Imagine yourself arguing with someone you've hurt. Imagine the wiggle words and excuses you might use. Imagine how hard it is to be quiet, to listen without arguing or defending yourself when someone tells you that you've hurt them.

Doing that, despite the voice inside you telling you to fight, telling you to try and get away clean, that's scary; that's difficult. That's a surrender.

Of course, surrendering is just the first step. It's best if you back it up with something of substance. My four-step algorithm for a proper surrender-apology goes:

1. How did I hurt them?

Sometimes people will tell you straight up how you hurt them. Others won't. And when you're proactively apologizing, you may know that you did something likely to hurt someone, but not exactly...
Making Exceptions to General Rules
by Chris_Leong — 2018-01-17T23:47:09.156Z

Suppose you make a general rule, ie. "I won't eat any cookies". Then you encounter a situation that legitimately feels exceptional , "These are generally considered the best cookies in the entire state". This tends to make people torn between two threads of reasoning:

1) Clearly the optimal strategy is to make an exception this one time and then follow the rule the rest of the time.

2) If you break the rule this one time, then you are likely to dismantle the rule and ending up not following it at all, so you should whether the rule is good on the whole, instead of for this particular opportunity

How can we resolve this? For a very, very long time I didn't want to take option 2) because I felt that taking a sub-optimal strategy was irrational. It may seem silly, but I found this incredibly emotionally compelling and I had no idea of how to respond to this with logic. Surely rationality should never require you to be irrational? It took me literally years to work out, but 1) is an incredibly misleading way of framing the situation. Instead, we should be thinking:

3) If future you will always make the most rational decision, the optimal strategy is going to clearly be to make the exception. If there is a chance that it will cause you to fall off the path, either temporarily or permanently, then we need to account for these probabilities in the expected value calculation.

As soon as I realised this was the more accurate way of framing 1), all of its rhetorical power disappeared. Who cares about what is best for a perfectly rational agent? That isn't you. The other key benefit of this framing is that it pushes you to think in terms of probability. Too often I've thought, "I can make an exception without it becoming a habit". This is bad practise. Instead of thinking in terms of a binary "Can I?" or "Can't I?", we should be thinking in terms of probabilities. Firstly, because 0% probability is unrealistic, secondly because making a probability estimate allows you to better calibrate over time.

We can also update 2) to make it a more sophisticated argument as well.

4) If you break the rule this one time, then you risk dismantling the rule and ending up not following it at all. Further, humans tend to heavily biased towards believing that their future selves will make the decisions that they want ...