trentmkelly/LessWrong-43k huggingface dataset:
Illness anxiety disorder: how to become more rational?
by completely_irrational — 2020-01-03T21:05:34.703Z

Disclaimer: When I think about dying I mostly think about cancer (I don't know much about other diseases). In the following, I'll be refering to "cancer" to point at deadly diseases where early detection is crucial.

[Epistemic status: thinking out loud; trying to debug my irrational mind.]

Situation I strongly believe that I have final stage cancer. When I touch my pectoral muscles, I can feel lumps. I have been suffering unbearable anxiety for the past one year and a haf or so, feeling that something very wrong is happening.

Medical Evidence One year ago, a cardiologist told me my heart was fine and didn't see anything from ultrasound. In the past two months, I checked two GPs. The first one said "it's probably just muscular". The second one (extremely talented, known as Dr. House) recommended additional ultrasound/computed tomography.

Internal Evidence It feels real. It hurts. It's cumbersome. It affects the left part of my chest and my left arm. Whenever I touch it I can feel that my left chest is of a very different consistency compared to my right one.

What happens I can't do any meaningful work. My mind is immersed in daily pain and the possibility of a final stage cancer. When I think about doing the additional tests, I'm TERRIFIED by the idea of having a doctor tell me "you have 4 months left". Yesterday, I managed to make an appointment for an ultrasound. However, I'm broke, and it turns out that the particular doctor is really expensive and has really bad reputation online (3/5 google reviews) so I ended up canceling it this morning. Also, I spend a lot of time in self-loathing, insulting myself for not having done any medical exams sooner, when the disease was "not yet deadly".

What I tell myself when I'm trying to be more rational I check the actual statistics of having some kind of cancer in my left chest. For males, breast cancer is 100 times less likely than females. I'm in my 20s, so the probability of having a cancer is smaller than for old people. When I look at actual symptoms of breast cancer, I have none. Most importantly, after checking actual cancer statistics, there's the chance of having a thing and still surviving (probability depends on the type of cancer and stage).

Also, I've been feeling pain for about one year and a half. So if it was...
What are the best self-help book summaries you've read?
by willbradshaw — 2020-01-03T17:45:46.102Z

There is an adage: "Every book should be a blog post."

I disagree with this adage as a general rule, but one specific context where it does seem fairly true is self-help books. Books in or close to the self-help domain seem reliably to be horribly padded, excessively anecdote-laden, and generally somewhat mawkish.

But they're so damn attractive, though. They promise so much, and some seem to have the capacity to be generally transformative to those best-suited to hearing their advice. There's a decent list of 10 or so self-help-ish books whose insights I'd genuinely like to have (in expectation), if I didn't have to wade through a self-help book to get them.

This combination of traits makes self-help books prime candidates for blogification. But any old summary post won't do; a lot of blog-post summaries of books manage to be just as badly written and excessively hype-y as the original while also being too short, vague or unconvincing to be helpful.

What we ideally want is some resource accumulating longer-form, high-quality (think Slate-Star-Codex-level) summaries of self-help books, from trustworthy authors who we can expect to apply some basic due-diligence to the claims being made.

Some day I may get around to co-ordinating a project like this (with enough interested parties we could cover a lot of books in a fairly short space of time) but in the meantime: what are some particularly good summaries of self-help (or self-help-adjacent) books you think more LessWrong readers should read?
CFAR Participant Handbook now available to all
by Duncan Sabien (Inactive) — 2020-01-03T15:43:44.618Z

Google Drive PDF

Hey, guys—I wrote this, and CFAR has recently decided to make it publicly available. Much of it involved rewriting the original work of others, such as Anna Salamon, Kenzie Ashkie, Val Smith, Dan Keys, and other influential CFAR founders and staff, but the actual content was filtered through me as single author as part of getting everything into a consistent and coherent shape.

I have mild intentions to update it in the future with a handful of other new chapters that were on the list, but which didn't get written before CFAR let me go. Note that such updates will likely not be current-CFAR-approved, but will still derive directly from my understanding of the curriculum as former Curriculum Director.
What cognitive biases feel like from the inside
by chaosmage — 2020-01-03T14:24:22.265Z

Building on the recent SSC post

Why Doctors Think They’re The Best

...

What it feels like for me	How I see others who feel the same
There is controversy on the subject but there shouldn't be because the side I am on is obviously right.	They have taken one side in a debate that is unresolved for good reason that they are struggling to understand
I have been studying this carefully	They preferentially seek out conforming evidence
The arguments for my side make obvious sense, they're almost boring.	They're very ready to accept any and all arguments for their side.
The arguments for the opposing side are contradictory, superficial, illogical or debunked.	They dismiss arguments for the opposing side at the earliest opportunity.
The people on the opposing side believe these arguments mostly because they are uninformed, have not thought about it enough or are being actively misled by people with bad motives.	The flawed way they perceive the opposing side makes them confused about how anyone could be on that side. They resolve that confusion by making strong assumptions that can approach conspiracy theories.

The scientific term for this mismatch is: confirmation bias

What it feels like for me	How I see others who feel the same
My customers/friends/relationships love me, so I am good for them, so I am probably just generally good.	They neglect the customers / friends / relationships that did not love them and have left, so they overestimate how good they are.
When customers / friends / relationships switch to me, they tell horror stories of who I'm replacing for them, so I'm better than those.	They don't see the people who are happy with who they have and therefore never become their customers / friends / relationships.

The scientific term for this mismatch is: selection bias

What it feels like for me	How I see others who feel the same
Although I am smart and friendly, people don't listen to me.	Although they are smart and friendly, they are hard to understand.
I have a deep understanding of the issue that people are too stupid or too disinterested to come to share.	They are failing to communicate their understanding, or to give unambiguous evidence they even have it.
This lack of being listened to affects several areas of my life but it is particularly jarring on topics that are very important to me.	This bad communication affects all areas of their life, but on the unimportant ones they don't

...
Excitement vs childishness
by Rafael Harth — 2020-01-03T13:47:44.964Z

I've heard Robin Hanson and others make the argument that people are often biased towards the fast takeoff scenario because it's exciting to think about (or "sexy"). On the other hand, there is a bias towards disbelieving the fast-takeoff scenario because it's childish.

I think most of us can agree that both are indeed biases, i.e., should be assigned zero weight, because both are about attributes that don't correlate with what's true. So we have the excitement-bias and the childishness-bias. The question is, how do they compare?

It feels obvious to me that the childishness bias is far stronger. I see people signaling maturity all the time, and childishness seems to be extremely low status in the relevant circles. It's so low status that it's not uncommon to see people say things about dangers from AI that are accurately summarized as "even though I don't know anything about this topic, I will evaluate its legitimacy based on how many childish sounding arguments I hear because obviously, they are not the real concern and people who defend them have zero credibility." Even among those who provide other arguments, it seems more common than not that they also assume that the fast takeoff scenario is less likely a priori because it's childish. Conversely, I've never heard anyone imply that fast takeoff must be true, or even likely, because it's exciting. It seems that, to have a similar effect, the excitement bias would have to do some heavy lifting in a purely subconscious way.

Given all this, the fact that I've seen far more discussion about the excitement-bias than the childishness-bias seems wrong.

Disclaimer: I don't think this is a strong argument that fast takeoff is likely, and I also don't think that bias towards excitement is weak in general – just that it's weak among the relevant class of people. Finally, I don't think this bias is common on LessWrong, only just about everywhere else.
Predictive coding & depression
by Steven Byrnes — 2020-01-03T02:38:04.530Z

Epistemic status: wild speculation.

This is a follow-up to the 2017 blog post Slate Star Codex: Toward a Predictive Theory of Depression. I think that post has fundamentally the right idea, but that it had some pieces of the puzzle that didn't quite fit satisfyingly.

Well I personally have no special knowledge about depression (happily!), but I have thought an awful lot about the how predictive coding, motivation, and emotions interact in the brain. So I think that I can flesh out Scott's basic story and make it more complete, coherent and plausible. Here goes!

Let's start with my cartoon of predictive coding as I see it:

For lots of details and discussion and caveats, see Predictive coding = RL + SL + Bayes + MPC.

In fact, don't bother reading on here until you've read that.

...

... done? OK, let's move on.

One more piece of background: In predictive coding, hypotheses have a "strength" term for the various predictions they make—basically, the confidence that this prediction will come true. If a strong prediction is erroneous, that's likelier to rise to attention, and to cause the offending hypothesis to be discarded or modified. So let's say you have the hypothesis:

Hypothesis: The thing I am looking at is a TV screen showing static.

This hypothesis...

...makes weak predictions about visual inputs within the screen area (where there's static),

...makes strong predictions about visual inputs within the thin black frame of the TV,

...makes infinitely weak (strength-zero) predictions of sensations coming from my foot.

The sensations from my foot are just not part of this hypothesis. Thus a separate hypothesis about my foot—say, that my foot is tapping the floor and will continue to do so—can be active simultaneously, and these two hypotheses won't conflict.

The proposed grand unified theory of depression in this post is the same as in Scott's post: Severe depression is when all the brain's hypotheses make anomalously weak predictions

I'm now going to go through the symptoms of depression listed in Scott's article, plus a couple others he left out:

Perception

... Depressed people describe the world as gray, washed-out, losing its contrast.

This one is just like Scott says. Normally we understand an apple via a hypothesis like "This thing is a bright red apple". But in severe depression, the hypotheses are weaker: "This thing is probably an apple, and it's probably red...
Becoming Unusually Truth-Oriented
by abramdemski — 2020-01-03T01:27:06.677Z

This is a post on "the basics" -- the simplest moment-to-moment attitudes one can take to orient toward truth, without any special calculations such as Fermi estimates or remembering priors to avoid base-rate neglect. At the same time, it's something almost everyone can fruitfully work on (I suspect), including myself.

Somewhat similar to track-back meditation.

Memory

Tip of the Tongue

The central claim here is that there's a special art associated with what you do when something is "on the tip of your tongue" and you can't quite remember it. Most people have the skill to some extent, but, it can be sharpened to a fine point.

Improved memory helps you become truth-oriented in a fact-oriented, detail-oriented sense. It works against inaccuracy. It also works against misspeaking, and thus propagating falsehoods.

Remembering Dreams

I first explicitly noticed the effectiveness of this technique for remembering dreams. When I wake up, I often have only one significant memory from my dreams. However, when I focus on the memory, explicitly naming each detail I can recall, and gently waiting for more, I can often unfold the memory into far, far more than I initially thought I could remember.

Each detail you recall can open up more details.
There's something special about explicitly naming details. I might have a general sense that there was a portal in the sky that looked a certain way, but explicitly confirming in my head that it looked as if the sky were broken glass, but at the same time the portal was perfectly round, might bring back more memories.

Writing things down on paper is probably a good way of making sure you're explicitly confirming each detail, if you want to go that far.

It's also very important to sit with memories and give them time to bring something more. Sometimes there will be a rush of memories, with each new item bringing more and more. Other times, you'll be stuck. It's easy to fail at that step, assuming that no more is coming. In my experience, if you sit with the memories, avoid getting distracted, and gently ask for more, more will often come to you fairly soon. You'll surprise yourself with what you can remember.

Sometimes I don't even remember any images from the dream at all, but have a vague sense of the dream (excitement, peace, more complicated emotions). I can still somet...
Dominic Cummings: "we’re hiring data scientists, project managers, policy experts, assorted weirdos"
by ESRogs — 2020-01-03T00:33:09.994Z
Link: https://dominiccummings.com/2020/01/02/two-hands-are-a-lot-were-hiring-data-scientists-project-managers-policy-experts-assorted-weirdos/

Dominic Cummings (discussed previously on LW, most recently here) is a Senior Advisor to the new UK PM, Boris Johnson. He also seems to be essentially a rationalist (at least in terms of what ideas he's paying attention to).

He has posted today that his team is hiring "data scientists, project managers, policy experts, assorted weirdos". Perhaps some LW readers should apply.

Extensive quotes below:

‘This is possibly the single largest design flaw contributing to the bad Nash equilibrium in which … many governments are stuck. Every individual high-functioning competent person knows they can’t make much difference by being one more face in that crowd.’ Eliezer Yudkowsky, AI expert, LessWrong etc.

[...]

Now there is a confluence of: a) Brexit requires many large changes in policy and in the structure of decision-making, b) some people in government are prepared to take risks to change things a lot, and c) a new government with a significant majority and little need to worry about short-term unpopularity while trying to make rapid progress with long-term problems.

There is a huge amount of low hanging fruit — trillion dollar bills lying on the street — in the intersection of:

the selection, education and training of people for high performance
the frontiers of the science of prediction
data science, AI and cognitive technologies (e.g Seeing Rooms, ‘authoring tools designed for arguing from evidence’, Tetlock/IARPA prediction tournaments that could easily be extended to consider ‘clusters’ of issues around themes like Brexit to improve policy and project management)
communication (e.g Cialdini)
decision-making institutions at the apex of government.

We want to hire an unusual set of people with different skills and backgrounds to work in Downing Street with the best officials, some as spads and perhaps some as officials. If you are already an official and you read this blog and think you fit one of these categories, get in touch.

The categories are roughly:

Data scientists and software developers
Economists
Policy experts
Project managers
Communication experts
Junior researchers one of whom will also be my personal assistant
Weirdos and misfits with odd skills

[...]

A. Unusual mathematicians, physicists, computer scientists, data scientists

You must have exceptional academic qualificatio...
Normalization of Deviance
by Gordon Seidoh Worley — 2020-01-02T22:58:41.716Z

An important, ongoing part of the rationalist project is to build richer mental models for understanding the world. To that end I'd like to briefly share part of my model of the world that seems to be outside the rationalist cannon in an explicit way, but which I think is known well to most, and talk a bit about how I think it is relevant to you, dear reader. Its name is "normalization of deviance".

If you've worked a job, attended school, driven a car, or even just grew up with a guardian, you've most likely experienced normalization of deviance. It happens when your boss tells you to do one thing but all your coworkers do something else and your boss expects you to do the same as them. It happens when the teacher gives you a deadline but lets everyone turn in the assignment late. It happens when you have to speed to keep up with traffic to avoid causing an accident. And it happens when parents lay down rules but routinely allow exceptions such that the rules might as well not even exist.

It took a much less mundane situation for the idea to crystalize and get a name. Diane Vaughan coined the term as part of her research into the causes of the Challenger explosion, where she described normalization of deviance as what happens when people within an organization become so used to deviant behavior that they don't see the deviance, even if that deviance is actively working against an important goal (in the case of Challenger, safety). From her work the idea has spread to considerations in healthcare, aeronautics, security, and, where I learned about it, software engineering. Along the way the idea has generalized from being specifically about organizations, violations of standard operating procedures, and safety to any situation where norms are so regularly violated that they are replaced by the de facto norms of the violations.

I think normalization of deviance shows up all over the place and is likely quietly happening in your life right now just outside where you are bothering to look. Here's some ways I think this might be relevant to you, and I encourage you to mention more in the comments:

If you are trying to establish a new habit, regular violations of the intended habit may result in a deviant, skewed version of the habit being adopted.
If you are trying to live up to an ideal (truth telling, vegetarianism, charitable givi

...
Has anyone used TAPs to combat BFRBs?
by Fauna M. — 2020-01-02T21:28:01.084Z

BFRB=body focused repetitive behavior such as skin picking or hair pulling.
What were your TAPs? Did you experience remission, and how did you handle that?

I managed to stop my BFRB in a few months ago using a multi-dimensional strategy, but I’ve had a few close calls with recurrence and was wondering if anyone had experience using TAPs to combat these.
[AN #80]: Why AI risk might be solved without additional intervention from longtermists
by Rohin Shah — 2020-01-02T18:20:01.686Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Welcome to another special edition of the newsletter! In this edition, I summarize four conversations that AI Impacts had with researchers who were optimistic that AI safety would be solved "by default". (Note that one of the conversations was with me.)

While all four of these conversations covered very different topics, I think there were three main points of convergence. First, we were relatively unconvinced by the traditional arguments for AI risk, and find discontinuities relatively unlikely. Second, we were more optimistic about solving the problem in the future, when we know more about the problem and have more evidence about powerful AI systems. And finally, we were more optimistic that as we get more evidence of the problem in the future, the existing ML community will actually try to fix that problem.

Conversation with Paul Christiano

(Paul Christiano, Asya Bergal, Ronny Fernandez, and Robert Long) (summarized by Rohin): There can't be too many things that reduce the expected value of the future by 10%; if there were, there would be no expected value left (ETA: see this comment). So, the prior that any particular thing has such an impact should be quite low. With AI in particular, obviously we're going to try to make AI systems that do what we want them to do. So starting from this position of optimism, we can then evaluate the arguments for doom. The two main arguments: first, we can't distinguish ahead of time between AIs that are trying to do the right thing, and AIs that are trying to kill us, because the latter will behave nicely until they can execute a treacherous turn. Second, since we don't have a crisp concept of "doing the right thing", we can't select AI systems on whether they are doing the right thing.

However, there are many "saving throws", or ways that the argument could break down, avoiding doom. Perhaps there's no problem at all, or perhaps we can cope with it with a little bit of effort, or perhaps we can coordinate to not build AIs that destroy value. Paul assigns a decent amount of p...
A reductio ad absurdum for naive Functional/Computational Theory-of-Mind (FCToM).
by David Scott Krueger (formerly: capybaralet) — 2020-01-02T17:16:35.566Z

Thesis: FCToM is underspecified, since they rely on a mapping (which I call the "level of analysis") from physical systems to mathematic functions or computational algorithms (respectively), but do not specify that mapping. (I'm collapsing functional and computational theories of mind, since I think the distinction isn't relevant here.

I believe this issue has great ethical significance: if we accept a naive version of FCToM, we may end up using a misleading level of analysis, and (e.g.) committing massive mind crime. One form of naive FCToM would ignore this issue and say: "if two systems can be described as performing the same computations, then they have the same 'mind' (and hence the same consciousness and same status as moral patients)"

The reductio ad absurdum: Imagine a future totalitarian society where individual humans are forced to play the role of logic gates in a computer which hosts an emulation of your brain. They communicate via snail-mail, and severe punishment, social isolation, and redundancies are used to ensure that they perform their task faithfully. If you endorse naive FCToM, you would say "that's just me!" But far more ethically relevant than the emulation is the experience of the many people enslaved in this system. Note: this is a thought experiment, and may not be physically realizable (for instance, the people playing the gates may be too difficult to control); I think exploring that issue can provide a complementary critique of FCToM, but I'll skip it for now.

Historical note: the idea for writing this post, although not the content, is somewhat inspired by a debate between Massimo Pigliucci and Eliezer Yudkowsky on blogging heads (EDIT: e.g., jumping in the middle here: https://www.youtube.com/watch?v=onvAl4SQ5-Q&t=2118s ). I think Massimo won that argument.