trentmkelly/LessWrong-43k huggingface dataset:
Inner alignment in the brain
by Steven Byrnes — 2020-04-22T13:14:08.049Z

Abstract: We can think of the brain crudely as (1) a neocortex which runs an amazingly capable quasi-general-purpose learning-and-planning algorithm, and (2) subcortical structures (midbrain, etc.), one of whose functions is to calculate rewards that get sent to up the neocortex to direct it. But the relationship is actually more complicated than that. "Reward" is not the only informational signal sent up to the neocortex; meanwhile information is also flowing back down in the opposite direction. What's going on? How does all this work? Where do emotions fit in? Well, I'm still confused on many points, but I think I'm making progress. In this post I will describe my current picture of this system.

Background & motivation

I'm interested in helping ensure a good post-AGI future. But how do we think concretely about AGI, when AGI doesn't exist and we don't know how to build it? Three paths:

We can think generally about the nature of intelligence and agency—a research program famously associated with MIRI, Marcus Hutter, etc.;

We can think about today's AI systems—a research program famously associated with OpenAI, DeepMind, CHAI, etc.;

We can start from the one "general intelligence" we know about, i.e. the human brain, and try to go from there to lessons about how AGI might be built, what it might look like, and how it might be safely and beneficially used and controlled.

I like this 3rd research program; it seems to be almost completely neglected,[1] and I think there's a ton of low-hanging fruit there. Also, this program will be especially important if we build AGI in part by reverse-engineering (or reinventing) high-level neocortical algorithms, which (as discussed below) I think is very plausible, maybe even likely—for better or worse.

Now, the brain is divided into the neocortex and the subcortex.

Start with the neocortex[2] The neocortex does essentially all the cool exciting intelligent things that humans do, like building an intelligent world-model involving composition and hierarchies and counterfactuals and analogies and meta-cognition etc., and using that thing to cure diseases and build rocket ships and create culture etc. Thus, both neuroscientists and AI researchers focus a lot of attention onto the neocortex, and on understanding and reverse-engineering its algorithms. Textbooks divide the neocortex into lots of functional regions like "moto...
How likely is the COVID-19 apocalyptic scenario?
by Roko Jelavić — 2020-04-22T10:28:58.388Z

This apocalyptic scenario:

1) We develop no long-term immunity, no vaccines, perhaps because the mutation rate is too high.

2) The disease causes permanent lung damage, making each successive infection worse for the patient.

3) No antivirals get developed, or at least not ones which can be produced in extremely large quantities.

is perhaps the most important consideration right now, and I'm not sure how much discussion there is about it. The probability of it seems hard to estimate, but it should nevertheless be attempted, because the stakes are so high.

Uncertainty

Covid seems to leave permanent lung damage even in young people. Let's say you are 30, get the disease now and recover. Say the vaccine does not get developed, the disease stays with us and you get it again in a few years. Now it's going it be worse. It still may not kill you. Let's say that no effective antivirals get developed which can help you. In a few years you get it again, and this straw breaks the camel's back, now you're dead.

The virus apparently has a mutation rate which is on the high end, unexpectedly large rate of mutation. This makes the vaccine less probable. How much less? No idea.

Some say that antivirals "look promising" but what does that actually mean? No idea. What are they going to cost? What is the theoretical maximum which could be produced on this planet? No idea.

As someone who is not an expert in medicine/biology, every estimate I attempt is going to have a large entropy. Information about this disease we got from various institutions in the past few months was highly biased, because of their incentives. What are the incentives of the people who claim that antivirals look promising? Are the companies just trying to make their product look better, in order to increase the funding provided to them?

The fact that I don't have a clear picture on this bothers me... Where to even get info on this? Are people talking about it but I'm not aware of it? Please, if you have an opinion, leave your subjective probability estimate in the comments, along with the reasoning behind it.

Society

The apocalyptic scenario would surely have huge impact on society. The totalitarian regimes would perhaps deal with this better than free ones, which would either tip the balance of power in favour of China, or it would offer an incentive for the free regime...
On Platitudes
by Ouroborus — 2020-04-22T05:55:59.668Z
Link: https://ouroborusdotblog.wordpress.com/2020/04/22/on-platitudes/

In Scott Alexander's review of Twelve Rules for Life he discusses how Jordan Peterson and CS. Lewis seem to have the ability to express cliches in ways that don't feel cliched.

Jordan Peterson’s superpower is saying cliches and having them sound meaningful. There are times – like when I have a desperate and grieving patient in front of me – that I would give almost anything for this talent. “You know that she wouldn’t have wanted you to be unhappy.” “Oh my God, you’re right! I’m wasting my life grieving when I could be helping others and making her proud of me, let me go out and do this right now!” If only.

This seems like an undervalued skill, particularly within the rationalist community where the focus tends to be on the new and flashy and exciting.

This may sound absurd, but I see it as entirely possible that there may be more wisdom in what we already know than what we don't know and that we just need to learn to extract it. After all, we should expect that almost all the low hanging fruit has been picked by someone in at some point and that if it is broadly applicable, that it'd spread. But if that's too strong, perhaps it might be true after someone has read a few personal development books and spent a few hours on understanding a couple of different religions. In any case, this theory seems worthy of attention as if it were true, then the consequences regarding how we should go about self-development would be truly momentous.

So if platitudes contain deep wisdom, why are they so often derided? Is it that everyone already knows them and so hearing them for the thousandth time is a waste?

I think it is related to idea inoculation. If you are convinced that you already know a concept but your understanding is incomplete then it becomes almost impossible to teach you.

There are many reasons why cliches fail to land:

Cliches are favoured by unoriginal thinkers who can't think their own thoughts. The low intellectual status of these thinkers attaches to these cliches
Someone who repeats a cliche demonstrates a low social awareness and this low social status attaches to the cliche
Unoriginal thinkers and people with low social awareness are less likely to express these thoughts in ways that are either original or which are persuasive
Once something is derided as a clic

...
[Site Meta] Feature Update: More Tags! (Experimental)
by Ruby — 2020-04-22T02:12:00.518Z

Edit: We're specifically seeking feedback right now on the tag relevance voting system, please chime in if you can. Thanks!
A month ago, the LessWrong team

launched our new tagging system

...with a whopping whole entire

single

tag

. We rushed it out quickly to give people a way to manage the deluge of Covid-19 posts which had grown to 50% of new posts by mid-March.

Now we’re ready to trial the tag system generally with all kinds of tags. With this launch people can now 1) use tag pages to find posts about their interests, and 2) filter their Latest Posts section on the frontpage for the content they’re most interested in. Soon tag filtering will applicable to Recent Discussion and All Posts page, as well as it being possible to subscribe to tags.

How experimental is this?

Very. The team is figuring out all kinds of questions related to tag ontology and tagging UI, and it’s all subject to change. We’d greatly appreciate feedback and user interviews (comment on this post or use the

usual channels

!)

Warning! For the next while, if you help us by tagging posts or voting on tag relevance, it’s possible you will lose your effort if we later decide to delete the tag due to our evolving tag policy. We appreciate anyone who tags anyway since this helps us develop the tagging system.

For example, we've created the World Optimization high-level core tag for everything which is about concretely improving the world. But the name isn't great– it doesn't imply that personal improvement is included. So we need to figure out a better name or maybe split the tag up.

Tagging Mini-FAQ

See the

Quick Guide to Tagging

for slightly more detail.

How do I use tags to filter content in our out?

Click on the tag filter gear icon next to Latest Posts.

Note the option for “soft filtering” which lets you titrate the amount you see posts with a given tag from Never through Normal Amount to Exclusively. (We’re still working to make this UI intuitive.)

Click on a Tag on the bottom of a post page to visit its Tag Page and find related posts.
Visit the Tag Index (Beta) page to see the list of high-quality existing tags.

Filter Options for Latest Posts

Tags at the Bottom of Post Page

Who can tag things?

Anyone can tag a post with an existing tag.

Ideally tags have a definition paragraph which explain when they do and don’t apply, but as we ramp up not all posts will contain this.

Who can vote on t

...
Wired Headsets for Video Calls
by jefftk — 2020-04-21T22:10:02.343Z

I've recently become a fan of wired headsets for video calls for two
reasons: headsets are good at keeping out background noise, and wired
is good for minimizing latency.

When I started working from home I would need to constantly mute and
unmute myself on video calls. I'm working from my bedroom, which is
near the kids bedroom's and their playing can be (intermittently)
loud. I would leave myself muted except to talk, but needing to
remember to unmute was annoying, and occasionally they would be being
loud when I needed to say something. A headset with a boom mic (Jabra
Evolve 40, via
the Wirecutter) helped enormously. The kids can be shouting in
the same room as me, and it essentially won't pick it up. For example,
here's audio of me speaking. When wearing the headset it's easily
audible, while if I speak even 1ft from the headset it's barely
audible (24dB quieter) and at 5ft it's an enormous 37dB quieter:

wearing
raw
+8dB

1ft
raw
+32dB

2ft
raw
+38dB

5ft
raw
+45dB

Similarly, I took a recording of me speaking and then closing my
bedroom door, with the headset on vs with my laptop mic. The peak of
the door closing is 28dB higher with the laptop mic:

Combined with the automatic gate in most video call software, I can
just leave my mic on all the time and talk frictionlessly.

With headsets one important question is wired vs wireless. With wired
you need to deal with a wire, while with wireless you have to deal
with charging, pairing, and higher latency. After my
experience with setting up wired
internet and realizing how much latency matters for feeling like
you're really in the same place, this seems like a major consideration
to me. But Bluetooth headset specs don't mention it, and the
Wirecutter doesn't
measure it or even discuss it. For gaming you can buy wireless
headsets that use custom
radio setups so this does sound like enough of a problem that some
people care about it. I found one site that
ran
latency tests and it looks like Bluetooth normally adds
~150-200ms of latency but if you can get both sides of the connection to
understand the aptX codec in low-latency mode you can get it down to
~25ms. Since we're talking about using a headset to participate in a
conversation then doubling these numbers sounds right.

My housemate lent me a Bose QC 35
II headset for testing and I measured its latency wi...
What are good defense mechanisms against dangerous bullet biting?
by Mati_Roy — 2020-04-21T20:51:53.983Z

Some beliefs seem to naively imply radical and dangerous actions. But there often are rational reasons to not act on those beliefs. Knowing those reasons is really important for those that don't have a natural defense mechanism.

Most people have a natural defense mechanism which is to not taking ideas seriously. If you just follow what others do, it's less likely that an error in your explicit reasoning will lead you to doing something radical and dangerous. The more likely you are to make such errors, the most (evolutionary and individually) advantageous it is for you to have a conformist instinct.

The answer to this question is mostly meant for people to which I want to share ideas that are dangerous if taken at face value / object-level (I want to make sure they have those defense mechanisms first; I encourage you do the same, and do your due diligences when discussing dangerous ideas; this post is not sufficient). I want to advocate to smart people to take ideas more seriously, but I don't want them to fully repress their conformist instincts, especially if they haven't built in explicit defense mechanisms. This post should also be useful for people already not having those defense mechanism. And also useful to people that want to better understand the function of conformity (although conformity is not the only defense mechanism).

Note that the defense mechanisms are not meant as fully general counterargument. They are not insurmontable (at least, not always), they just indicate when it's prudent to want more evidence.

As a small tangent, is also often has a positive externality to do exploration:

Like it’s rational for any individual to be pursuing much more heavily exploitation based strategy as long as someone somewhere else is creating the information and part of what I find kind of charming and counterintuitive about this is that you realize people who are very exploratory by nature are performing a public service.
(source: Computersciencealgorithmstacklefundamentalanduniversalproblems.Cantheyhelpuslivebetter,oristhatafalse hope?)

I will post my answer below.
Databases of human behaviour and preferences?
by Stuart_Armstrong — 2020-04-21T18:06:51.557Z

Hey there, lesserwrongers!

Starting on some of the computer science/neuroscience of my ideas for deducing human preference.

To do this, it would be useful to have datasets of human behaviour in relatively restricted situations. Possibly datasets of people playing simple games, or solving certain puzzles, or responding to messages, or something similar?

The question is intentionally vague, so that readers can come up with suggestions. What is needed is that the dataset be largish (more than a thousand humans at least), and be of real humans making non-trivial decisions in not-too-complicated circumstances.

Any suggestions?
Training Regime Day 17: Deflinching and Lines of Retreat
by Mark Xu — 2020-04-21T17:45:34.766Z

Introduction

Declaring commitment to Crocker's rules means that other people can optimize their messages to you for information, disregarding our cultures normal rules against potentially causing offense. The reason why one might commit to said rules is to maximize growth - in order to change yourself, one must first acknowledge that you're currently insufficient, which is made easier if someone is allowed to tell you so.

However, being able to abstractly reason that accepting blunt criticism is good does not allow you to accept criticism. When someone directly critiques you, it hurts. For most people, blunt criticism is net negative because the emotional state it induces is not conducive to change. For some people, normal criticism is also net negative for the same reasons.

Deflinching is a technique that attempts to create space to listen to criticism and Lines of Retreat is a technique to create the mindset to accept it.

Deflinching

At its core, Deflinching is just a TAP. It goes like this:

Trigger: notice that you're triggered/offended/bristling

Action: make space

The heart of the technique lies in "making space". This is akin to a micro-focusing move, where the goal is to determine what part of you got triggered/offended/bristly.

In general, there are good reasons for you to get offended. If someone I respect tells me I commited a social faux pas, the internal wince I experience is real. In my case, it's because I don't like being told that I hurt someone because I don't like hurting people. The temptation is to respond "no, I didn't do that" because I obviously didn't intend to hurt them. If there's something your instinctive reaction is trying to defend, you want to surface that thing. You don't want to squish it down and let it eek out the sides, but you don't want to let it dominate either.

The generalized temptation is to direct the wince-type-feeling back at the person who gave you the criticism. This is bad for a few reasons, but one of the main reasons is that this negatively reinforces people giving you criticism, which means you stop finding out about possible improvements you can make. The point of "making space" is to figure out what the wince-type-feeling is actually directed at. In many cases, it's directed at the world state in which the criticism is true - in which case, don't kill the messenger. In other cases, it can be directed at the f

...
Solar system colonisation might not be driven by economics
by Stuart_Armstrong — 2020-04-21T17:10:32.845Z

This is a stub of an argument that I don't necessarily agree with, but that I think deserves to be thought about and analysed more.

Why would we expand into the solar system? What's the actual point - what would we get out there?

I'll argue here that there may not be good economic reasons to go to space; therefore space colonisation would be driven by non-economic reasons, which suggests a different pattern of expansion.

The revenues of space

Mining

How could we make money in space? The obvious answer is mining. Asteroids and planets are full of resources, carbon, iron, gold, rare earths, and so on; the value of these resources, at current prices, is staggering.

But note "at current prices". Obviously, if we brought down a few trillions of tons of copper, the price of that metal might suffer a slight dip.

So what's the economic value of current mining? Well, mining represented 1.2% of world GDP in 2016; but roughly 50% of this is coal, which is mainly used for energy (we'll get to that later). This is a small fraction of the current economy, and one that is in decline (over the century span, if not the decade span).

The attraction of space mining is that we have access to immense resources; but what we need is access to resources at a cheap marginal cost. If space mining doesn't offer that, then we just won't use it, just as there are ways of getting resources on Earth (eg some forms of recycling, distilling resources from the sea, very deep mining) that we are just not using because they're currently too expensive.

Or, put another way, if you're thirsty, the glass of water a metre away is of more interest than the vast reservoir just over the mountain range.

Other revenue generators

There are other things we can do in space. Some scientific experiments require micro-gravity; we can harvest energy via solar panels in space; new manufacturing methods may be usable in zero-G.

But these things can be done in near-Earth orbit, on some nearby space station, or on the moon if we really need some gravity or some mining. Deeper space seems to have little economic advantage. Is it any easier to design AI in space versus doing it on Earth?

The incidental economics of a colony

It's possible to do some agriculture is space; solar energy collection can be done almost anywhere; water is a useful resource for a colony, and, of course, once you have a colony anyway, shipping reso

...
Don't Use Facebook Blocking
by Davis_Kingsley — 2020-04-21T15:50:43.769Z

This might seem like an odd and trivial thing to write a LW post about, but it bears mentioning after seeing another notable incident:

The Facebook block feature is implemented poorly and you should not use it if you intend to participate in community discussions on Facebook. When you block someone on Facebook, they cannot see any of your posts or comments. However, they also cannot see other people's comments in reply to you.

The failure mode that I've seen multiple times now looks something like this:

Person A starts a thread on a community topic that gets a lot of engagement
Person B, who has blocked many people, makes a comment on that post that itself gets a lot of engagement in sub-replies.
Person C, who is blocked by Person B, wants to participate in the discussion but is missing significant context because they cannot see either Person B's comments or any other comments made in reply to the thread person B started, even those made by Person A or other people who have not blocked Person C.

Yes, that's right -- if you block someone on Facebook, not only are you preventing them from seeing your content, but also in some cases you're preventing them from seeing *other people's content* because it happens to have been posted in reply to a thread that you initiated.

As a result of this dynamic, I think people are somewhat obligated to use the block feature sparingly or not at all if they intend to participate in serious community conversations on Facebook, as using it more widely can quite impede conversations.

(Of course, it would be better still if we just didn't have important community conversations on Facebook at all, but that's a different story...)
Equilibrium point of the social 
evolution under the selective pressure of COVID-19?
by Yandong Zhang — 2020-04-21T15:06:46.453Z

As discussed and analyses in the below posts, "enough people wearing respirators" could be an equilibrium point of the final social evolution under the selective pressure of COVID-19, because people wearing resprator (and gloves) could block the propagating path of the virus and lead to smarespiratorll R0.
https://www.lesswrong.com/posts/oHT2WZxFA9CSPMicb/the-hammer-and-the-mask-a-call-to-action
https://www.lesswrong.com/posts/yKYg6D7HNxLuJDcLS/hammer-and-mask-wide-spread-use-of-reusable-particle
Could anybody provide other candidates of the final equilibrium point?
What are some fun ways to spend $100,000?
by Sausage Vector Machine — 2020-04-20T22:45:50.353Z

I'm re-reading The Fun Theory Sequence and enjoying it. But my question is not about theory:

What are some concrete, practical ways to spend $100,000 to generate a non-negligible amount of fun?

Underlying assumptions:

"We're not nearly as unique as we think, at least when it comes to emotional responses to events"[1]. (Therefore it makes sense to ask this question.)
The hypothetical person who has the money and wants the fun doesn't possess any specific skills that would make answers too unique (e.g., buy an exquisite antique violin to happily play it every day).
This hypothetical person is predisposed to boredom (although not depressed or particularly unhappy, just easily bored). This is meant to exclude a wide range of things that some people consider fun but that are actually not fun at all. :)