trentmkelly/LessWrong-43k huggingface dataset:
Uber Self-Driving Crash
by jefftk — 2019-11-07T15:00:01.625Z

Content warning: discussion of death

A year and a half ago an Uber self-driving car hit and killed Elaine
Herzberg. I wrote at
the time:

The dashcam video from the Uber crash has been released. It's really
bad. The pedestrian is slowly walking their bike left to right across
a two lane street with streetlights, and manages to get to the right
side of the right lane before being hit. The car doesn't slow down at
all. A human driver would have vision with more dynamic range than
this camera, and it looks to me like they would have seen the
pedestrian about 2s out, time to slow down dramatically even if not
stop entirely. But that doesn't matter here, because this car has
LIDAR, which generates its own light. I'm expecting that when the full
sensor data is released it will be very clear that the system had all
the information it needed to stop in time.

This is the sort of situation where LIDAR should shine, equivalent to
a driver on an open road in broad daylight. That the car took no
action here means things are very wrong with their system. If it were
a company I trusted more than Uber I would say "at least two things
going wrong, like not being able to identify a person pushing a bike
and then not being cautious enough about unknown input" but with Uber
I think they may be just aggressively pushing out immature tech.

On Tuesday
the NTSB released their report (pdf)
and it's clear that the system could easily have avoided this accident
if it had been better designed. Major issues include:

"If we see a problem, wait and hope it goes away". The car was
programmed to, when it determined things were very wrong,
wait one second. Literally. Not even gently apply the brakes. This
is absolutely nuts. If your system has so many false alarms that you
need to include this kind of hack to keep it from acting erratically,
you are not ready to test on public roads.

"If I can't stop in time, why bother?" When the car concluded
emergency braking was needed, and after waiting one second to make
sure it was still needed, it decided not to engage emergency braking
because that wouldn't be sufficient to prevent impact. Since
lower-speed crashes are far more survivable, you definitely still want
to brake hard even if it won't be enough.

"If I'm not sure what it is, how can I remember what it was
doing?" The car wasn't sure whether Herzberg

...
Building Intuitions On Non-Empirical Arguments In Science
by Scott Alexander — 2019-11-07T06:50:02.354Z

I.

Aeon: Post-Empirical Science Is An Oxymoron And It is Dangerous:

There is no agreed criterion to distinguish science from pseudoscience, or just plain ordinary bullshit, opening the door to all manner of metaphysics masquerading as science. This is ‘post-empirical’ science, where truth no longer matters, and it is potentially very dangerous.

It’s not difficult to find recent examples. On 8 June 2019, the front cover of New Scientist magazine boldly declared that we’re ‘Inside the Mirrorverse’. Its editors bid us ‘Welcome to the parallel reality that’s hiding in plain sight’. […]

[Some physicists] claim that neutrons [are] flitting between parallel universes. They admit that the chances of proving this are ‘low’, or even ‘zero’, but it doesn’t really matter. When it comes to grabbing attention, inviting that all-important click, or purchase, speculative metaphysics wins hands down.

These theories are based on the notion that our Universe is not unique, that there exists a large number of other universes that somehow sit alongside or parallel to our own. For example, in the so-called Many-Worlds interpretation of quantum mechanics, there are universes containing our parallel selves, identical to us but for their different experiences of quantum physics. These theories are attractive to some few theoretical physicists and philosophers, but there is absolutely no empirical evidence for them. And, as it seems we can’t ever experience these other universes, there will never be any evidence for them. As Broussard explained, these theories are sufficiently slippery to duck any kind of challenge that experimentalists might try to throw at them, and there’s always someone happy to keep the idea alive.

Is this really science? The answer depends on what you think society needs from science. In our post-truth age of casual lies, fake news and alternative facts, society is under extraordinary pressure from those pushing potentially dangerous antiscientific propaganda – ranging from climate-change denial to the anti-vaxxer movement to homeopathic medicines. I, for one, prefer a science that is rational and based on evidence, a science that is concerned with theories and empirical facts, a science that promotes the search for truth, no matter how transient or contingent. I prefer a science that does not readily admit theories so vague and slippery that empirical tests are eit

...
Personal quality experimentation
by KatjaGrace — 2019-11-06T21:50:01.565Z

Different people seem to have different strategies, which they use systematically across different parts of their lives, and that we recognize and talk about. For instance people vary on:

Spontaneity

Inclination toward explicit calculations

Tendency to go meta

Skepticism

Optimism

Tendency to look at the big picture vs. the details

Expressed confidence

Enacted patience

I don’t know of almost anyone experimenting with varying these axes, to see which setting is best for them, or even what different settings are like. Which seems like a natural thing to do in some sense, given the variation in starting positions and lack of consensus on which positions are best.

Possibly it is just very hard to change them, but my impression is that for at least some of them it is not hard to try, or to change them a bit for a short period, with some effort. (I have briefly tried making decisions faster and expressing more confidence.) And my guess is that that is enough to often be interesting. Also that if you effortfully force yourself to be more skeptical and it seems to go really well, you will find that it becomes appealing and thus easier to keep up and then get used to.

I also haven’t done this much, and it isn’t very clear to me why. Maybe it just doesn’t occur to people that much for some reason. (It also doesn’t occur to people to choose their value of time via experimentation I think, a related suggestion I like, I think from Tyler Cowen a long time ago.) So here, I suggest it. Fun date activity, maybe: randomly reselect one personality trait each, and both try to guess which one the other person is putting on.
A mechanistic model of meditation
by Kaj_Sotala — 2019-11-06T21:37:03.819Z

Meditation has been claimed to have all kinds of transformative effects on the psyche, such as improving concentration ability, healing trauma, cleaning up delusions, allowing one to

track their subconscious strategies

, and

making one’s nervous system more efficient

. However, an explanation for why and how exactly this would happen has typically been lacking. This makes people reasonably skeptical of such claims.

In this post, I want to offer an explanation for one kind of a mechanism: meditation increasing the degree of a person’s introspective awareness, and thus leading to increasing psychological unity as internal conflicts are detected and resolved.

Note that this post does not discuss “enlightenment”. That is a related but separate topic. It is possible to pursue meditation mainly for its ordinary psychological benefits while being uninterested in enlightenment, and vice versa.

What is introspective awareness?

In an earlier

post on introspective awareness

, I distinguished between being aware of something, and being aware of having been aware of something. My example involved that of a robot whose consciousness contains one mental object at a time, and which is aware of different things at different times:

Robot’s thought at time 1: It’s raining outside

Robot’s thought at time 2: Battery low

Robot’s thought at time 3: Technological unemployment protestors are outside

Robot’s thought at time 4: Battery low

Robot’s thought at time 5: I’m now recharging my battery

At times 2-5, the robot has no awareness of the fact that it was thinking about rain at time 1. As soon as something else captures its attention, it has no idea of this earlier conscious content - unless a particular subsystem happens to record the fact, and can later re-present the content in an appropriately tagged form:

Time 6: At time 1, there was the thought that [It’s raining outside]

I said that at time 6, the robot had a moment of introspective awareness: a mental object containing a summary of its previous thoughts, which can then be separately examined and acted upon.

Humans are not robots. But I previously summarized the neuroscience book

Consciousness and the Brain

, and its global neuronal workspace (GNW) model of consciousness. According to this model, the contents of consciousness correspond to what is being re...
AI Alignment Research Overview (by Jacob Steinhardt)
by Ben Pace — 2019-11-06T19:24:50.240Z
Link: https://docs.google.com/document/d/1FbTuRvC4TFWzGYerTKpBU7FJlyvjeOvVYF2uYNFSlOc/edit

I'm really excited to see someone outline all the work they think needs solving in AI alignment - to describe what the problem looks like, what a solution looks like, and what work has been done so far. Especially from Jacob, who is a coauthor of the Concrete Problems in AI Safety paper.

Below, I've included some excerpts from doc. I've included the introduction, the following section describing the categories of technical work, and some high-level information from the long sections on 'technical alignment problem' and the 'detecting failures in advance'.

Introduction

This document gives an overview of different areas of technical work that seem necessary, or at least desirable, for creating safe and aligned AI systems. The focus is on safety and alignment of powerful AI systems, i.e. systems that may exceed human capabilities in a broad variety of domains, and which likely act on a large scale. Correspondingly, there is an emphasis on approaches that seem scalable to such systems.

By “aligned”, I mean that the actions it pursues move the world towards states that humans want, and away from states that humans don’t want. Some issues with this definition are that different humans might have different preferences (I will mostly ignore this issue), and that there are differences between stated preferences, “revealed” preferences as implied by actions, and preferences that one endorses upon reflection (I won’t ignore this issue).

I think it is quite plausible that some topics are missing, and I welcome comments to that regard. My goal is to outline a critical mass of topics in enough detail that someone with knowledge of ML and some limited familiarity with AI alignment as an area would have a collection of promising research directions, a mechanistic understanding of why they are promising, and some pointers for what work on them might look like.

To that end, below I outline four broad categories of technical work: technical alignment (the overcoming of conceptual or engineering issues needed to create aligned AI), detecting failures (the development of tools for proactively assessing the safety/alignment of a system or approach), methodological understanding (best practices backed up by experience), and system-building (how to tie together the three preceding categories in the context of many engineers working on a large system). These are described in more detail i...
[AN #72]: Alignment, robustness, methodology, and system building as research priorities for AI safety
by Rohin Shah — 2019-11-06T18:10:01.604Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

AI Alignment Research Overview

(Jacob Steinhardt) (summarized by Dan H): It has been over three years since

Concrete Problems in AI Safety

. Since that time we have learned more about the structure of the safety problem. This document represents an updated taxonomy of problems relevant for AI alignment. Jacob Steinhardt decomposes the remaining technical work into “technical alignment (the overcoming of conceptual or engineering issues needed to create aligned AI), detecting failures (the development of tools for proactively assessing the safety/alignment of a system or approach), methodological understanding (best practices backed up by experience), and system-building (how to tie together the three preceding categories in the context of many engineers working on a large system).”

The first topic under “technical alignment” is “Out-of-Distribution Robustness,” which receives more emphasis than it did in Concrete Problems. Out-of-Distribution Robustness is in part motivated by the fact that transformative AI will lead to substantial changes to the real world, and we should like our systems to perform well even under these large and possibly rapid data shifts. Specific subproblems include some work on adversarial examples and out-of-distribution detection. Next, the problem of Reward Learning is described. For this, there are challenges including learning human values and ensuring those lossily represented human values can remain aligned under extreme optimization. While we have attained more conceptual clarity about reward learning since Concrete Problems, reward learning still remains largely “uncharted,” and it is still not clear “how approach the problem.” The next section on Scalable Reward Generation points out that, in the future, labeling meaning or providing human oversight will prove increasingly difficult. Next, he proposes that we ought to study how to make systems “act conservatively,” such as endowing systems with the ability to activate a conservative fallback routi...
Judgment, Punishment, and the Information-Suppression Field
by Benquo — 2019-11-06T17:44:38.908Z
Link: http://benjaminrosshoffman.com/judgment-punishment-and-the-information-suppression-field/

There are a lot of senses in which I or the people around me can be considered unsafe. Many-tonned hunks of metal whiz by us on the same streets we have to navigate on foot to buy our groceries. The social infrastructure by which we have access to clean drinking water is gradually being adulterated. Our country is run by occasionally genocidal white nationalists. And, of course, The Bomb. But when I hear people talk about feeling unsafe, they are almost never describing a concrete threat to their physical well-being. (As usual, life may be different for the less privileged classes, who have reason to fear the authorities, and behave accordingly.) "Safety" does not come up as a motive for actions taken or avoided in order to mitigate such threats. Instead, it seems that "safety" nearly always means a nonjudgmental context (the exact opposite of what I would naively expect to be able to ensure clean drinking water or keep the cars from colliding with us), and "feeling unsafe" is generally used to explain only why they're trying to withhold information (mainly "vulnerable," i.e. relevant-to-their-interests, information) in a way that seems out of proportion to actually existing risks and opportunities.

Judgment and Punishment

Consider a simple model: information about social censure consists of two parts. Each socially legible action is assigned a vulnerability score based on how often, empirically, someone responds by revealing the intent to punish the actor. Actions are sometimes defined contextually, so that talking loudly in a crowded bar or on the street is different than talking loudly in a library or theater, but it's not a different action depending on who is present - only impersonal context cues and stereotyped identities (e.g. some things are inappropriate "in mixed company"). Vulnerability is a global variable with respect to persons.

If Cato is observed to punish singing but not dancing, and Flaccus is observed to punish dancing but not singing, this is treated as unpredictable random variation - possibly just measurement error. Cato and Flaccus both acquire a reputation for judginess, and both singing and dancing start to feel like vulnerable activities, so people will feel inhibited about doing either activity in the presence of either censor.

At the same time, each known person is evaluated for their generic propensity to punish, or judginess. Som

...
Is my perception of reality bounded by my intelligence?
by  — 2019-11-05T20:40:50.246Z

Hello all,

Here I am living in the Netherlands, 2019 AD. In my perception there is no two-way explicit communication with non-human intelligence, no space travel. Artifical intelligence is in it's infancy (at least, as seen by the general public). All my evidence to today tells me I this current objective reality is certain.

But now I am asking myself from a rationality perspective: How is this perception of reality is bounded by my intelligence (in whatever way my biological system has implemented this intelligence) and (unaware) filtering of evidence?

If I were able to enhance my intelligence by just 1 bit, changing some personal absolute certainty into something probabilistic, what would be the effect on my objective reality? What would be the effect on our shared objective reality? What experiment could be designed to find out?
Normative reductionism
by KatjaGrace — 2019-11-05T20:20:02.083Z

Here’s a concept that seems useful, but that I don’t remember ever hearing explicitly referred to (with my own tentative name for it—if it turns out to not already have one in some extensive philosophical literature, I might think more about whether it is a good name):

Normative reductionism: The value of a world history is equal to the value of its parts (for some definition of relevant parts).

For instance, if two world histories only differ between time t and time t’, according to NR you do not need to know what happened at other times to evaluate them in full. Similarly, the value of Alice’s life, or the value of Alice enjoying a nap, depend on the nature of her life or the nap, and not for instance on other people’s lives or events that took place before she was born with no effect on her (unless perhaps she has preferences about those events or they involve people having preferences about her, but still the total value can be decomposed into the value of different preferences being fulfilled or not). Straightforward hedonistic utilitarianism probably implies normative reductionism.

My impression is that people have different intuitions about this and vary in how much they assume it, and that it mostly isn’t entirely aligned with other axes of ethical view, either logically or sociologically, though is related to them. So it seems maybe worth noting explicitly.
Lite Blocking
by jefftk — 2019-11-05T13:50:02.402Z

There are people I would like to interact less with online: perhaps
they post inane things, perhaps they pull comment threads off in bad
directions, perhaps they make terrible arguments for things I agree
with. The standard tools social networks offer for this sort of
situation are:

Blocking: you can't see their stuff, they can't see your stuff,
you can't interact.

Hiding: you don't see their stuff, or you see less of it. You
can still interact.

I'd like another tool:

Lite blocking: as if they'd hidden you.

Social networks generally have far more things they could show you
than you'll be able to look at. To prioritize they use inscrutable
algorithms that boil down to "we show you the things we predict you're
going to like". You can think of hiding as "dramatically lower the
prediction that I would like seeing their stuff", and lite blocking as
"dramatically lower the prediction that they would like seeing my
stuff".

Lite blocking could be symmetrical or not, but the important thing to
me is that the network would stop encouraging people to interact with
me if I don't want those interactions.

Perhaps I should just block people? I'm glad blocking exists, and
there are times when it's the right tool. But other times it's much
too powerful:

Blocking is obvious, but with the level of "who knows
why the things decided to show me what it did" lite blocking maintains
plausible deniability. This is important for people you want to avoid
interacting with but need to stay on good terms with for other
reasons.

Blocking is too thorough. Maybe I don't like the way you tend
to come into threads I host and derail them, but I'd still like you to
be able to find our past discussions and reference them if
intentionally seek them out.

Groups often have high thresholds for kicking people out, but
an in-between level where someone would just see fewer group posts in
their feed would be helpful in cases where the moderators think
someone's posts are generally making the group worse.

Since lite blocking explicitly overrides the network's prediction
about how much someone will like things, you could imagine a change to
add it being difficult to get past launch review. It would probably
look bad on the core metrics, with a decrease in estimated user
satisfaction and engagement. But the metrics don't capture the ways
"so and so isn't sho...
Book Review: Design Principles of Biological Circuits
by johnswentworth — 2019-11-05T06:49:58.329Z

I remember seeing a talk by a synthetic biologist, almost a decade ago. The biologist used a genetic algorithm to evolve an electronic circuit, something like this:

(

source

)

He then printed out the evolved circuit, brought it to his colleague in the electrical engineering department, and asked the engineer to analyze the circuit and figure out what it did.

“I refuse to analyze this circuit,” the colleague replied, “because it was not designed to be understandable by humans.” He has a point - that circuit is a big, opaque mess.

This, the biologist argued, is the root problem of biology: evolution builds things from random mutation, connecting things up without rhyme or reason, into one giant

spaghetti tower

. We can take it apart and look at all the pieces, we can simulate the whole thing and see what happens, but there’s no reason to expect any deeper understanding. Organisms did not evolve to be understandable by humans.

I used to agree with this position. I used to argue that there was no reason to expect human-intelligible structure inside biological organisms, or deep neural networks, or other systems not designed to be understandable. But over the next few years after that biologist’s talk, I changed my mind, and one major reason for the change is Uri Alon’s book

An Introduction to Systems Biology: Design Principles of Biological Circuits

.

Alon’s book is the ideal counterargument to the idea that organisms are inherently human-opaque: it directly demonstrates the human-understandable structures which comprise real biological systems. Right from the first page of the introduction:

… one can, in fact, formulate general laws that apply to biological networks. Because it has evolved to perform functions, biological circuitry is far from random or haphazard. ... Although evolution works by random tinkering, it converges again and again onto a defined set of circuit elements that obey general design principles.

The goal of this book is to highlight some of the design principles of biological systems... The main message is that biological systems contain an inherent simplicity. Although cells evolved to function and did not evolve to be comprehensible, simplifying principles make biological design understandable to us.

It’s hard to update one’s gut-level instinct that biology is a giant mess of spaghetti without seeing the structure first hand, so the goal of this post ...