trentmkelly/LessWrong-43k huggingface dataset:
[AN #94]: AI alignment as translation between humans and machines
by Rohin Shah — 2020-04-08T17:10:02.654Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Alignment as Translation (John S Wentworth) (summarized by Rohin): At a very high level, we can model powerful AI systems as moving closer and closer to omniscience. As we move in that direction, what becomes the new constraint on technology? This post argues that the constraint is good interfaces, that is, something that allows us to specify what the AI should do. As with most interfaces, the primary challenge is dealing with the discrepancy between the user's abstractions (how humans think about the world) and the AI system's abstractions, which could be very alien to us (e.g. perhaps the AI system uses detailed low-level simulations). The author believes that this is the central problem of AI alignment: how to translate between these abstractions in a way that accurately preserves meaning.

The post goes through a few ways in which we could attempt to do this translation, but all of them seem to only reduce the amount of translation that is necessary: none of them solve the chicken-and-egg problem of how you do the very first translation between the abstractions.

Rohin's opinion: I like this view on alignment, but I don't know if I would call it the central problem of alignment. It sure seems important that the AI is optimizing something: this is what prevents solutions like "make sure the AI has an undo button / off switch", which would be my preferred line of attack if the main source of AI risk were bad translations between abstractions. There's a longer discussion on this point here.

TECHNICAL AI ALIGNMENT

AGENT FOUNDATIONS

Two Alternatives to Logical Counterfactuals (Jessica Taylor)

LEARNING HUMAN INTENT

State-only Imitation with Transition Dynamics Mismatch (Tanmay Gangwani et al) (summarized by Zach): Most existing imitation learning algorithms rely on the availability of expert demonstrations that come from the same MDP as the one the imitator will be evaluated in. With the advent of adversarial inverse reinforcement learning (AIRL) (AN #17), it has become possible to learn general behaviors. However, algo...
Open thread: Language
by Chris_Leong — 2020-04-08T14:57:36.798Z

I believe that the discussion of language is currently an underrated component of rationality. The language we use constantly biases us and restricts the way that we see the world. At the same time, many of these influences are so small that they aren't worth writing a separate post about. So what are some examples you've seen recently of where language is misleading? Try to keep it to one example or one set of related examples per top-level post.
Assessing Kurzweil's 1999 predictions for 2019
by Stuart_Armstrong — 2020-04-08T14:27:21.689Z

Thanks everyone who volunteered! No longer accepting new volunteers now.

Results coming soon.

First of all, I want to thank all the volunteers who agreed to do this assessment. It's important to check the track record, especially in a period when expert predictions are so important and so debated. If you have any questions, add a comment here or PM me on LessWrong.

Logistics:

Go to this Google form (asking for your name/pseudonym and email) and I will send you a TSV (tab-separated values) file with all 105 questions in random order.

If you don't want to share your email with me, or share it on a Google form, send me a PM on LessWrong and we'll see what else we can do.

Assessment:

For each prediction, please answer on a five point scale: 1=True, 2=Weakly True, 3=Cannot decide, 4=Weakly False, 5=False. Use your intuitive assessment of Kurzweil's words if there is ambiguity in them.

Use your best estimate for the accuracy of the prediction only; do not consider how specific or trivial the prediction is. "People will breath oxygen" is true; "there will be a major pandemic in 2018, driving people to quarantine and crashing economies and stock markets" is false or weakly false, because of the specific date. If it said "around 2018", then it would be true or weakly true.

The prediction is true if it was true in 2019; thus, ignore any recent effects of COVID-19.

Procedure:

Do predictions on the .tsv file I sent you, adding another column.

Do them in the order of that file, starting from the top.

Try and attempt at least 10 predictions.

Some predictions start with a piece in square brackets []. This is to provide context for the prediction. Ignore it when assessing the accuracy.

I'll include the ordered list of predictions as a PDF. Use this if you need yet more context on the prediction.

Once you're done, send me the file back, and I will acknowledge receiving it. If you don't get an acknowledgement in the next two working days, ping me to see if I got it.

Cheers!
Link Retrospective for 2020 Q1
by mraxilus — 2020-04-08T12:38:09.925Z
Link: https://write.as/mraxilus/curated-links-for-2020-q1

Below are a list of interesting content I came across in the first quarter of 2020. I subdivided the links into the type of medium they use, namely auditory, textual, and visual. The links are in no particular order.

1. Auditory

D. McRaney. How a Divisive Photograph of a Perceptually Ambiguous Dress Led Two Researchers to Build the Nuclear Bomb of Cognitive Science out of Socks and Crocs. You Are Not So Smart. 2020:

...the science behind The Dress, why some people see it as black and blue, and others see it as white and gold. But it’s also about how the scientific investigation of The Dress lead to the scientific investigation of socks and Crocs, and how the scientific investigation of socks and Crocs may be, as one researcher told me, the nuclear bomb of cognitive neuroscience.

When facing a novel and uncertain situation, the brain secretly disambiguates the ambiguous without letting you know it was ever uncertain in the first place, leading people who disambiguate differently to seem iNsAnE.

C. Connor. Psychoacoustics. YouTube. 2020:

00:00 Psychoacoustics is the study of the perception of sound. These videos attempt to gather all of the various interesting phenomena that fall in to this category in one condensed series, including many neat illusions. We will also cover a few fascinating geeky topics relating to hearing.

MIT, 15.ai, fifteen.ai, 2020:

This is a text-to-speech tool that you can use to generate 44.1 kHz voices of various characters. The voices are generated in real time using multiple audio synthesis algorithms and customized deep neural networks trained on very little available data (between 30 15 and 120 minutes of clean dialogue for each character). This project demonstrates a significant reduction in the amount of audio required to realistically clone voices while retaining their affective prosodies.

2. Textual

AI Impacts. Interviews on Plausibility of AI Safety by Default. AI Impacts Blog. 2020:

AI Impacts conducted interviews with several thinkers on AI safety in 2019 as part of a project exploring arguments for expecting advanced AI to be safe by default. The interviews also covered other AI safety topics, such as timelines to advanced AI, the likelihood of current techniques leading to AGI, and currently promising AI safety interventions.

Before taking into account other researchers’ opinions, Shah guesses an extremely rough

...
How do you determine temperature cutoffs for women's body-temperature?
by ChristianKl — 2020-04-08T12:11:33.977Z

I do have a female primary partner with whom I'm not cohabating. Even through I'm mostly in quarantine and my partner is as well, I do want to minimze the risk of getting COVID-19.

Given that fever is frequent in most cases of COVID-19, I consider daily temperature measurement a good way to decide whether or not to see my partner. I likely want to use a relatively low value because I'm more afraid of false negatives then false positives.

For women the temperature fluctuates during the cycle and I haven't read much about that. How do I set a good cutoff for abnormal temperatures for a woman?
Idea: Create a podcast of admin tagged posts using AI TTS like Amazon Polly
by Rudi C — 2020-04-08T10:07:52.694Z

Considering current AI TTS systems have reached "pleasant" performances (as evidenced by https://danwahl.net/unsong-audiobook/ and usage of the same Polly API by Pocket), it can be a worthwhile endeavor to create a system that automatically creates podcasts episodes from posts curated by admins and tagged auto-podcast.

PS: I feel that I should mention that current AI TTS services are UNFREE software and by using them we are perpetuating bad equilibria. See Free Software, Free Society - Selected Essays of Richard M. Stallman, 2nd Edition.
Would 2014-2016 Ebola ring the alarm bell?
by DirectedEvolution — 2020-04-08T02:01:47.031Z

Introduction

My project is to create an "alarm bell" questionnaire to help coordinate an earlier response to the next pandemic. Please see the original document for a fuller explanation. In brief, if 13 of the 16 criteria below are met after a disease makes front-page news, this questionnaire anticipates that we should see a major stock market crash due to the disease. The goal, though, is not precisely to predict that crash. It's to stimulate discussion so that we don't behave complacently and have some intellectual tracks laid for assessing the new situation.

I am testing it by seeing whether it would have rung, and whether it should have, during a set of historical epidemics and pandemics. My first test on 2009 swine flu was as success, as was this second test on the Ebola outbreak of 2014-2016.

Assessment

Once again, the initial questionnaire performed correctly. The alarm bell should not have rung for Ebola, and indeed it didn't ring.

Three criteria received a "no" throughout the epidemic. It's not airborn and there's no asymptomatic transmission. It also never made it to a significant number of large, industrialized nations. That puts it on the borderline.

Prior to September or later, the death toll was below the threshold, and it hadn't yet reached major cities with strong international travel links or led to major economies restricting travel.

In September, better estimates for overall transmissibility made it no longer comparable with COVID-19. The same month, the first treatment became available.

Those shifting and coinciding trends mean that there was no month where at least 13/16 criteria were met simultaneously.

My explanation for why Ebola did not crash the stock market is that it's a disease with a terrifying case fatality rate, but a transmissibility rate too low to make it a world-shaker. Most of its fatalities were in nations without much economic clout. It's relatively easy for a well-equipped nation to prevent its spread. The disease is horrible for the affected nations. I think it was probably neglected due to being an African illness, and I'm glad that at least we've produced both a treatment and a vaccine for it now.

Below is my analysis for the questionnaire and the stock market.

Stock market trends in 2014

October 2014 market the peak of the Ebola epidemic, although that'...
Problems with p-values
by  — 2020-04-08T00:58:19.399Z
Link: https://mlu.red/muse/53264266310.html

I'm taking a really cool course right now on statistics education, which covers:

replication

p-values being bad

bayesian statistics

meta-analyses

A lot of our readings are really fun, and I'll likely write more posts based on them, maybe into a "why social science studies are bad" sequence.

Anyway, this is the first entry that goes over into misinterpretations and issues with p-values and null hypothesis significance testing.
Hanson & Mowshowitz Debate: COVID-19 Variolation
by Ben Pace — 2020-04-08T00:07:28.315Z

On Sunday 29th March, Robin Hanson and Zvi Mowshowitz debated Robin's policy proposal of deliberate infection (now named "variolation"), a proposal that Robin argues can reduce deaths from coronavirus by somewhere between 3x to 30x.

They talked on a YouTube livestream for 90 mins, followed by a 30 minute Q&A moderated by me.

Full recording on YouTube.

This is a proposal Robin wrote about first on February 17th. Since the debate he wrote a new full summary of his position in the post Variolation (+ Isolation) May Cut Covid19 Deaths 3-30x and continues to write on the topic. He has also now had a debate with Greg Cochran (link), plus a back-and-forth with Tyler Cowen about it (Tyler's critique, Robin's response).

Zvi's post-debate posts include Taking Initial Viral Load Seriously, and On R0 (in part a response to Robin's post on the topic) and continues to write on the topic.

This was an experiment in online events by the LessWrong team, with more to come.

Some of the topics discussed

What is the basic case for deliberate infection?
Is there still time for this strategy to be useful in places so close to the peak (e.g. New York)?
Is the average viral load currently very high?
What information can we get from deliberate exposure experiments?
What leverage do we have in analysing this policy proposal?
Why do many people have an ick reaction to this strategy?
What would deliberate infection look like concretely?
What could go horribly wrong with this strategy?
Is this worth working on relative to various other existential risks?
What effect will this shock have on our institutions?
What is the likely outcome of the coronavirus crisis?

Highlights

Where deliberate infection fits into the picture

Robin: The basic idea here is to have a better Plan B. Lots of people in the last month had been really hopeful that even though we really screwed up in the first month or two of this, especially in the United States compared to many other places in the world, that we're the dynamic powerhouse and we're going to fix that and we're going to, therefore, create unprecedented levels of testing and tracing and isolation and we're just going to do a heroic job. And hey, it's our moonshot, it's our World War II and this generation is going to do it. And you know what? That's a hope, it's real and it should be pursued. But we should also be planning for what happens if that doesn't wo

...
On R0
by Zvi — 2020-04-07T23:20:01.205Z

Epistemic Status: As with all of my Coronavirus posts, I am not any kind of expert. I am a person thinking out loud, who will doubtless make many mistakes. Treat accordingly. However, the concrete policy proposal contained herein seems right and I endorse it strongly.

Partly a response to (Overcoming Bias): Beware R0 Variance

Previously (not required): Taking Initial Viral Load Seriously

Also related to Covid-19: Coronavirus is Here, An Open Letter To The Congregation Regarding The Upcoming Holiday, Let My People Stay Home,

Ultimately, it’s always all been about R0.

If you get and keep R0 substantially below one, infections fall off. Covid-19 is squashed.

If you can keep R0 below one and let normal life happen, life can return to normal. If you can’t, life can’t.

If R0 remains above one, infections continue to rise until something changes.

What will it take to do that?

Interventions that reduce contact and exposure, or reduce the danger of each contact or exposure, or change their dynamics in useful ways, reduce R0. Every person already infected is almost certainly immune (at least for now) which also reduces R0.

You need some combination of interventions and immunity from previous infection that sufficiently reduces the initial R0.

This is not an o-ring production function. You are not as vulnerable as your weakest link. Not everything you do has to be correct. This isn’t a designed puzzle where there are exactly enough interventions available to solve the problem and you need to do them all. Instead, we have a variety of possible actions and need to pick the cheapest basket that reliably accomplishes the mission.

What Was Initial R0?

As Robin notes, R0 does not start out and never is one number. It depends on the surrounding disease environment. In some places it will start out very high. I’ve seen plausible estimates for New York City as high as 8. In other places R0 might be lower than one to begin with, because people barely interact with each other in normal times. That hermit with fifty years experience social distancing? His local region’s R0 is about zilch.

To get intuition pumping one still needs a good ‘default’ R0 before any substantial interventions take place, so we can adjust after that for interventions and anything about a particular place that changes R0 substantially versus the default.

This document collects, among other useful things, a bunch of esti...
An Orthodox Case Against Utility Functions
by abramdemski — 2020-04-07T19:18:12.043Z

This post has benefitted from discussion with Sam Eisenstat, Scott Garrabrant, Tsvi Benson-Tilsen, Daniel Demski, Daniel Kokotajlo, and Stuart Armstrong. It started out as a thought about Stuart Armstrong's research agenda.

In this post, I hope to say something about what it means for a rational agent to have preferences. The view I am putting forward is relatively new to me, but it is not very radical. It is, dare I say, a conservative view -- I hold close to Bayesian expected utility theory. However, my impression is that it differs greatly from common impressions of Bayesian expected utility theory.

I will argue against a particular view of expected utility theory -- a view which I'll call reductive utility. I do not recall seeing this view explicitly laid out and defended (except in in-person conversations). However, I expect at least a good chunk of the assumptions are commonly made.

Reductive Utility

The core tenets of reductive utility are as follows:

The sample space Ω of a rational agent's beliefs is, more or less, the set of possible ways the world could be -- which is to say, the set of possible physical configurations of the universe. Hence, each world ω∈Ω is one such configuration.
The preferences of a rational agent are represented by a utility function U:Ω→R from worlds to real numbers.
Furthermore, the utility function should be a computable function of worlds.

Since I'm setting up the view which I'm knocking down, there is a risk I'm striking at a straw man. However, I think there are some good reasons to find the view appealing. The following subsections will expand on the three tenets, and attempt to provide some motivation for them.

If the three points seem obvious to you, you might just skip to the next section.

Worlds Are Basically Physical

What I mean here resembles the standard physical-reductionist view. However, my emphasis is on certain features of this view:

There is some "basic stuff" -- like like quarks or vibrating strings or what-have-you.
What there is to know about the world is some set of statements about this basic stuff -- particle locations and momentums, or wave-form function values, or what-have-you.
These special atomic statements should be logically independent from each other (though they may of course be probabilistically related), and together, fully

...
Body Mass and Risk from COVID-19 and Influenza
by Radford Neal — 2020-04-07T18:18:24.610Z
Link: https://radfordneal.wordpress.com/2020/04/06/body-mass-and-risk-from-covid-19-and-influenza/

I've written a blog post on "Body Mass and Risk from COVID-19 and Influenza", available at https://radfordneal.wordpress.com/2020/04/06/body-mass-and-risk-from-covid-19-and-influenza/

Here's the intro:

Understanding the factors affecting whether someone infected with COVID-19 will become seriously ill is important for treatment of patients, for forecasting and planning, and — with factors that can be changed — for personal decisions aimed at reducing risk. Despite our current focus, influenza also remains a serious disease, so understanding its risk factors is also important.

Here, I’ll look at some of the evidence on how body mass — formalized as Body Mass Index (BMI, weight in kilograms divided by squared height in metres) — influences prognosis for respiratory diseases. Information specific to COVID-19 is still scant, but there is more data on influenza and on other respiratory infections (which includes coronaviruses other than COVID-19). Information on how BMI relates to general mortality should also be helpful.

Below, I’ll look at two relevant papers, plus a preliminary report on COVID-19. To preview my conclusions, it seems that being underweight and being seriously obese are both risk factors for serious respiratory illness. Furthermore, it seems that “underweight” should include the lower part of the “normal weight” category as defined by the WHO. Official advice in this respect seems dangerously misleading.
Mosh
by jefftk — 2020-04-07T17:40:02.588Z

I remember when

mosh first came out
and thinking "that sounds like a great idea, maybe I'll set it up
sometime" and then never getting around to it. Several times over the
years I've thought similarly, but it never seemed worth it. A week
ago a coworker asked if there were good tools for handling remote
terminal use over laggy internet connections, and I suggested mosh.
They really liked it, which then had me feeling silly for never trying
it, and got me to actually do it. So: I'm typing this over mosh, and
it's excellent! I should have done this years ago.

With a traditional remote shell like ssh you are sending characters
back and forth and the state of your terminal is maintained
client-side. This means that when you press a 'q' you don't see it
appear on the screen until a message has gone to the server saying
"they pressed 'q'" and a response comes back saying "ok, put a 'q' on
the screen". If you have a nice fast connection this happens so
quickly you can't even tell, but if you're on flaky wifi it can be
very painful.

Mosh turns this around, maintains the terminal on the server, and
syncs the state of the terminal from the server to the client as
efficiently as it can. Additionally, it echos your typing back as you
go, and then overwrites it with the state from the server once it gets
it. It even handles cases where you're typing text into the middle of
an existing line, sliding the rest of the line forwards to make room.
It shows locally echoed text with a little underline so you know it's
just a prediction.

Working from my third floor
bedroom connected to wifi in the basement, in a house with seven
other people who might all be streaming video at once, with a workflow
that is built entirely around being ssh'd into a remote server, this
is just amazing.

It's funny looking back at the world before, and seeing people being
dismissive of these ideas. For example, about six months before mosh
came out someone asked
on StackOverflow how to avoid the typing delay on SSH. The
accepted answer is saying this is impossible and just suggesting that
people type without seeing feedback (this probably works a lot better
for people who are more accurate typists than I am). Similarly, the
HackerNews
discussion has several people saying that you can patch something
similar together with existing tools.

This does leave me wondering whet...
What are the Nth Order Effects of the Coronavirus?
by Yoav Ravid — 2020-04-07T13:55:08.456Z

Some were easily foreseeable - like travel bans, lockdowns, and economic damage.

Some were made total sense in retrospect but were not an obvious think to think about - Like an increase in domestic violence (and, consequentially, shelters being overwhelmed).

Let's make a list of both the easily foreseeable, and especial the not easily foreseeable, nth order effects from the virus.

If you can add a source or extra details it's good, but the main point is to just mention the possible effects so someone doesn't have to try thinking of all of them on his own.

If you want to say what order effect something is (second order, third order, etc..) that's extra cool.