trentmkelly/LessWrong-43k huggingface dataset:
AI Boxing for Hardware-bound agents (aka the China alignment problem)
by Logan Zoellner — 2020-05-08T15:50:12.915Z

I feel like I fit into a small group of people who are both:

1) Very confident AI will be developed in the next few decades

2) Not terribly worried about AI alignment

Most people around here seem to fall into either the camp "AI is always 50 years away" or "Oh my gosh Skynet is going to kill us all!".

This is my attempt to explain why

A) I'm less worried than a lot of people around here.

B) I think a lot of the AI alignment work is following a pretty silly research path.

BLUF

Sorry this is so long.

The short version is: I believe we will see a slow takeoff in which AI is developed simultaneously in several places around the globe. This means we need to focus on building institutions not software.

!!!Edit: Important Clarification

I apparently did a poor job writing this, since multiple people have commented "Wow, you sure hate China!" or "Are you saying China taking over the world is worse than a paperclip maximizer!?"

That is not what this post is about!

What this post is about is:

Suppose you were aware that an entity was soon to come into existence which would be much more powerful than you are. Suppose further that you had limited faith in your ability to influence the goals and values of that entity. How would you attempt to engineer the world so that nonetheless after the rise of that entity, your values and existence continue to be protected?

What this post is not about:

An AI perfectly aligned with the interests of the Chinese government would not be worse than a paperclip maximizer (or your preferred bad outcome to the singularity). An AI perfectly aligned with the consistently extrapolated values of the Chinese would probably be pretty okay, since the Chinese are human and share many of the same values as I do. However in a world with a slow takeoff I think it is unlikely any single AI will dominate, much less one that happens to perfectly extrapolate the values of any one group or individual.

Foom, the classic AI-risk scenario

Generally speaking the AI risk crowd tells a very simple story that goes like this:

1) Eventually AI will be developed capable of doing all human-like intellectual tasks

2) Improving AI is one of these tasks

3) Goto 1

The claim is that "Any AI worth its salt will be capable of writing an even better AI, which will be capable of building an even better AI, which will be capable of...." s...
Kids and Moral Agency
by jefftk — 2020-05-08T12:50:02.684Z

We were having some sort of
discussion where the idea of a "moral agent" came up, and

Lily asked what one was. I
described the idea, including something along the lines of "if
it hurts you is that it's fault?", and then asked her what counted.
Her answers, with my ordering:

Rock: no

Ocean: no

Tree: no

Ants: no

Pigs: no

Monkeys: no

Babies: sort of

Morgan (9m cousin): sort of

Felix (18m cousin): in between

Oli (3y cousin): yes

Anna (4y sister): yes

Ruthie: yes

Fairies: yes

Anna pretending to be a baby: yes

I'm curious whether adults would give the same answers, or whether
adults would typically consider moral agency to start later?
Failures in technology forecasting? A reply to Ord and Yudkowsky
by MichaelA — 2020-05-08T12:41:39.371Z

In The Precipice, Toby Ord writes:

we need to remember how quickly new technologies can be upon us, and to be wary of assertions that they are either impossible or so distant in time that we have no cause for concern. Confident denouncements by eminent scientists should certainly give us reason to be sceptical of a technology, but not to bet our lives against it - their track record just isn’t good enough for that.

I strongly agree with those claims, think they’re very important in relation to estimating existential risk,[1] and appreciate the nuanced way in which they’re stated. (There’s also a lot more nuance around this passage which I haven’t quoted.) I also largely agree with similar claims made in Eliezer Yudkowsky’s earlier essay There's No Fire Alarm for Artificial General Intelligence.

But both Ord and Yudkowsky provide the same set of three specific historical cases as evidence of the poor track record of such “confident denouncements”. And I think those cases provide less clear evidence than those authors seem to suggest. So in this post, I’ll:

Quote Ord and/or Yudkowsky’s descriptions of those three cases, as well as one case mentioned by Yudkowsky but not Ord

Highlight ways in which those cases may be murkier than Ord and Yudkowsky suggest

Discuss how much we could conclude about technology forecasting in general from such a small and likely unrepresentative sample of cases, even if those cases weren’t murky

I should note that I don’t think that these historical cases are necessary to support claims like those Ord and Yudkowsky make. And I suspect there might be better evidence for those claims out there. But those cases were the main evidence Ord provided, and among the main evidence Yudkowsky provided. So those cases are being used as key planks supporting beliefs that are important to many EAs and longtermists. Thus, it seems healthy to prod at each suspicious plank on its own terms, and update incrementally.

Case: Rutherford and atomic energy

Ord writes:

One night in 1933, the world’s pre-eminent expert on atomic science, Ernest Rutherford, declared the idea of harnessing atomic energy to be ‘moonshine’. And the very next morning Leo Szilard discovered the idea of the chain reaction.

Yudkowsky also uses the same case to support similar claims to Ord’s.

However, in a footnote, Ord adds:

[Rutherford’s] prediction was in fact partly self-d

...
A non-mystical explanation of "no-self" (three characteristics series)
by Kaj_Sotala — 2020-05-08T10:37:06.591Z

This is the second post of the "a non-mystical explanation of insight meditation and the three characteristics of existence" series. You can read the first post, explaining my general intent and approach, here.

On the three characteristics

So, just what are

the three characteristics of existence

?

My take is that they are a rough way of clustering the kinds of insights that you may get from insight meditation: in one way or another, most insights about the structure of your mind can be said to be about no-self, impermanence, unsatisfactoriness, or some combination of them. As upcoming posts should hopefully make obvious, this is not a very clear-cut distinction: the three are deeply intertwined with each other, and you can’t fully explain one without explaining the others. I am starting with a discussion of no-self, then moving to unsatisfactoriness, then coming back to no-self, moving between the different characteristics in a way that seems most clear.

I think that what’s called “enlightenment” refers to the gradual accumulation of these kinds of insights, combined with practices aimed at exploiting an understanding of them. There are many different insights and ways of exploring them, as well as many general

approaches for making use of them

. Different traditions also seem to have different enlightenments [

1

,

2

]. Thus, rather than providing any definitive explanation of “this is enlightenment”, I attempt to focus on exploring how various cognitive mechanisms behind different enlightenments work. My intent is to cover enough of different things to give a taste of what's out there and what kinds of outcomes might be possible, while acknowledging that there's also a lot that I have no clue of yet.

So this is not trying to be anything like “a definitive and complete explanation of the three characteristics”; I don’t think anyone could write such a thing, as nobody can have explored all the aspects of all the three. Rather, this is more of a sketch of those aspects of the three characteristics which I think I have some understanding of.

In particular, this explanation strongly emphasizes no-self and unsatisfactoriness, which I feel I have a better understanding of. Impermanence, which some approaches consider the very core characteristic, ends up relatively neglected. Apologies to any impermanence fans - maybe some day I’ll come back to write more about it.

But let’s...
What fraction of your lifetime (0-80 years old) egoist budget would you (want to) spend on a pill that made you live for as long as you wanted (perfect invincibility), as healthily as you wanted if you knew it would become available to you once you're 80 years old (and that you would otherwise irreversibly die)?
by Mati_Roy — 2020-05-08T08:50:40.815Z

Question type: poll

If you're a pure altruist, this question doesn't apply to you.

Number higher than 100% are possible if you'd borrow money, although it can become weird in scenarios where you stop being economically valuable and/or money becomes obsolete.

This question assumes you have an "egoist budget" separate from an "altruist budget" which might not be the case. If it's not the case, you can answer a similar question that works for you, such as "the fraction of your total budget".
What would you do differently if you were less concerned with looking weird?
by Mati_Roy — 2020-05-07T23:29:16.618Z

EtA 2020-09-16: ah dang, I just realized I had already asked that question here: https://www.lesswrong.com/posts/mengYutEGfzyeA6Xi/what-would-you-do-differently-if-you-were-less-concerned
Covid-19 5/7: Fighting Limbo
by Zvi — 2020-05-07T19:00:01.414Z

Last week: Covid-19 4/30: Stuck in Limbo

Recently: Covid-19: New York’s Antibody Tests 2, On “COVID-19 Superspreader Events in 28 Countries: Critical Patterns and Lessons”

Background Assumptions: On R0, Taking Initial Viral Load Seriously, On New York’s Antibody Tests, My Covid-19 Thinking: 4/23 pre-Cuomo Data

Spreadsheet where I do work: Access it here as read only

Deaths By Week in the 5 Big Regions:

WEST
MIDWEST
SOUTH
NE ex-NY
NY

Mar 26-Apr 1
164
450
182
143
364

Mar 19-Mar 25
424
1894
667
856
1988

Apr 2-8
764
3873
1331
2248
4694

Apr 9-15
890
4853
1596
3605
5318

Apr 16-22
1033
5189
1727
5165
3716

Apr 23-29
1128
5234
1694
4742
2666

Apr 30-May 6
988
5126
1740
4921
1862

Positive Tests By Week By Region:

WEST
MIDWEST
SOUTH
NE ex-NY
NY

Mar 26-Apr 1
7176
7812
9927
11923
33106

Mar 19-Mar 25
16665
22121
28412
37339
55123

Apr 2-8
19925
31460
38989
60433
67556

Apr 9-15
16291
29267
35570
62293
64463

Apr 16-22
20065
34130
33932
65080
43437

Apr 23-29
21873
42343
34683
62355
42475

Apr 30-May 6
23424
49205
37595
52232
24287

Tests each week (New York’s 5/6 negative rate assumed to be same as 5/5):

USA Pos%
USA ex-NY Pos %
NY Pos %
USA Tests
USA ex-NY Tests
NY Tests

Mar 26-Apr 1
16.2%
10.7%
32.0%
347577
258695
88882

Mar 19-Mar 25
20.2%
15.4%
45.1%
728474
611073
117401

Apr 2-8
19.8%
15.8%
45.5%
1067220
922947
144273

Apr 9-15
20.1%
16.5%
40.1%
1039790
878931
160859

Apr 16-22
15.7%
13.8%
30.2%
1253535
1109565
143970

Apr 23-29
13.7%
12.6%
21.0%
1480101
1277602
202499

Apr 30-May 6
10.6%
10.3%
13.3%
1733601
1550320
183281

Total Positive Tests:

USA Positives
USA Ex-NY Pos

Mar 26-Apr 1
56198
27769

Mar 19-Mar 25
146888
93987

Apr 2-8
211095
145491

Apr 9-15
209067
144604

Apr 16-22
197111
153674

Apr 23-29
202934
160459

Apr 30-May 6
184614
160327

New York: Rapidly Making It Here

The news keeps getting better, with each day seeming to be a more rapid decline in positive test rates and the implied R0. Last week I had R0 ~ 0.78, and now I have more like R0 ~ 0.74.

I believe that this reflects a real drop in the effective R0 over time. Some of that is shifts in behavior. Some of that is additional herd immunity, especially in the places it is most valuable. Some of that is selection, as the infection shifts to where it can better be contained. Some of that is that we have more resourc...
Meditation skill: Surfing the Urge
by Max Hodges — 2020-05-07T17:30:04.659Z

Surfing the Urge: A distress tolerance skill

I learned this meditation technique from my freediving coach. It's a technique we employ during a breath-hold to help us deal with an increasingly strong urge to breath.

Understand that the urge to breath is driven by the body’s desire to rid itself of carbon dioxide (CO2)--not (as some assume) your body's desire to take in oxygen (O2). The urge to breath is a warning, however. It can be safely ignore up to a certain limit. That limit, for our purposes, is the loss of consciousness (cerebral hypoxia); however, the urge to breath becomes quite strong well before your blood oxygen saturation low enough to cause a blackout.

Under normal conditions, the human body has a blood-oxygen saturation of around 98 to 100 percent (the higher number being the most oxygen that the blood could possibly contain). Physical stress or sickness can decrease oxygen saturation to about 95 percent. Few healthy people will ever go below this, but during dives, expert divers have registered oxygen-saturation levels as low as 50 percent, an extraordinarily low number. Oxygen saturations below 85 percent generally cause an increased heart rate and impaired vision; 65 percent and below greatly impairs basic brain functions; 55 percent results in unconsciousness. But well-trained divers have not only remained conscious with oxygen saturations of 50 percent but maintained muscle control and extremely low heart rates, reportedly as low as seven beats per minute.

Hold your breath too long, and you start to feel malaise. The urge to breath appears and it seems as if the malaise will only increase in intensity until you submit to releasing you breath and starting to breath again. But what happens if you don't give in and keep holding your breath? Ultimately static apnea (breath hold) is a matter of pure, mind-over-matter willpower.

After a while, along with the discomfort, you're start to experience involuntary contractions in your diagram. These is where the meditation technique comes in. When you feel a contraction, just observe it. As they increase in frequency, simple witness them as if you were a third-party.

At first, you may feel like these urges will rise up, stronger and stronger and become more and more unbearable, but if you simple observe and remain detached, you'll notice a pattern. The urges come in waves. ...
Kurzweil's predictions' individual scores
by Stuart_Armstrong — 2020-05-07T17:10:36.637Z

In my post on Kurzweil's 1999 predictions for 2019, I wrote:

The (anonymised) data can be found here, and I encourage people to download and assess it themselves.

But it is, of course, not trivial for everyone to download a spreadsheet and wade through it, and you might want something in between a 5-number summary and a full spreadsheet.

So I decided to post a more detailed summary here, that should give people an easier overview. This table will show the actual prediction, the mean (ie how true or false the predictor found it), the standard deviation (how much spread there was in predictions) and the proportion of "3", ie "Cannot Decide" answers. This data is also available as a tab-separated file here.
Multiple Arguments, Multiple Comments
by lc — 2020-05-07T09:30:17.494Z

This is partly a recommendation on how to format multiple different independent points or replies in any discussion forum, but it's also a post I'm going to use as a link to explain why I'm leaving comments all over the place so that I don't seem as weird/feel self conscious and don't have to explain it each time.

Oftentimes, when someone disagrees or simply wants to comment on something you've said, they'll have multiple independent critiques of it. The standard way I see this handled rhetorically is to quote your relevant material once, and write a long post, mixing those critiques together. Other times, someone will make multiple different points or arguments in favor of a single underlying idea, or even write completely unrelated paragraphs contained in the same post, that are replied to all at once, in quote->response, quote->response, quote->response format. This makes arguments hard to track as layers of->response->counter-response->counter-counter-response start to pile up each iteration of the comment chain.

To remedy this, when I can, I like to split up these independent points into multiple comments. This makes it much easier to follow the chain of argument, and to see which objections or justifications stand on their own, instead of hiding behind each other. It's much easier to see when this is done that someone doesn't reply, or gives up a particular idea. Even when people are correct, often most or all of their individual reasons for believing what they do are flawed, and even when they're incorrect, often the objections others raise are unsound, too. Leaving each argument naked to individual attack makes it easier to see, both for the writer and the commenter, that a paragraph is either silly or the main rhetorical pillar behind an idea or rebuttal.
[link] How many humans will have their brain preserved? Forecasts and trends
by Mati_Roy — 2020-05-07T06:11:00.176Z

http://lessdead.com/how-many-humans-will-have-their-brain-preserved-forecasts-and-trends

Summary:

Doubling time for the number of people that got cryopreserved has been pretty consistently 9 years since the beginning.
Prioritizing COVID-19 interventions & individual donations
by Ian David Moss — 2020-05-07T01:44:27.250Z

NOTE: The COVID-19 funding landscape is evolving rapidly, and we are making updates to reflect new developments and revised donation recommendations. Please check the EA Forum version of this post for the latest information.

Authors: Catherine Olsson and Ian David Moss, with contributions from the collective members of the "Funding Rational Actors Promptly" Pandemic Endowment (FRAPPE).

At the beginning of April, a group of friends pulled together a messenger chat to discuss how to most effectively spend personal donation funds towards mitigating global suffering caused by COVID-19. We're about 20 people, with somewhere in the ballpark of $200k to donate. We gave out one chunk of ~$50k in mid-April, and are currently in the process of distributing a second round of funds.

A defining motivation of our group was to find time-sensitive and neglected bottlenecks to effective COVID response that could be eased with rapid funding or other supportive actions. Fast action can be an important source of philanthropic leverage in responses to the current pandemic, a factor that we did not see explored in depth in available analyses of COVID-related giving opportunities. Accordingly, we have summarized our research here in hopes that others can use it to inform their own giving.

This article is organized in two parts. The first shares our working framework for prioritizing interventions, which helped us get oriented in a fast-changing and otherwise confusing landscape.

In the second part, we enumerate specific giving opportunities (jump to section) we have found that currently rate highly on this framework as of right now (early May 2020). One of these has a high minimum donation; if you would like to join a pooled gift, please reach out.

We've written this post primarily for the benefit of donors who have already decided to focus on COVID-19 for their own reasons. We haven't made it a priority to weigh the relative value of COVID-related donations as compared to other issues or causes, although we address this briefly at the end.

Some disclaimers: this research is being done and our donations are being made in a purely personal capacity, and none of us is acting as an employee, representative, or spokesperson of our employer or any other organization. Furthermore, because we don't have complete information on many opportunities and the situation i...
Specification gaming: the flip side of AI ingenuity
by Vika — 2020-05-06T23:51:58.171Z

(Originally posted to the Deepmind Blog)

Specification gaming is a behaviour that satisfies the literal specification of an objective without achieving the intended outcome. We have all had experiences with specification gaming, even if not by this name. Readers may have heard the myth of King Midas and the golden touch, in which the king asks that anything he touches be turned to gold - but soon finds that even food and drink turn to metal in his hands. In the real world, when rewarded for doing well on a homework assignment, a student might copy another student to get the right answers, rather than learning the material - and thus exploit a loophole in the task specification.

This problem also arises in the design of artificial agents. For example, a reinforcement learning agent can find a shortcut to getting lots of reward without completing the task as intended by the human designer. These behaviours are common, and we have collected around 60 examples so far (aggregating existing lists and ongoing contributions from the AI community). In this post, we review possible causes for specification gaming, share examples of where this happens in practice, and argue for further work on principled approaches to overcoming specification problems.

Let's look at an example. In a Lego stacking task, the desired outcome was for a red block to end up on top of a blue block. The agent was rewarded for the height of the bottom face of the red block when it is not touching the block. Instead of performing the relatively difficult maneuver of picking up the red block and placing it on top of the blue one, the agent simply flipped over the red block to collect the reward. This behaviour achieved the stated objective (high bottom face of the red block) at the expense of what the designer actually cares about (stacking it on top of the blue one).

SOURCE: DATA-EFFICIENT DEEP REINFORCEMENT LEARNING FOR DEXTEROUS MANIPULATION (POPOV ET AL, 2017)

We can consider specification gaming from two different perspectives. Within the scope of developing reinforcement learning (RL) algorithms, the goal is to build agents that learn to achieve the given objective. For example, when we use Atari games as a benchmark for training RL algorithms, the goal is to evaluate whether our algorithms can solve difficult tasks. Whether or not the agent solves the task by exploiting a loophole is unim

...
Tips/tricks/notes on optimizing investments
by Wei Dai — 2020-05-06T23:21:53.153Z

I've been optimizing various aspects of my investment setup recently, and will write up some tips and tricks that I've found in the form of "answers" here. Others are welcome to share their own here if they'd like. (Disclaimer: I’m not a lawyer, accountant, or investment advisor, and everything here is for general informational purposes only.)