trentmkelly/LessWrong-43k huggingface dataset:
Training Regime Day 14: Traffic Jams
by Mark Xu — 2020-02-28T17:52:28.354Z

Introduction

A traffic jam in a city is a situation where cars want to move but cannot, usually because they are impeded by other cars. Sometimes, the cars are impeded by an obstacle, like a crashed car or a broken road. Other times, cars are impeded by other cars that are impeded by other cars that are impeded by the first set of cards, resulting in gridlock. Gridlock is particularly interesting because all the cars want to move, but none of them can move because they're arranged in a way that makes them all stuck.

A traffic jam in your brain is a situation where you have a lot of thoughts, all of which want to move, but none of them can move. Internally, this might feel like being yanked in many directions, some sort of mental stuttering or maybe running in circles really fast or maybe like you're a wounded animal. Noticing traffic jams is a precondition for being able to resolve them.

A traffic jam might look something like "I feel kind of tired, so many I should shower, but I'm planning on going to the gym later, so I want to shower after I go to the gym, but I want to eat before I go to the gym - oh that reminds me, I have to pay my friend back for the dinner they bought me yesterday - oh yeah, I also have to pay my credit card bill and cancel my netflix subscription and I really should watch the last few episodes of Television Show, but I can't because I have to write my thesis - oh man I'm so worried by my research isn't good enough and I want to solve a bigger problem so I can impress my mentor and..."

Related concepts are decision paralysis and thrashing.

Traffic Jams

How do you stop a traffic jam? The zeroth step is to figure out you're in a traffic jam. I've listed some of my/others phenomenology above; try and find your own.

Brain Dump

The first step is to get a piece of paper/some computational device and brain dump everything that you want/are thinking/feeling. It is absolutely critical that the thing that you put the stuff onto isn't your brain. Note that you're not trying to generate a list of things that you should do, you're just trying to catalog all the things that you already want to do that are floating around in your brain.

For instance, a list for for the above might look something like:

shower

gym

eat

pay back friend for dinner

pay credit card bills

cancel netflix subscription

watch Television Show

write thesis

what if my research is ba

...
Stretching the Beat
by jefftk — 2020-02-28T16:30:02.356Z

Initially when I started playing around with having my rhythm setup

subdivide the beat I went with
a very simple division of four equal pieces. This works well for some
things, but it is also a very rigid sound. When I'm playing on the
mandolin I often stretch the beat a bit, pulling subbeats earlier and
pushing others later. This is something I learned to do by ear; I
just play it.

But what am I actually doing? I took a recording of myself playing
mandolin and measured exactly how far through the beat each of the
four subbeats fell. I measured three repetitions worth:

1
0.000
0.000
0.000

2
0.154
0.139
0.168

3
0.275
0.253
0.264

4
0.444
0.421
0.420

1
0.574
0.535
0.549

Converting these into percentages, and calculating an average:

1
00.0%
00.0%
00.0%
00.0%

2
26.9%
26.0%
30.6%
27.8%

3
47.9%
47.3%
48.1%
47.8%

4
77.4%
78.7%
76.5%
77.5%

Here's another way of looking at it:

strict
stretched
delta

2
25.0%
27.8%
11% late

3
50.0%
47.8%
9% early

4
75.0%
77.5%
10% late

Here's a video walking through how this sounds and how it fits into
the rest of my setup: youtube.

Comment via: facebook
Conclusion to 'Reframing Impact'
by TurnTrout — 2020-02-28T16:05:40.656Z

Epistemic Status

I've made many claims in these posts. All views are my own.

Elicit Prediction (elicit.org/binary/questions/7SoL5DPRf)

Elicit Prediction (elicit.org/binary/questions/AevXOS1Rj)

Confident (75%). The theorems on power-seeking only apply to optimal policies in fully observable environments, which isn't realistic for real-world agents. However, I think they're still informative. There are also strong intuitive arguments for power-seeking.

Elicit Prediction (elicit.org/binary/questions/javyyEd8C)

Fairly confident (70%). There seems to be a dichotomy between "catastrophe directly incentivized by goal" and "catastrophe indirectly incentivized by goal through power-seeking", although Vika provides intuitions in the other direction.

Elicit Prediction (elicit.org/binary/questions/iYT69bLl9)

Elicit Prediction (elicit.org/binary/questions/GFGG5plOQ)

Elicit Prediction (elicit.org/binary/questions/8Rr-YFSWi)

Elicit Prediction (elicit.org/binary/questions/GFurWKpJn)

Acknowledgements

After ~700 hours of work over the course of ~9 months, the sequence is finally complete.

This work was made possible by the Center for Human-Compatible AI, the Berkeley Existential Risk Initiative, and the Long-Term Future Fund. Deep thanks to Rohin Shah, Abram Demski, Logan Smith, Evan Hubinger, TheMajor, Chase Denecke, Victoria Krakovna, Alper Dumanli, Cody Wild, Matthew Barnett, Daniel Blank, Sara Haxhia, Connor Flexman, Zack M. Davis, Jasmine Wang, Matthew Olson, Rob Bensinger, William Ellsworth, Davide Zagami, Ben Pace, and a million other people for giving feedback on this sequence.

Appendix: Easter Eggs

The big art pieces (and especially the last illustration in this post) were designed to convey a specific meaning, the interpretation of which I leave to the reader.

There are a few pop culture references which I think are obvious enough to not need pointing out, and a lot of hidden smaller playfulness which doesn't quite rise to the level of "easter egg".

Reframing Impact

The bird's nest contains a literal easter egg.

The paperclip-Balrog drawing contains a Tengwar inscription which reads "one measure to bind them", with "measure" in impact-blue and "them" in utility-pink.

"Towards a New Impact Measure" was the title of the post in which AUP was introduc...
Refactoring EMH – Thoughts following the latest market crash
by Alex_Shleizer — 2020-02-28T15:53:28.298Z

In a twitter thread, Eliezer Yudkowsky challenges the belief in EMH due to the seemingly illogical behavior of the markets with respect to the COVID-19. In the replies, Robin Hanson asks the obvious question: “what exactly is your better theory?”

I've been thinking a lot about EMH in recent years and would like to propose a different formulation of EMH - a one that I believe resolves some of the obvious issues with the current theory while preserving many of its conclusions and underlying logic at large. I will also suggest in which situations profit can be made and why the market was indeed inefficient in the COVID-19 situation.

Consider the following question:

If you feed a powerful AGI with almost unlimited computation resources all the public data we have regarding stocks and businesses - will it be able to beat the market?

If you believe in EMH your answer should be a “no.” But this answer seems wrong.

It’s hard to believe that a god-like AGI wouldn’t be able to find any competitive advantage in analyzing the data given practically unlimited resources. The underlying cognitive process of beating the market is the same as any other sufficiently hard cognitive task, a search in highly complex multidimensional space.

Having large amounts of computing power should help you to achieve better results, even without private information.

But yet we have those empirical results that show that traders having a very hard time to beat the market using public data. So I would like to suggest a different version of EMH that solves both of these issues. And the rule can be phrased in the following way:

“In the long run, the costs of analyzing public data in order to obtain alpha will be equal to the gained alpha.”

Intuition:

Think of crypto mining as an equivalent, if you are able to mine crypto for less money than what it’s worth you can make a profit (equivalent to alpha). But your competitors will do the same – raising the difficulty of mining, so in the long run in a competitive market, you can expect the mining expenses will be very close to the value of the mined crypto.
In a similar way, companies that invest in the market spend money on analysts and experts to find inefficiencies using public data and as long this is profitable the market should hire analysts to analyze data until the cost of a

...
If I were a well-intentioned AI...
III: Extremal Goodhart
by Stuart_Armstrong — 2020-02-28T11:24:23.090Z

In this post, I’ll be looking at a more extreme version of the problem in the previous post.

The Extremal Goodhart problem is the fact that “Worlds in which the proxy takes an extreme value may be very different from the ordinary worlds in which the correlation between the proxy and the goal was observed.”

And one of the easiest ways to reach extreme values, and very different worlds, is to have a powerful optimiser at work. So forget about navigating in contained mazes: imagine me as an AI that has a great deal of power over the world.

Running example: curing cancer

Suppose I was supposed to cure cancer. To that end, I’ve been shown examples of surgeons cutting out cancer growths. After the operation, I got a reward score, indicating, let’s say, roughly how many cancerous cells remained in the patient.

Now I’m deployed, and have to infer the best course of action from this dataset. Assume I have three main courses of action open to me: cutting out the cancerous growths with a scalpel, cutting them out much more effectively with a laser, or dissolving the patient in acid.

Apprenticeship learning

Cutting out the growth with a scalpel is what I would do under apprenticeship learning. I’d be just trying to imitate the human surgeons as best I can, reproducing their actions and their outcomes.

This is a relatively well-defined problem, but the issue is that I can’t do much better than a human. Since the score gives a grading to the surgeries, I can use that information as well, allowing me to imitate the best surgeries.

But I can’t do much better than that. I can only do as well as the mix of the best features of the surgeries I’ve seen. Nevertheless, apprenticeship learning is the “safe” base from which to explore more effective approaches.

Going out of distribution, tentatively

Laser vs acid

Laser surgery is something that gives me a high score, in a new and more effective way. Unfortunately, dissolving the patient in acid is also something that fulfils the requirements of getting rid of the cancer cells, in a new and (much) more effective way. Both methods involve me going off-distribution compared with the training examples. Is there any way of distinguishing between the two?

First, I could get the follow-up data on the patients (and the pre-operation data, though that is less relevant). I want to get a distribution of the outcomes of the operation - make sure tha

...
Epistea Workshop Series: Epistemics Workshop, May 2020, UK
by Nora_Ammann — 2020-02-28T10:37:34.229Z

Hi all!

We are excited to announce the Epistea Epistemics Workshop! This is the first out of a series of 3 workshops happening in Europe, organized and run by the Epistea team.

Tl,dr:

What: a 3.5 day workshop focusing on improving our epistemics and habits of belief formation
When: May 14th (Thu) - May 18th (Sun) (With optional and recommended stay until Monday morning)
Where: in proximity to Oxford, UK
Price: 400£ (In case you hesitate to apply because of financial issues, let us know.)
Who are we: the

Epistea team

and external instructors
Application deadline: Sun, 15th of March

Sign up

here!

For more information, visit our

website

or send us an email at

rationality@epistea.org

.

We are looking forward to spending some exciting workshop days with you!

Best,

The Epistea team
Understandable vs justifiable vs useful
by MichaelA — 2020-02-28T07:43:06.123Z

This is a quickly written post, just sharing a rough conceptual distinction that I’ve found useful in everyday life and that I think others might find useful too.

Your boss just expected you to stay late yesterday. Your partner just shot you a pretty damn sharp comment. Someone just cut you off in traffic.

Are these actions intrinsically, incomprehensibly evil?

Or can you at least comprehend what led these people to such sins, despite there clearly being no adequate justification for them?

Or are these actions in some sense justified - perhaps from some sort of common sense or deontological perspective - but still just not helpful or not what should’ve been done?

For another example, maybe you’re about to shoot your partner a pretty damn sharp comment. Anyone would surely understand why you’d say it, in the circumstances - and it seems fair enough, really, given what they just said about The Event, which you’ve already apologised for. But is saying it going to make either of you any happier, or just spark another spiral upwards through the decibel charts?

Since I was a teenager, I’ve regularly leaned on a rough distinction between understandable, justifiable, and useful actions. I’m not sure when I “came up with it”, or if I stole it from somewhere. It’s also very possible everyone already uses something like this distinction themselves. But in case not everyone does, I thought I’d share the distinction, as I’ve found it very helpful.

I won’t define my terms; their standard definitions work quite well here, and I’ll let examples do the rest.

Examples

Let’s say someone’s complaining about something their boss did. I might respond with something like:

To be honest, I do think I can understand why your boss did that, because of incentive X and norm Y. But, I still think it was uncalled for and unfair on you [unjustifiable], and that it just made things worse [not useful].

Or I might say:

To be honest, I do think I can understand why your boss did that, because of incentive X and norm Y. In fact, I can even see how, from a certain angle, it might have been ‘fair enough’, given past-bad-behaviour-of-yours Z [justifiable]. But I still think it was just fighting fire with fire and making things worse [not useful], and I can see why it’s pissed

...
Coronavirus: Justified Practical Advice Thread
by Ben Pace — 2020-02-28T06:43:41.139Z

(Added: To see the best advice in this thread, read this summary.)

This is a thread for practical advice for preparing for the coronavirus in places where it might substantially grow.

We'd like this thread to be a source of advice that attempts to explain itself. This is not a thread to drop links to recommendations that don’t explain why the advice is accurate or useful. That’s not to say that explanation-less advice isn’t useful, but this isn't the place for it.

Please include in your answers some advice and an explanation of the advice, an explicit model under which it makes sense. We will move answers to the comments if they don't explain their recommendations clearly. (Added: We have moved at least 4 comments so far.)

The more concrete the explanation the better. Speculation is fine, uncertain models are fine; sources, explicit models and numbers for variables that other people can play with based on their own beliefs are excellent.

Here are some examples of things that we'd like to see:

It is safe to mostly but not entirely rely on food that requires heating or other prep, because a pandemic is unlikely to take out utilities, although if if they are taken out for other reasons they will be slower to come back on
CDC estimates of prevalence are likely to be significant underestimates due to

their narrow testing criteria

.

A guesstimate model

of the risks of accepting packages and delivery food

One piece of information that has been lacking in most advice we’ve seen is when to take a particular action. Sure, I can stock up on food ahead of time, but not going to work may be costly– what’s your model for the costs of going so I can decide when the costs outweigh the benefits for me? This is especially true for advice that has inherent trade-offs– total quarantine means eating your food stockpiles that you hopefully have, which means not having them later.
Draft: Models of Risks of Delivery Under Coronavirus
by Elizabeth — 2020-02-28T04:10:01.310Z

I’ve never considered prophylactically quarantining myself before, but now that I’m considering it I find it contains many more choices than I would have imagined. Let’s take my need to eat- I could go to a supermarket, but that’s full of people. I could get delivery, but that still has a human touch. I could eat my stores, but then I won’t have them later. This makes “when do I stop ordering delivery?” an important question. To attempt a more informed answer, I made a guesstimate model. As of writing this (2/27) the numbers are completely made up: I just wanted to get comments on the underlying model. I’m working to fill in the variables with actual answers. If you want to follow along you can do so at my Roam page. I am exceedingly grateful for comments on either the abstract model or information that could help me fill in variables.

Here are some general factors going into my thinking:

COVID-19 seems to have a long dormant period during which people are contagious but not symptomatic

Some additional portion of people have only mild symptoms

The economics of pink-collar work are such that a lot of people will go to work until they are on death’s door.

1+2+3 = if the virus is prevalent in the population, there will be a lot of contagious people handling stuff I order.

The American government’s monitoring provides, at best, an extremely lagging indicator of prevalence, and is at worst made up.

Here are images of the model and Roam page now, for posterity

Note that this shows food delivery as less risky than package delivery, which is clearly wrong.
How to Write a News article on the Dangers of Artificial General Intelligence
by  — 2020-02-28T02:14:48.419Z

Inspired by this response.

I noticed that a lot of news articles/op-eds on AGI follow the same scheme. Not sure if this has been written about before so I'm just going to put it out there.

1. The strawman

"Let me explain to you how AIs today are totally not AGI by pointing out all the simple things even a human child can do with ease that AIs struggle with. Have I mentioned we don't have real AGIs yet? How silly of you to still think that we already have AGI! No, really, I assure you AlphaGo is definitely not an AGI. It's not even as smart as a five year old/lab rat/retarded cockroach."

2. Impress me.

"Talk to me once you've solved natural language processing/causal reasoning/AI doctors/*insert any AGI-complete problem*. Anything before that is just fear-mongering/sensationalism"

3. Serious problems for serious people

In the mean time, we need to focus on real issues serious researchers worry about, such as:

Trolley problems for self-driving cars (How convenient that human drivers are too slow to make a decision anyway)
Adversarial examples for image classifiers (But when humans fail, it's because the image is genuinely confusing, you see)
Pranksters altering traffic signs to confuse self-driving cars (How about I just steal the sign? Checkmate.)
Racist AIs (Not gonna touch that issue with a ten-yard pole)

Of course this is just a caricature of AI articles in the news media not to be taken too seriously; there are also plenty of articles with more nuance.

(Someone train GPT-2 to write AGI news articles)
Reasons for Excitement about Impact of Impact Measure Research
by TurnTrout — 2020-02-27T21:42:18.903Z

Can we get impact measurement right? Does there exist One Equation To Rule Them All?

I think there’s a decent chance there isn’t a simple airtight way to implement AUP which lines up with AUPconceptual, mostly because it’s just incredibly difficult in general to perfectly specify the reward function.

Reasons why it might be feasible: we’re trying to get the agent to do the goal without it becoming more able to do the goal, which is conceptually simple and natural; since we’ve been able to handle previous problems with AUP with clever design choice modifications, it’s plausible we can do the same for all future problems; since there are a lot of ways to measure power due to instrumental convergence, that increases the chance at least one of them will work; intuitively, this sounds like the kind of thing which could work (if you told me “you can build superintelligent agents which don’t try to seek power by penalizing them for becoming more able to achieve their own goal”, I wouldn’t exactly die of shock).

Even so, I am (perhaps surprisingly) not that excited about actually using impact measures to restrain advanced AI systems. Let’s review some concerns I provided in Reasons for Pessimism about Impact of Impact Measures:

Competitive and social pressures incentivize people to cut corners on safety measures, especially those which add overhead. Especially so for training time, assuming the designers slowly increase aggressiveness until they get a reasonable policy.

In a world where we know how to build powerful AI but not how to align it (which is actually probably the scenario in which impact measures do the most work), we play a very unfavorable game while we use low-impact agents to somehow transition to a stable, good future: the first person to set the aggressiveness too high, or to discard the impact measure entirely, ends the game.

In a What Failure Looks Like-esque scenario, it isn't clear how impact-limiting any single agent helps prevent the world from "gradually drifting off the rails".

You might therefore wonder why I’m working on impact measurement.

Deconfusion

Within Matthew Barnett’s breakdown of how impact measures could help with alignment, I'm most excited about impact measure research as deconfusion. Nate Soares explains:

By deconfusion, I mean something like “making it so that you can think about a given topic without continuously accidental

...
Abstract Plans Lead to Failure
by Chris_Leong — 2020-02-27T21:20:11.554Z

Epistemic Status: Probably not generalisable to everyone, applies especially strong to me

It's very easy to decide to do something in the abstract on the basis that it should give you some particular skills. Here are some personal examples where I didn't get what I wanted out of my time:

Taking a class on art history
Reading a bunch of articles on philosophy
Studying a degree in psychology
Watching a bunch of short videos about history
Reading a book on human rights

These failure were basically my fault as I either just consumed the information or crammed any study I did do. Obviously I knew that this was less than ideal.

What I didn't realise was that I'd get basically nothing out of these activities. That I wouldn't be any better about talking about art, that I'd end up with only vague recollections of most of the philosophers, that I wouldn't have much more insight into psychology than someone who spent their time on Reddit, ect.

This isn't just limited to intellectual activities. I had the same experience with Salsa where I did a lot of classes, but never really put time into the basic skills. I learned more moves, but my basic skills really didn't improve.

The assumption I made in each of these cases was, "Surely doing X makes people better at Y, so I just have to do X and I'll get better at Y, maybe not as much as if I did Z, but it should still have a reasonable effect".

However, in my experience it doesn't work like that. You'll likely think it's working at the time, but any gains are unlikely to persist.

Instead, it's important to be iterative. Instead I suggest the following: Do something, see where you are improving and where you are not and then adjust. Repeat until you get the results you want.

This probably sounds stupidly obvious, but these things are easy to overlook when you are inside a situation.