trentmkelly/LessWrong-43k huggingface dataset:
Propagating Facts into Aesthetics
by Raemon — 2019-12-19T04:09:17.816Z

Epistemic status: Tentative. I’ve been practicing this on-and-off for a year and it’s seemed valuable, but it’s the sort of thing I might look back on and say “hmm, that wasn’t really the right frame to approach it from.”

In doublecrux, the focus is on “what observations would change my mind?”

In some cases this is (relatively) straightforward. If you believe minimum wage helps workers, or harms them, there are some fairly obvious experiments you might run. “Which places have instituted minimum wage laws? What happened to wages? What happened to unemployment? What happened to worker migration?”

The details will matter a lot. The results of the experiment might be

weird and confusing

. If I ran the experiment myself I’d probably get a lot of things wrong, misuse statistics and forget to account for some confounding factors. But I don’t feel confused about how to learn better statistics, account for more confounders, etc.

But there’s a problem that seems harder to me, which is how to change my mind about aesthetics. Sarah Constantin first brought this up in

Naming the Nameless

, and I’ve been thinking about it ever since.

I think a lot of deep disagreements have to do with “what is beautiful, and what is ugly?”, and inability to directly address this is part of what prevents those disagreements from resolving.

In the case of the minimum wage example, you might run an experiment, and find overwhelming evidence that minimum wage helps or hurts workers. But because there’s lots of confounders, the evidence might be mixed and confusing. How you interpret it will depend on how it fits into your existing worldview.

Part of this has to do with your ontological frame. But I think a lot has to do with aesthetics judgments, such as:

Is capitalism ugly and/or distasteful? You might have very salient examples of how capitalism can result in exploitation, pollution, or people becoming trapped in unhealthy power structures.
Is capitalism beautiful? Alternately, it might be salient that capitalism creates supermarkets, gains from trade, and vast surplus. Economic efficiency isn’t just pretty numbers on a graph, it’s real value being created.

These don’t directly bear on the minimum wage question, but might make it harder to resolve.

In some cases, your aesthetic taste might make it harder to update on new information properly. In other cases, your aesthetic taste might help y...
What's going on with this failure of Bayes to converge?
by Orborde — 2019-12-19T03:58:13.883Z
Link: https://a-point-in-tumblspace.tumblr.com/post/189588000957/bayes-trubs-part-1

There are circumstances (which might only occur with infinitesimal probability, which would be a relief) under which a perfect Bayesian reasoner with an accurate model and reasonable priors – that is to say, somebody doing everything right – will become more and more convinced of a very wrong conclusion, approaching certainty as they gather more data.

(click through the notes on that post to see some previous discussion)

I have two major questions:

1. Is this exposition correctly capturing Freedman's counterexample?

2. If using a uniform prior sometimes breaks, what prior should I be using, and, more importantly, how do I arrive at that prior?
2019 AI Alignment Literature Review and Charity Comparison
by Larks — 2019-12-19T03:00:54.708Z

Cross-posted to the EA forum here.

Introduction

As in

2016

,

2017

and

2018

, I have attempted to review the research that has been produced by various organisations working on AI safety, to help potential donors gain a better understanding of the landscape. This is a similar role to that which GiveWell performs for global health charities, and somewhat similar to a securities analyst with regards to possible investments.

My aim is basically to judge the output of each organisation in 2019 and compare it to their budget. This should give a sense of the organisations' average cost-effectiveness. We can also compare their financial reserves to their 2019 budgets to get a sense of urgency.

I’d like to apologize in advance to everyone doing useful AI Safety work whose contributions I may have overlooked or misconstrued. As ever I am painfully aware of the various corners I have had to cut due to time constraints from my job, as well as being distracted by 1) another existential risk capital allocation project, 2) the miracle of life and 3) computer games.

How to read this document

This document is fairly extensive, and some parts (particularly the methodology section) are the same as last year, so I don’t recommend reading from start to finish. Instead, I recommend navigating to the sections of most interest to you.

If you are interested in a specific research organisation, you can use the table of contents to navigate to the appropriate section. You might then also want to Ctrl+F for the organisation acronym in case they are mentioned elsewhere as well.

If you are interested in a specific topic, I have added a tag to each paper, so you can Ctrl+F for a tag to find associated work. The tags were chosen somewhat informally so you might want to search more than one, especially as a piece might seem to fit in multiple categories.

Here are the un-scientifically-chosen hashtags:

Agent Foundations
AI_Theory
Amplification
Careers
CIRL
Decision_Theory
Ethical_Theory
Forecasting
Introduction
Misc
ML_safety
Other_Xrisk
Overview
Philosophy
Politics
RL
Security
Shortterm
Strategy

New to Artificial Intelligence as an existential risk?

If you are new to the idea of General Artificial Intelligence as presenting a major risk to the survival of human value, I recommend

this Vox piece

by Kelsey Piper.

If you are already convinced and are interested in contributing technically, I recommend ...
One Million Dollars
by jefftk — 2019-12-19T02:10:02.034Z

On Giving Tuesday, Julia and I

donated our
millionth dollar to

GiveWell's top charities
and

growing the Effective
Altruism movement. It was more dramatic than you might have guessed,
since we were trying to donate as close as possible to 8:00:00 to get
the

FB donation
match.

I remember when we were starting out thinking that if we kept at this
$1M might be a
good lifetime goal; it's really exciting that we've gotten there
within ten years! More than that, however, what I really didn't expect
was how much of a community there now is around EA. In September 2011
we hosted our first EA meetup; now we're having them about once a
month. Most of the universities around us have EA student groups, and
some of the students come to our meetups. There are EA conferences,
which Julia helps organize, and I went to ones in 2015, 2016, and 2017. The EA Forum has a lot of
interesting discussion, and I've met many good friends via talking
about EA ideas.

I don't know whether I'll keep earning to give long term. It's a good
fit for my situation now, but the question of giving vs doing is still a really
important one. So while I'm tempted to set a new long-term goal, I
think that would risk pushing a future me towards staying in earning
to give even if that started making less sense.

Comment via: facebook
"You can't possibly succeed without [My Pet Issue]"
by Raemon — 2019-12-19T01:12:15.502Z

There's a particular conversational move that I've noticed people making over the past couple years. I've also noticed myself making it. The move goes:

"You can't possibly succeed without X", where X is whatever principle the person is arguing for.

(Where "succeed" means "have a functioning rationality community / have a functioning organization / solve friendly AI / etc")

This is not always false. But, I am pretty suspicious of this move.

(I've seen it from people from a variety of worldviews. This is not a dig on any one particular faction from local-politics. And again, I do this myself).

When I do the move, my current introspective TAP goes something like: "Hmm. Okay, is this actually true? Is it impossible to succeed without my pet-issue-of-the-day? Upon reflection, obviously not. I legit think it's harder. There's a reason I started caring about my pet-issue in the first place. But 'impossible' is a word that was clearly generated by my political rationalization mind. How much harder is it, exactly? Why do I believe that?")

In general, there are incentives (and cognitive biases) to exaggerate the importance of your plans. I think this is partly for political reasons, and partly for motivational reasons – it's hard to get excited enough about your own plans if you don't believe they'll have outsized effects. (A smaller version of this, common on my web development team, is someone saying "if we just implemented Feature X we're get a 20% improvement on Metric Y", and the actual answer was we got, like, a 2% improvement, and it was worth it. But, like, the 20% figure was clearly ridiculous).

"It's impossible" is an easier yellow-flag to notice than "my numbers are bigger than what other people think are reasonable". But in both cases, I think it's a useful thing to train yourself to notice, and I think "try to build an explicit quantitative model" is a good immune response. Sometimes the thing is actually impossible, and your model checks out. But I'm willing to bet if you're bringing this up in a social context where you think an abstract principle is at stake, it's probably wrong.
Abstraction, Causality, and Embedded Maps: Here Be Monsters
by johnswentworth — 2019-12-18T20:25:04.584Z

In a lot of examples of abstract causality, the abstract causal structure generally mirrors the ground-level causal structure. Think about fluid flow, for instance. At the (classical) ground level, we have a bunch of particles whose interactions are local in time and space. Then we glom together a bunch of particles, abstract it all into the Navier-Stokes equations, and we see… interactions which are local in time and space. If we picture the whole thing as a causal DAG, the abstract DAG (Navier-Stokes) is basically just a glommed-together version of the ground DAG (particle interactions), with some information thrown out.

But once the system contains embedded maps and controllers, that all goes out the window. Abstract causal structure can be completely different from ground-level causal structure.

This post is just a few examples of the weird stuff which can happen. We’ll run through four examples:

A model in which an embedded map impersonates another variable
A model in which an embedded controller impersonates another variable
A thermostat, as an example of feedback control making a system behave as if it follows a different causal model
A counterfactual-detector: a node in a causal model which detects whether a counterfactual is running on the model

Map-Impersonator

First, a simple example with an embedded map. We have some background variables B, which are upstream of an “embedded territory” X. We’ll assume that almost all the variation in X is accounted for by B (although B may be high-dimensional, and the X(B) function may be quite complicated).

A map sees all the background information B, and then tries to estimate the value of X (or at least some summary information about X). If the map node has a sufficiently accurate model of the function X(B), then it can act-as-if it were reading from X directly, rather than just inferring X.

Out-of-the-box, the abstract causal model is not perfect - it will not correctly predict how the map responds to counterfactual changes in X. However, it will make other counterfactual predictions correctly: counterfactual changes in B or the map, as well as correlational queries, will all produce answers matching the answers on the ground model.

We can make the abstract model support counterfactual surgery on X by mapping it to counterfactual surgery on both X and the map in the ground level model....
Against Premature Abstraction of Political Issues
by Wei Dai — 2019-12-18T20:19:53.909Z

A few days ago romeostevensit wrote in response to me asking about downvotes on a post:

I didn’t down­vote, but I do think that con­ver­sa­tions like this at­tract peo­ple who aren’t in­ter­ested in ar­gu­ing in good faith. I pre­fer that such dis­cus­sions oc­cur at one ab­strac­tion level up so that they don’t need to men­tion any ob­ject level be­liefs like so­cial jus­tice in or­der to talk about the pat­tern that the au­thor wants to talk about.

And I replied:

This seems like a rea­son­able worry. Maybe one way to ad­dress it would be to make posts tagged as “poli­tics” (by ei­ther the au­thor or a mod­er­a­tor) visi­ble only to logged in users above a cer­tain karma thresh­old or speci­fi­cally ap­proved by mod­er­a­tors. Talk­ing at the meta-level is also good, but I think at some point x-risk peo­ple have to start dis­cussing ob­ject-level poli­tics and we need some place to prac­tice that.

Since writing that, I've had the thought (because of this conversation) that only talking about political issues at a meta level has another downside: premature abstraction. That is, it takes work to find the right abstraction for any issue or problem, and forcing people to move to the meta level right away means that we can't all participate in doing that work, and any errors or suboptimal choices in the abstraction can't be detected and fixed by the community, leading to avoidable frustrations and wasted efforts down the line.

As an example, consider a big political debate on LW back in 2009, when "a portion of comments here were found to be offensive by some members of this community, while others denied their offensive nature or professed to be puzzled by why they are considered offensive." By the time I took my shot at finding the right abstraction for thinking about this problem, three other veteran LWers had already tried to do the same thing. Now imagine if the object level issue was hidden from everyone except a few people. How would we have been able to make the intellectual progress necessary to settle upon the right abstraction in that case?

One problem that exacerbates premature abstraction is that people are often motivated to talk about a political issue because they have a strong intuitive position on it, and when they find what they think is the right abstraction for thinking about it, they'll rationalize an argument for their position within th

...
Inductive biases stick around
by evhub — 2019-12-18T19:52:36.136Z

This post is a follow-up to Understanding “Deep Double Descent”.

I was talking to Rohin at NeurIPS about my post on double descent, and he asked the very reasonable question of why exactly I think double descent is so important. I realized that I hadn't fully explained that in my previous post, so the goal of this post is to further address the question of why you should care about double descent from an AI safety standpoint. This post assumes you've read my Understanding “Deep Double Descent” post, so you should read that first before reading this if you haven't already.

Specifically, I think double descent demonstrates the in my opinion very important yet counterintuitive result that larger models can actually be simpler than smaller models. On its face, this sounds somewhat crazy—how can a model with more parameters be simpler? But in fact I think this is just a very straightforward consequence of double descent: in the double descent paradigm, larger models with zero training error generalize better than smaller models with zero training error because they do better on SGD's inductive biases. And if you buy that SGD's inductive biases are approximately simplicity, that means that larger models with zero training error are simpler than smaller models with zero training error.

Obviously, larger models do have more parameters than smaller ones, so if that's your measure of simplicity, larger models will always be more complicated, but for other measures of simplicity that's not necessarily the case. For example, it could hypothetically be the case that larger models have lower Kolmogorov complexity. Though I don't actually think that's true in the case of K-complexity, I think that's only for the boring reason that model weights have a lot of noise. If you had a way of somehow only counting the “essential complexity,” I suspect larger models would actually have lower K-complexity.

Really, what I I'm trying to do here is dispel what I see as the myth that as ML models get more powerful simplicity will stop mattering for them. In a Bayesian setting, it is a fact that the impact of your prior on your posterior (for those regions where your prior is non-zero[1]) becomes negligible as you update on more and more data. I have sometimes heard it claimed that as a consequence of this result, as we move to doing machine learning with ever larger datasets and ever bigger

...
[AN #77]: Double descent: a unification of statistical theory and modern ML practice
by Rohin Shah — 2019-12-18T18:30:01.862Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

Deep Double Descent

(Preetum Nakkiran et al) (summarized by Rohin): This blog post provides empirical evidence for the existence of the double descent phenomenon, proposed in an earlier paper summarized below. Define the effective model complexity (EMC) of a training procedure and a dataset to be the maximum size of training set such that the training procedure achieves a train error of at most ε (they use ε = 0.1). Let's suppose you start with a small, underparameterized model with low EMC. Then initially, as you increase the EMC, the model will achieve a better fit to the data, leading to lower test error. However, once the EMC is approximately equal to the size of the actual training set, then the model can "just barely" fit the training set, and the test error can increase or decrease. Finally, as you increase the EMC even further, so that the training procedure can easily fit the training set, the test error will once again decrease, causing a second descent in test error. This unifies the perspectives of statistics, where larger models are predicted to overfit, leading to increasing test error with higher EMC, and modern machine learning, where the common empirical wisdom is to make models as big as possible and test error will continue decreasing.

They show that this pattern arises in a variety of simple settings. As you increase the width of a ResNet up to 64, you can observe double descent in the final test error of the trained model. In addition, if you fix a large overparameterized model and change the number of epochs for which it is trained, you see another double descent curve, which means that simply training longer can actually correct overfitting. Finally, if you fix a training procedure and change the size of the dataset, you can see a double descent curve as the size of the dataset decreases. This actually implies that there are points in which more data is worse, because the training procedure is in the critical interpolation region where test error can increase. Note that most of these results only occur when ther...
The Vostok Ice Cores Revisited pt. II
by Lymphatera — 2019-12-18T11:50:04.365Z

Speculation, suggestion and mystery

Causes and effects of climate change

[Continued from The Vostok Ice Cores pt. 1: An Investigation]

Finding that atmospheric CO2 levels and surface temperatures on Earth (the average of which, according to this graph, seems to have quite regularly leveled off -regardless of partially more than tenfold higher than present carbon dioxide levels- at around 25 °C absolute, or 10-15 °C higher than today; which in itself is quite a lot) correlate during less than half of the observable time period of 500 million years, but do so very well under current and more recent conditions (i. e. with both values historically low, absolutely, and the relative difference to today near zero), it can be speculated as to why.

There are many possible factors, and it should help to eliminate those that are not recent and look for those that are.

There seems to be open dissent on which regions of the world are (and therefore were) most affected by climate change; and perhaps one should not so much focus on the direct effects of absolute temperatures, which are in dispute, but rather on the amount of heat or energy stored in the environment, which can be indirectly measured by the amount of ice in a region - where there is ice; atmospheric water vapor would be an indicator where there is not. These would, in either direction, buffer changes in temperature.

Then there are the oceans, absorbing and releasing carbon dioxide, dependent on temperature, and probably a host of other factors. Especially the thawing and freezing in the intermediate zones of this planet will have a direct effect on the amount of plant, and with that, animal life; both of these then have an effect on the amount of climate-affecting gases retained in the atmosphere.

In other words, global 'warming' or 'cooling' can take place without a rise or fall in temperature, but still affect all living systems; global thawing should not be confused with global warming. That will follow.

However, twice a year, the hemispheres of this planet alternately experience an externally induced short-term climate change, from winter to summer and back, where CO2 levels rise with the cold in winter (by about 20 ppm as far as can be discerned) due to the lower temperatures causing the levels of photosynthesis to diminish, and fall in summer due to the higher temperatures enhancing photosynthesis - the oppo...
The Vostok Ice Cores Revisited pt. I
by Lymphatera — 2019-12-18T11:42:47.243Z

An Investigation

Introduction

It's impressive how fast the focus in things climate change has recently shifted from Thunberg back to Milankovic again in just a few weeks.

Even more impressive is the fact that, though astronomic or cosmic cycles and positions can be predicted with utmost precision, weather and climate on Earth still and most obviously can not. It seems there are many other factors involved; like - well, perhaps, indeed, the makeup of a planet's atmosphere.

So, the following is premised on the assumption that, just as does the Sun's intensity, the position of Earth in Space, and even its own geothermal heat, atmospheric carbon dioxide can and does influence this planet's surface temperature; at least it is doing so under the current circumstances.

If it doesn't, then other explanations have to be found for the phenomena observed.

If it does, the question is how, and how much; the increase in temperature induced by atmospheric CO2, as given by different authors, lies somewhere in the range of +0 °C to +30 °C, which almost renders any discussion useless.

The following, in parts 1 & 2, is therefore hypothetical in its attempt. It presents one possible way of interpreting the depictions of the data as found in public.

CO2: Photosynthesis and Climate Change

The following shall be discussed here:

- There has been a natural climate fluctuation for at least a million years

- This could, at least in part, be biological by cause
- Humans have most possibly played a role in the latest stages

There has been much discussion about climate change, and most of it refers in some way or another to the Vostok ice cores and similar evaluations.

Under the premise that the data gleaned from these is correct, their interpretation is correct (which is by no means certain, as the recent controversy of Mann vs. Ball shows), and their publicly available depiction in the graphs referred to below as well as other public statements are correct (which is by no means certain either), it would seem to follow that for the last million years, at least, the Earth's climate (rather: it's mean average surface temperature) has not only constantly changed, but that this change was a) confined in range and b) quite regular in nature.

During this time, the publicly acknowledged culprit behind climate change, the amount of carbon dioxide in the ...