trentmkelly/LessWrong-43k huggingface dataset:
Preface to the sequence on economic growth
by Matthew Barnett — 2020-08-27T20:29:24.517Z

On Lesswrong, when we talk about artificial intelligence, we tend to focus on the technical aspects, such as potential designs, specific developments, and future capabilities. From an engineering perspective, this focus makes sense. But most people here aren't interested in artificial intelligence because they want to know how AI will be designed; the reason we're here is because AI has the potential to radically reshape the world around us.

Longtermists have often emphasized the role economic growth plays as perhaps the most important phenomena of human history. In a quite real sense, economic growth is what distinguishes 21st century humanity from our distant ancestors who had no technology or civilization. Nick Bostrom summarizes this point well,

You could argue that if we look back over history, there have really only been two events that have fundamentally changed the human condition, the first being the Agricultural Revolution some 10,000 or 12,000 years ago in Mesopotamia, where we transitioned from being hunter-gatherers, small bands roaming around, to settling into cities, growing, domesticating crops and animals. [...]

The second fundamental change in the human condition, Industrial Revolution, where for the first time, you have the rate of economic and technological growth outstripping population growth, and so only when this happens can you have an increase in average income. Before that, there was technological growth and economic growth, but the economy grew 10%, the population grew 10%, everybody's still in a Malthusian condition.

Many theorists anticipate that there will be a third fundamental change in the human condition, roughly timed with the development of advanced artificial intelligence. In line with these predictions, economic growth is the primary specific benchmark people have used to characterize potential future AI takeoff.

If economic growth is the essential variable we should pay most attention to when it comes to AI, then our understanding of AI takeoff will be woefully incomplete without a grasp of what drives economic growth in the first place. To help mitigate this issue, in this sequence I will explore the underpinnings of modern economic growth theory, and then try to relate economic theory to AI developments. In doing so, I aim to identify crucial pieces of information that may help answer questions like,

How muc

...
Notes on "The Anthropology of Childhood"
by juliawise — 2020-08-27T17:11:19.205Z

Crossposted from The Whole Sky.

I read David Lancy’s “

The Anthropology of Childhood: Cherubs, Chattel, and Changelings

” and highlighted some passages. A lot of passages, it turns out.

[content note: discussion of abortion and infanticide, including infanticide of children with disabilities, in “Life and Death” section but not elsewhere]

I was a sociology major and understood anthropology to be basically “like sociology, but in Papua New Guinea.” This is the first cultural anthropology book I’ve read, and that was pretty much right. I found it very accessible as a first dive into anthropology. The first chapter summarizes all his points without the examples, so you could try that if you want to get the gist without reading the whole book.

I enjoyed it and would recommend it to people interested in this topic. A few things that shifted for me:

I feel less obliged to entertain my children and intervene in their conflicts. We don’t live with a tribe of extended family, but my two children play with each other all day, which is how most people throughout time have spent their childhoods. Lancy isn’t a child development expert, but I buy his argument that handling conflict (for example about the rules of a game) is a skill children need to learn, rather than having conflicts always mediated by adults.
Even though it doesn’t change anything concrete, I feel some relief that not having endless patience for toddlers seems to be normal. Except where families were very isolated, it’s not normal in traditional societies for one or two adults to watch their own children all day every day. And childcare has traditionally looked mostly like “being sure they don’t hurt themselves too badly.”
It surprised me that childcare by non-parents was so common. Some more modern views treat women’s childcare work as basically free, but traditional cultures have valued women’s labor enough that the society wants to free up their time from childcare. It was striking to me that the expectation that stay-at-home mothers will be responsible for all childcare was a relatively short historical blip. But of course, having childcare done by teenagers and grandmothers requires that those people’s time be available, which usually isn’t the reality we live in.
I was surpris

...
Covid 8/27: The Fall of the CDC
by Zvi — 2020-08-27T14:10:01.636Z

Most weeks, the disaster that was the head of the FDA not having any understanding of statistics and not making any attempt to think about the world would have been the headline. Then the CDC decided to revise its guidelines on testing from being for it to largely being against it, under pressure from the White House, and suddenly it’s the B-story.

It seems that every day there is a new thing surfacing to enrage me. The difference is that early in the pandemic, every day something would terrify me. I’m still periodically scared in an existential or civilization-is-collapsing-in-general kind of way, but not in a ‘the economy is about to collapse’ or ‘millions of Americans are about to die’ kind of way.

I’m not sure whether this is progress.

Either way, a reminder that I’ve started a sports, sports gambling and sports modeling substack to avoid cluttering up this blog, so check it out here if you have yet to do so and that is relevant to your interests.

That’s out of the way. Let’s run the numbers.

Positive Test Counts

Date	WEST	MIDWEST	SOUTH	NORTHEAST
June 11-June 17	41976	22510	75787	17891
June 18-June 24	66292	26792	107221	15446
June 25-July 1	85761	34974	163472	16303
July 2-July 8	103879	40139	202863	18226
July 9-July 15	108395	53229	250072	20276
July 16-July 22	117506	57797	265221	20917
July 23-July 29	110219	67903	240667	26008
July 30-Aug 5	91002	64462	212945	23784
Aug 6-Aug 12	93042	61931	188486	21569
Aug 13-Aug 19	80887	63384	156998	20857
Aug 20-Aug 26	67545	66540	132322	18707

Apologies for the lack of charts last week – it seems they didn’t copy properly from Google docs to WordPress and I didn’t notice.

Positive test counts are declining rapidly in the West and South, and slowly in the Northeast. Before, I would have considered that strong evidence that things are going great in those areas. Now, with testing on the decline, it’s not that simple, and we’ll have to dig into the positive test percentages and deaths to know for sure.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST
June 11-June 17	778	1040	1207	1495
June 18-June 24	831	859	1204	1061
June 25-July 1	858	658	1285	818
July 2-July 8	894	559	1503	761
July 9-July 15	1380	539	2278	650
July 16-July 22	1469	674	3106	524
July 23-July 29	1707	700	4443	568
July 30-Aug 5	1831	719	4379	365
Aug 6-Aug 12	1738	663	4554	453
Aug 13-Aug 19	1576	850	4264	422
Aug 20-Aug 26	1503	745	3876	375

Deaths continue to tell the story that the South and West have turned the corner, the Northeast is still making steady pr...
Technical model refinement formalism
by Stuart_Armstrong — 2020-08-27T11:54:22.534Z

This post has been mainly superceeded by this one.

Introduction

This post aims to formalise the models and model changes/model splintering described in this post. As explained there, the idea is to have a meta-model sufficiently general to be able to directly capture the process of moving from one imperfect model to another.

A note on infinity

For simplicity of exposition, I'll not talk about issues of infinite sets, continuity, convergence, etc... Just assume that any infinite set that comes up is just a finite set, large enough for whatever practical purpose we need it for.

Features, worlds, environments

A model M is defined by three object, the set F of features, the set E of environments, and a probability distribution Q. We'll define the first two in this section.

Features

Features are things that might be true or not about worlds, or might take certain values in worlds. For example, "the universe is open" is a possible feature about our universe, "the temperature is 250K" is another possible feature, but instead of returning true or false, it returns the temperature value. Adding more details, such as "the temperature is 250K in room 3, at 12:01" show that features should also be able to take inputs: features are functions.

But what about "the frequency of white light"? That's something that makes sense in many models - white light is used extensively in many contexts, and light has a frequency. The problem with that statement is that light has multiple frequencies; so we should allow features to be, at least in some cases, multivalued functions.

To top that off, sometimes there will be no correct value for a function; "the height of white light" is something that doesn't mean anything. So features have to include partial functions as well.

Fortunately, multivalued and partial functions are even simpler than functions at the formal level: they are just relations. And since the sets in the relations can consist of a single element, in even more generality, a feature is a predicate on a set. We just need to know which set.

So, formally, a feature F∈F consists of an (implicit) label defining what F is (eg "open universe", "temperature in some location") and a set on which it is a predicate. Thus, for example, the features above could be:

Fopen universe={0} (features which are simply true or false are functions of a single element).

Ftemperature={R+}.

Ftemper

...
Model splintering: moving from one imperfect model to another
by Stuart_Armstrong — 2020-08-27T11:53:58.784Z

1. The big problem

In the last few months, I've become convinced that there is a key meta-issue in AI safety; a problem that seems to come up in all sorts of areas.

It's hard to summarise, but my best phrasing would be:

Many problems in AI safety seem to be variations of "this approach seems safe in this imperfect model, but when we generalise the model more, it becomes dangerously underdefined". Call this model splintering.

It is intrinsically worth studying how to (safely) transition from one imperfect model to another. This is worth doing, independently of whatever "perfect" or "ideal" model might be in the background of the imperfect models.

This sprawling post will be presenting examples of model splintering, arguments for its importance, a formal setting allowing us to talk about it, and some uses we can put this setting to.

1.1 In the language of traditional ML

In the language of traditional ML, we could connect all these issues to "out-of-distribution" behaviour. This is the problems that algorithms encounter when the set they are operating on is drawn from a different distribution than the training set they were trained on.

Humans can often see that the algorithm is out-of-distribution and correct it, because we have a more general distribution in mind than the one the algorithm was trained on.

In these terms, the issues of this post can be phrased as:

When the AI finds itself mildly out-of-distribution, how best can it extend its prior knowledge to the new situation?

What should the AI do if it finds itself strongly out-of-distribution?

What should the AI do if it finds itself strongly out-of-distribution, and humans don't know the correct distribution either?

1.2 Model splintering examples

Let's build a more general framework. Say that you start with some brilliant idea for AI safety/alignment/effectiveness. This idea is phrased in some (imperfect) model. Then "model splintering" happens when you or the AI move to a new (also imperfect) model, such that the brilliant idea is undermined or underdefined.

Here are a few examples:

You design an AI CEO as a money maximiser. Given typical assumptions about the human world (legal systems, difficulties in one person achieving massive power, human fallibilities), this results in an AI that behaves like a human CEO. But when those assumptions fail, the AI can end up feeding the universe to a money-making p

...
Are We Right about How Effective Mockery Is?
by Ronny Fernandez — 2020-08-27T10:48:18.288Z

Crossposted from Figuring Figuring.

I did a second survey that fixed some of the flaws of the first survey. The results from the second survey significantly color the interpretation of the results from the first survey given in the first “Conclusion and Discussion” section. Please continue reading past the section titled “Second Survey” to get a full picture of the results from all surveys.

Intro

A couple days ago a friend of mine on facebook asked about arguments in favor of mockery. They pointed out that they had noticed a lot of facebook posts mocking people for not wearing masks in the covid-19 era, and wondered whether this was an effective way to change people’s behaviors.

I said in the comment section of that post that I would make a survey that worked as follows. Roughly half of the survey takers would be randomly assigned to answer the following questions:

Do you think that mockery is an effective way to change people’s minds?
Do you think that mockery is an effective way to change people’s behaviors?

The other half would be randomly assigned to answer these questions:

Has being mocked ever caused you to change your mind about something?
Has being mocked ever caused you to change your habits or behaviors?

No survey respondent was permitted to see all four questions. The possible answers to each question were “Yes”, “No”, and “Not sure”.

I made this survey using GuidedTrack. I posted it on my facebook wall, and also posted it to Positly and paid people to participate.

A total of 145 people responded to any of the questions on positly. 74 were asked the first set of questions, and 71 were asked the second set of questions. A total of 66 people responded to any of the questions on facebook. 31 were asked the first set of questions, and 35 were asked the second set of questions.

Before I go on to tell you the results of the survey and the predictions me and some of my friends made, you might want to make your own predictions. I suggest you quickly scribble them down. Some particular questions you might want to make predictions about:

Did more people answer yes to the first set of questions than the second set of questions, or is the reverse true, or were they about the same?
Were facebook respondents (presumably people who are friends, or friends of friends of mind on facebook) more or less likely to say yes to the first set of questions?
Were facebook respondent

...
Basic Inframeasure Theory
by Diffractor — 2020-08-27T08:02:06.109Z

a

Our task in this post will be to develop the basic theory and notation for inframeasures and sa-measures. The proofs and concepts require some topology and functional analysis. We assume the reader is familiar with topology and linear algebra but not functional analysis, and will explain the functional analysis concepts more. If you wish to read through these posts, PM me to get a link to MIRIxDiscord, we'll be doing a group readthrough where I or Vanessa can answer questions. Here's the previous post and here are the proof sections. Beware, the proof sections are hard.

Notation Reference

Feel free to skip this segment and refer back to it when needed. Duplicate the tab, keep one of them on this section, and you can look up notation here.

X,Y: some compact metric space. Points in this are denoted by x or y.

d: Some distance metric.

M±(X): The topological vector space of finite signed measures over X equipped with the weak topology. A sequence of measures mn converges to m in the weak topology iff, for all bounded continuous functions f of type X→R, ∫Xf(x)dmn limits to ∫Xf(x)dm. Elements are denoted by m. By Jordan decomposition, we can uniquely split it into m++m− where the former is all-positive and the latter is all-negative.

C(X),C(X,[0,1]): The Banach space of continuous functions X→R. The latter one is the space of continuous functions bounded in [0,1]. Elements of C(X,[0,1]) are typically denoted by f.

m(f): We can interpret signed measures as continuous linear functionals on C(X). This is given by ∫Xf(x)dm. If m was actually a probability distribution, this would just be Eμ(f). They're generalized expectations.

b: used to refer to the number component of an a-measure or sa-measure.

Msa(X): The closed convex cone of sa-measures. An sa-meaure is a pair (m,b) where b+m−(1)≥0. Elements of this (sa-measures) are denoted by M.

f+: A positive functional. A continuous linear functional that's nonnegative for all sa-measures.

B: A set of sa-measures.

EB(f): The expectation of a function f relative to a set of sa-measures. Defined as inf(m,b)∈B(m(f)+b).

Bmin,Buc: The set of minimal points of B (points that can't be written as a different point in B plus a nonzero sa-measure), and the upper...
Belief Functions And Decision Theory
by Diffractor — 2020-08-27T08:00:51.665Z

So, if you haven't read the first two posts, do so now.

In this post, we'll be going over the basic theory of belief functions, which are functions that map policies to sets of sa-measures, much like how an environment can be viewed as a function that maps policies to probability distributions over histories. Also, we'll be showing some nifty decision theory results at the end. The proofs for this post are in the following three posts (1,2,3), though it's inessential to read them and quite difficult.

Now, it's time to address desideratum 1 (dynamic consistency), and desideratum 3 (how do we formalize the Nirvana trick to capture policy selection problems) from the first post. We'll be taking the path where Nirvana counts as infinite reward, instead of counting as 1 reward forever. The latter option is cleaner mathematically, ties into learning theory better, and makes it clearer what a pseudocausal hypothesis is. But this post was long enough as-is, so we'll just be presenting the path where Nirvana counts as infinite reward, as that allows for more flexibility in encoding policy selection problems and a full tie-in with UDT.

Definition 1: Nirvana

A special observation N which may occur at any time, counts as infinite reward, and brings the history to an end. An a-measure or sa-measure may be described as Nirvana-free when the set of histories ending in Nirvana has measure 0. The set of Nirvana-free a-measures is denoted as NF.

Our ambient mathematical setting is one where: At each point, there are finitely many discrete actions or observations available (no continuous observation spaces here, though the available space of actions or observations may vary depending on the history so far). In any situation, Nirvana may occur, and it ends the history. All policies are deterministic (refer back to the first post to see why this isn't much of an imposition).

We'll be working with bounded inframeasures (like a bounded infradistribution in all ways except normalization), instead of inframeasures, because it gets harder if we don't. Also, we'll be working within the cone of a-measures, but use the notion of upper-completion that uses the cone of sa-measures. This odd restriction is in place because Nirvana doesn't interact well with negative measures, as that would be −∞ reward and Murphy minimizes your reward, which force...
Proofs Section 1.1 (Initial results to LF-duality)
by Diffractor — 2020-08-27T07:59:12.512Z

Fair upfront warning: This is not a particularly readable proof section (though much better than Section 2 about belief functions). There's dense notation, logical leaps due to illusion of transparency since I've spent a month getting fluent with these concepts, and a relative lack of editing since it's long. If you really want to read this, I'd suggest PM-ing me to get a link to MIRIxDiscord, where I'd be able to guide you through it and answer questions.

Proposition 1: If f∈C(X,[0,1]) then f+:(m,b)↦m(f)+b is a positive functional on Msa(X).

Proof Sketch: We just check three conditions. Linearity, being nonnegative on Msa(X), and continuity.

Linearity proof. Using a,a′ for constants,

f+(a(m,b)+a′(m′,b′))=f+(am+a′m′,ab+ab′)=(am+a′m′)(f)+ab+a′b′

=a(m(f)+b)+a′(m′(f)+b′)=af+(m,b)+a′f+(m′,b′)

So we have verified that f+(aM+a′M′)=af+(M)+a′f+(M′) and we have linearity.

Positivity proof: An sa-measure M, writeable as (m,b) has m uniquely writeable as a pair of finite measures m+ (all the positive regions) and a m− (all the negative regions) by the Jordan Decomposition Theorem, and b+m−(1)≥0. So,

f+(M)=m(f)+b=m+(f)+m−(f)+b≥0+m−(1)+b≥0

The first ≥ by 1≥f≥0, so the expectation of f is positive and m− is negative so taking the expectation of 1 is more negative. The second ≥ is by the condition on how m− relates to b.

Continuity proof: Fix a sequence (mn,bn) converging to (m,b). Obviously the b part converges, so now we just need to show that mn(f) converges to m(f). This is immediate because "mn(f) converges to m(f) for all bounded continuous f" is the definition of mn weakly converging to m.

Theorem 1: Every positive functional on Msa(X) can be written as (m,b)↦c(m(f)+b), where c≥0, and f∈C(X,[0,1])

Proof Sketch: The first part is showing that it's impossible to have a positive functional where the b term doesn't matter, without the positive functional being the one that maps everything to 0. The second part of the proof is recovering our f by applying the positive functional to Dirac-delta measures δx, to see what the function must be on point x.

Part 1: Let's say f+ ...
Proofs Section 1.2 (Mixtures, Updates, Pushforwards)
by Diffractor — 2020-08-27T07:57:27.622Z

The previous proofs are here.

Proposition 5: If Bmin⊆Ma(X), then the condition "there is a λ⊙ where, ∀(λμ,b)∈Bmin:λ≤λ⊙" is equivalent to "there is a compact C s.t. Bmin⊆C"

Proof sketch: One direction is immediate from the Compactness Lemma. For showing that just a bound on the λ values suffices to be contained in a compact set, instead of a bound on the λ and b values to invoke the Compactness Lemma, we use a proof by contradiction where we can get a bound on the b values of the minimal points from just a bound on the λ values.

Proof: In one direction, assume there's a compact C s.t. Bmin⊆C, and yet there's no upper-bounding λ⊙ on the λ values. This is impossible by the Compactness Lemma, since (λμ)+(1)=λμ+(1)=λμ(1)=λ.

In the other direction, assume there's a λ⊙ bound on λ for the minimal points. Fix some arbitrary (λμ,b)∈Bmin for the rest of the proof. Now, we will show that all minimal points (λ′μ′,b′)∈Bmin have λ′≤λ⊙, and b′≤λ⊙+b, letting us invoke the Compactness Lemma to get that everything is in a suitable compact set C. The first bound is obvious. Since λ′ came from a minimal point, it must have λ⊙ as an upper bound.

For the other one, by contradiction, let's assume that there's a minimal point (λ′μ′,b′) where b′>λ⊙+b. Then, we can write (λ′μ′,b′) as: (λμ,b)+(−λμ,λ⊙)+(λ′μ′,b′−λ⊙−b)

The first component, (λμ,b) is our fixed minimal point of interest. The second component is an sa-measure, because λ⊙−λ≥0, due to the λ⊙ upper bound on the λ value of minimal points. The third component is also a nonzero sa-measure, because λ′ is nonnegative (it came from a minimal point), and by assumption, b′>λ⊙+b. Hang on, we wrote a minimal point (λ′μ′,b′) as another minimal point (λμ,b), plus two sa-measures (one of which is nonzero), so (λ′μ′,b′) can't be minimal, and we have a contradiction.

Therefore, all (λ′μ′,b′)∈Bmin have b′≤λ⊙+b. Now that we have bounds on λ and b for minimal points, we can invoke the Compactness Lemma to conclude that everything is in a compact set.

Proposition 6: EB(0)...