trentmkelly/LessWrong-43k huggingface dataset:
Anatomy of a Gear
by johnswentworth — 2020-11-16T16:34:44.279Z

Thankyou to Sisi Cheng (of the Working as Intended comic) for the excellent drawings.

When modelling a physical gearbox, why do we chunk the system into gears? We’re essentially treating each gear as a black box, a discrete component without internal structure of its own, interesting mainly for its interactions with other components. Why not zoom out, and model the entire gearbox as a black box? Why not zoom in, and model the individual chunks of metal which comprise each gear, or even each individual atom?

It seems like gears are a natural choice of abstraction for a gearbox. But why that choice? In general, what makes a good “gear” - a good lowest-level component in a

“gears-level” model

?

Why choose “gears” as the components of our model? Why chunk the world that way, rather than chunk multiple gears into subsystems, or break up gears into even smaller subsystems?

More generally: we want to build gears-level models, but we don’t want to model things down to the last atom. At some point, we have to accept some black boxes in our model components. So how do we decide where that point is?

This post answers that question. We’ll start with a relatively-detailed discussion of a physical gear, then extend the reasoning to “gears” in other kinds of models.

A Physical Gear

Picture a gear, turning in a gearbox. We can pick out a little patch of metal on that gear, zoom in, and see lots of atoms vibrating around. Those vibrations aren’t completely random - vibrations of one atom are correlated with the atoms next to it, although the correlations do typically fall off over a short distance. If we look at the motions of all the atoms in one little chunk of the gear, it won’t tell us much about the motions of the atoms in some other chunk far away in the gear.

… with one exception. If we average together the motion of all the atoms in our little chunk, we should get a decent estimate of the overall rotation speed of the gear. And that does tell us something about all the other little chunks of metal: it tells us roughly what the average motion in all those other chunks is.

If we look at all the atoms in one little chunk of one gear, only the average motion of all the atoms will tell us much about motion of atoms in a neighboring gear.
Caution: physical accuracy not a priority in this visual.

Likewise, if we look at the motion of all the individual atoms in one little chunk of our ...
How Roodman's GWP model translates to TAI timelines
by Daniel Kokotajlo — 2020-11-16T14:05:45.654Z

How does

David Roodman’s world GDP model

translate to TAI timelines?

Now, before I go any further, let me be the first to say that I don’t think we should use this model to predict TAI. This model takes a very broad outside view and is thus inferior to models like

Ajeya Cotra’s

which make use of more relevant information. (However, it is still useful for rebutting claims that TAI is unprecedented, inconsistent with historical trends, low-prior, etc.) Nevertheless, out of curiosity I thought I’d calculate what the model implies for TAI timelines.

Here is the projection made by Roodman’s model. The red line is real historic GWP data; the splay of grey shades that continues it is the splay of possible futures calculated by the model. The median trajectory is the black line.

I messed around with a ruler to make some rough calculations, marking up the image with blue lines as I went. The big blue line indicates the point on the median trajectory where GWP is 10x what is was in 2019. Eyeballing it, it looks like it happens around 2040, give or take a year. The small vertical blue line indicates the year 2037. The small horizontal blue line indicates GWP in 2037 on the median trajectory.

Thus, it seems that between 2037 and 2040 on the median trajectory, GWP doubles. (One-ninth the distance between 1,000 and 1,000,000 is crossed, which is one-third of an order of magnitude, which is about one doubling).

This means that TAI happens around 2037 on the median trajectory according to this model, at least according to

Ajeya Cotra’s definition of transformative AI

as “software which causes a tenfold acceleration in the rate of growth of the world economy (assuming that it is used everywhere that it would be economically profitable to use it)... This means that if TAI is developed in year Y, the entire world economy would more than double by year Y + 4.”

What about the non-median trajectories? Each shade of grey represents 5 percent of the simulated future trajectories, so it looks like there’s about a 20% chance that GWP will be near-infinite by 2040 (and 10% by 2037). So, perhaps-too-hastily extrapolating backwards, maybe this means about a 20% chance of TAI by 2030 (and 10% by 2027).

At this point, I should mention that I disagree with this definition of TAI; I think the point of no return

(which is what matters for planning)

is reasonably likely to come several y...
Brigaded Rounds
by jefftk — 2020-11-16T01:50:05.191Z

A group of

round
singers were interested in trying out the

bucket brigade singing program, and
I realized that with a small tweak it could support rounds directly.
We just needed to write audio both at the place it belonged to, and an
appropriate distance into the future. This would let everyone hear
everyone else, though the leader still has to wait a bit before they
start to hear others.

The system already needs to know BPM (beats per minute) so it can run
a metronome, but the support around it also needs to know BPR (beats
per repeat). For example, "Row, Row, Row, Your Boat" is a 16 beat
round with participants entering every four beats, so you would
probably set BPR to be 16. Four (or more) people could sing, with one
person leading and each other person joining after the right number of
beats as if they were singing together in person. Once everyone has
been singing for 16 beats they all hear everyone else, and also
themself.

You can get interesting effects by setting the BPR to some other
multiple of the part interval, such as 12. This lets you sing along
with yourself an offset, so that you appear to be one of the other
singing voices.

It's also possible to configure the system with multiple repeats, so
you would hear each iteration of your voice, say, three times. This
can be fun if you have a small group of people and want to sing many
parts, or even if you're playing around by yourself.

One thing this made me realize is that the latency calibration is
really very important, and the previous way we were handling it was
not sufficient. Originally, it started off trying to get a very
accurate estimation of your latency, but if it wasn't doing very well
it would slowly lower its standards. After about 30 seconds, it would
let in people even with pretty inaccurate estimates. I've updated it
now to be very careful, and if it can't estimate your latency to
within 2 ms, it lets you know and gives you the choice between trying
again, and joining without sending any of your audio to the server.

Feel free to play with it: echo.jefftk.com.

Comment via: facebook
When socializing, to what extent does walking reduce the risk of contracting Covid as opposed to being stationary?
by Adam Zerner — 2020-11-16T00:39:30.182Z

I would expect that when stationary, the aerosol particles have more time to accumulate whereas when walking they don't, and thus walking would offer a pretty nice reduction of risk. Along the same lines, I would expect jogging and biking to offer an even larger risk reduction.
Announcing the Forecasting Innovation Prize
by ozziegooen — 2020-11-15T21:12:39.009Z

Motivation

There is already a fair amount of interest around Effective Altruism in judgemental forecasting. We think there’s a whole lot of good research left to be done.

The valuable research seems to be all over the place. We could use people to speculate on research directions, outline incentive mechanisms, try novel forecasting questions with friends, and outline new questions that deserve forecasts. Some of this requires a fair amount of background knowledge, but a lot doesn’t.

The EA and LW communities have a history of using

prizes

to encourage work in exciting areas. We’re going to try one in forecasting research. If this goes well, we’d like to continue and expand this going forward.

Prize

This prize will total $1000 between multiple recipients, with a minimum first place prize of $500. We will aim for 2-5 recipients in total. The prize will be paid for by the

Quantified Uncertainty Research Institute

(QURI).

Rules

To enter, first make a public post online between now and Jan 1, 2021. We encourage you to either post directly or make a link post to either LessWrong or the EA Forum. Second, complete

this form

, also before Jan 1, 2021.

Research Feedback

If you’d like feedback or would care to discuss possible research projects, please do reach out! To do so, fill out

this form

. We’re happy to advise at any stages of the process.

Judges

The judges will be

AlexRJL

,

Nuño Sempere

,

Eric Neyman

,

Tamay Besiroglu

,

Linch Zhang

and

Ozzie Gooen

. The details of the judging process will vary depending on how many submissions we get. We’ll try to select winners for their importance, novelty, and presentation.

Some Possible Research Areas

Areas of work we would be excited to see explored:

Operationalizing questions in important domains so that they can be predicted in e.g., Metaculus. This is currently a significant bottleneck; it’s surprisingly difficult to write good questions. Examples in the past have been the

Ragnarök

or the

Animal Welfare series

. A possible suggestion might be to try to come up with forecastable

fire alarms

for AGI. Tamay Besiroglu has suggested a “S&P 500 but for AI forecasts,” i.e., a group of forecasting questions which track something useful for AI (or for other domains.)
Small experiments where you and/or a group of people use forecasting for your own decision making, and write up what you’ve learned. For example, set up a

Foretold

...
Notes on Loyalty
by David Gross — 2020-11-15T19:30:05.025Z

This post examines the virtue of loyalty. It is meant mostly as an exploration of what other people have learned about this virtue, rather than as me expressing my own opinions about it, though I’ve been selective about what I found interesting or credible, according to my own inclinations. I wrote this not as an expert on the topic, but as someone who wants to learn more about it. I hope it will be helpful to people who want to know more about this virtue and how to nurture it.

What is loyalty?

These are two senses of loyalty:

loyalty as reciprocity — Α is loyal to Β in this way because either Α has reason to feel gratitude to Β for some previously-granted favor, or expects to be able to get such a favor from Β at some time in the future. “You pulled the thorn from my paw, and now I shall certainly come to your aid.”
loyalty as partiality — Α is loyal to Β in this way by reliably making the promotion and defense of Β’s interests a priority to Α when that would not be the case absent the loyalty. “I would never rat out a fellow member of the Water Buffalo lodge.”

Loyalty may suggest both “I’ll be in your corner” (unlike other non-loyal people), and “I’ll be in your corner” (not in your adversary’s), but sometimes one more than the other.

Phrases like that one that describe loyalty commonly include descriptions of body language and relative positions of bodies: “I won’t turn my back on you,” “Now I know where you stand,” “I’ve got your back,” “He stayed by her side through it all.”

Issues of loyalty provide the story arc of many a popular classic — for example movies like Casablanca, Yojimbo / A Fistful of Dollars, or the Star Wars films (will Annakin go over to the dark side? will Han abandon his comrades in their hour of need?). Betrayal and false loyalty define many a classic villain (e.g. Macbeth). Loyalties that make incompatible demands have been a staple of tragedy at least since Sophocles and the Mahabharata. We seem to take particular interest in stories that involve shifts in loyalty, hidden loyalties being uncovered, loyalties being put to the test, the disloyal getting their comeuppance, and that sort of thing. This suggests that careful attention to the loyalties of those around us may have been an important skill to have in the history of our species.

Synonyms and related virtues

The words “faithfulness” and “fidelity” are sometimes used more-or-less syn...
Spend twice as much effort every time you attempt to solve a problem
by Jsevillamol — 2020-11-15T18:37:24.372Z

In brief: in order to iteratively solve problems of unknown difficulty a good heuristic is to double your efforts every time you attempt it.

Imagine you want to solve a problem, such as getting an organization off the ground or solving a research problem. The problem in question has a difficulty rating d, which you cannot know in advance.

You can try to solve the problem as many times as you want, but you need to precommit in advance how much effort e you want to put into each attempt - this might be for example because you need to plan in advance how you are going to spend your time.

We are going to assume that there is little transfer of knowledge between attempts, so that each attempt succeeds iff the amount of effort you spend on the problem is greater than its difficulty rating e>d.

The question is - how much effort should you precommit to spend on each attempt so that you solve the problem spending as little effort as possible?

Let's consider one straightforward strategy. We first spend 1 unit of effort, then 2, then 3, and so on.

We will have to spend in total 1+2+⋯+d=12(d+1)d∈O(d2). The amount of effort spent scales quadratically with the difficulty of the problem.

Can we do better?

What if we double the effort spent each time?

We will have to spend 1+2+4+⋯+2⌈log2d⌉=2⋅2⌈log2d⌉−1≤4d−1∈O(d). Much better than before! The amount of effort spent now scales linearly with the difficulty of the problem.

In complexity terms, you cannot possibly do better - even if we knew in advance the difficulty rating we would need to spend d∈O(d) to solve it.

Is doubling the best scaling factor? Maybe if we increase the effort by 1.5 each time we will spend less total effort.

In general, if we scaled our efforts by a factor b then we will spend in total 1+b+b2+⋯+b⌈logbd⌉=b⌈logbd⌉+1−1b−1≤b2d−1b−1. This upper bound is tight at infinitely many points, and its minimum is found for b=1+√d−1d - which asymptotically approaches 2 as d grows. Hence 2 seems like a good heuristic scaling factor without additional information on the expected difficulty. But this matters less than the core principle of increasing your effort exponentially.

Total effort spent vs scaling factor, for difficulties 10, 1e4, 1e9. The effort zigzags up and

...
A guide to Iterated Amplification & Debate
by Rafael Harth — 2020-11-15T17:14:55.175Z

This post is about two proposals for aligning AI systems in a scalable way:

Iterated Distillation and Amplification (often just called 'Iterated Amplification'), or IDA for short,[1] is a proposal by Paul Christiano.

Debate is an IDA-inspired proposal by Geoffrey Irving.

This post is written to be as easy to understand as possible, so if you found existing explanations of IDA confusing, or if you just never bothered because it seemed intimidating, this post is for you. The only prerequisite is knowing about the concept of outer alignment (and knowing about inner alignment is helpful as well). Roughly,

Outer alignment is aligning the training signal or training data we give to our model with what we want.

If the model we find implements its own optimization process, then inner alignment is aligning [the thing the model is optimizing for] with the training signal.

See also this post for an overview and this paper or my ELI12 edition for more details on inner alignment.

1. Motivation / Reframing AI Risk

Why do we need a fancy alignment scheme?

There has been some debate a few months back about whether the classical arguments of the kind made in Superintelligence for why AI is dangerous hold up to scrutiny. I think a charitable reading of the book can interpret it as primarily defending one claim, which is also an answer to the leading question. Namely,

It is hard to define a scalable training procedure that is not outer-misaligned.

For example, a language model (GPT-3 style) is outer-misaligned because the objective we train for is to predict the most likely next word, which says nothing about being 'useful' or 'friendly'. Similarly, a question-answering system trained with Reinforcement Learning is outer-misaligned because the objective we train for is 'optimize how much the human likes the answer', not 'optimize for a true and useful answer'.

I'll refer to this claim as (∗). If (∗) true, it is a problem even under the most optimistic assumptions. For example, we can suppose that

progress is gradual all the way, and we can test everything before we deploy it;

we are likely to maintain control of AI systems (and can turn them off whenever we want to) for a while after they exceed our capabilities;

it takes at least another 50 years for AI to exceed human capabilities across a broad set of tasks.

Even then, (∗) remains a problem. The only way to bui...
Beware Experiments Without Evaluation
by Davis_Kingsley — 2020-11-15T08:18:33.830Z

Sometimes, people propose "experiments" with new norms, policies, etc. that don't have any real means of evaluating whether or not the policy actually succeeded or not.

This should be viewed with deep skepticism -- it often seems to me that such an "experiment" isn't really an experiment at all, but rather a means of sneaking a policy in by implying that it will be rolled back if it doesn't work, while also making no real provision for evaluating whether it will be successful or not.

In the worst cases, the process of running the experiment can involve taking measures that prevent the experiment from actually being implemented!

Here are some examples of the sorts of thing I mean:

Management at a company decides that it's going to "experiment with" an open floor plan at a new office. The office layout and space chosen makes it so that even if the open floor plan proves detrimental, it will be very difficult to switch back to a standard office configuration.
The administration of an online forum decides that it is going to "experiment with" a new set of rules in the hopes of improving the quality of discourse, but doesn't set any clear criteria or timeline for evaluating the "experiment" or what measures might actually indicate "improved discourse quality".
A small group that gathers for weekly chats decides to "experiment with" adding a few new people to the group, but doesn't have any probationary period, method for evaluating whether someone's a good fit or removing them if they aren't, etc.

Now, I'm not saying that one should have to register a formal plan for evaluation with timelines, metrics, etc. for any new change being made or program you want to try out -- but you should have at least some idea of what it would look like for the experiment to succeed and what it would look like for it to fail, and for things that are enough of a shakeup more formal or established metrics might well be justified.
Examples of Measures
by Maxwell Peterson — 2020-11-15T01:44:39.593Z

I recently started learning measure theory but had a pretty hard time finding real-world examples of the different measures. So this is a list of real-world examples of some different, common measures.

The count measure

If I have 4 apples, their count measure is 4. The count measure is just normal counting.

The Dirac measure

You are camping in a forest with some trees. You've set up your tent and it was a lot of work, so you're not going to move. It's daytime. The trees cast shadows, but the tree cover is sparse enough that there's still some sunlight that hits the ground. As the day goes on, the shadows move, at times covering your camp, at times leaving you in the sun. The dirac measure for whether a shadow is on your tent is something that outputs 1 when a shadow is on your tent, and 0 when there isn't a shadow on your tent.

Imagine the shadows (green circles) changing position over the course of a day. The point marked 'you' stays where it is. Whenever a green circle overlaps the red dot, the dirac measure is 1; whenever not, it's 0.

Formally, if δx means "Tent is in shadow", and A is the collection of shadow regions, the mathematical expression of this dirac measure is δx(A)={0,x∉A1,x∈A.

The dirac measure always looks like this, whatever meanings you attach to δx and the set A.

(Since x is a point, the analogy breaks down a little: shadows are not allowed to partially overlap the tent. They cover it fully, or they don't touch it. We could fix this above by talking about "the northwesternmost atom of fabric of the tent" instead of "the tent").

(The dirac measure is similar to the indicator function (not measure!) 1A(x), which takes a point x and returns whether or not it's in A. For example, if you were walking around the forest instead of entented in it - that is, if the red point in the picture was moving around and the shadows didn't move, instead of the other way around - you could set up an indicator function 1a(x) to be 1 when you're inside a shadow, and 0 when you're outside one. A previous version of this post confused the dirac measure with the indicator function; the difference is that δx takes a set A as input and assumes x fixed, while 1A takes a point x as input and assumes the set A fixed. Thanks to deluks917 in the comments for the correction.)

The

...
Early Thoughts on Ontology/Grounding Problems
by johnswentworth — 2020-11-14T23:19:36.000Z

These all seem to be pointing to different aspects of the same problem.

Cross-ontology goal translation: given a utility function over a latent variable in one model, find an equivalent utility function over latent variables in another model with a different ontology. One subquestion here is how the first model’s input data channels and action variables correspond to the other model’s input data channels and action variables - after all, the two may not be “in” the same universe at all, or they may represent entirely separate agents in the same universe who may or may not know of each other's existence.

Correspondence

theorems

: quantum mechanics should reduce to classical mechanics in places where classical worked well, special relativity should reduce to Galilean relativity in places where Galilean worked well, etc. As we move to new models with new ontologies, when and how should the structure of the old models be reproduced?
The

indexing problem

: I have some system containing three key variables A, B, and C. I hire someone to study these variables, and after considerable effort they report that X is 2.438. Apparently they are using different naming conventions! What is this variable X? Is it A? B? C? Something else entirely? Where does their X fit in my model?
How do different people ever manage to point to the same thing with the same word in the first place? Clearly the word “tree” is not a data structure representing the concept of a tree; it’s just a pointer. What’s the data structure? What’s its type signature? Similarly, when I point to a particular tree, what’s the data structure for the concept of that particular tree? How does the “pointer” aspect of these data structures work?
When two people are using different words for the same thing, how do they figure that out? What about the same word for different things?
I see a photograph of a distinctive building, and wonder “Where is this?”. I have some data - i.e. I see the distinctive building - but I don’t know where in the world the data came from, so I don’t know where in my world-model to perform an update. Presumably I need to start building a little side-model of “wherever this picture was taken”, and then patch that side-model into my main world model once I figure out “where it goes”.
Distributed models and learning: a bunch of different agents study different (but partially overlapping) subsystems

...