trentmkelly/LessWrong-43k huggingface dataset:
[AN #156]: The scaling hypothesis: a plan for building AGI
by Rohin Shah — 2021-07-16T17:10:05.809Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

The Scaling Hypothesis (Gwern Branwen) (summarized by Rohin): This post centers around the scaling hypothesis:

Once we find a scalable architecture which can be applied fairly uniformly, we can simply train ever larger networks and ever more sophisticated behavior will emerge naturally as the easiest way to optimize for all the tasks and data. More powerful NNs are “just” scaled-up weak NNs, in much the same way that human brains look much like scaled-up primate brains⁠.

Importantly, we can get this sophisticated behavior just by training on simple objectives, such as “predict the next word”, as long as the data is sufficiently diverse. So, a priori, why might we expect the scaling hypothesis to be true?

The core reason is that optimal (or human-level) prediction of text really does require knowledge, reasoning, causality, etc. If you don’t know how to perform addition, you are probably not going to be able to predict the next word in the sentence “Though he started with six eggs, he found another fourteen, bringing his total to ____”. However, since any specific fact is only useful in a tiny, tiny number of cases, it only reduces the expected loss by a tiny amount. So, you’ll only see models learn this sort of behavior once they have exhausted all the other “easy wins” for predicting text; this will only happen when the models and dataset are huge.

Consider a model tasked with predicting characters in text with a set of 64 characters (52 uppercase and lowercase letters, along with some punctuation). Initially it outputs random characters, assigning a probability of 1/64 to the correct character, resulting in a loss of 6 bits. Once you start training, the easiest win is to simply notice how frequent each character is; just noticing that uppercase letters are rare, spaces are common, vowels are common, etc. could get your error down to 4-5 bits. After this, it might start to learn what words actually exist; this...
Underlying model of an imperfect morphism
by Stuart_Armstrong — 2021-07-16T13:13:10.483Z

We've already seen that if M0=(F0,Q0) and M1=(F1,Q1) are generalised models, with the relation r⊂W0×W1 a Q-preserving morphism between them, then there is an underlying model Mr=(F0⊔F1,Qr) between them.

Since r⊂W0×W1, Qr is defined on r; indeed, it is non-zero on r only. The underlying model has functions r0 and r1 to M0 and M1, which push forward Qr in a unique way - to Q0 and Q1 respectively. Essentially:

There is an underlying reality Mr of which M0 and M1 are different, consistent, facets.

Illustrated, for gas laws:

Underlying model of imperfect morphisms

But we've seen that relations r need not be Q-preserving; there are weaker conditions that also form categories.

Indeed, even in the toy example above, the ideal gas laws and the "atoms bouncing around" model don't have a Q-preserving morphism between them. The atoms bouncing model is more accurate, and the idea gas laws are just an approximation of these (for example, they ignore molar mass).

Let's make the much weaker assumption that r is Q-birelational - essentially that if any wi has non-zero Qi-measure (i.e. Qi(wi)>0), then r relates it to at least one other wj which also has non-zero Qj-measure. Equivalently, if we ignore all elements with zero Qi-measure, then r and r−1 are surjective relations between what's left. Then we have a more general underlying morphism result:

Statement of the theorem

Let r be a Q-birelational morphism between M0=(F0,Q0) and M1=(F1,Q1), and pick any 0≤α≤1.

Then there exists a generalised model Mαr=(F0⊔F1,Qαr), with Qαr=0 off of r⊂W0×W1 (this Qαr is not necessarily uniquely defined). This has natural functional morphisms r0:Mαr→M0 and r1:Mαr→M1.

Those ri push forward Qαr to Mi, such that for the distance metric L defined on morphisms,

|r0(Qαr)−Q0|1=αL(r),

|r1(Qαr)−Q1|1=(1−α)L(r).

By the definition of L, this is the minimum |r0(Qαr)−Q0|1+|r1(Qαr)−Q1|1 we can get. The proof is in this footnote[1].

Accuracy of models

If α=0, we're saying that M0 is a correct model, and that M1 is an approximation. Then the underlying model reflects this, with M0 a true facet of the underlying model, and M1 the closest-to-accurate facet that's possible given the connection with M0. If α=1, then it's reversed: M0 is an approximation, and M1 a correct model. For α between those two value, we see both M0 and M1 as approximations of the underlying reality Mr.

Measuring ontology change

This...
When beliefs become identities, truth-seeking becomes hard
by Writer — 2021-07-16T09:02:45.257Z
Link: https://youtu.be/NxqTOm3TzsY

This is Rational Animations' script for the video linked above. This time, the topic is the "Rethinking Identity" section in the book "The Scout Mindset" by Julia Galef. As always, feedback is welcome.
Previously, we made a video about how scorn for ideas and communities is often damaging for truth-seeking and truth-telling. In this video, we’ll talk about another way in which social reality might interfere with your ability to pursue what's true and what's best for you to do: identity.

Julia Galef talks about this topic in her book “The Scout Mindset.” The scout mindset is the mindset of truth-seeking, and she opposes it to the soldier mindset. Having a soldier mindset means defending your beliefs for reasons other than truth as if you were fighting a war of ideas instead of trying to figure out what’s going on. She makes the case that the scout mindset is way more helpful than the soldier mindset.
The soldier mindset has advantages, such as avoiding unpleasant emotions, increasing motivation, and persuasion. But Julia Galef outlines strategies to keep the advantages of the soldier mindset without actually sacrificing truth-seeking and truth-telling. Seems cool, right?
In her book, there are also useful sections about how to be a better scout. One of those sections is called “rethinking identity”, which is the topic of this video.
Julia observes that it’s often the case that certain kinds of beliefs are prone to become identities. Communities form around them and, in a sense, fight to defend them. She makes the example of the “mommy wars”, in which one faction believes that it's better to feed babies with breast milk and the other thinks that baby formula is fine. In theory, this should be a straightforward empirical question, but you can observe the two factions arguing passionately and calling each other epithets in an ideological battle that spans years and transcends geography.
There are many other examples you could come up with [on screen].

When your beliefs are central to your identity, it is way more difficult to change your mind if they turn out to be wrong. And the discussions you have will not be about the facts that make those beliefs true or false, but about either how much you identify with them or how your opponent is stupid. For example, you may wear t-shirts with slogans or engage in personal attacks. When someone mak...
Benchmarking an old chess engine on new hardware
by hippke — 2021-07-16T07:58:14.565Z

I previously explored the performance of a modern chess engine on old hardware (1, 2). Paul Christiano asked for the case of an old engine running on modern hardware. This is the topic of the present post.

State of the art

Through an online search, I found the CCRL Blitz Rating list. It is run on an i7-4770k at 9.2 MNodes/s. The time controls are 2min+1s per move, i.e. 160s per 40 moves, or 4s per move. On the 4770k, that's 36.8 MNodes/move. The current number one on that list is Stockfish 14 at 3745 ELO. The list includes Fritz 5.32 from 1997, but on old hardware (Pentium 90. Over the years, CCRL moved through P90-P200-K62/450-Athlon1200-Athlon4600-4770k). I screened the list, but found no case of old-engine on modern hardware.

One solution may be to reach out to the CCRL and ask for a special test run, calibrated against several modern engines.
I can also imagine that the Swedish Chess Computer Association is able to perform the test directly, using their Rating List procedure. It covers 155,019 games played by 397 computers, going back to 2 MHz machines from the year 1984. They may be able to transplant an old version onto a new machine for cross-calibration. The right person to contact would be Lars Sandin.

But the devil is in the details...

Making own experiments

In principle, the experiment should be trivial. The standard tool to compare chess engines is the command-line interface cutechess-cli. Set up old and new and get it running.

Problem 1: Find an old engine with a working interface

Paul and I independently favored the program Fritz from the most famous chess year 1997, because it was at the time well respected, had seen serious development effort over many years, and won competitions. The problem is that it only supports today's standard interface, UCI, since version 7 from the year 2001. So with that we can go back 20 years, but no more. Earlier versions used a proprietary interface (to connect to its GUI and to ChessBase), for which I found no converter.

I was then adviced by Stefan Pohl to try Rebel 6.0 instead. Rebel was amongst the strongest engines between 1980 and 2005. For example, it won the WCCC 1992 in Madrid. It was converted to UCI by its author Ed Schröder. I believe that old Rebel makes for a similarly good comparison as old Fritz.

A series of other old engines that support UCI are available for download. The following experiments shoul...
Lafayette: empty traffic signals
by KatjaGrace — 2021-07-16T04:30:15.759Z

Seeking to cross a road on the walk into downtown Lafayette, instead of the normal pedestrian crossing situation, we met a button with a sign, ‘Push button to turn on warning lights’. I wondered, if I pressed it, would it then be my turn to cross? Or would there just be some warning lights? What was the difference? Do traffic buttons normally do something other than change the lights? I clearly believe they do. They make it my turn. But they don’t send a wire into the ‘whose turn is it’ variable deep in the ought-sphere, so what exactly do they do?

I suspected that this button didn’t change whose turn it was, and it felt empty and devoid of some special potence of being a traffic button.

I liked to imagine that it was just a normal traffic button, but taking a more nihilistic view of its role. In which case, its nihilistic view seemed to have practical consequences! It wasn’t being as good a good traffic button while saying that it didn’t change whose turn it is. It genuinely fails to coordinate the traffic so well, because here am I unable to garner the ‘right’ to cross with confidence, and there are the drivers unsure what I’m doing. But shouldn’t a traffic button be able to do its job regardless of its philosophical commitments, or without pretending to have philosophical commitments it doesn’t have?

One might say that the thing going on is that it being ‘my turn’ is a fact about everyone’s expectations. For instance, if the drivers will expect me to cross, then it is ‘my turn’. (I’m tempted to say ‘if the drivers think it is my turn, then it is my turn’, but what are the drivers then thinking?) This doesn’t seem quite right, in that the drivers themselves are asking whether this light means that it is the pedestrian’s turn, and all of us seem to be asking something about the underlying truth, not about each other. Also, if I run erratically into the road, the drivers and I may both come to expect that I am going to cross, but it still isn’t my turn.

I fantasized that I had woken up in a new world which was just like the old world, but where everything was like the traffic light. I would phone the doctor’s office later to ask if it was ok to cancel my appointment this late, they would just say, ‘I’ll change what it says in this schedule’.

‘But is it ok?’

‘I will not record your cancellation.’

‘Should I pay you?’

‘I am not charging you’

‘But is that like a favor, o...
How to Sleep Better
by Marius Hobbhahn — 2021-07-16T00:00:29.425Z

This is a cross-post from my personal blog.

What is this post about?

Over the summer of 2020, I slept pretty badly for a long period of time and it had pretty noticeable effects on my productivity, perceived energy levels, and well-being in general. This was mostly due to the fact that my room was under the southern roof of the house and it had around 28 degrees Celcius during "cool" summer nights which made it effectively impossible for me to get more than 6 hours of light sleep per night. In autumn, I moved to a new flat where I had to get a new bed, mattress, pillow, etc.

Since I had felt the negative consequences of bad sleep pretty recently, I figured I could conduct a bit of research and use science to improve my own sleep. In the following, you can find a summary of what I found out about the right equipment and behavior to improve sleep mixed with my own considerations.

One piece of framing that I want to provide before diving into the main part is about the money and effort one should be willing to spend to improve our sleep. The longer I thought about this question the more I thought the right answer should be: "a lot". Just think of the effect that good vs. bad sleep has on your life and quantify it roughly. Let's assume the difference between good vs. bad sleep results in 30 more productive minutes per day. If you value your time at the level of the minimum wage (as a conservative estimate) which is around 10€ in Germany this means you should be willing to pay up to 5€ per day. Over a year that makes 5 * 365 = 1825€. If you additionally consider all the negative side effects of bad sleep this number gets larger very quickly. If you have, for example, a mattress that is way too soft for you, your spine might suffer which can have large consequences for the rest of your life, e.g. while sitting, driving a car, or during exercise. Treating these outcomes, which are often only revealed after years and take a long time to fix, is often very expensive and emotionally draining or painful. A second example would be our perceived well-being throughout the day. I think we all know that there is a noticeable difference between a day after a good night and a bad one. One day feels slow, annoying, and unproductive while the other feels faster, more energetic, and more productive. While this is harder to quantify financially it is still something that I would be w...
Generalizing Koopman-Pitman-Darmois
by johnswentworth — 2021-07-15T22:33:03.772Z

Suppose we want to estimate the mean and variance of a normal distribution from some samples X1…Xn from the distribution. It turns out that we don’t need to keep around all of the data: only the sample mean ^μ=1n∑iXi and sample variance ^Σ=1n∑i(Xi−^μ)2 are needed for the estimation problem. From those, we can calculate the entire posterior distribution of μ,Σ. The pair (^μ,^Σ) is a “sufficient statistic” for X1…Xn: a statistic which summarizes all the estimation-relevant information from the data.

In principle, every estimation problem has a sufficient statistic: just let the summary be all the data. But this isn’t very useful. We really want to compress the data, pull out the actually-useful parts and throw away the rest. In the normal distribution example, our n samples of data are each 1-dimensional for a total of n dimensions of data, but the summary is only 2-dimensional, no matter how large n grows.

When will an estimation problem have a fixed-dimensional sufficient statistic as the number of data points increases, like the normal-distribution estimation problem? The Koopman-Pitman-Darmois (KPD) Theorem says that, when estimating distribution parameters from IID samples, there is a fixed-dimensional sufficient statistic if-and-only-if the distribution can be expressed in

exponential family

form:

P[X|Θ]=1Z(Θ)ef(Θ)T∑ig(Xi)∏ih(Xi)

… for some vector-valued functions f,g and positive-real-valued function h. The sufficient statistic is then ∑ni=1g(Xi). For instance, for a normal distribution, the parameters Θ are (μ,Σ), and we can use g(x)=(x,x2) and f(Θ)=f(μ,Σ)=(−Σ−1μ,−12Σ−1).

Intuitively, this says the “coupling” between X and Θ is only via the fixed-dimensional “summary” vectors f(Θ) and ∑ig(Xi).

That’s the original KPD Theorem. We’ll see in this post that the theorem generalizes a lot. It’s not just estimation of distribution parameters from IID samples - KPD can also generalize to estimation of parameters for Markov chains, or causal models with symmetry - aka stochastic programs. The theorem also extends to causal models without any symmetry, albeit slightly weaker: a small number of variables can violate the exponential form, but (roughly speaking) the number of variables which can violate the e...
Fractional progress estimates for AI timelines and implied resource requirements
by Mark Xu — 2021-07-15T18:43:10.163Z

This post was written by Mark Xu based on interviews with Carl Shulman. It was paid for by Open Philanthropy but is not representative of their views. A draft was sent to Robin Hanson for review but received no response.

Summary

Robin Hanson estimates the time until human-level AI by surveying experts about the percentage progress to human-level that has happened in their particular subfield in the last 20 years, and dividing the number of years by the percentage progress.

Such surveys look back on a period of extremely rapid growth of compute from both hardware improvements and more recently skyrocketing spending.

Hanson favors using estimates from subsets of researchers with lower progress estimates to infer AI timelines requiring centuries worth of recent growth, implying truly extraordinary sustained compute growth is necessary to surpass human performance.

Extrapolated compute levels are very large to astronomically large compared to the neural computation that took place in evolution on Earth, and thus likely far overestimate AI requirements and timelines.

Introduction

Suppose that you start with $1 that grows at 10% per year. At this rate, it will take ~241 years to get $10 billion ($1010). When will you think that you’re ten percent of the way there?

You might say that you’re ten percent of the way to $10 billion when you have $1 billion. However, since your money is growing exponentially, it takes 217 years to go from $1 to $1 billion and only 24 more to go from $1 billion to $10 billion, even though the latter gap is larger in absolute terms. If you tried to guess when you would have $10 billion by taking 10x the amount of time to $1 billion, you would guess 2174 years, off by a factor of nine.

Instead, you might say you’re ten percent of the way to $1010 when you have $101, equally spacing the percentile markers along the exponent and measuring progress in terms of log(wealth). Since your money is growing perfectly exponentially, multiplying the number of years it takes to go from $1 to $10 by ten will produce the correct amount of time it will take to go from $1 to $1010.

When employing linear extrapolations, choosing a suitable metric that better tracks progress, like log wealth over wealth for investment, can make an enormous difference to forecast accuracy.

Hanson’s AI timelines estimation methodology

Hanson’s preferred method for estimating AI...
Covid 7/15: Rates of Change
by Zvi — 2021-07-15T17:10:01.985Z

Cases rose by over 60% in America this week, and we’re seeing large jumps in cases around the world. I am highly suspicious about the jump in the rate of increase, but Delta certainly seems to be the real deal, and this was well above my expectations.

I worry that recently I’ve lacked sufficient skin in the game. Everyone I personally care about is vaccinated or young enough that they don’t need vaccination, so the real sense of danger is largely gone. The worry is about the reaction to Covid, rather than about Covid itself. But that’s a very real danger, and I have back that sense of ‘oh no, things could go very wrong’ because there’s the danger that we really will blow up our way of life over all this, and go into a permanent dystopia of sorts. That’s what we need to ensure does not happen.

Thus, the bulk of this post is a numbers analysis trying to figure out what we know about Delta’s transmissibility and the effectiveness of vaccines in reducing that transmissibility, using data from a variety of sources. Others are encouraged to continue this analysis and try to get to the bottom of this.

So let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 3.3% (up 0.4%) and deaths increase by 7%.

Result: Positivity rate of 4.8% (!) and deaths increase by 15%.

Prediction for next week: Positivity rate of 4.7% (down 0.1%) and deaths unchanged.

The null prediction is always an option, here two distinct null predictions with distinct reasoning. For deaths it’s clear that there was a reporting gap as predicted, so I do not think the death rate last week represents things getting worse, but they likely should start to get worse given Delta is deadlier and cases have stopped dropping within the required time window, and it doesn’t seem like last week’s number was too artificially high.

The case number is trickier, as there’s good reasons to think the data is distorted, either by July 4 or otherwise:

That giant spike represents going from an average of 2.6% to an average of 5.0% over two days. That’s not a thing that should happen to seven day averages. If it does, then the next five days things should continue to rise as old data cycles out for new, but that didn’t happen so far.

Perhaps July 4 was truly a superspreader event in the way that previous holidays were almost always underwhelming, as much as the previous h...