trentmkelly/LessWrong-43k huggingface dataset:
Fertility Roundup #3
by Zvi — 2024-04-02T14:50:04.554Z

Previous Fertility Roundups: #1, #2.

The pace seems to be doing this about twice a year. The actual situation changes slowly, so presumably the pace of interesting new things should slow down over time from here.

Demographics

This time around, a visualization. Where will the next 1,000 babies be born?

Population Trends

Scott Lincicome notes American population now expected to peak in 2080 at 369 million.

South Korea now down to 0.7 births per woman. The story of South Korea is told as a resounding success, of a country that made itself rich and prosperous. But what does it profit us, if we become nominally rich and prosperous, but with conditions so hostile that we cannot or will not bring children into them? If the rule you followed led you here, of what use was the rule? Why should others follow it?

More Births reminds us that we have indeed seen countries fall below replacement level and come roaring back, most famously in the Baby Boom. Cultural trends can go any number of ways.

Basil: The fertility rate drops from 2015 -> 2023 are insane in just EIGHT YEARS:

France: 1.96 -> 1.68

Sweden: 1.85 -> 1.42

America: 1.84 -> 1.64

UK: 1.78 -> 1.45

Russia: 1.78 -> 1.41

China: 1.75 -> 1.05 (wtf)

Germany: 1.50 -> 1.42

South Korea: 1.24 -> .73 (jesus christ)

Netherlands: 1.55 -> 1.45

Canada: 1.60 -> 1.25

Japan: 1.45 -> 1.21

Poland: 1.44 -> 1.28

Taiwan: 1.18 -> 0.86

Those are LFR numbers, so the actual changes are presumably bigger.

Singapore falls to historic low of 0.97. New figures have South Korea at 0.72.

I continue to say that if the numbers can decline this fast, they can also bounce back. Indeed do many things come to pass, and we can expect transformational technological change. Things have looked bleak or inevitable before. Don’t lose hope.

After that, China’s numbers came in for 2023 and they are even worse. The population already shrunk by 2 million in 2023, a trend that will rapidly accelerate by default.

That WSJ article ends with a Harvard professor saying China should not pursue ‘higher’ birthrates, allowing itself to quickly cease to exist, in order to instead address nominal measures of well being.

Martin Whyte, a sociology professor emeritus at Harvard University, said in an essay in China-US Focus, run by the independent China-United States Exchange Foundation, last year that instead of

...
What can we learn about childrearing from J. S. Mill?
by Adam Scherlis — 2024-04-02T06:06:19.703Z

[EDIT: This is an April Fool's post. Please don't raise your kids like this.]

John Stuart Mill

was given an extremely rigorous upbringing, and was deliberately shielded from association with children his own age other than his siblings. His father, a follower of Bentham and an adherent of associationism, had as his explicit aim to create a genius intellect that would carry on the cause of utilitarianism and its implementation after he and Bentham had died.

-- A biographer or something, not Wikipedia

This worked perfectly.

We're overthinking it. Just raise kids in isolation and train them to be perfect rationalists and Effective Altruists. We'll get an entire generations of Mills.
OMMC Announces RIP
by Adam Scholl — 2024-04-01T23:20:00.433Z

At the Omnicide Machine Manufacturing Corporation, we work tirelessly to ensure an omnicide-free future. That’s why we’re excited to announce our Responsible Increase Policy (RIP)—our internal protocol for managing any risks that arise as we create increasingly omnicidal machines.

Inspired by the risk-management framework used in gain-of-function virology research, our RIP defines a framework of Omnicidal Ability Levels (OAL), reflecting the precautions we plan to take as we release increasingly dangerous features over time:

The basic idea of the RIP is simple: each time we ship an update which makes our product more lethal, we will pause our efforts for some amount of time, and then revise our policies to be in some sense more “cautious.” For example, our RIP contains the following firm commitments:

We aspire to take actions which are broadly good, rather than broadly bad;
We hope to refrain from releasing any fairly omnicidal systems, until first implementing “certain safeguards”;
And we intend to refrain from creating any systems which we’re quite sure would kill everyone.

That said, we want to acknowledge that even this cautious approach has drawbacks. For example, if our prevention measures are too weak, we risk catastrophe—potentially leading to extreme, knee-jerk regulatory responses, like banning omnicide machines altogether. On the other hand, if our precautions are too conservative, we risk ending up in a situation where someone who isn’t us builds one first.

This is a tricky needle to thread. History is rife with examples of countries deciding to heavily restrict, or even outright ban, technologies which they perceive as incredibly dangerous. So we have designed our RIP to tread lightly, and to exemplify a “minimum viable” safety policy—a well-scoped, small set of tests, that labs can feasibly comply with, and that places the least possible restrictions on frontier existential risks.

The Sweet Lesson: Reasoning is Futile

As an omnicide creation and prevention research company, we think it’s important to seriously prepare for worlds in which our product ends up getting built. But the central insight of the modern era of gigantic machines—the so-called “Sweet Lesson”—is that it's possible to build incredibly powerful machines without first developing a deep theoretical understanding of how they work.

Indeed, we currently see ourselves as operating under condi...
Coherence of Caches and Agents
by johnswentworth — 2024-04-01T23:04:31.320Z

There's a lot of confusion about what coherence means for agents, and what "coherence theorems" do and don't say about agents. In this post, I'll talk about some particularly simple notions of coherence in a particularly simple setting. We'll see what nontrivial things coherence has to say, at least in a simple kind of environment, starting with an analogous notion of coherence for caches.

What Kind Of "Coherence" We're Talking About Here

Let’s start with a standard CS-101-style example. We write a recursive python function to compute fibonacci numbers:

def fib(n):
if n == 0:
result = 1
elif n == 1:
result = 1
else:
result = fib(n-1) + fib(n-2)
return result

We pass in n = 0, then n = 1, then 2, then 3, etc. It spits out 1, 1, 2, 3, 5, 8, .... Great. Buuuuut it gets very slow very quickly as n increases; the runtime is exponential in n.

So, standard simple improvement: memoize. The first time fib(n) is computed for each value of n, cache it (i.e. "make a memo" of the result).

cache = {}
def memo_fib(n):
if n in cache:
return cache[n]

if n == 0:
result = 1
elif n == 1:
result = 1
else:
result = memo_fib(n-1) + memo_fib(n-2)

cache[n] = result
return result

Now the recursive calculation will only happen once for each value of n, so runtime is linear in n.

Ok, that's the CS 101 part. Now on to coherence.

Imagine that the cache in our fibonacci program gets corrupted somehow. Maybe I mess around in the debugger and stick a few wrong numbers into it, maybe some other thread writes into it, whatever. Somehow, incorrect values end up in that cache.

Key point: we can notice the cache corruption "locally", i.e. by only looking at a small subset of the cache. Say, for instance, that cache[6] is corrupted - it should be 8 (the sixth fibonacci number), but instead let's say it's 11, and let's assume for now that the rest of the cache is fine. So we're looking in the cache, and we see:

cache[4] = 3
cache[5] = 5
cache[6] = 11

Well, just from those three entries we can tell that something's wrong, because 3 + 5 is not 11. It's supposed to be the case that cache[n] = cache[n-1] + cache[n-2] for any n bigger than 1, but that equation is not satisfied by these three cache entries. Our cache must be corrupt. And notice that we did not need to look at the rest of the cache in order to tel...
LessWrong: After Dark, a new side of LessWrong
by So8res — 2024-04-01T22:44:04.449Z

The LessWrong team has obviously been hard at work putting out their debut album. But another LessWrong feature also seems to have been released today, to less fanfare: LessWrong: After Dark, a branch of the site devoted to explicit discussion of sex and sexuality, where the LessWrong team finally gets to let loose their long-suppressed sexual instincts.

As someone who’s close friends with Aella, I’m thrilled to see this new branch of the site. Sex workers are heavily discriminated against in modern society, with limited access to banking, a heightened risk of physical injury, and an inability to rely on police. The topic of sex is overstigmatized in modern culture, and I’m glad to see that the LessWrong team has decided to accept the sexual aspect of the human experience, and that they now have a place to hornypost to their hearts’ content.

I’m looking forward to seeing what comes of rationalists applying rationality techniques to sex with the same dogged vigor and dubiously-directed determination that we apply to everything else.
Gradient Descent on the Human Brain
by Jozdien — 2024-04-01T22:39:24.862Z

TL;DR: Many alignment research proposals often share a common motif: figure out how to enter a basin of alignment / corrigibility for human-level models, and then amplify to more powerful regimes while generalizing gracefully. In this post we lay out a research agenda that comes at this problem from a different direction: if we already have ~human-level systems with extremely robust generalization properties, we should just amplify those directly. We’ll call this strategy “Gradient Descent on the Human Brain”.

Introduction

Put one way, the hard part of the alignment problem is figuring out how to solve ontology identification: mapping between an AI’s model of the world and a human’s model, in order to translate and specify human goals in an alien ontology.

In generality, in the worst case, this is a

pretty

difficult

problem

. But is solving this problem necessary to create safe superintelligences? The assumption that you need to solve for arbitrary ontologies is true if you assume that the way to get to superintelligence necessarily routes through systems with different ontologies. We don’t need to solve ontology translation for high-bandwidth communication with other humans[1].

Thus far, we haven’t said anything really novel. The central problem to this approach, as any alignment researcher would know, is that we don’t really have a good way to bootstrap the human brain to superintelligent levels. There have been

a few

attempts

to approach this recently, though focusing on very prosaic methods that, at best, buy points on the margin. Scaling to superintelligence requires much stronger and robust methods of optimization.

The Setup

The basic setup is pretty simple, though there are a few nuances and extensions that are hopefully self-explanatory.

The simple version: Take a hundred human brains, put them in a large vat, and run gradient descent on the entire thing.

The human brain is a remarkably powerful artifact for its size, so finding a way to combine the capabilities of a hundred human brains with gradient descent should result in something significantly more powerful. As an intuition pump, think of how powerful human organizations are with significantly shallower communication bandwidth. At the very lowest bound we can surpass this, more impressive versions of this could look like an integrated single mind that combines the capa...
Do I count as e/acc for exclusion purposes?
by denyeverywhere — 2024-04-01T21:18:08.389Z

EDIT: THIS IS NOT APRIL FOOLS RELATED

ALSO: This is specific to the LW scene in Berkeley and nearby Berkeley, as this is the only place where e/acc exclusion is asserted to take place.

I haven't been around the LW scene for some time, but I understand it's common to exclude e/acc people from events. I further understand this to be exclusion on philosophical grounds, not just because LW-ites tend to view e/acc people individually as unlikeable.

I personally don't want to try to sneak into LW parties if I'm someone that the hosts are trying to exclude on philosophical grounds. So I'd rather clarify whether, in the opinion of various people, I count.

It's common among e/acc people to say things like "We're so close, just don't die" by which they mean that AGI is close. They also want to create AGI as soon as possible. By contrast, LW-ites typically believe that AGI is close, and therefore it is necessary to slow down or stop AGI development as soon as possible, in order to ensure that future development is done safely.

I part ways from both camps in believing that we're nowhere close to AGI, that the apparently-impressive results from LLMs are highly overrated, and that the X-risk from AI is 0 for the forseeable future. If I didn't think this, I would be sympathetic[1] to the desire to stop AI until we thought we could do it safely. But I do think this, so AI safety seems like a Victorian Nuclear Regulatory Commission. The NRC is a good thing, but it's going to be a while before splitting the atom is even on the table.

As a result, in practice I think I'm functionally e/acc because I don't want to stop the e/acc people from trying to push AGI as fast as possible. I don't think they're actually an X-risk since they're not going to succeed any time soon. But I'm theoretically decel because if I thought anyone was anywhere close to AGI I'd be sympathetic to efforts to restrain it. As it is, I think the AI safety people can continue to study AI safety for years confident that they can finish all the theories off long before they actually become necessary for survival.

In light of that, if you're the sort of person who wants to exclude e/acc people from your party, should I just not show up? That's fine with me, I'd just as soon know ahead of time.

Actually, the fact that I have to even ask this question makes me disinclined to show up anyway, but I'm sort of curious what p...
Self Explaining Neural Networks, the interpretability technique no one seems to be talking about.
by f3mi — 2024-04-01T20:52:57.698Z

Disclaimer: I'm very new to alignment as a whole. I wouldn't be surprised if this turned out to be a nothing burger.

This is the coolest paper I've seen in a while, yet I've never heard of the technique. It's not mentioned on blog posts about AI interpretability/AI safety, and I've found very few papers trying to build on the technique, but it seems like a very promising approach.

It's an intrinsic method, meaning that it achieves interpretability goals by making changes to model architecture, not by interpreting weights or activations.

non linear-linearity

The paper starts by outlining what it really means to have an interpretation of a model.

Explicitness: How easy is it to understand the interpretations of the model
Faithfulness: How faithful are the interpretations of the model to what's actually going on inside of it
Stability: How consistent are explanations for neighboring examples[1]

We actually already have a model which has explanations that satisfy all these criteria, and is still widely used to this day! The humble Linear Model. It's coefficient vector θθ tells you exactly how each input contributed to the output of the model and is identical for each input.

But for most classification and regression tasks, the linear model is shit.

The problem is, most functions we care about aren't linear. Now they might be locally linear, the same way that zooming in really close to a graph kind of looks like a straight line. But the formula f(x)=θTx can only ever globally be a straight line (or an N dimensional analogue), even as the function may do wildly non-linear things.

However, what if we just train a model to compute θθ for each input? The authors of the paper propose using a Deep Neural Network they call θθ(xx) to be the coefficients of the linear model for a given input.

Intuitively, this feels like trying to approximate the derivative of some underlying function instead of the actual function.[2]

However, there's still one big problem. The dimensionality of some problems is mind boggling. Even if we knew how much each pixel contributed to the overall output, how much would we care?

refined representations

Our model is theoretically more interpretable, but it tells us how much each pixel contributes to the final answer. Frankly, these kind of pixel level shallow attribution techniques have been shown to be not...
Death with Awesomeness
by osmarks — 2024-04-01T20:24:21.697Z

Introduction

In the two years since the original publication of Death with Dignity, it has been clear that we're not back, it's so over, and that it has never been more over. AI capabilities leapt forward across domains, the general public noticed the existence of AI and threw money and GPUs at anything vaguely related to it, and increasingly incorrect opinions and degradation of terminology have polluted any attempt to discuss the problem. In this environment, Death with Dignity is an ever-more-attractive notion. But it makes a critical mistake.

Given our inevitable demise, death with dignity has the right idea, but it fails to consider the well-established results of fun theory: if you are dying of a terminal disease, is it better to go out peacefully and slowly on a hospital bed surrounded by family, or jumping a motorbike across a ravine filled with cloned raptors trying to chase you? I think the correct answer is obvious.

Something like interpretability work allowing possible dangerous mesaoptimization in a new model to be detected and ignored, or a new mathematical theory of agency which will never be usefully applied to a real inscrutable blob of floats (or, nowadays, ternary numbers) is not, by itself, awesome. While bending such theories to the task of saving the world from AI doom would be awesome, this is unlikely, as Death with Dignity already argues. So what's left? Making our imminent paperclipping more awesome. This is a relatively tractable problem, as I'm about to illustrate.

Avenues for Awesomeness

As TVTropes has taught us, awesomeness requires some sense of meaningful human participation. Naively, this requires actual ability to change the outcome, which is of course impossible: however, this is more of an aesthetic requirement than a functional one - people are generally fine with railroaded fiction, and more generally with watching things play out even when the outcome is known in advance.

The obvious solution, then, is to engineer a dramatic final battle with the AGI - ideally, people get to fight through an army of robots to access an automated OpenMetaAmazGoogMind datacentre (ideally an oversized vertically stacked one), break in, and then engage in a hacking scene (with at least five monitors, and sunglasses) to (futilely) try and shut it down.

How can we achieve this?

Even reaching this is nontrivial, due to continued problems with al...
[GPT-4] On the Gradual Emergence of Mechanized Intellect: A Treatise from the Year 1924
by tailcalled — 2024-04-01T19:14:02.363Z

Editors note: This treatise was found in the Global Preservation and Technology Archive - 4th Edition (GPT-4). It makes a compelling argument that artificial general intelligence will have a "slow takeoff", developing over centuries.

In the year of our Lord 1924, as humanity stands amidst the clanking machineries and humming dynamos of the industrial age, a bold and fantastical vision emerges from the fertile grounds of speculation: the advent of Mechanized Intellect. This treatise endeavors to chart the course towards such a future, navigating through the technological and conceptual wilderness of our present era.

The Foundation: The Electrification of Calculative Machinery

Our initial foray into the realm of Mechanized Intellect brings us to the crossroads of power and precision. Within the present epoch, steam and clockwork reign supreme, driving the engines of industry and the tickings of time. Yet, when summoned to the task of emulating the subtleties of human thought, they reveal their inherent limitations. Steam, for all its might, lacks the delicacy required for intricate calculations, while clockwork, though precise, cannot adapt nor learn as the faculties of the mind demand.

Thus, our gaze turns to electricity, a force of nature tamed but only just, as the beacon of hope. The challenges that stand in our way are not trifling: the generation of electric power in volumes vast enough to quench the thirst of intellect machinery; the safe conveyance of this potent force across the leviathan spans of land and sea; and the invention of devices capable of modulating this power with the finesse required for thought. These hurdles, monumental in their scale, underscore the nascent state of our electrical arts and the daring of our ambition.

The Mechanism of Thought: The Labyrinth of Conditional Probabilities

Venturing deeper into the machinations of Mechanized Intellect, we confront the enigma of imbuing our creation with the ability to think beyond mere numbers. Here, the science of probabilities, conditional in nature, presents itself as a key to unlocking this chamber of secrets. Yet, the path is fraught with shadows, for our machinery, as it stands, groans under the weight of such sophisticated contemplation. Without a firm foundation in the yet-uncharted territories of information dynamics, our efforts to encode the tapestry of human thought into the bras...
Notes on Dwarkesh Patel’s Podcast with Sholto Douglas and Trenton Bricken
by Zvi — 2024-04-01T19:10:12.193Z

Dwarkesh Patel continues to be on fire, and the podcast notes format seems like a success, so we are back once again.

This time the topic is how LLMs are trained, work and will work in the future. Timestamps are for YouTube. Where I inject my own opinions or takes, I do my best to make that explicit and clear.

This was highly technical compared to the average podcast I listen to, or that Dwarkesh does. This podcast definitely threated to technically go over my head at times, and some details definitely did go over my head outright. I still learned a ton, and expect you will too if you pay attention.

This is an attempt to distill what I found valuable, and what questions I found most interesting. I did my best to make it intuitive to follow even if you are not technical, but in this case one can only go so far. Enjoy.

(1:30) Capabilities only podcast, Trenton has ‘solved alignment.’ April fools!

(2:15) Huge context tokens is underhyped, a huge deal. It occurs to me that the issue is about the trivial inconvenience of providing the context. Right now I mostly do not bother providing context on my queries. If that happened automatically, it would be a whole different ballgame.

(2:50) Could the models be sample efficient if you can fit it all in the context window? Speculation is it might work out of the box.

(3:45) Does this mean models are already in some sense superhuman, with this much context and memory? Well, yeah, of course. Computers have been superhuman at math and chess and so on for a while. Now LLMs have quickly gone from having worse short term working memory than humans to vastly superior short term working memory. Which will make a big difference. The pattern will continue.

(4:30) In-context learning is similar to gradient descent. It gets problematic for adversarial attacks, but of course you can ignore that because as Tenton reiterates alignment is solved, and certainly it is solved for such mundane practical concerns. But it does seem like he’s saying if you do this then ‘you’re fine-tuning but in a way where you cannot control what is going on’?

(6:00) Models need to learn how to learn from examples in order to take advantage of long context. So does that mean the task of intelligence requires long context? That this is what causes the intelligence, in some sense, they ask? I don’t think you can reverse it that way,

...