trentmkelly/LessWrong-43k huggingface dataset:
Altruism and Selfishness
by Zero Contradictions — 2024-08-09T06:15:41.854Z
Link: https://thewaywardaxolotl.blogspot.com/2015/04/altruism-and-selfishness.html

The PDF version can be read here.

The video version can be watched here:

This is how I define “altruism” and “selfishness”:

Altruism: Acting for the benefit of others.
Selfishness: Acting for your own benefit.

These definitions match the common usage of the words. They are as simple and clear as possible.

Of course, like all definitions, they depend on the meanings of other words. So, now we need to dig a bit deeper.

They both depend on the definition of “benefit”. Selfishness and altruism are relative to a definition of value: what is good or bad for an individual. There are different types of altruism and selfishness, depending on how we define “benefit”.

Here are some ways that “benefit” could be defined:

Biological: Reproduction is the measure of value.
Psychological: Desire is the measure of value.
Energetic: Energy is the measure of value.

Whether an action is altruistic or selfish depends on which definition is used. An action that is altruistic when viewed energetically could be selfish when viewed biologically. For example, parenting is energetically altruistic but biologically selfish.

(see the rest of the post in the link)
Altruism and Vitalism Aren't Fellow Travelers
by Arjun Panickssery — 2024-08-09T02:01:11.361Z
Link: https://arjunpanickssery.substack.com/p/altruism-and-nietzscheanism-arent

Summary: In his most recent post (

“Altruism and Vitalism as Fellow Travelers”

), Scott Alexander tries to reconcile EA-style altruism with Nietzschean “vitalism.” He claims that the philosophies argue for different goals only in extreme cases, while having similar goals in reality. But his definition of vitalism misses the point: the goal isn’t to increase vital traits in the population, but to uplift rather than impede the vital type of person. So he fails to realize that the philosophies have different goals even in reality. They can still be reconciled but not in the way he thinks.

Definitions

The problem comes from his definitions:

Define altruism as “try to increase happiness and decrease suffering across a society” and vitalism as “try to increase strength and decrease weakness across a society”, where “strength” is defined as ability to achieve goals (and, in a tie-breaker, ability to win wars).

But this misunderstands Nietzsche. From

“Nietzsche’s Morality in Plain English”

:

Nietzsche’s overarching project is the “revaluation of all values”: a critique of herd morality (which he typically just refers to as “morality”) on the grounds that it’s hostile to the flourishing of the best type of person.

…

He advances what Leiter calls a “doctrine of types” where everyone is some type of guy and the type of guy you are determines the kind of life you can lead, and that you’ll hold whatever philosophical or moral beliefs will favor your interests. He doesn’t hold any extreme “determinist” position but is broadly fatalistic about how your type-facts circumscribe and set limits on the kind of person you’ll be and the beliefs you’ll hold, within which you can be influenced by your environment and values.

The vitalist position would more likely hold that some people are capable of doing great things, and others aren’t. So better definitions are:

Altruism: Try to maximize the total happiness and decrease suffering.
Vitalism: Uplift or at least don’t impede the flourishing of the highest type of person.

Don’t tell them that their suffering is bad, and don’t try to reduce their suffering if it could interrupt their work.
Don’t encourage (through culture) or force (through taxes) altruism, which could limit them from pursuing their demanding obsessions.
Don’t scrutinize them as “tall poppies” in general.

Differences

Scott gives the extreme example of altruism as WALL-E’s “morbidly ob...
The Hessian rank bounds the learning coefficient
by Lucius Bushnaq — 2024-08-08T20:55:36.960Z

TL;DR: In a neural network with d parameters, the (local) learning coefficient λ can be upper and lower bounded by the rank of the network's Hessian d1:

d12≤λ≤d12+d−d14.

The lower bound is a known result. The upper bound is a claim by me, and this post contains the proof for it.[1] If you find any problems, do point them out.

Edit 16.08.2024: The original version of this post had a three in the denominator of the upper bound. Dmitry Vaintrob spotted an improvement to make it a four.

Introduction

The learning coefficient λ is a measure of loss basin volume and model complexity. You can think of it sort of like an effective parameter count of the neural network. Simpler models that do less stuff have smaller λ.

Calculating λ for real networks people actually use is a pain. My hope is that these bounds help make estimating it a bit easier.

In a network with d parameters, the learning coefficient is always a number

0≤λ≤d2.

An existing result in the literature says that if you’ve calculated the rank of the network’s Hessian d1,[2] you get a tighter lower bound

d12≤λ.

I claim that we can also get a tighter upper bound

λ≤d12+d−d14,

where d−d1 will be the dimension of the Hessian kernel, meaning the number of zero eigenvalues it has.[3]

This is neat because it means we can get some idea of how large λ is using only standard linear algebra methods. All we need to know is how many zero eigenvalues the Hessian has.[4] Singular Learning Theory introductions often stress that just looking at the Hessian isn’t enough to measure basin volume correctly. But here we see that if you do it right, the Hessian eigenspectrum can give you a pretty good idea of how large λ is. Especially if there aren't that many zero eigenvalues.

Intuitively, the lower bound works because a direction in the parameters w that isn't free to vary to second order in the Taylor expansion won't become any more free to vary if you pile on a bunch of higher order terms. The Second order term strictly dominates the higher order ones, they can't cancel it out.

Qualitatively speaking, the upper bound works for the same reason. The higher order terms in the Taylor expansion of the loss can only matter so much. The Hessian is the leading term, so it can impact λ the most, adding 12...
GPT-4o System Card
by Zach Stein-Perlman — 2024-08-08T20:30:52.633Z
Link: https://openai.com/index/gpt-4o-system-card/

At last. Yay OpenAI for publishing this. Highlights: some details on Preparedness Framework evals + evals (post-deployment) by METR and Apollo.

Preparedness framework evaluations

You should follow the link and read this section.

Brief comments:

Cyber: the setup sounds good (but maybe substantially more powerful scaffolding/prompting is possible). Separately, I wish OpenAI shared the tasks (or a small random sample of them) or at least said more about where they came from. (Recall that DeepMind shared CTF tasks.)
Bio uplift: GPT-4o clearly boosts users on biological threat creation tasks — OpenAI doesn't say that but shows a graph. (It continues to be puzzling that novices score similarly to experts.) (I kinda worry that this is the wrong threat model—most bio risk from near-future models comes from a process that looks pretty different from a bigger boost to users like these—but I don't have better ideas for evals.)
Persuasion: unclear whether substantially more powerful scaffolding/prompting is possible.
Autonomy: unclear whether substantially more powerful scaffolding/prompting is possible.

I'm looking forward to seeing others' takes on how good these evals are (given the information OpenAI published) and how good it would be for OpenAI to share more info.

Third party assessments

Following the text output only deployment of GPT-4o, we worked with independent third party labs,

METR

and

Apollo Research

[,] to add an additional layer of validation for key risks from general autonomous capabilities. . . .

METR ran a GPT-4o-based simple LLM agent on a suite of long-horizon multi-step end-to-end tasks in virtual environments. The 77 tasks (across 30 task “families”) (See

Appendix B

) are designed to capture activities with real-world impact, across the domains of software engineering, machine learning, and cybersecurity, as well as general research and computer use. They are intended to be prerequisites for autonomy-related threat models like self-proliferation or accelerating ML R&D. METR compared models’ performance with that of humans given different time limits. See METR’s

full report

for methodological details and additional results, including information about the tasks, human performance, simple elicitation attempts and qualitative failure analysis. . . .

Apollo Research evaluated capabilities of schemingN in GPT-4o. They tested whether GPT-4o can model itself

...
Parasites (not a metaphor)
by lemonhope — 2024-08-08T20:07:13.593Z

A few of my new-parent friends started having terrible gut problems and figured they have colon cancer. Their doctors agreed it's the first thing to check. But the colonoscopies were negative for colon cancer. The tissue was inflamed though. One doctor called this "pre-cancer" (??)

Hmm what could be causing inflammation in the colon, but wouldn't show up on camera after you fasted and had medically-induced diarrhea for 24 hours?

The babies were born over a year before symptoms appeared, so it can't be related to pregnancy. No change in diet. No family history.

What happens a year or two after a kid is born? They go outside and immediately eat as much dirt as they can. What lives in dirt? Everything!

Me and my boy's mother ate some combantrin 3 months ago and have been clear since.

I'm currently trying to convince my friends that they didn't all get colon cancer in the same year at a young age. If I get them to eat the poison chocolate, then I'll write a follow up post in a few months.

I've actually had some very odd food issues since 2019 (eg seizures & fainting after garlic) which disappeared since the combantrin.

So if you randomly got food/gut/brain issues one day years ago you should consider taking a dewormer. Note that all the tests suck (insensitive) and the medicine is cheap and safe and sold online without prescription. (Albendazole available too but slightly less safe.) Worms are much easier to kill than bacteria, viruses, fungus, etc.

Also note that at least 5 million people in the US (ie 1.5%) have parasites according to the most conservative estimates offered here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7847297/ (table 1, the big table)

This seems to be a blind spot. No doctors or friends or family ever considered this or even mentioned the word "parasite" to me in the last 5 years.

Kind of funny that dewormers took off in poor countries but not here.

Edit sept 2: I think I am slowly gaining weight. It is well-known that malnourished kids gain weight after deworming. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1502184/ Good for them, but if you have overweight family members, you should do a cost-benefit analysis.
Some Unorthodox Ways To Achieve High GDP Growth
by johnswentworth — 2024-08-08T18:58:56.046Z

GDP growth, as traditionally calculated,

is a weird metric

. People interpret it as measuring “economic growth”, but… well, think about electronics. Electronics which would have cost millions of dollars (or more) in 1984 are now commonplace, everyone carries them around in their pockets. So if we calculate GDP growth based on 1984 prices, then GDP has grown multiple orders of magnitude since then, everyone now owns things which would make 1984’s wealthiest people jealous, and practically all of that growth has come from electronics. On the other hand, if we calculate GDP based on 2024 prices, then all of the digital electronics produced before, say, 2004 are worth almost nothing, so electronics contributed near-zero GDP growth throughout the entire internet boom.

Economists didn’t like either of those conclusions, so back in the 90’s, they

mostly switched

to a different way of calculating GDP growth: “chaining”. Basically, we calculate 1984-1985 GDP growth using prices from 1984-1985, then 1985-1986 GDP growth using prices from 1985-1986, and so forth. At the end, we multiply them all together (i.e. “chain” the yearly growth numbers) to get a long-term growth line. Chaining gives less dramatic GDP growth numbers when technological changes make previously-expensive things very cheap.

Chaining also opens up some interesting new methods for achieving high GDP growth.

A Toy Example

Suppose we have two goods, A and B. Over the course of five years, the price of each good and the amount consumed evolve as follows:

Year	Price A	Amount A	Price B	Amount B

1

$1

10

$10

1

2

$1

1

$10

10

3

$10

1

$1

10

4

$10

10

$1

1

5

$1

10

$10

1

The main thing to notice about this table is that year 5 is exactly the same as year 1; our toy economy goes full-circle back to where it started.

Now let’s calculate the GDP growth for this toy economy, using the same standard chaining method

adopted by

the Bureau of Economic Analysis for calculating US GDP back in the 90’s.

Calculation details (click to expand)

To calculate the GDP growth from year t to year t+1, we calculate the ratio of year t+1 to year t consumption using year t prices, then calculate the ratio of year t+1 to year t consumption using year t+1 prices, then average those together using a geometric mean. So the formula is:

Δt+1t=√∑iptiqt+1i∑iptiqti∑ipt+1iqt+1i∑ipt+1iqti

where:

i&nb

...
You can remove GPT2’s LayerNorm by fine-tuning for an hour
by StefanHex — 2024-08-08T18:33:38.803Z

This work was produced at Apollo Research, based on initial research done at MATS.
Edit: arXiv version available at https://arxiv.org/abs/2409.13710

LayerNorm is annoying for mechanistic interpretability research (“[...] reason #78 for why interpretability researchers hate LayerNorm” – Anthropic, 2023).

Here’s a Hugging Face link to a GPT2-small model without any LayerNorm.

The final model is only slightly worse than a GPT2 with LayerNorm[1]:

Dataset	Original GPT2	Fine-tuned GPT2 with LayerNorm	Fine-tuned GPT without LayerNorm
OpenWebText (ce_loss)	3.095	2.989	3.014 (+0.025)
ThePile (ce_loss)	2.856	2.880	2.926 (+0.046)
HellaSwag (accuracy)	29.56%	29.82%	29.54%

I fine-tuned GPT2-small on OpenWebText while slowly removing its LayerNorm layers, waiting for the loss to go back down after reach removal:

Introduction

LayerNorm (LN) is a component in Transformer models that normalizes embedding vectors to have constant length; specifically it divides the embeddings by their standard deviation taken over the hidden dimension. It was originally introduced to stabilize and speed up training of models (as a replacement for batch normalization). It is active during training and inference.

LN(x)=x−E[x]√Var[x]+ϵ⋅γ+β

The equation includes the standard deviation (std) √Var[x]+ϵ which makes it a non-linear operation. This hinders interpretability in a variety of ways, from annoyances and inaccuracies such as

attributing residual stream directions to logit effects (e.g. SAE features, direct logit attribution),[2]

being annoying to deal with Attribution Patching, or

being difficult to deal with in Apollo’s LIB method.

In the Docstring circuit analysis we seriously considered whether the model might be using LN in its algorithm. This post even shows that LN can be used as the sole non-linearity to solve non-linear classification problems (see also this related work).

Recently, with progress in Sparse Dictionary Learning, agendas (e.g. this one) imagine decomposing networks into sets of sparsely connected components (SAEs, Transcoders, etc.). A core difficulty to “putting it all together” is that the interactions between different components often route through LayerNorm whose effect we do not understand.

Motivation

It would be pretty neat to have an LLM that still works (speaks English etc.) while less or no LN layers. One option would be to train a ...
Leaving MIRI, Seeking Funding
by abramdemski — 2024-08-08T18:32:20.387Z

This is slightly old news at this point, but: as part of MIRI's recent strategy pivot, they've eliminated the Agent Foundations research team. I've been out of a job for a little over a month now. Much of my research time in the first half of the year was eaten up by engaging with the decision process that resulted in this, and later, applying to grants and looking for jobs.

I haven't secured funding yet, but for my own sanity & happiness, I am (mostly) taking a break from worrying about that, and getting back to thinking about the most important things.

However, in an effort to try the obvious, I have set up a Patreon where you can fund my work directly. I don't expect it to become my main source of income, but if it does, that could be a pretty good scenario for me; it would be much nicer to get money directly from a bunch of people who think my work is good and important, as opposed to try to justify my work regularly in grant applications.

What I'm (probably) Doing Going Forward

I've been told by several people within MIRI and outside of MIRI that it seems better for me to do roughly what I've been doing, rather than pivot to something else. As such, I mainly expect to continue doing Agent Foundations research.

I think of my main research program as the Tiling Agents program. You can think of this as the question of when agents will preserve certain desirable properties (such as safety-relevant properties) when given the opportunity to self-modify. Another way to think about it is the slightly broader question: when can one intelligence trust another? The bottleneck for avoiding harmful self-modifications is self-trust; so getting tiling results is mainly a matter of finding conditions for trust.

The search for tiling results has two main motivations:

AI-AI tiling, for the purpose of finding conditions under which AI systems will want to preserve safety-relevant properties.
Human-AI tiling, for the purpose of understanding when we can justifiably trust AI systems.

While I see this as the biggest priority, I also expect to continue a broader project of deconfusion. The bottleneck to progress in AI safety continues to be our confusion about many of the relevant concepts, such as human values.

I'm also still interested in doing some work on accelerating AI safety research using modern AI.

Thoughts on Public vs Private Research

Some work that is worth doing sh...
Does VETLM solve AI superalignment?
by Oleg Trott — 2024-08-08T18:22:26.905Z

Eliezer Yudkowsky’s main message to his Twitter fans is:

Aligning human-level or superhuman AI with its creators’ objectives is also called “superalignment”. And a month ago, I proposed a solution to that. One might call it Volition Extrapolated by Language Models (VELM).

Apparently, the idea was novel (not the “extrapolated volition” part):

But it suffers from the fact that language models are trained on large bodies of Internet text. And this includes falsehoods. So even in the case of a superior learning algorithm[1], a language model using it on Internet text would be prone to generating falsehoods, mimicking those who generated the training data.

So a week later, I proposed a solution to that problem too. Perhaps one could call it Truthful Language Models (TLM). That idea was apparently novel too. At least no one seems to be able to link prior art.

Its combination with the first idea might be called Volition Extrapolated by Truthful Language Models (VETLM). And this is what I was hoping to discuss.

But this community’s response was rather disinterested. When I posted it, it started at +3 points, and it’s still there. Assuming that AGI is inevitable, shouldn’t superalignment solution proposals be almost infinitely important, rationally-speaking?

I can think of five possible critiques:

It’s not novel. If so, please do post links to prior art.
It doesn’t have an “experiments” section. But do experimental results using modern AI transfer to future AGI, possibly using very different algorithms?
It’s a hand-waving argument. There is no mathematical proof. But many human concepts are hard to pin down, mathematically. Even if a mathematical proof can be found, your disinterest does not exactly help.
Promoting the idea that a solution to superalignment exists doesn’t jibe with the message “stop AGI”. But why do you want to stop it, if it can be aligned? Humanity has other existential risks. They should be weighed against each other.
Entertaining the idea that a possible solution to superalignment exists does not help AI safety folks’ job security. Tucker Carlson released an episode recently arguing that "AI safety" is a grift and/or a cult. But I disagree with both. People should try to find the mathematical proofs, if possible, and flaws, if any.

^

Think “AIXI running on a hypothetical quantum supercomputer”, if this helps your imagination. But I think that super

...
Toy Models of Superposition: what about BitNets?
by Alejandro Tlaie — 2024-08-08T16:29:02.054Z

Summary

In this post I want to briefly share some results I have got after experimenting with the equivalent version of the simple neural networks that the authors used here to study how superposition and poly-semantic neurons come about in neural networks trained with gradient descent.

The take-home message is that BitLinear networks are similar to their dense equivalents in terms of how feature superposition emerges. Perhaps, BitNets have a slight upper hand (less feature superposition and a more structured geometry) in some sparsity regimes. It is definitely too little work to extract strong conclusions from this, though!

Motivation

I will skip the majority of the details, as they are exactly the same as in the original work. However, for the sake of completeness, I will motivate this study and the main experimental parameters.

Why is superposition interesting?: In an ideal world, after we trained an artificial neural network, each neuron univocally mapped onto (humanly) interpretable features of the input. For example, in the context of computer vision, if one trains a system to classify cats vs dogs, it would be desirable for interpretability (i.e., so that these ANNs are not black boxes anymore) that we found that a given neuron is the "left-eye" neuron, another one the "nose" neuron, etc. This would imply that each of these units care about (and only about) a given feature of the input that we have concepts for. In contrast, if these neurons cared about more than one feature or if these features are not easily mappable to human ontologies, it would question the feasibility of interpretability.

Why are BitNets promising?: As it has been recently shown and reproduced, modifying dense weight matrices in regular Transformer-based architectures can lead do dramatic improvements in efficiency while keeping performance basically untouched. I wondered whether they would also add any nice property to the interpretability of neural activation.

Results

I took one of the simplest experiments of the original paper: I trained a few ReLU output models with different sparsity levels and visualize the results. For each sparsity level, I also trained a BitLinear network.

BitLinear networks seem to show less feature superposition

In the original paper, they[1] found that, as sparsity increases, feature superposition emerges. The explanation they give is that:

"the model r

...