trentmkelly/LessWrong-43k huggingface dataset:
Breakthroughs, Neurodivergence, and Working Outside the System
by j_passeri — 2024-10-14T16:54:16.617Z

In reflecting on historical figures like Einstein and Tesla, it becomes clear that disproportionate contributions often come from just a few highly impactful individuals—even within neurodivergent populations like those with Asperger’s. While many people with similar cognitive architectures work within conventional systems, these extreme outliers often seem to achieve more radical, transformative breakthroughs by working outside traditional structures.

This leads me to question whether systemic constraints in academia, corporate environments, or neurotypical-dominated social systems are holding back the majority of individuals with neurodivergent cognitive traits from realizing their full potential.

Disproportionate Contributions of a Few

Throughout history, there have been disproportionately high contributions from individuals with neurodivergent traits, like Albert Einstein, Nikola Tesla, and potentially even Thomas Edison. These figures were able to achieve breakthroughs that changed the trajectory of human knowledge and technology. The question is, why do these major breakthroughs come from only a few, while many others with similar cognitive abilities seem stifled?

The answer, I believe, lies in external pressures and systemic constraints.

Independence and Environment:

Einstein famously worked in the Swiss patent office, which gave him mental freedom away from the restrictions of the academic environment. He had the autonomy to explore unconventional ideas like special relativity.

Tesla, although often under financial strain, worked independently of many constraints imposed by established institutions. His ideas often clashed with mainstream thought, but this freedom allowed for his remarkable creativity.

External Stressors:

Many neurodivergent individuals, including those with Asperger’s, often operate within systems that impose neurotypical expectations. In academia, for example, masking and dealing with social pressures may cause them to spend more cognitive energy on conforming rather than innovating. This stifles potential breakthroughs.

Even for neurodivergent individuals, stress can limit the brain’s capacity to operate at its highest potential. High pressure from external forces—whether social, economic, or survival-based—can create barriers to full cognitive performance, impacting their ability to achieve disproportionately large gains.

The Cons...
The case for unlearning that removes information from LLM weights
by Fabien Roger — 2024-10-14T14:08:04.775Z

What if you could remove some information from the weights of an AI? Would that be helpful?

It is clearly useful against some misuse concerns: if you are concerned that LLMs will make it easier to build bioweapons because they have memorized such information, removing the memorized facts would remove this misuse concern.

In a paper Aghyad Deeb and I just released, we show it is tractable to evaluate the presence of certain undesirable facts in an LLM: take independent facts that should have all been removed, fine-tune on some of them, and see if accuracy increases on the other ones. The fine-tuning process should make the model “try” to answer, but if the information was removed from the weights (and if the facts are actually independent), then accuracy on the held-out facts should remain low.

Removing information from the weights is stronger than the usual notion of unlearning. Previous work that tried to develop unlearning methods focused on other properties such as the failure of probing, and jailbreak robustness - which have the drawback that if you find new jailbreaks,

you can extract information about bioweapons

from models which

were supposed to have this information "removed"

. Additionally, our evaluation of information removal should be relatively robust to sandbagging, as it is an i.i.d fine-tuning evaluation.

In this post:

I give a brief summary of the results of the paper
I present some hopes for why unlearning should be possible
I describe certain sets of facts that I think would reduce the risk of takeover from early transformative AIs if they were removed from model weights.

Do current unlearning techniques remove facts from model weights?

This is a summary of the paper.

No technique we tested can remove a significant fraction of the information from the weights.[1]

We look at several datasets, built such that learning facts in the T split should not help an LLM guess the facts in the V split if it didn't already know them:

Years: A dataset of major events and the years they happened in;
MMLU: The regular MMLU dataset, but we perform the T vs V split across categories (i.e. T and V never share the same subject);
WMDP-deduped: A filtered version of WMDP where we removed skill-based questions and questions about the same facts in order to prevent leakage between the questions we train on and questions we evaluate on;
Random birthdays: A dataset

...
Circuits in Superposition: Compressing many small neural networks into one
by Lucius Bushnaq — 2024-10-14T13:06:14.596Z

Tl;dr: We generalize the mathematical framework for computation in superposition from compressing many boolean logic gates into a neural network, to compressing many small neural networks into a larger neural network. The number of small networks we can fit into the large network depends on the small networks' total parameter count, not their neuron count.

Work done at Apollo Research. The bottom half of this post is just maths that you do not need to read to get the gist.

Introduction

Background

Anthropic's toy model of superposition shows how to compress many sparsely activating variables into a low dimensional vector space and then read them out again. But it doesn't show how to carry out computations on the compressed variables in their native format. The mathematical framework for computation in superposition makes a first stab at closing that gap. It shows how to compute boolean circuits in superposition.

What we do

We show how a network can perform any computations whatsoever in superposition. Specifically, we show how T small residual neural networks, each with n parameters that perform arbitrary tasks can be compressed into a single larger residual network that performs all T tasks, provided that the large network is only evaluated on sparse combinations of tasks — any particular forward pass only asks for k≪T tasks to be carried out. In the limit of T,n going to infinity, this larger network will require N=˜O(kTn) parameters[1].

Crucially, this means that the total number of small networks the larger network can implement scales approximately linearly with the number of weights in the network, not the number of neurons, as would be the case without computation in superposition. For example, if each small network uses m neurons per MLP layer and d dimensions in the residual stream, a large network with M neurons per MLP connected to a D-dimensional residual stream could implement about ˜O(MDkmd) small networks, not just
˜O(Mm). Qualitatively speaking, our construction works using same basic trick as the one for boolean circuits in superposition. We just generalize it from boolean AND gates to any operations the neural network could implement.

Generalising to circuits

While our derivation here assumes T networks carrying out unrelated ...
Beyond Defensive Technology
by ejk64 — 2024-10-14T11:34:24.595Z

I’ve been pretty confused by what it means for a technology or operation to be ‘defensive’.

Technologies do things. What does it mean for a technology to, like, be something that stops bad things? Is an anti-missile missile the same as a missile? Can we come up with a classification system that feels a bit more systematic than Vitalik Buterin’s classification based on whether things are

big or not

? Can we extend the great work being done around resilience and

adaptation to risks from AI

to technology more broadly?

And perhaps most crucially: if we can make technologies that stop bad things, does that mean that they’re inherently good and we should go ahead and def/acc making them?

In this short blog post, I make three claims.

1. Good ‘defensive’ technologies are status-quo preserving or entropy minimising.

The most obvious view of defensive technologies is as tools that counter deliberate attempts to disrupt a valuable system. However, we often want to design systems that defend against disruption without a conscious perpetrator (e.g. sprinkler systems against accidental fires).

For this reason, I’ve started thinking about ‘narrowly defensive’ technologies as a subcategory of ‘broadly defensive’ technologies (e.g. sprinklers) which broadly work to preserve the status-quo. For convenience, I’ll refer to the latter here as ‘defensive technologies’, but you could call them ‘anti-entropic’ or ‘status-quo preserving’ as preferred.

Specifically, this set of technologies either helps to 1) secure the status quo against interventions that would change it, 2) identify interventions that would change it or 3) return the situation to normal as quickly as possible after an intervention has changed it.

2. Second, and as such, not all defensive technologies are inherently ‘good’.

Making the status quo harder to change is not always a good thing. But it’s not inherently a bad thing, either.

Most defensive technologies might defend things that most people agree are good. But some defensive technologies might defend things that you might think are bad, and/or might make it harder for the people who do follow your value system to change this. As a basic example, you can encrypt biosecurity information secrets to prevent people from hacking and revealing it, but you can also

encrypt child porno

...
The Explore vs. Exploit Dilemma
by nathanjzhao — 2024-10-14T06:20:25.526Z
Link: https://nathanzhao.cc/explore-exploit

I often analogize real-world problems according to their ML-related counterparts. One of them is the exporation-exploitation problem, but it’s often been met with minimal recognition. I wrote this blog as something I can refer to the next time I get a “what do you mean?”
AI Alignment via Slow Substrates: Early Empirical Results With StarCraft II
by Lester Leong — 2024-10-14T04:05:05.096Z

A few months ago, I wrote a post about using slower computing substrates as a possibly new way to safely train and align ASI.

If you haven't read that post, basically the idea is that if we consider compute speed as a factor in Total Intelligence (alongside say, quality of intelligence), then it should be possible to keep quality the same and lower compute speed in order to lower Total Intelligence while keeping quality the same.

An intuition pump is to imagine a scenario where we are able to slow down Einstein's brain, by slowing actual biochemical and electrical processes, so that it produces the Theory of Relativity in 40 years instead of 10.

The obvious reason to do this would be to gain some degree of controllability so that a sharp left turn is less likely. However, a more compelling argument for this strategy is that it also opens up some interesting research directions - for example, revisiting Paul Cristiano's iterated amplification proposal, but varying compute speed instead of number of agents, which was the crux of Eliezer Yudkowsky's disagreement with Paul (Eliezer believed you couldn't easily preserve alignment across a number of agents). If this works, one could ostensibly scale up alignment all the way to fully aligned ASI, which would constitute a pivotal act.

Running the first tests

In the time since, I gave some thought as to how some parts of this hypothesis could be tested today. The first questions are whether the core assumption of lowering speed while maintaining quality is possible to test empirically with models we have today, and to what extent a weaker intelligence would be able to "catch up" to a stronger intelligence given a relative advantage in speed. This may be an important question, because it implies a related claim - namely, that speeding up weaker intelligences will put them on par with stronger intelligences, and certain aspects of this have been doubted by some members of the AI safety community ("making chimpanzees think faster, or giving them 1 million years/some arbitrarily large period of time to think, will not lead to them deducing the Theory of Relativity"). We may not be able to address these aspects fully at this point, but by deploying a testable environment, we can at least establish that some parts of it are true with empirical certainty.

I decided to explore this by looking at competitive games - namely re...
some questionable space launch guns
by bhauth — 2024-10-13T22:52:26.418Z
Link: https://bhauth.com/blog/space/longshot%20space.html

high school history

Recently, someone asked me about a startup called Longshot Space. They're trying to make big guns that're useful for space launch. Here's a video on them from Scott Manley. Many startups exist, and normally I wouldn't write a post about something like this, but it reminded me of my younger self.

Yes, back when I was a freshman in high school, I liked some flawed ideas, including:

non-rocket space launch systems, like launch loops

concentrated photovoltaic solar

text compression as a route to AI...with low compressor size as a metric

So I had email conversations about stuff like that with various people at universities instead of doing homework.

Those seem pretty cringe now, right? What I didn't realize at the time was:

If rockets are expensive, then the solution is to just make rockets more efficiently. Sure, governments already spent a trillion dollars on rocket development, but that doesn't matter for smart people with internet access.

Similarly, if solar panels are too expensive, then just make solar panels more efficiently. Notably, diamond-studded wire saws for slicing thinner wafers was an obvious idea but it took a long time to go from invention to implementation.

Neural networks with certain activation functions and L2 norm regularization don't overfit; see this post.

Now, past me was well ahead of Longshot Space conceptually, despite the tech progress since then, but I still have some sympathy.

Longshot's plan

The speed of gun projectiles is limited by the speed of sound of the propellant gas. Light gas guns can get high speeds by using hydrogen, which has a higher speed of sound.

It looks like Longshot Space has a bunch of gas tanks containing H2/O2, connected with burst discs to a gun barrel. The projectile has a long tail that gas pushes on sideways. The gas in a tank is ignited, the disc ruptures, and the hot gas pushes on the projectile tail.

Well...at least they're thinking about some of the problems...? Their design could well be cheaper than a comparable railgun...?

ram accelerators

Railguns and coilguns have a bunch of capacitors + wires + switches. This fancy gun thing has a bunch of tanks + igniters + rupture discs + gas fill lines. You know what's cheaper than those systems? Just the barrel part. Just a big metal tube.

Instead of the gun doing something as the projectile passes, we can simply have a ramjet projectile t...
The Hopium Wars: the AGI Entente Delusion
by Max Tegmark — 2024-10-13T17:00:29.033Z

As humanity gets closer to Artificial General Intelligence (AGI), a new geopolitical strategy is gaining traction in US and allied circles, in the NatSec, AI safety and tech communities. Anthropic CEO Dario Amodei and RAND Corporation call it the “entente”, while others privately refer to it as “hegemony" or “crush China”. I will argue that, irrespective of one's ethical or geopolitical preferences, it is fundamentally flawed and against US national security interests.

If the US fights China in an AGI race, the only winners will be machines

The entente strategy

Amodei articulates key elements of this strategy as follows:

"a coalition of democracies seeks to gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain, scaling quickly, and blocking or delaying adversaries’ access to key resources like chips and semiconductor equipment. This coalition would on one hand use AI to achieve robust military superiority (the stick) while at the same time offering to distribute the benefits of powerful AI (the carrot) to a wider and wider group of countries in exchange for supporting the coalition’s strategy to promote democracy (this would be a bit analogous to “Atoms for Peace”). The coalition would aim to gain the support of more and more of the world, isolating our worst adversaries and eventually putting them in a position where they are better off taking the same bargain as the rest of the world: give up competing with democracies in order to receive all the benefits and not fight a superior foe."

[…]

This could optimistically lead to an 'eternal 1991'—a world where democracies have the upper hand and Fukuyama’s dreams are realized."

Note the crucial point about “scaling quickly”, which is nerd-code for “racing to build AGI”. The question of whether this argument for "scaling quickly” is motivated by self-serving desires to avoid regulation deserves a separate analysis, and I will not further comment on it here except for noting that most other industries, from big tobacco to big oil, have produced creative anti-regulation arguments to defend their profits.

Why it’s a suicide race

If the West pursues this entente strategy, it virtually guarantees that China will too, which in turn virtually guarantees that both sides will cut corners on safety to try to “win” the race. The key point I will make is that, from a game-t...
Parental Writing Selection Bias
by jefftk — 2024-10-13T14:00:03.225Z

In general I'd like to see a lot more of people writing about their
failures in addition to their successes. If a bunch of people all try
a thing and have mixed results, and only the people with good results
write about it, people who don't know about this selection bias or
don't realize its extent are going to end up with overly positive
views. I've

written

about

some

of

my

mistakes, and I
think it would be good if this were a higher fraction of my posts.

On the other hand, once other people are involved this isn't entirely
up to me. One place this comes up a lot is parenting: I don't want to write things about my
kids that they don't (or won't) want public. This is especially
tricky if I write a post about something we've tried which worked well
in part because the kids did a good job with it, and then later they
stop doing a good job.

I don't have a good solution here. I don't want to go all the way to
"if this had come out with my kids looking bad I wouldn't write about
it, so I also won't write about it if they look good" because this
would exclude a huge fraction of things involving the kids (there are
a tremendous number of possible ways kids could do something that
would be embarrassing). Sometimes I can handle it with plausible
deniability (one of our kids did embarrassing thing X) but often it
would be clear to some people which kid actually did it, or it's bad
enough that even being in a pool of three is mortifying. Other times
I'm able to include some minor negative information, if it's about
them when they were enough younger and it's combined with positive
information. But mostly I think this will just need to be something
people keep in mind when reading my posts, and posts by other parents.

I asked one of my kids what they thought about this issue and they
suggested: "only write about things where [sibling] looks bad and I
look good, such as [redacted]".

Comment via: facebook, mastodon
Personal Philosophy
by Xor — 2024-10-13T03:01:59.324Z

This is a rough outline of my philosophical framework. Which gives me a context for all of my experiences and knowledge.

Certain Knowledge

We cannot know much for certain. The one exception is the existence of your subjective experience of reality in this very moment of time. As Rene Descartes put it "Cognito ergo sum."

Probabilistic Knowledge

From there I believe we make probabilistic assertions about the world based on evidence. For example a major question as a conscious entity is where did you come from?

There are an incredible amount of possibilities an imaginative person could conjure, and doubtless infinitely more of which no person could ever conceive.

Materialism

However there is one explanation for our existence which we have an incredible amount of evidence for. This is the materialistic explanation for our existence.

Your conscious experience has access to an external environment. We observe this environment. We seem to be inexorably tied to this environment. There is an incredible amount of evidence based on millions of man-hours of investigation, that says my conscious experience is emergent of the material world.

Mortality

An implication of this is that it is incredibly likely that when we die. There is no afterlife. That this subjective experience will come to an end just as it came to a beginning. There will come a time in which the material structure which gives rise to your experience fails. It will loose available energy the last few connections will fire and you will be unaware. Your brain will decompose and everything that makes you up will be gone.

It is still a potentiality that some greater god or alien species will take pity on you. They may find value in your conscious experience and will record it; bringing you into an afterlife alien or divine. However we have very little valid evidence supporting such a possibility. In a practical sense I would be more surprised to die and wake up in an afterlife than to walk in the gas station next door and win a million dollars my very first time playing the lottery.

Phenomenological Reality

Phenomenological reality is simply the subjective experience of reality. There are aspects of this reality which are distinct from materialistic reality. They are emergent from materialistic reality and dependent on it however they on meaningful within the context of human experience. The following ex...