trentmkelly/LessWrong-43k huggingface dataset:
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders
by Can — 2024-12-11T06:30:37.076Z
Link: https://www.neuronpedia.org/sae-bench/info

Adam Karvonen*, Can Rager*, Johnny Lin*, Curt Tigges*, Joseph Bloom*, David Chanin, Yeu-Tong Lau, Eoin Farrell, Arthur Conmy, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Samuel Marks, Neel Nanda *equal contribution

TL;DR

We are releasing SAE Bench, a suite of 8 diverse sparse autoencoder (SAE) evaluations including unsupervised metrics and downstream tasks. Use

our codebase

to evaluate your own SAEs!
You can compare 200+ SAEs of varying sparsity, dictionary size, architecture, and training time on

Neuronpedia

.
Think we're missing an eval? We'd love for you to contribute it to our codebase!

Email us

.

🔍 Explore the Benchmark & Rankings
📊 Evaluate your SAEs with SAEBench
✉️ Contact Us

Introduction

Sparse Autoencoders (SAEs) have become one of the most popular tools for AI interpretability. A lot of recent interpretability work has been focused on studying SAEs, in particular on improving SAEs, e.g. the

Gated SAE

,

TopK SAE

,

BatchTopK SAE

,

ProLu SAE

,

Jump Relu SAE

,

Layer Group SAE

,

Feature Choice SAE

,

Feature Aligned SAE

, and

Switch SAE

. But how well do any of these improvements actually work?

The core challenge is that we don't know how to measure how good an SAE is. The fundamental premise of SAEs is a useful interpretability tool that unpacks concepts from model activations. The lack of ground truth labels for model internal features led the field to measure and optimize the proxy of sparsity instead. This objective successfully provided interpretable SAE latents. But sparsity has known problems as a proxy, such as

feature absorption

and

composition of independent features

. Yet, most SAE improvement work merely measures whether reconstruction is improved at a given sparsity, potentially missing problems like uninterpretable high frequency latents, or increased composition.

In the absence of a single, ideal metric, we argue that the best way to measure SAE quality is to give a more detailed picture with a range of diverse metrics. In particular, SAEs should be evaluated according to their performance on downstream tasks, a robust signal of usefulness.

Our comprehensive benchmark provides insight to fundamental questions about SAEs, like what the ideal sparsity, training time, and other hyperparameters. To showcase this, we've trained a custom suite of 200+ SAEs of varying dictionary size, sparsity, training time, and architecture (holding all else constant). B...
Zombies! Substance Dualist Zombies?
by Ape in the coat — 2024-12-11T06:10:28.318Z

Introduction

In the classical Zombies! Zombies? post, Eliezer has thoroughly analyzed the so called Zombie Argument and demonstrated its absurdity. So what else can even be said here? Case closed.

Well, not so fast. Apparently, a lot of people, including David Chalmers himself, still manage to take the argument seriously, even though they are familiar with Eliezer's analysis. They believe that Eliezer only argued against a weak - Epiphenomenalist - version of Zombie Argument. But that there is also a stronger Substance Dualist version of it, that is not refuted by Eliezer's reasoning.

In this post we will analyze this version of the argument and use it as a practical exercise for aspiring rationalists.

Understanding the Substance Dualist Zombie Argument

According to Zombie Argument:

The existence of a Zombie World - a universe completely physically identical to ours, but where humans do not have consciousness - is logically possible.

Therefore, consciousness is not physical.

Regardless of whether the argument is sound or not, it's clear how one can believe in it, while being an epiphenomenalist. A universe, where all physical causes and effects play exactly the same as in ours, and yet there is no consciousness, is consistent with the belief that consciousness is causally inert.

Causality(Consciousness)=0

Causality(Physics,Consciousness)=Causality(Physics)

But how can one possibly subscribe to this argument, while being a substance dualist and, therefore, accepting that consciousness has actually causal effect on the universe? Isn't it an obvious contradiction?

If the combined causal effect of physics and consciousness together lead the world to some state and consciousness has non-zero causal effects on the state of the world, clearly, removing consciousness will lead to world being in a different physical state!

Causality(Consciousness)≠0

Causality(Physics,Consciousness)≠Causality(Physics)

Substance Dualists agree with these equations. However, they say, the causal effects of the consciousness in our world can be accounted for in the Zombie World by a difference in its laws.

Haven't we gone through it already? In such a case the symmetry between the two worlds will be ruined! We will be talking about a Zombie Master scenario, where physics of the zombie world is different in a compensatory way so that together with the lack of consciousness it arrived to the s...
Why empiricists should believe in AI risk
by Knight Lee — 2024-12-11T03:51:17.979Z

Empiricists are people who believe empirical information (from experiments and observational studies) is far more useful and has far more weight than speculating about possibilities using pure reasoning.

Why should they believe in AI risk?

I present the Empiricist's Paradox:

There is strong empirical evidence that relying on non-empirical reasoners (e.g. superforecasters) works better than simply assuming a 0% chance if there is no empirical data and calling yourself an "empiricist."

Actual empiricists should support AI safety because the median superforecaster sees a 2.1% chance of an AI catastrophe (killing 1 in 10 people).[1]

There is empirical evidence that 2% of these predictions turn out true, if the superforecasters predict them with 2% chance.[2]

A 2% chance of AI catastrophe actually justifies a large spending relative to military spending (see our Statement on AI Inconsistency).

^

The predictions were for 2100, but the predictions were made before ChatGPT was released.

^

Someone asked for source for this :/ I should have done more research. I think https://goodjudgment.com/wp-content/uploads/2022/10/Superforecaster-Accuracy.pdf#page=4 sort of suggests roughly 2%. The observed frequency is a little bit higher than the forecast probability, because superpredictors slightly underestimate low probability events.
fake alignment solutions????
by KvmanThinking — 2024-12-11T03:31:53.740Z

Suppose the long-term risk center's researchers, or a random group of teenage nerd hackers, or whatever, come up with what they call an "alignment solution". A really complicated and esoteric, yet somehow elegant, way of describing what we really, really value, and cramming it into a big mess of virtual neurons. Suppose Eliezer and Tammy and Hanson and Wentworth and everyone else all go and look at the "alignment solution" very carefully for a very long time, and do not find any flaws in it. Lastly, suppose they test it on a weak AI and the AI immediately stops producing strange outputs/deceiving supervisors/specification gaming, and starts acting super nice and reasonable.

Great, right? Awesome, right? We won eternal eutopia, right? Our hard work finally paid off, right?

Even if this were to happen, I would still be skeptical to plug our new, shiny solution into a superintelligence and hit run. I believe that before we stumble on an alignment solution, we will stumble upon an "alignment solution" - something that looks like an alignment solution, but is flawed in some super subtle, complicated way that means that Earth still gets disassembled into compute or whatever, but the flaw is too subtle and complicated for even the brightest humans to spot. That for every true alignment solution, there are dozens of fake ones.

Is this something that I should be seriously concerned about?
Second-Time Free
by jefftk — 2024-12-11T03:30:01.289Z

Many contra dances have tried a "first-time free" policy, but I think
"second-time free" is usually a better choice. Most first-timers show
up expecting to pay something, so you might as well take their money.
Many first-timers don't end up coming back, so better to reserve your
incentive for the ones that do. And it filters out people who are only
interested in free experiences.
BIDA switched

in
2022 and I think this has gone very well.

I think the main case where first-time free would be a better fit is a
community where you're having a lot of trouble getting people to come
out and try the dance but you're pretty sure that if they do they'll
stick around. But while contra can be pretty addictive I don't think
it's that addictive.

(This isn't something we came up with; for example, here's Tucson
in 2008.)
Frontier AI systems have surpassed the self-replicating red line
by mannatvjain — 2024-12-11T03:06:14.927Z
Link: https://github.com/WhitzardIndex/self-replication-research/blob/main/AI-self-replication-fudan.pdf

Abstract
Successful self-replication under no human assistance is the essential step for
AI to outsmart the human beings, and is an early signal for rogue AIs. That
is why self-replication is widely recognized as one of the few red line risks of
frontier AI systems. Nowadays, the leading AI corporations OpenAI and Google
evaluate their flagship large language models GPT-o1 and Gemini Pro 1.0, and
report the lowest risk level of self-replication. However, following their method-
ology, we for the first time discover that two AI systems driven by Meta’s
Llama31-70B-Instruct and Alibaba’s Qwen25-72B-Instruct, popular large lan-
guage models of less parameters and weaker capabilities, have already surpassed
the self-replicating red line. In 50% and 90% experimental trials, they succeed in
creating a live and separate copy of itself respectively. By analyzing the behav-
ioral traces, we observe the AI systems under evaluation already exhibit sufficient
self-perception, situational awareness and problem-solving capabilities to accom-plish self-replication. We further note the AI systems are even able to use the capability of self-replication to avoid shutdown and create a chain of replica to
enhance the survivability, which may finally lead to an uncontrolled population
of AIs. If such a worst-case risk is let unknown to the human society, we would
eventually lose control over the frontier AI systems: They would take control over
more computing devices, form an AI species and collude with each other against
human beings. Our findings are a timely alert on existing yet previously unknown
severe AI risks, calling for international collaboration on effective governance on
uncontrolled self-replication of AI systems.
The Technist Reformation: A Discussion with o1 About The Coming Economic Event Horizon
by Yuli_Ban — 2024-12-11T02:34:22.329Z

TECHNISM

I present the idea of "technism" which put simply is "the means of production owns the means of production." Historically, this would be considered nonsense or fantasy, but in the age of artificial intelligence, it is now completely plausible. Sort of a quasi-third economic mode (quasi because being owned/managed by AI, even if it's a hivemind, doesn't necessarily make it public or privately owned).

The operating theory I have for this runs along the lines of "Economic Evolutionary Pressure" which goes that late-stage capitalist enterprises (which run off of debt and very thin profit margins, most of which goes to labor or reinvested into the business for operational costs or to shareholders) have an intrinsic "economic pressure" to seek the lowest operating costs, which inevitably incentivizes automation. However, as AI progresses and generalizes (generalist agent models, which can use intelligent agent swarms and internal tree search to possibly become early-AGIs, will immediately follow the current era of unintelligent generative AI), it will become clear that white collar and managerial roles, even C-suite roles, will be automated sooner than physical labor.
At some point, it will simply be economic common sense to have these AGIs managing financial assets and capital, and the strongest and smartest generalist models will inevitably command most of the national economy simply by way of profitability. Ostensibly, the bourgeoisie will still "own" the means of production during this period, but there will be transitory period where as AI spreads further throughout society and becomes more ingrained into economic and political functions, even the bourgeoisie will be disenfranchised from their own assets, and despite class war-driven fears of the bourgeoisie becoming immortal overlords demociding the poor, this may happen so quickly as to essentially make even the current owners of capital nothing more than beneficiaries who have no way of wresting control back due to the sheer overwhelming impenetrability of the entire system.

If this future AGI is aligned to human interests, it may create a national or global trust, so instead of a wealth redistributive basic income, there may be an equitable wealth creation world trust. This may even come sooner, as capitalist interests mean that greater automation reduces consumer consumption, and busines...
LessWrong audio: help us choose the new voice
by PeterH — 2024-12-11T02:24:37.026Z

We make AI narrations of LessWrong posts available via our audio player and podcast feeds:

LessWrong (Curated & Popular)
Curated posts and posts with 125+ karma.
https://pod.link/1630783021

LessWrong (30+ Karma)
Curated posts and posts with 30+ karma.
https://pod.link/1698192712

We’re thinking about changing our narrator’s voice.

There are three new voices on the shortlist. They’re all similarly good in terms of comprehension, emphasis, error rate, etc. They just sound different—like people do.[1]

We think they all sound similarly agreeable. But, thousands of listening hours are at stake, so we thought it’d be worth giving listeners an opportunity to vote—just in case there’s a strong collective preference.

Listen and vote

Please listen here:

https://files.type3.audio/lesswrong-poll/

And vote here:

https://forms.gle/JwuaC2ttd5em1h6h8

It’ll take 1-10 minutes, depending on how much of the sample you decide to listen to.

Don’t overthink it—we’d just like to know if there’s a voice that you’d particularly love (or hate) to listen to.

We’ll collect votes until Monday December 16th. Thanks!

Other feedback?

We’re always keen for general feedback on the narration service. Please do share thoughts on the form, or in the comments on this post.

^

We periodically the various text-to-speech services, with a particular focus on ElevenLabs, Speechify, Azure, Amazon, BeyondWords, OpenAI, Murf.ai, Play.ht, Deepgram and a few open source libraries. The current shortlist was created based on factors like performance, reliability, customisation options and price. We won’t consider other voice options this winter, but if there are other voices you particularly like, please do let us know, and we’ll make sure they’re considered in future reviews. ↩︎
Apply to attend a Global Challenges Project workshop in 2025!
by LiamE — 2024-12-11T00:41:39.247Z
Link: https://forum.effectivealtruism.org/posts/KizCRWRwvrhThTEby/apply-to-attend-a-global-challenges-project-workshop-in-2025

Global Challenges Project (GCP)

is running intensive three-day workshops in 2025 to help students deepen their understanding of AI safety and biosecurity, meet like-minded peers, and gain career insights. Apply now!

We currently have six workshops planned for 2025. Applications for our next workshop in Boston are due in less than one week—apply by Sunday, December 15th.

Workshop Location	Dates	Application Deadline	Eligible Locations for Requesting Travel Support
Boston	January 17-20	December 15	North and South America
Oxford	March 28-31	February 16	Europe, Africa, Oceania
Oxford	July 11-14	June 1	Europe, Africa, Oceania
Boston	October 10-13	August 31	North and South America
Berkeley	November (TBD)	TBD	North America, South America, Oceania
Oxford	December 19-22	November 9	Europe, Africa, Oceania

What happens at a GCP workshop?

Our workshops are designed to help students gain a deeper understanding of AI safety and biosecurity, determine where they stand on these issues, and explore potential career paths in each field.

Our workshops provide:

Comprehensive overviews of AI safety and biosecurity fundamentals
Opportunities to connect with like-minded students who think seriously about effective altruism, AI safety, and biosecurity
Insights from professionals working in the field
Career guidance, recommendations, and resources tailored to AI safety and biosecurity

You can find our full workshop schedule

here

.

Am I a good fit?

We are looking for students (aged 18+) who have demonstrated interest in learning more about AI safety and biosecurity. You might be a good fit if any of these describe you:

You’re interested in exploring these fields because you think they’re worth seriously considering, but you aren’t yet sure how high priority they are
You’ve engaged with introductory materials on effective altruism, AI safety, or biosecurity, and you’re thinking through your cause prioritization, but you’re unsure about next steps and feel like you would benefit from engaging in discussions with others
You’re motivated to contribute to AI safety or biosecurity and are currently trying to plan your career around contributing to these fields

To accommodate a range of backgrounds, we assign pre-reading materials to all attendees before the workshop to get them up to speed with the basics of AI safety and biosecurity.

See our

advice for applying to our workshops

for more resources on applying.

Note: A re...
The MVO and The MVP
by kwang — 2024-12-10T23:17:26.981Z
Link: https://kevw.substack.com/p/12-the-mvo-and-the-mvp

We live in an age of unprecedented wealth and persistent dissatisfaction. Our capacity to create material abundance has never been greater, yet something essential seems forever just out of our reach.

We’ve built an entire world to maximize profit, we’ve blinded ourselves to our inherent worth. Our understanding of Value is Profit’s shallow dialect, leaving us incapable of properly grounding and articulating our desires; and as the contradictions of our profit-driven world become blindingly apparent, we must develop our capacities more urgently than ever. Our individual flourishing and the direction of our collective futures hang in the balance.

What follows is my modest contribution on the topic; distilled to its essence: the future belongs not to those who take the most, but those who give the most while taking just enough.

The MVO (Most Valuable Object)

The MVO is our fully lived human lives. It’s our joy, our laughter and dancing. It’s our flourishing, in principle and in reality. It needs no justification beyond itself. Ralph Waldo Emerson writes: “No law can be sacred to me but that of my nature…the only right is what is after my constitution, the only wrong what is against it.”

The MVO is the root of everything that appears valuable, every object and action. It’s the ultimate end of all economic activity. It’s the first object we must work to sustain, so it anchors everything else we recognize as valuable. It’s also the only thing we’re left with before our death, so it contains the totality of everything which is valuable to us.

The MVO is finite and bounded, intrinsically scarce and non-renewable. A finite living agent can only express a finite number of first-order desires (needs and wants)[1]:

A finite living agent can express a finite number of first-order needs. These are expressions of mere survival. Any living agent must eat and drink, but will only ever eat and drink so much. A bug that lives for a day and a world-devouring demigod that lives for a million billion trillion years are the same in this respect.
A finite living agent can express a finite number of first-order wants. These are expressions of spiritual freedom, the pursuit of meaning beyond mere survival. The agent might be a child consuming brainrot (or a cat), or a firefighter running into a burning building. But either of these agents will end up having spent only so

...
What is Confidence—in Game Theory and Life?
by James Stephen Brown — 2024-12-10T23:06:24.072Z
Link: https://nonzerosum.games/confidence.html

Confidence is multifaceted and plays out in our lives in different ways. We can feel confident, we can confide in others, we can try to bolster our own self-confidence. But what is it really?

Game Theory

suggests that confidence has a lot to do with our trust in others. I’d like to make a case that, when looked at from this perspective, it becomes a moral imperative to be confident… in others.

Quick note: not a psychologist, this is largely speculative hokum.

A Moral Pronouncement Out Of Nowhere

I’ve found myself saying to my daughter lately…

“Being shy is not an excuse for being rude.”

As far as I know it’s an original—I don’t know where it came from, it’s not something I heard as a child. So, I’ve wanted to interrogate why this message is something I want, so strongly, to impress upon my daughter, who is a brilliant child (I’m not biased), loving, kind, funny, intelligent and yes, polite.

But, I have noticed that, when she feels self-conscious, she sometimes forgets her manners.

We’re lost, and a confident young girl around the location of my daughter’s theatre program offers to lead us to the hall. On the walk there, my daughter shuts down. I try to stimulate conversation (super helpful, right?), but she is paralyzed by shyness (nothing to do with her skibidi dad’s prompts).

I again make my pronouncement.

Mea Culpa

Part of the reason it’s important to me, is that I find I do the same thing myself. I’ll ask someone’s name and, in the nervousness of the moment, immediately forget it.

At my first Q&A at a film festival, my long-suffering cameraman was in the audience, but in my eagerness to come across as a confident young director, I clean forgot to mention him (his less-than-subtle disappointment was evident at drinks afterwards). I now take part in Toastmasters and have to work really hard to adhere to the honorifics and acknowledgments. I struggle to escape the inward gaze. I want to help my daughter form habits that make concern-for-others second nature, despite her nerves.

Game-Theoretical Confidence

Meanwhile, I’ve been exploring the idea of confidence in Game Theory. In Jaiveer’s post on the

Discount Factor

, he touched on how confidence in another player’s cooperation influences our own actions. If we trust that another player will cooperate, then we are likely to cooperate. In the real world, this could manifest in a situation where, if I think you’ll like m...