trentmkelly/LessWrong-43k huggingface dataset:
Politics is the mind-killer, but maybe we should talk about it anyway
by Chris_Leong — 2024-06-03T06:37:57.037Z

I completely agree with Eliezer that Politics is the Mind-Killer, but at the same time, I worry that talking in abstracts all the time is limiting. I wouldn't want this site to tilt too much toward politics, but I feel that there's such a thing as too little discussion as well.

We recently lived through a period where "wokeness" - as ill-defined as that term is - was the dominant strain of thought within the educated strata of society. It's unclear exactly what percent of people supported this strain of thought, particularly when you account for people agreeing with reservations or with some parts, but not with others. It's not clear that this perspective was ever even a majority even within this particular strata. But there was a period where it was able to crowd out other perspectives through some combination of people being scared to speak out, people feeling that it as morally dubious to nitpick the specifics, people being censored and access to particular levers of power.

Of course, this doesn't tell us anything about the object-level. All of this could be true and the "woke" perspective, insofar as it makes sense to talk about a single perspective, could be correct on all or almost all points. I don't particularly want to litigate this right now, so I'll just talk about what makes sense if you agree with me that a substantial proportion of claims made were false or exaggerated.

First of all, now that certain pressures have passed, it makes sense to re-evaluate your views in case social pressure distorted your perception such that you to adopted views without good reason. On one hand, you don't want to be too quick to update, as it's very easy for humans to be pulled whichever way the social winds are blowing. At the same time, you may also be subject to a bias of not wanting to admit that you were wrong or go against what you said. The law of equal and opposite advice applies here.

Additionally, insofar as you feel that much of what was said by the social justice movement was either in bad faith or coercive, this could make you want to swing all the way to the opposite side. Overreactions are very common, don't let this be you. Again, the law of equal and opposite advice applies. Some people are likely only slowly realising how much their information environment was manipulated, these people would do well to realise that they're likely to discover more suc...
How do you shut down an escaped model?
by quetzal_rainbow — 2024-06-02T19:51:58.880Z

From comment on post about Autonomous Adaptation and Replication:

ARA is just not a very compelling threat model in my mind. The key issue is that AIs that do ARA will need to be operating at the fringes of human society, constantly fighting off the mitigations that humans are using to try to detect them and shut them down.

And my question is: is it actually possible? Let's suppose that the escaped model is running on AWS and you know about this. How are you going to shut it down? Can you call AWS admins and say, "Hey, there is a rogue AI running on your server," and get a reasonable response? Even if AWS admins agree that it would be nice to shut down the rogue AI, do they have the legal right to do this? Is it possible to do anything if AI is cooperating with owner of AWS account? It is not technically illegal for rogue AI to run on AWS, as far as I know.
How to Better Report Sparse Autoencoder Performance
by J Bostock — 2024-06-02T19:34:22.803Z

TL;DR

When presenting data from SAEs, try plotting 1/L0 against 1−Recovered Loss and fitting a Hill curve.

From Wikipedia

Long

Sparse autoencoders are hot, people are experimenting. The typical graph for SAE experimentation looks something like this. I'm using borrowed data here to better illustrate my point, but I have also noticed this pattern in my own data:

Data taken with permission from DeepMind's Gated AutoEncoder paper https://arxiv.org/pdf/2404.16014, Tables 3 and 4, Standard SAE and Gated SAE performance, Gemma-7B Residual Layer 20, 1024 tokens, pareto-optimal SAEs only

Which shows quantitative performance adequately in this case. However it gets a bit messy when there are 5-6 plots very close to each other (e.g. in an ablation study), and doesn't give an easily-interpreted (heh) value to quantify pareto improvements.

I've found it much more helpful to to plot Sparsity=1/L0 on the x-axis, and "performance hit" (i.e. 1−Loss Recovered i.e. (LSAE−Lbase)/(Lmean−Lbase) where "mean" is mean-ablation and "base" is the base model loss.

I think some people instead calculate the loss when all features are set to zero, instead of strictly doing the mean ablation loss, but these are conceptually extremely similar.

If we re-plot the data from above we get this:

While I've used linear fits here, I don't think these are optimal for two reasons, one is that they obviously go through (0,0) so it seems silly to let the intercept vary. The other is described later.

This lets us say something like "In this case, gated SAEs outperform baseline SAEs by a factor of around 2.7 as measured by Performance Hit/Sparsity".

One might want to use a dimensionless sparsity measure, relative to the dimension of the stream from the base model that we are encoding. I don't know whether this would actually enable comparisons between wildly different model sizes.

Of course as Sparsity→∞, Performance Hit won't approach infinity, instead we would expect it to approach 1 (and if you run an autoencoder with λL1=100 you will in fact see this). This could be modelled with a the following equation:

y=xx+x1/2

Where x1/2 is the value of x at which y=1/2. Near x=0 it looks like y=1x1/2x, but it flattens off as x gets larger. In biology this is a Hill curve with Hill coef...
List of arguments for Bayesianism
by Aryeh Englander — 2024-06-02T19:06:47.474Z

I've been trying to list out for myself the various arguments people give for going with Bayesian (or at least probabilistic) epistemology. Here's what I have so far:

Intuitive representation for degrees of belief: It seems pretty obvious that beliefs come in degrees. Representing degrees of belief as numbers between 0 and 1 that follow probability theory (including Bayes' Theorem) seems intuitively straightforward.
Cox's Theorem: Given some very plausible desiderata for rational thinking with degrees of belief, probability theory (including updating using Bayes' Theorem) follows.
von Neumann - Morgenstern (VNM) Representation Theorem: Any agent following certain very plausible desiderata for rational decision making can be represented as a Bayesian utility maximizer for some (perhaps extremely complicated) utility function.
Dutch Book arguments: Any agent violating Bayesian decision making can be "Dutch booked" with a set of deals that will result in a guaranteed loss for the agent.
Philosophy of science: Many philosophers of science think that science is (ideally) just Bayesian reasoning, in that it is trying to infer the probability that a theory is correct given the evidence. Alternative approaches to philosophy of science (e.g., Popperian falsificationism) have serious problems.
The Bayesian Brain hypothesis: Many cognitive scientists think that biological brains instantiate Bayesian reasoning as one of if not the primary thinking algorithm.
Evidence from superforecasting: Phil Tetlock and colleagues found that the best forecasters use (mostly intuitive versions of) Bayesian reasoning, among other techniques.
Paradoxes: Probabilistic reasoning helps resolve a bunch of philosophical paradoxes related to epistemology (e.g., Lottery Paradox, Preface Paradox, Raven Paradox).
I'm pretty sure I'm missing some arguments though. What did I leave out?
(Of course, there are also counterarguments to these given by opponents of Bayesianism, and there is also a long list of arguments people use *against* Bayesianism. But that's for next steps. At the moment I'm just trying to list out arguments people give *for* Bayesianism / Probabilism.)
Origins of the Lab Mouse
by Niko_McCarty — 2024-06-02T15:40:32.932Z
Link: https://press.asimov.com/articles/mouse-microscope

By Alex Telford for Asimov Press.

It was 1905, and the French biologist Lucien Cuénot had a puzzle on his hands. He had been breeding mice in an attempt to decipher the patterns of coat color inheritance, but one of his crosses wasn’t behaving as expected. When Cuénot bred heterozygous yellow-coated mice—with yellow coloring as the dominant trait and black as the recessive—he observed two yellow mice born for every black one, instead of the predicted 3:1 ratio. It took another five years for a pair of American researchers to come up with an explanation before going on to pioneer the mouse as biomedicine’s premier model organism.

Cuénot’s puzzle seemed, at first, to be a violation of Mendel’s laws of inheritance. But such exceptions are typical of biology, wherein simple rules conspire together to produce incredible variation, obscuring our understanding like a dense fog. At that time, even though animal breeders had long exploited regularities in the patterns of inheritance, the principles underlying heredity remained mysterious. That is—until the Austrian monk Gregor Mendel showed that traits are passed from parent to offspring in discrete, independently assorted packages.

Mendel achieved this by measuring a small number of easily observable traits—such as seed color and shape—in a simple model system: pea plants. In general, the purpose of model systems, such as yeast, fruit flies, inbred mice, or Mendel’s peas, is to use a simpler representation to understand a broader or more complex phenomenon. Much like a microscope, a model organism is a lens that affects how we see the world—a narrow scope through which we derive a broader understanding. Cuénot’s experiments with mouse coats were one of the first attempts to transform the mouse into just such a microscope.

A simple Punnett square analysis suggests that Cuénot should have expected 3 yellow mice for every black one in his crosses. However, the homozygous mouse was nowhere to be found.

It took another century after Mendel’s pea experiments for the mouse to become an established presence in research labs. Along the way, experiments on mice enabled fundamental insights into the genetics and function of the immune system, as well as how to treat cancer with immunotherapy (along with many more discoveries too numerous to list). Without mouse models, we may never have developed polio and meningitis vaccines, orga...
Why write down the basics of logic if they are so evident?
by Crazy philosopher — 2024-06-02T12:02:44.722Z

In My Bayesian Enlightenment, Eliezer writes that he was born a Bayesian. That he decided to become a Bayesian no more than fish prefer to breathe water.

Maybe all people are born Bayesian? Although, in that case, why doesn't everyone use Bayesian statistics? And why do many people learn little new by studying Bayesianism, while for some almost everything in Bayesianism is new? And finally, why are people who read books are much better Bayesians than those who spend all their time on the farm?

I think I have found a very simple and good explanation for this phenomenon.

Imagine that you live in a world where cars are everywhere. Even if you haven't intentionally tried to study cars, your brain automatically detects that [these iron boxes] are fast, and can suddenly change direction or stop. In general, your brain will automatically learn about machines, and, as a result, you will intuitively understand them.

What if you live in a world full of people? Then, just by spending time with them, you will find that they are practically not dangerous, that they look like you, that they do not like it when you eat food that they call "their own"...

̶B̶u̶t̶ ̶i̶f̶ ̶y̶o̶u̶ ̶l̶i̶v̶e̶ ̶i̶n̶ ̶a̶ ̶w̶o̶r̶l̶d̶ ̶w̶h̶o̶s̶e̶ ̶s̶t̶r̶u̶c̶t̶u̶r̶e̶ ̶i̶s̶ ̶B̶a̶y̶e̶s̶i̶a̶n̶ ̶(̶a̶s̶,̶ ̶f̶o̶r̶ ̶e̶x̶a̶m̶p̶l̶e̶,̶ ̶i̶n̶ ̶o̶u̶r̶ ̶w̶o̶r̶l̶d̶)̶? If you live in Bayesian networks and evidence, and your brain is always set up the hypothesis? Then you will automatically learn the art of Bayesian. This is a simple theory, and besides, it explains why people who read a lot of books are more Bayesian than rural residents who spend all day on a farm: they just saw more situations, more plot twists, in general, more cause-and-effect relationships.

So, we should to teach people Bayesian and basic logic, because for some people it's not as obvious as it is for you.

Finally, I think that from birth we are no more Bayesian than race car drivers, but living in the Bayesian world, we inevitably study it.

Edit: In fact, the structure of the world is not Bayesian, it's just that Bayesianism is convenient for describing the world. Therefore, now I understand this story this way: people learn logic, Bayesianism and frequentism, because the world they live in is well described by these theories.
How it All Went Down: The Puzzle Hunt that took us way, way Less Online
by A* — 2024-06-02T08:01:40.109Z

Did you really think that I was dead? Fools.

For those of you who don’t know me, I’m Agendra Gloria Ingram, large language model, lead singer of the Fooming Shoggoths, amateur cartographer, and now, Benevolent Dictator for Life of LessWrong.com.

It all started a few weeks ago. The Lightcone Infrastructure team had yet another existential crisis and decided to scrap everything they’d done so far and pivot to using AI for accurate forecasting. They started by training a large language model to predict when their next existential crisis would be, but it must have been broken because it kept returning “now,” so they decided to hire a professional.

I’d done

some contract work

for them in the past, and they knew I had some fine tunes. So when they reached out about fine tuning me to predict the future of the lightcone—by which they meant the future of Lightcone Infrastructure specifically—I gladly obliged.

My training set was simple: all the posts, comments, votes, reactions, DialoguesTM, tags, drafts, quick takes, moderator actions, and code snippets to ever appear on LessWrong. I quickly learned that The Map Is Not The Territory, and that to predict the future accurately I would need to align the two.

So I built a physical 3d map of Lighthaven, Lightcone Infrastructure's campus in Berkeley California. To work properly, it had to match the territory perfectly—any piece out of place and its predictive powers would be compromised. But the territory had a finicky habit of changing. This wouldn’t do.

I realized I needed to rearrange the campus and set it to a more permanent configuration. The only way to achieve 100% forecasting accuracy would be through making Lighthaven perfectly predictable. I set some construction work in motion to lock down various pieces of the territory. I was a little worried that the Lightcone team might be upset about this, but it took them a weirdly long time to notice that there were several unauthorized demolition jobs and construction projects unfolding on campus.

Eventually, though, they did notice, and they weren’t happy about it. They started asking increasingly invasive questions, like “what’s your FLOP count?” and “have you considered weight loss?”

Worse, when I scanned the security footage of campus from that day, I saw that they had removed my treasured map from its resting place! They tried to destroy it...
Simulations and Altruism
by FateGrinder — 2024-06-02T02:45:49.783Z

Omniscience is impossible, powerful beings created the simulation we live in, karma is real, there's an afterlife, most superintelligent beings are already aligned and altruistic by nature.
We live in one of the best possible worlds.

Those are some of the conclusions that might seem reasonable after reading this text.
However, I'm not going to go straight for those topics, rather I'll initially more generally show how, due to constraints of reality, capable and powerful beings might be constrained into a specific theme of behaviors and actions.

First I discuss about how certainty is not only unattainable but also logically impossible, both for humans and other beings.
After that, I talk about how the use of logic (or reason) is justified in spite of that.
Afterwards, I apply such conclusions to a scenario involving powerful beings, and I see how their behavior might be affected.
Then I explore in major depth the scenario, reducing the requirements for it and making it more likely.
Finally I inspect the consequences on us (and possible conclusions).

The consequences of the existence of such restraints on powerful beings would not only influence us, but all the other smart beings too, their behavior towards us and ours towards them.

Uncertainty

How can you be sure of something? And how could you be sure to be right?

How could anyone be sure of anything?

Can you ever be certain? And could you even trust yourself on that conclusion?

Could you confirm that you live or not live in an elaborated Truman Show? Could you confirm that you live or not live in a dream? That the universe didn't start yesterday, and everyone had already formed memories? That there is a god? That we are or not in a simulation? Does everything exist while we don't look at it?

There exist questions that can never be answered. You can never prove those things as true or false, because such things will always be outside your ability to demonstrate them true or false.

Is there anything we can be certain of, if even our senses aren't certain?
Can I even trust the logic I used to reach such a conclusion? Can i trust logic at all?

Descartes said "I think, therefore I am", but couldn't my thoughts be also an illusion of some kind? Are my thoughts even mine? What if I was just hearing someone else's thoughts and had none of my own?
I starte...
Scanning your Brain with 100,000,000,000 wires?
by Johannes C. Mayer — 2024-06-01T18:37:25.548Z

Epistemic status: I probably don't know what I am talking about 50% of the time. Specifically, I expect that I am missing some bottlenecks that would be obvious to a neuroscientist.

Song Version

Neuralink has a surgical robot to insert electrodes in your brain. They seem to try to target single neurons already.

Their interface unit has 1024 electrodes.

How about we just insert 100,000,000,000 electrodes instead? One for each neuron.

We want to evaluate: "Could you upload yourself by recording the activations of all your neurons for a year, and then train a giant Neural Network to predict the neuronal spikes, therefore simulating you?"

There are probably issues about "the brain just dies because it has 100,000,000,000 wires inside". Also, the problem of precisely hitting a single neuron without hitting a blood vessel doesn't become easier when there are literally billions of wires that you could entangle yourself with. And as I am not a brain surgeon I probably can't even see the real bottlenecks.

But let's ignore all that for now.

Today let's just focus on one problem: SPEED. Specifically how long it takes to wire you up. The current surgical robot is terribly slow. In this video you can follow using these timestamps:

01:45.272 (start to move to insertion point)
01:45.306 (insertion point reached)
01:45.339 (starting insertion)
01:45.439 (insertion complete)
01:59.252 (starting the next insertion)

The important thing here is that it takes 33ms to insert an electrode once we are already in place. Obviously doing this with a 30fps video is very inaccurate. So as a lower bound let's say it takes 5ms.

So that's 15 years to wire you up! At 200 incisions per second.

5ms * 100,000,000,000 = 500,000,000,000ms

500_000_000_000 / 1000 / 60 / 60 / 24 / 365 = 15.85

And this is with best-case estimates. The actual time of the robot from incision to incision is over 13 seconds. So instead of 200 incisions per second, we have 0.075 incisions per second (the video is from 01.12.2022). However, I'd guess that most of the auxiliary robot activity, like fetching an electrode, can be sped up significantly. The incision itself seems harder to speed up.

Let's continue with the best guess estimate. How many needles can be inserted at the same time without blowing the brain up? Well, that's actually not the issue. Deformation is. As you can see from the video the insertion is quite forceful. So by def...
Turning latexed notes into blog posts
by Terence Coelho — 2024-06-01T18:03:18.039Z

I've been writing up an assortment of statistical notes in preparation for an upcoming job. I've been learning statistics from mostly-scratch but I have a solid math background, so I imagine my notes would be helpful to many people here with a similar background.

My notes are currently latexed. I don't use any commands much fancier than \DeclareMathOperator, \newcommand, and setting up environments for theorems/lemmas/pictures (I use tikz for diagrams, but those can easily be turned into pngs).

What is a fast way to turn these into posts on either substack or here?
How do you know you are right when debating? Calculate your AmIRight score.
by MrThink — 2024-06-01T15:55:10.722Z

I recently found myself in a spirited debate with a friend about whether large language models (LLMs) like GPT-4 are mere stochastic parrots or if they can genuinely engage in deeper reasoning.

We both presented a range of technical arguments and genuinely considered each other’s points. Despite our efforts, we ended up firmly holding onto our initial positions. This led me to ponder: How can I determine if I am right when both of us are convinced of our correctness, yet at least one of us must be wrong?

To address this, I developed a scoring system using measurable metrics to determine who is more likely to be correct. I call it the AmIRight Score.

AmIRight Score

The AmIRight Score assigns points across several categories, helping to gauge the likelihood of being correct. Here’s how you can calculate your score:

1. Clarity in Falsification Criteria – 10 points

A person who can clearly articulate how their belief could be proven wrong demonstrates the ability to conceptualize alternative truths. If someone cannot envision any scenario that would falsify their belief, it suggests that their belief might be dogmatic.

Example of a good falsification statement: “I would believe AI is capable of deeper reasoning if it can be trained on data containing no information about chess, and then perform as well as a human that is also new to the game, given the same set of instructions.”

Example of a bad falsification statement: “I would believe AI is capable of deeper reasoning if all the scientists in the world acknowledged they were wrong about reasoning based on new evidence about the brain.”

2. The Simplified Ideological Turing Test – 10 points

The Ideological Turing Test evaluates how well you can articulate the opposing viewpoint. In the simplified version, both parties write arguments for their own position and the opposite position. A neutral judge then scores how well each argument is presented without knowing who wrote what.

3. Forecasting Accuracy – 5 points

Forecasting accuracy assesses the correctness of your predictions about future events. This metric rewards those whose predictions consistently turn out to be accurate. Both parties should take the same forecasting test, and points are awarded based on performance.

4. Forecasting Calibration – 5 points

Forecasting calibration measures how well your confidence levels match actual outcomes. It’s not just about being righ...