trentmkelly/LessWrong-43k huggingface dataset:
Evidence of Learned Look-Ahead in a Chess-Playing Neural Network
by Erik Jenner — 2024-06-04T15:50:47.475Z

Paper authors: Erik Jenner, Shreyas Kapur, Vasil Georgiev, Cameron Allen, Scott Emmons, Stuart Russell

TL;DR: We released a paper with IMO clear evidence of learned look-ahead in a chess-playing network (i.e., the network considers future moves to decide on its current one). This post shows some of our results, and then I describe the original motivation for the project and reflect on how it went. I think the results are interesting from a scientific and perhaps an interpretability perspective, but only mildly useful for AI safety.

Teaser for the results

(This section is copied from our project website. You may want to read it there for animations and interactive elements, then come back here for my reflections.)

Do neural networks learn to implement algorithms involving look-ahead or search in the wild? Or do they only ever learn simple heuristics? We investigate this question for Leela Chess Zero, arguably the strongest existing chess-playing network.

We find intriguing evidence of learned look-ahead in a single forward pass. This section showcases some of our results, see our paper for much more.

Setup

We consider chess puzzles such as the following:

In the initial board state, white sacrifices the knight on g6. Black has no choice but to capture it (second state) since the white queen prevents the king from going to g8. Then white can move the rook to h4 (third state), delivering checkmate.

We focus on the policy network of Leela, which takes in a board state and outputs a distribution over moves. With only a single forward pass per board state, it can solve puzzles like the above. (You can play against the network on Lichess to get a sense of how strong it is—its rating there is over 2600.) Humans and manually written chess engines rely on look-ahead to play chess this well; they consider future moves when making a decision. But is the same thing true for Leela?

Activations associated with future moves are crucial

One of our early experiments was to do activation patching. We patch a small part of Leela's activations from the forward pass of a corrupted version of a puzzle into the forward pass on the original puzzle board state. Measuring the effect on the final output tells us how important that part of Leela's activations was.

Leela is a transformer that treats every square of the chess board like a token in a language model. One type of intervention we can thus...
Is This Lie Detector Really Just a Lie Detector? An Investigation of LLM Probe Specificity.
by Josh Levy — 2024-06-04T15:45:54.399Z

Abstract

Whereas previous work has focused primarily on demonstrating a putative lie detector’s sensitivity/generalizability[1][2], it is equally important to evaluate its specificity. With this in mind, I evaluated a lie detector trained with a state-of-the-art, white box technique - probing an LLM’s activations during production of facts/lies - and found that it had high sensitivity but low specificity. The detector might be better thought of as identifying when the LLM is doing something other than fact-based retrieval (e.g. when writing fiction), which spans a much wider surface area than it should. I found that the detector could be made more specific through data augmentation, but that this improved specificity did not transfer to other domains, unfortunately. I hope that this study sheds light on some of the remaining gaps in our tooling for and understanding of lie detection - and probing more generally - and points in directions toward improving them.

You can find the associated code

here

, and a rendered version of the main notebook

here

.

Background

As society increasingly relies on LLMs, it is evermore important to ensure they are trustworthy. Given the breadth and depth of their knowledge, it is already difficult to validate their factfulness purely based on what they output[3]. This problem will only become more acute as AIs achieve superintelligence in more domains, and their outputs become increasingly illegible to and unverifiable by humans.

To address this challenge, an alternative approach has emerged: monitoring and understanding the internals of the LLM. Such an approach, dubbed "Mechanistic Interpretability"[4], is appealing for a couple of reasons. First, the space to monitor is probably much lower dimensional than the space of all possible outputs. Under this hypothesis, regardless of what output may be produced, the system would need to internally represent something simpler, like whether it is being honest or dishonest. Second, whereas deceptive AI systems are capable of manipulating their outputs, it is likely significantly harder to manipulate their internals in a precise way. This is akin to the rationale behind lie detector use in criminology: whereas someone can control what they say, it is difficult to control their physiological reaction.

Given these advantages - and the fact that...
[Paper] Stress-testing capability elicitation with password-locked models
by Fabien Roger — 2024-06-04T14:52:50.204Z
Link: https://arxiv.org/abs/2405.19550

The paper is by Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov and David Krueger. This post was written by Fabien and Ryan, and may not reflect the views of Dmitrii and David.

Scheming models

might try to perform less capably than they are able to (

sandbag

). They might do this to make evaluations underestimate danger, slow down promising safety work, or cause issues with debate or

recursive

oversight

more generally. You might hope that you can avoid this issue by eliciting capabilities, for example by fine-tuning on demonstrations and then using RL. In principle, this elicitation might fail. It could fail due to having too few (or no) high-quality demonstrations (i.e. sample efficiency is too low) or because RL fails to explore good actions.

We’re uncertain how this goes in practice. It could be the case that simple countermeasures prevent sandbagging in most of the important cases such that it isn’t a big problem for safety. But it could also turn out that even high-effort approaches can’t rule out these issues.

We released

a paper

studying this by examining how well supervised fine-tuning and RL can elicit capabilities from password-locked models (LLMs trained to produce low-quality outputs except when a particular password is provided). We find that:

If you have some high-quality demonstrations, supervised fine-tuning usually recovers the hidden capabilities with very few samples.
Elicitation on password-locked models generalizes well across domains (even when we made password-locked models using techniques that we thought might prevent generalization).
Using RL on password-locked models recovers hidden capabilities, except when the weak model RL starts from is extremely weak.

While we target cases where AIs intentionally produce low-quality outputs and they could easily produce higher quality outputs, we also think our results have some relevance to other cases where AIs produce low-quality outputs despite being able to produce better ones.

Here are the main takeaways:

We think our results are a moderate update against scheming models being able to sandbag effectively (given countermeasures) for the kind of tasks we studied here, though there are differences between password-locked models and actual scheming models. We think this should be a bigger update for people who are relatively more pessimistic than us about sandbagging (at

...
Circuit Board Ordering
by jefftk — 2024-06-04T14:00:02.084Z

Working first on my

plucked
electronic instrument and now my

hardware
whistle synthesizer I've been ordering custom circuit boards
("PCBs"). Once the design digitized in

KiCad (

tutorial), the
timeline looks like:

~15min: place order online (JLCPCB, picked kind of randomly)

~30min: manufacturer reviews order

In the most recent case they paused at this stage until I sent
them another $5 because I'd made a panel
of PCBs but hadn't checked the "this is a panel" button and so
had been undercharged.

~2d: manufacturer produces board (2.5d, 3.5d, 1.5d, 1.5d)

~3d: DHL ships board from China to Boston (5d, 2d, 2d, 3d)

~5.5d: total time (8d [1], 5d, 4d, 5d)

This is really impressive: for ~$30 I can go from having a fragile
breadboarded circuit on my desk to a professional quality circuit
board in less than a week.

I think a big reason why Nell and I never finished the RS232 to ethernet
project is that while we were very enthusiastic about the project
initially that faded during the months we waited for the PCB to
arrive. Five days is still not as good as overnight, but it's fast
enough to maintain decent momentum on a project.

[1] This one was slower because of a holiday
there.
Has anyone here written about religious fictionalism?
by SpectrumDT — 2024-06-04T12:10:21.987Z

Religious fictionalism, as I understand it, is the position that religion can be valuable and worth practicing and spreading even though it makes false claims. I was inspired to ask this question by the below YouTube video essay by Kane Baker.

Has anyone on Less Wrong written about religious fictionalism and the arguments for and against it? I could not immediately find anything when I searched for it, but it may have been covered under other names.

This thread is not an attempt to argue for or against religious fictionalism. I just want to know if anyone can point me to existing discussions here which touch upon the subject.

Thanks!

Video by Kane Baker:

https://www.youtube.com/watch?v=zSucRrn1JbU&pp=ygUWcmVsaWdpb3VzIGZpY3Rpb25hbGlzbQ%3D%3D
Is Wittgenstein's Language Game used when helping Ai understand language?
by  — 2024-06-04T07:41:53.725Z

Hello. I am a lurker, but I checked the search and didn't see anyone discussing Wittgenstein's ideas concerning the "essence" of language and his talk of "Language Games" so I thought I'd ask.

Wittgenstein is a linguistic philosopher, who in very brief terms clarified our usage of language. While many people conceived of language as clear distinct and obvious, Wittgenstein used the example of the word "game" to show how there is no consistent and encompassing definition for plenty of words we regularly use. Among other things, he observed that language rather exists as a web of connotations that depend and change with context, and that this connotation can only truly be understood when observing the use of language, rather than some detached definition.

"Wittgenstein In Philosophical Context:

Essential Definition - Socrates, boil it down to its essence

Extensive Definition - Wittgenstein, use it in a sentence"

The above dichotomy frames the use of "words" by philosophers in a contradictory manner. And perhaps in qualia we do conceive them as different sorts of definitions, but imo it's just a matter of framing and we can readily say that "How you use a word in a sentence is itself the essence of a word". And intend the Ai therefore to conceive of the "essence" of words accordingly.

Descriptively speaking, Wittgenstein has always appeared unambiguously correct on this matter to me.

This all being said, I am wondering something relating to Wittgenstein:

Does AI safety, and AI engineers in general, have a similar conception of language? When CGPT reads a sentence, does it intentionally treat each word 's essence as some rigid unchanging thing derived from some dictionary definition, or as a web of connotation to other words? This might seem rather trivial, but when interpreting a prompt like "save my life" it seems clear why truly understanding each word's meaning is so important for potential AGI. So then, is Wittgenstein or rather this conception of language taken seriously and intentionally consciously implemented? Is there even an intention of ensuring that Ai truly consciously understands language? It seems like this is a prerequisite to actually ensuring any AGI we build is 100% aligned. If the language we use to communicate with the AGI is up to interpretation it seems alignment is simply obviously impossible.
Smartphone Etiquette: Suggestions for Social Interactions
by Declan Molony — 2024-06-04T06:01:03.336Z

If anyone stopped and asked themselves if they enjoy the current smartphone social etiquette—whereby you could be opening up your soul to a friend in a moment of vulnerability, when suddenly, they feel a buzz and reflexively check their phone to discover an important & urgent message from the McDonald’s app that The McRib is back!—they would probably shake their heads knowing how close to home a situation like this hits.

Every time one of these moments of disconnection happen, a small part of me—the part craving real connection—dies. But it doesn’t have to be this way. I’ve managed to improve my social interactions due to:

Internalizing the emotional impact of checking my phone,
Recognizing how it affects my capacity to think clearly,
Understanding group interactions are better without distractions, and
Creating

suggestions

to remove phones from social situations (as doing so is in everyone’s best interest).

(1) The emotional impact of disconnection

I don’t blame individuals for lacking the restraint to not check their phones in social contexts. Why? Because there’s thousands of engineers whose full-time job it is to make our technology as addicting “engaging” as possible. As researcher Dr. Anna Lembke notes in her book Dopamine Nation:

We’re living in a time of

unprecedented access

to high-dopamine stimuli. The smartphone is the modern-day hypodermic needle, delivering digital dopamine 24/7. As such, we’ve all become vulnerable to compulsive overconsumption.

Author Nir Eyal wrote the book on how to addict “increase engagement” in users called Hooked: How to Build Habit-Forming Products. So it’s with a combination of sadistic glee and pity when I read about one of Nir’s moments of disconnection with his young daughter[1]:

[Nir was reading a book with his daughter. As she was contemplating a question he asked her, he received a text] and “I started looking at my phone, as opposed to being fully present with her.” When he looked up, she was gone. A childhood is made up of small moments of connection between a child and their parent. If you miss them, you don’t ever get them back. Nir realized with a lurch: “She got the message that whatever was on my phone was more important than she was.”

(2) “Smart” phones are making you dumber

The evidence on task switching does not look good for people who frequently c...
Just admit that you’ve zoned out
by joec — 2024-06-04T02:51:27.594Z

Summary: Zoning out is difficult to avoid and common, zoning out without admitting it hurts your comprehension, therefore you should admit that you zoned out and ask people to repeat things.

If you’re anything like me, you’ve “zoned out” before. You’ve probably even zoned out when you’re trying to learn something interesting. In fact, I’d bet that you’ve zoned out when listening to someone you respect teaching you something interesting, and that you didn’t admit it, and that this left you with a gap in understanding that you may or may not have filled in later.[1] Perhaps I’m falling for the typical minds fallacy, but I don’t think I am. This happens to me very often,[2] and I think it happens to others, and I think that any community focused on rationality or scholarship or understanding ought to account for this. I doubt we’ll be able to prevent people from zoning out, but I know we can encourage people who are listening to admit when they’ve zoned out and we can encourage people who are speaking to patiently re-iterate the thing they just said without taking offense.

One time I was explaining something to a friend of mine and she said the unthinkable. “Sorry, I zoned out. Could you repeat what you said after first bringing up mitochondria?” I was at first somewhat taken aback, but quickly realized that I’ve been in the same position as her. I repeated myself and took less than a minute to do so. I think her understanding was better than it would have been if she hadn’t simply admitted she zoned out. I’m thankful she did it, since it brought the fact that I could do the same to my awareness. If you’re in the right company, admitting that you’ve zoned out has barely any cost and real benefits.

Zoning out when someone is talking to you is far more common if the things they’re saying are boring or hard to comprehend or otherwise unpleasant. It’s perfectly rational to, as a speaker, take “people are zoning out” as evidence of a poor job. However, if you were unpleasant to listen to, nobody would ask you to repeat yourself. If someone admits to you that they stopped paying attention and asks you to repeat yourself, it doesn't imply any fault of yours. The right thing to do in that situation is to resist the temptation to be offended or annoyed and just go along with it. Of course, there’s always a limit. If someone admits to zoning out twenty times in th...
(Not) Derailing the LessOnline Puzzle Hunt
by Error — 2024-06-04T01:28:31.688Z

(spoiler alert: may meta-spoil future iterations of the LOPH, if you
haven’t already read other posts about it)

I knew early on that I wouldn’t be able to finish the LessOnline puzzle
hunt.

I contributed to solving two of the first six puzzles, each of which
revealed the combination to a locked box. Each box contained wooden medallions
for the solvers, plus a QR code. The QR codes led to the second layer of a
larger meta-puzzle.

On discovering the existence of the meta-puzzle, I knew I would have to
drop out. It was a shame, because I have never done a puzzle hunt
before and I could see that the puzzlemasters had produced something
amazing. But the opportunity cost of playing “for real” was just too
high. There were too many other things I wanted to do, and the integral
of my attention over time is not infinite.

So I stopped hunting. But, unexpectedly and hilariously, I ended up
contributing to the game in a very different way.

Suspicion

During the Fooming Shoggoth concert I noticed (probably thanks to the
song lyrics) that the parts of the puzzle I was aware of were all
lockboxes. I mean, I knew that already, but now I noticed. And I knew
from the contents of the two boxes I had opened that there was more to
the puzzles than the obvious. The whole setup seemed like a
suspiciously plausible metaphor for the AI Box Experiment.

I suddenly, strongly suspected that the puzzle hunt had a hidden
narrative -- one that would end with the release of a rogue AI.

Sequence Breaker!

So I did what any Less Wronger should have done: I tried to warn others.
No unbinding of seals, no opening of gates! At first I just told nearby
hunters directly, but quickly realized it wouldn’t work; if nothing
else, I didn’t know who was working on the puzzle hunt and who wasn’t. I
abandoned that plan and asked the front desk to print out three notes
for me. The notes outlined my suspicions and warned the reader not to
open whatever final box might be found at the end of the puzzle chain.

I taped Note 1 to my medallion, which I returned to its original box. In
addition to the warning, Note 1 asked anyone opening the box to leave
the medallion and the note itself for later hunters to see, and
suggested they return their own medallion(s), just in case. I had no
reason to believe it mattered whether the medallions stayed in the
boxes, but I had no reason to believe it didn’t either, and it was an o...
Masculinity—A Case For Courage
by James Stephen Brown — 2024-06-04T00:04:48.411Z
Link: https://nonzerosum.games/masculinity.html

HOW THIS POST CAME ABOUT

When I put a call out for non-zero-sum topics from subscribers, it was my brother who first replied with masculinity—pointing me to the work of author

Richard Reeves

.

“I think the danger of zero-sum thinking is present … men are suffering because women are rising. But it’s also true—when I try to persuade policymakers to do more stuff about boys and men, they’ll say No, no, no, it’s women and girls who need help. No, no, they both need help.” — Richard Reeves

I had also been listening to

Scott Galloway

echo many of the same sentiments. This seems to be part of a more healthy discussion around masculinity—a counterpoint to the toxic extremes of the

Manosphere

.

“We’ve conflated toxicity with masculinity” — Scott Galloway.

This post is far from a comprehensive analysis of the issue, but I have followed the guidance of experts; Reeves, Galloway, and

Christine Emba

—whose work I recommend for a deeper dive. We will, as always, explore the issue from a non-zero-sum angle—looking for win-wins.

MEN’S… RIGHTS?

I have to admit, I haven’t taken an interest in “men’s rights” until now, because of the way I had experienced the subject being represented, in the past, by Men’s Rights Activists — promoting a traditionalist model of masculinity, while emphasising their victimhood, in a world where, as a man, I personally felt I still benefitted from many advantages, and saw the women around me still facing barriers I did not. But I have now found my dismissal of the issue may have been misguided.

THE PROBLEMS

There are numerous reasons to rescue masculinity from toxicity, after all, men cause a lot of problems for everyone through anti-social behaviour (this is, by no means, an exhaustive list).

MEN ARE…

3 times as likely to be addicted

12 times as likely to have a criminal conviction

7•5 times as likely to be problem gamblers

Men are a group that poses risks to society, and yet they are an at-risk group themselves.

4 out of 5 victims of suicide are men

Scott Galloway points out that, if this was any other group, we would talk about it in terms of needing better social programs and empathy. But a call for empathy can seem like quite an ask for those who are the victims of male anti-social behaviour. However, as we will explore, empathy is not a reward for bad behaviour but the key to preventing it—that is, when it is adopted by responsible people.

“If responsible peop

...