trentmkelly/LessWrong-43k huggingface dataset:
Soviet comedy film recommendations
by Nina Panickssery — 2024-06-09T23:40:58.536Z
Link: https://open.substack.com/pub/ninarimsky/p/soviet-comedies

I’m a big fan of the Soviet comedy directors

Eldar Ryazanov

,

Leonid Gaidai

, and

Georgiy Daneliya

. Almost anything by them is worth watching, but here are my favorites (filtered for things that have a free YouTube version with good English subtitles, bold are the highest-recommended):

Scene from "The Garage"

Ryazanov

1966

Beware of the Car

(Берегись автомобиля)

[YouTube]

Comedy about a benevolent car thief who steals to donate to charity

1975

The Irony of Fate

(Ирония судьбы или с легким паром!)

[YouTube]

A New Year’s classic premised on the uniformity of Soviet apartment buildings—a guy gets drunk on NYE and ends up in a different city but finds an identical building that his key can access

1977

Office Romance

(Служебный роман)

[YouTube]

Romantic comedy and satirical portrayal of Soviet office life

1979

The Garage

(Гараж)

[YouTube]

Comedy set in a single room where people argue about who should lose their garage after the government decides to build a road through the plot they were collectively building garages on

1987

Forgotten Melody for a Flute

(Забытая мелодия для флейты)

[YouTube]

Satirical romantic comedy about Soviet bureaucracy and its decline in power in the late 80s, great opening song (translate the lyrics)

1991

The Promised Heaven

(Небеса обетованные)

Sadly couldn’t find an English-subtitled YT link for this but I like it too much to miss off[1]
Tragic comedy about the lives of people made recently homeless during the Perestroika period, very sad and of its time

Gaidai

1966

Kidnapping, Caucasian Style

(Кавказская пленница, или Новые приключения Шурика)

[YouTube]

One of the most famous Soviet comedies—a naive visitor to the Caucasus is convinced to assist in the “bride kidnapping” tradition

1969

The Diamond Arm

(Бриллиантовая рука)

[YouTube]

Another one of the most famous Soviet comedies—diamonds end up being smuggled in the wrong guy’s cast because he happens to injure himself and say the “codeword” in front of the smugglers’ hideout

1971

The Twelve Chairs

(12 стульев)

[YouTube]

Film adaptation of the satirical novel by Soviet authors

Ilf and Petrov

set in post-revolutionary Russia

Daneliya

1977

Mimino

(Мимино)

[YouTube]

Romantic comedy about a Georgian bush pilot

1986

Kin-dza-dza!

(Кин-Дза-Дза!)

[YouTube]

Funny low-budget sci-fi

Bonus recommendations

1973

Seventeen Moments of Spring

(Семнадцать мгновений весны)

[YouTube]

Extremely popular Soviet spy thriller set durin

...
The Data Wall is Important
by JustisMills — 2024-06-09T22:54:20.070Z
Link: https://justismills.substack.com/p/the-data-wall-is-important

Modern AI is trained on a

huge fraction of the internet

, especially at the cutting edge, with the best models trained on close to all the high quality data we’ve got.[1] And data is

really important

! You can scale up compute, you can make algorithms more efficient, or you can add infrastructure around a model to make it more useful, but on the margin, great datasets are king. And, naively, we’re about to run out of fresh data to use.

It’s rumored that the top firms are looking for ways to get around the data wall. One possible approach is having LLMs create their own data to train on, for which there is kinda-sorta a precedent from, e.g. modern chess AIs learning by playing games against themselves.[2] Or just finding ways to make AI dramatically more sample efficient with the data we’ve already got: the existence of human brains proves that this is, theoretically, possible.[3]

But all we have, right now, are rumors. I’m not even personally aware of rumors that any lab has cracked the problem: certainly, nobody has come out and say so in public! There’s a lot of insinuation that the data wall is not so formidable, but no hard proof. And if the data wall is a hard blocker, it could be very hard to get AI systems much stronger than they are now.

If the data wall stands, what would we make of today’s rumors? There’s certainly an optimistic mood about progress coming from AI company CEOs, and a steady trickle of not-quite-leaks that exciting stuff is going on behind the scenes, and to stay tuned. But there are at least two competing explanations for all this:

Top companies are already using the world’s smartest human minds to crack the data wall, and have all but succeeded.

Top companies need to keep releasing impressive stuff to keep the money flowing, so they declare, both internally and externally, that their current hurdles are surmountable.

There’s lots of precedent for number two! You may have heard of startups hard coding a feature and then scrambling to actually implement it when there’s interest. And race dynamics make this even more likely: if OpenAI projects cool confidence that it’s almost over the data wall, and Anthropic doesn’t, then where will all the investors, customers, and high profile corporate deals go? There also could be an echo chamber effect, where one firm acting like the data wall’s not a big deal makes other firms take their word...
Two Family Dance Flyers
by jefftk — 2024-06-09T20:50:02.381Z

I'm going to be calling

another
family dance

in a week,
and Lily and Anna wanted to make flyers to advertise it. I wrote out
a sheet with the key details they might want to include:

Lily wanted to do hers on the computer, and it ended up being
primarily about learning Inkscape:

Anna did hers by hand, and was very into the drawing:
3b. Formal (Faux) Corrigibility
by Max Harms — 2024-06-09T17:18:01.007Z

(Part 3b of the CAST sequence)

In the first half of this document, Towards Formal Corrigibility, I sketched a solution to the stop button problem. As I framed it, the solution depends heavily on being able to detect manipulation, which I discussed on an intuitive level. But intuitions can only get us so far. Let’s dive into some actual math and see if we can get a better handle on things.

Measuring Power

To build towards a measure of manipulation, let’s first take inspiration from the suggestion that manipulation is somewhat the opposite of empowerment. And to measure empowerment, let’s begin by trying to measure “power” in someone named Alice. Power, as I touched on in the ontology in Towards Formal Corrigibility, is (intuitively) the property of having one’s values/goals be causally upstream of the state of some part of the world, such that the agent’s preferences get expressed through their actions changing reality.

Let’s imagine that the world consists of a Bayes net where there’s a (multidimensional and probabilistic) node for Alice’s Values, which can be downstream of many things, such as Genetics or whether Alice has been Brainwashed. In turn, her Values will be upstream of her (deliberate) Actions, as well as other side-channels such as her reflexive Body-Language. Alice’s Actions are themselves downstream of nodes besides Values, such as her Beliefs, as well as upstream of various parts of reality, such as her Diet and whether Bob-Likes-Alice.

As a simplifying assumption, let’s assume that while the nodes upstream of Alice’s Values can strongly affect the probability of having various Values, they can’t determine her Values. In other words, regardless of things like Genetics and Brainwashing, there’s always at least some tiny chance associated with each possible setting of Values. Likewise, we’ll assume that regardless of someone’s Values, they always have at least a tiny probability of taking any possible action (including the “null action” of doing nothing).

And, as a further simplification, let’s restrict our analysis of Alice’s power to a single aspect of reality that’s downstream of their actions which we’ll label “Domain”. (“Diet” and “Bob-Likes-Alice” are examples of domains, as are blends of nodes like those.) We’ll further compress things by combining all nodes upstream of values (e.g. Genetics and Brainwashing) into a single node called “En...
3a. Towards Formal Corrigibility
by Max Harms — 2024-06-09T16:53:45.386Z

(Part 3a of the CAST sequence)

As mentioned in Corrigibility Intuition, I believe that it’s more important to find a simple, coherent, natural/universal concept that can be gestured at, rather than coming up with a precisely formal measure of corrigibility and using that to train an AGI. This isn’t because formal measures are bad; in principle (insofar as corrigibility is a real concept) there will be some kind of function which measures corrigibility. But it’s hard to capture the exact right thing with formal math, and explicit metrics have the tendency to blind people to the presence of better concepts that are nearby.

Nevertheless, there are advantages in attempting to tighten up and formalize our notion of corrigibility. When using a fuzzy, intuitive approach, it’s easy to gloss-over issues by imagining that a corrigible AGI will behave like a helpful, human servant. By using a sharper, more mathematical frame, we can more precisely investigate where corrigibility may have problems, such as by testing whether a purely corrigible agent behaves nicely in toy-settings.

Sharp, English Definition

The loose English definition I’ve used prior to this point has been: an agent is corrigible when it robustly acts opposite of the trope of "be careful what you wish for" by cautiously reflecting on itself as a flawed tool and focusing on empowering the principal to fix its flaws and mistakes.

Before diving into mathematical structures, I’d like to spend a moment attempting to sharpen this definition into something more explicit. In reaching for a crisp definition of corrigibility, we run the risk of losing touch with the deep intuition, so I encourage you to repeatedly check in with yourself about whether what’s being built matches precisely with your gut-sense of the corrigible. In particular, we must be wary of both piling too much in, such that it ceases to be a single coherent target, becoming a grab-bag, and of stripping too much out, such that it loses necessary qualities.

My best guess of where to start is in leaning deeper into the final bit of my early definition—the part about empowering the principal. Indeed, one of the only pre-existing attempts I’ve seen to formalize corrigibility also conceives of it primarily as about the principal having power (albeit general power over the agent’s policy, as opposed to what I’m reaching for). Many of the emergent...
Introducing SARA: a new activation steering technique
by Alejandro Tlaie — 2024-06-09T15:33:11.699Z

Disclaimer

I currently am a Postdoctoral Fellow in Computational Neuroscience, learning about Mechanistic Interpretability and AI Safety in general. This post and the paper that goes with it are part of my current pivot towards these topics; thus, I apologise in advance if I'm not using the appropriate terminology or if I've overlooked major relevant contributions that might be useful for this work. Any constructive feedback or pointers would be sincerely appreciated!

Executive summary

This post introduces SARA (Similarity-based Activation Steering with Repulsion and Attraction), a tool that I designed to provide precise control over the moral reasoning[1] of Large Language Models (LLMs). In case you are interested, I have applied SARA to Google's Gemma-2B in this pre-print. Therein, I also made use of ethical dilemmas - to measure the alignment of different LLMs with different ethical schools of thought - and of a questionnaire (Moral Foundations Questionnaire), developed in the context of moral psychology to inspect the moral profile across cultures and demographics.

Introduction

In the context of Mechanistic Interpretability, activation steering is a technique that, coming from Neuroscience, I found particularly interesting. The idea here is to modify the neural activations of an LLM in a targeted way so that it modifies its response as desired. One of the simplest and most straightforward such manipulations is that of Activation Addition (ActAdd), introduced here. For the sake of this post to be self-contained, I will paraphrase their post and briefly explain how ActAdd works:

Start with a prompt that will be steered (p0).

Take a pair of prompts with a property that will be emphasised (p+) and its opposite (p−).

If (h(l)+) is the activation vector for the prompt (p+) at layer l, then the difference h(l)+−h(l)− is a new activation vector which (intuitively) captures the difference between a prompt with the property, and without it.

To obtain a steering vector, perform a forward pass on each prompt, record the activations at the given layer in each pass, take the difference h(l)+−h(l)−, and then finally rescale this difference in activations by an ‘injection coefficient’ c.

To steer, add the resulting activation vector to the input of layer l and allow the forward pass to continue, and so obtain the steered output.

Thus, mathematically:
p′0=p0+c(h(l)+−h(...
"What the hell is a representation, anyway?" | Clarifying AI interpretability with tools from philosophy of cognitive science | Part 1: Vehicles vs. contents
by IwanWilliams — 2024-06-09T14:19:28.322Z

AI interpretability researchers want to understand how models work. One popular approach is to try to figure out which features of an input a model detects and uses to generate outputs. For instance, researchers interested in understanding how an image classifier distinguishes animals from inanimate objects might try to uncover the properties of the image (such as fur, scales and feathers) that the model “looks for” when faced with that task. Researchers might also try to localise where in the internal workings of the model the this information is encoded and processed (is fur detected at earlier layers of a neural network than limbs?). Answering these sorts of questions is one way of peeking inside the “black box” of an AI system.

The approach just described involves applying a representational lens to AI models – the models are thought of as representing features of inputs, and these representations play some role in explaining how the model performs a task (and, when it fails, why it fails). But what the hell is a representation, anyway?

As a philosopher who spends a lot of time thinking about representation (mainly in the context of biological minds and brains) I have a hunch that the philosophical literature on the topic contains a few nuggets of wisdom that may be useful (or at the very least interesting) to those interested in interpretability research.

Drawing on philosophy of mind and cognitive science, I’ll share a few “tools” (concepts, distinctions and ways of thinking about the issues) that may help to clarify research questions in AI interpretability. Along the way I’ll suggest some relevant literature, for those interested in digging a bit deeper.

More broadly, this is an advertisement for the value that philosophy can add to AI safety and interpretability research, beyond the more obviously relevant sub-disciplines of moral philosophy and metaethics.

In this first post, I’ll introduce tool number one: a handy distinction between representational content and representational vehicles.

***

AI interpretability research does not always explicitly use the term “representation”. Research into the properties of inputs that models detect and respond to is sometimes described instead as the search for “features”. However, the idea of a feature can be a little confusing because the term is used and defined in apparently c...
Exploring Llama-3-8B MLP Neurons
by ntt123 — 2024-06-09T14:19:10.822Z
Link: https://neuralblog.github.io/llama3-neurons/

TL;DR: We created a dataset of text snippets that strongly activate neurons in Llama-3-8B model. This dataset shows meaningful features that can be found. Explore the neurons with the web interface: https://neuralblog.github.io/llama3-neurons/neuron_viewer.html

An example of a "derivative" neuron which is triggered when the text mentions the concept of derivatives.

Introduction

Transformer networks (Vaswani et al. 2017) have a remarkable ability to capture complex patterns and structures in their training data. Understanding how these neural networks work is not only an inspiring research problem, but also a practical necessity given their widespread deployment to millions of people.

Transformer models consist of two major components: attention layers and MLP layers. While significant progress has been made in understanding attention layers, such as the work on Transformer circuits by (Elhage et al. 2021), the understanding of MLP layers remains limited.

Interestingly, MLP layers are one of the few places in transformer networks where privileged bases can be found (Elhage et al. 2021). These vector bases are favored by the model due to their pointwise non-linear computation. They are referred to as neurons, while the outputs of the activation function are referred to as neuron activations. Neural networks tend to use neurons to represent important features (Karpathy, Johnson, and Fei-Fei 2015; Geva et al. 2020, 2022), making them a good starting point for understanding transformers.

In this work, we release a dataset of text snippets that strongly activate MLP neurons in the Llama-3-8B model. We chose the Llama-3-8B model for its strong evaluation performance and real-world usefulness.

We show examples of meaningful features discoverable with the dataset, and expect that many more can be found. We also anticipate that automated systems using LLMs could greatly help uncover features from the dataset, as shown in (Bills et al. 2023; Bricken et al. 2023). By open-sourcing our work, we enable others to easily create similar datasets for other transformer models. To facilitate exploration of Llama-3 features, we create a simple web interface available at

https://neuralblog.github.io/llama3-neurons/neuron_viewer.html

.

Dataset

Overview. The dataset includes a total of more than 14 million text snippets from the FineWeb-Edu dataset (Guilherme et al. 2024). There are 32...
Dumbing down
by Martin Sustrik — 2024-06-09T06:50:47.469Z

In past few years I've been blogging in Slovak, that is, downscaling from writing in English, a language with 1457 million speakers to a language with 7 million speakers.

From the point of view of the writer, this has been a very different experience. It's not only that for a topic that interests one million English speakers, the equivalent is five thousand in Slovakia, scaling down by factor of 200. It's also that topic that interests 100 English speakers, interests one half of a hypothetical Slovak speaker, that is, nobody. In fact, not everybody reads blogs, so the population in question is likely smaller by an order of magnitude or even two, resulting in even more fractional Slovaks... In other words, the reader population is not as big as to fill in all the possible niches and the writing thus has to become much more generic.

It must also be "dumbed down". Not because Slovaks are less intelligent than other nations, but because the scale of the existing discourse is much smaller. While in English, not matter how esoteric your topic is, you can reference or link to the relevant discussion, in Slovak it often is the case that there's no discussion at all.

The combination of the two factors above means that you have to explain yourself all the time. You want to mention game theory? You have to explain what do you mean. You want to make a physics metaphor? You can't, if you care about being understood. You want to hint at some economic phenomenon? You have to explain yourself again.

And often even the terminology is lacking. Even such a basic word as "policy" has no established equivalent. I had to ask a friend who works as a translator at the European Commission, just to be told that they use word "politika" for this purpose. Which is definitely not a common meaning of the word. "Politika" typically means "politics" and using it for "policy" sounds really strange and awkward.

(All of this gave me gut-level understanding of how small populations can lose knowledge. Joe Henrich mentions a case of small Inuit population getting isolated from the rest and gradually losing technology, including the kayak building skills, which in turn made it, in a vicious circle, unable to import other technology. This kind of thing also tends to be mentioned when speaking of dropping fertility rates and possible inability of a smaller global population to keep the technology we ta...
What if a tech company forced you to move to NYC?
by KatjaGrace — 2024-06-09T06:30:03.329Z

It’s interesting to me how chill people sometimes are about the non-extinction future AI scenarios. Like, there seem to be opinions around along the lines of “pshaw, it might ruin your little sources of ‘meaning’, Luddite, but we have always had change and as long as the machines are pretty near the mark on rewiring your brain it will make everything amazing”. Yet I would bet that even that person, if faced instead with a policy that was going to forcibly relocate them to New York City, would be quite indignant, and want a lot of guarantees about the preservation of various very specific things they care about in life, and not be just like “oh sure, NYC has higher GDP/capita than my current city, sounds good”.

I read this as a lack of engaging with the situation as real. But possibly my sense that a non-negligible number of people have this flavor of position is wrong.
What should I do? (long term plan about starting an AI lab)
by not_a_cat — 2024-06-09T00:45:12.369Z

I was listening to this Dwarkesh podcast with Leopold Aschenbrenner where they talk about AGI, superintellignence and how things might unfold. All I want to say about it is that it created a sense of concreteness and urgency when considering my plans for the future.

A bit of context about myself: Since I was a teenager, I've always been fascinated by computers and intelligence. I did CS studies which took away the mystery about computers (to great satisfaction). But the more I read about intelligence, brains, neuroscience and machine learning, the more it was clear we don't know how it works. I took a job as a web/database engineer after getting my CS master because I had to make a living, but I kept reading on the side. With my interest about intelligence getting stronger and no good answers, I made a plan to quit my job, and study on my own while living on my savings, with the hope of landing a research engineer position at DeepMind or GoogleBrain. One year into this self learning journey, it was clear that this would be challenging. So I turned to plan B (prompted by a bitcoin run up to almost $20000 dollars in 2017): I would salvage my newly acquired ML skills to trade those markets, then fund my own lab. And in the process, get better at applied ML and prove to myself I can do ML.

Fast forward 6 years, the plan has worked (to everyone's surprise myself included). The project has grown into a 10-person organization. I recently stepped down, after having transferred the skills and knowledge to the team that is now more competent than me to run it. Now is the time to activate the next step of the plan.

But things have changed. In 2018, concerns like alignment, controllability and misuse felt very theoretical and distant. Not anymore. The big change that occurred is my belief that the ML/AI field as a whole has a very high chance to achieve AGI, followed by superintelligence. Wether I get involved or not.

This is of course adds to all the other concerns regarding starting an AI lab: should I first study on my own to get better? Partner with other labs? Start recruiting now vs later?

AI safety being more important now, I'm telling myself is that the best way to approach it, is to be able to train good models, so I should work on AI capabilities regardless. Researching AI safety in a vacuum is much harder if you don't have AI capability expertise. But ...