trentmkelly/LessWrong-43k huggingface dataset:
The AI regulator’s toolbox: A list of concrete AI governance practices
by Adam Jones — 2024-08-10T21:15:09.265Z
Link: https://adamjones.me/blog/ai-regulator-toolbox/

This article explains concrete AI governance practices people are exploring as of August 2024.

Prior summaries have mapped out high-level areas of work, but rarely dive into concrete practice details. For example, they might describe roles in policy development, advocacy and implementation - but don’t specify what practices these policies might try to implement.

This summary explores specific practices addressing risks from advanced AI systems. Practices are grouped into categories based on where in the AI lifecycle they best ‘fit’ - although many practices are relevant at multiple stages. Within each group, practices are simply sorted alphabetically.

Each practice's explanation should be readable independently. You can navigate and get a direct link to a section by clicking the section in the left sidebar.

The primary goal of this article is to help newcomers contribute to the field.[1] Readers are encouraged to leave comments where they are confused. A secondary goal is to help align the use of terms within the field, to help even experienced researchers understand each other.

So, without further ado let’s jump in!

Pre-training

Compute governance

TLDR: Regulate companies in the highly concentrated AI chip supply chain, given AI chips are key inputs to developing frontier AI models.

Training AI systems requires applying algorithms to lots of data, using lots of computing power (also known as ‘compute’ - effectively the number of operations like addition or multiplication you can do). The algorithms are generally known,[2] and most of the data used is publicly available on the internet.

Computing power on the other hand is generally limited to far fewer actors: to be at the cutting edge you generally need thousands of AI chips which are incredibly costly (e.g. one chip costs $30-40k, and companies like Meta intend to buy hundreds of thousands of them). In addition to being very costly, there are very tight bottlenecks in the supply chain: NVIDIA designs 80-95% of AI chips, TSMC manufactures 90% of the best chips, and ASML makes 100% of EUV machines (used to create the best AI chips).[3]

Governments could therefore intervene at different parts of the compute supply chain to control the use of compute for training AI models. For example, a government could:

Know which organisations use the most compute, so regulators can prioritise supervising those most likely to de

...
Diffusion Guided NLP: better steering, mostly a good thing
by Nathan Helm-Burger — 2024-08-10T19:49:50.963Z
Link: https://arxiv.org/html/2408.04220v1

I think this is a very promising method for improving the steering of LLMs. Which is great for reducing risk from model-originating harms like deception.

The flipside is that it increases misuse potential.

This is yet another possibility for the widening of the safety gap between closed-weight models with locked-down controls, and open weight models.
Tall tales and long odds
by Solenoid_Entity — 2024-08-10T15:22:16.958Z

IgorV: Волчар с миней?! Bro, srsly? LMAO

KatyaL: Волчар возвращн? With its минй?!

randomguy1337: Chigura again? lol haven't herd in awhile

MishaD: So its all bout Chigura now? Lmaoo.

IgorV: Another Chigura thery? C'mon... 😂

fnorsch: Pryidite k nashemu Discordu dlya Роблокс! Ekspertiz y obsuzhdny!

MishaD: Chigura tales never gt old do they? 😆

Once upon a time, there was a quaint village that skirted the edge of a vast dark forest, nestled between foothills. The villagers' sheep fed on the grass near the fringe of the woods, watched over during the long summer days by a young child with sharp eyes and a natural proclivity for discerning patterns and focusing on details, perhaps at the expense of his ability to communicate — not that he couldn't speak, and not that anything he said was wrong, just that when he did speak, many of the villagers felt, for reasons they couldn't themselves understand, that he should be kept away.
One morning among mornings, while the village was still awakening, the young shepherd led his sheep to the forest's edge. As he watched over his flock, a glimmer in the grass caught his eye. He approached and found a small coin nestled among the dewy blades. Because his mind was the way that it was, he effortlessly noticed the coin showing heads. He left it where it lay, for no reason he could quite justify.
That evening, the boy mentioned the coin to his mother, who just smiled warmly as she tucked him into bed, her soft voice thrumming as she recounted old fables until he could cling on no longer.

A few months later, while guiding the sheep over the foothills, the boy found his gaze drawn towards the forest, though he couldn't understand why.

Suddenly, a chill gripped him as a dark form seemed to materialize from the backdrop of the woods, its faceless focus falling on the boy's flock. Though his instincts whispered "wolf," his eyes struggled to make any sense of this... entity. It appeared more an amorphous blur, an enigmatic void, wasted potential for sense, akin to a flaw in a tapestry.

Heart racing, muscles poised to carry him shouting and screaming back down the road to the village, the boy watched as the figure suddenly retreated, melting into the forest. Doubt clouded his mind for a moment, but as he cautiously investigated the spot, two coins lay there: one displaying heads, the other tails.

With the immediate danger dissipated, he hurr...
The Great Organism Theory of Evolution
by rogersbacon — 2024-08-10T12:26:02.434Z
Link: https://www.secretorum.life/p/the-great-organism-theory-of-evolution

From Becoming Animal (David Abrams, 2010):

Many months earlier, in a village near a wild forest preserve on the south coast of Java, I’d been warned by the local fishermen that there was an unusually bold individual among the bands of monkeys that roam the forest canopy, a particular monkey much more daring and skillful than the others, especially at stealing things from humans. Because I wear glasses, I was urged by the fishermen not to enter that forest, for that sly monkey was known to silently accompany people in the branches far overhead, waiting for an opportune moment to swing low and snatch the glasses from their face. The villagers had had to organize several search parties for missing visitors who turned out simply to have been wandering half blind for several days, unable to find their way out of the woods.

Similarly, when I lived in the northern Rockies there was an old bull elk who was legendary among the local hunters. Larger than the other males thereabouts, he had once had the biggest rack of any bull in those mountains, although in recent years (folks said) his antlers were smaller. He was glimpsed often, yet no one had ever succeeded in planting a bullet anywhere on his person. His ability to elude hunters was uncanny, enabling him to melt away and vanish even as the hunter registered the glimpse. In earlier years the locals had taken the large bull’s readiness to show himself as a challenge, with each hunter eager to finally shoot him and be able to boast about the fact. But after so many years, the old one’s continued defiance of hunters had made him not only a legend but a revered spirit among the hunters and everyone else in the region.

Hiking one October evening with a friend who’d grown up in that area and hearing now and then the most beautiful of all earth-born sounds, which is the autumn bugling of elk—there abruptly sounded from far off the most heart-wrenchingly lovely of any call I'd ever heard, a bugling that was the most full-throated and deep and at the same time the most ethereal, ascending slowly upward through a sequence of clear overtones before ending in a series of gutteral grunts. I looked wide-eyed at my friend. It was the unmistakable call, he said, of that great elder, the phantom.

In these cases, and I could mention many others, the uniqueness of the individuals seems to reside not just in their intelligence but

...
Emergence, The Blind Spot of GenAI Interpretability?
by Quentin FEUILLADE--MONTIXI — 2024-08-10T10:07:53.654Z

Epistemic status: This post was planned to be part of a broader "Holistic Interpretability" post but this isn't going as fast as I'd like so I am releasing the foreword to get some early feedback on whether I should pursue this or not. I haven't had a lot of red teaming on this emergence idea, but I'd still put a 80% confidence that this is important. I also tried to use the word emergence only where it made sense, but I put a 40% chance that I might have used it a bit wrong.

Emergence: More Than Just a Buzzword?

When I think about large auto-regressive models, I can't help but return to the concept of emergence. Now, I know what you're probably thinking - "emergence" has become something of a fuzzy buzzword in AI discussions, and I mostly agree with the critics, but I think there is more to unpack here.

Let's start with a classic example: water. A single H2O molecule is just a simple arrangement of atoms, but combine gazillions of these molecules, and suddenly you have a substance that flows, splashes, and takes the shape of its container. While individual molecules have properties like polarity and can form hydrogen bonds, the macroscopic properties we associate with liquids only become apparent when vast numbers of these molecules interact.

Now, you might argue, "But we can simulate water behavior perfectly if we know all the properties of individual molecules!" And you'd probably be right. However, even with a perfect simulation, to predict how water will behave in a new scenario, you'd still need to run the entire simulation. You can't shortcut it by just looking at a few molecules or an over-simplified model.

This illustrates why I believe emergence matters when discussing AIs. It's not about mystical properties appearing out of nowhere. Rather, it's about the necessity of studying the system as a whole to fully understand its behavior, even with complete knowledge of its components.

When I look at current efforts to interpret LLMs[1], I often have this impression that they bet on the idea that LLM behaviors, cognition, and capabilities are NOT emergent properties of auto-regressive deep neural networks and it's training. There seem to be an implicit belief that everything could be pieced out just by being good enough at understanding all the internal circuitry and components.

The default trajectory?

This approach seems to mirrors a common scientific reaction ...
Rowing vs steering
by Saul Munn — 2024-08-10T07:00:17.594Z
Link: https://www.brasstacks.blog/rowing-steering/

Alex Lawsen used a great metaphor on the

80k After Hours podcast

:[1]

[1:38:14] …you’re rowing a boat on your own, and you’re trying to get somewhere, and you’ve got some map that you need to look at to see where you’re going, I imagine like a map and compass. […] When you’re rowing, you’re facing back; you can’t see where you’re going. You’ve just got to sit there and pull both of the oars, and do that a bunch of times, and then the boat goes forwards. […] You steer [… by pulling] harder with one side, something like that.
I can imagine […] you sitting forwards in the boat, and trying to hold the map with your left hand while it’s gripping one oar, and trying to hold the compass with your right hand while it’s gripping the other; pushing them rather than pulling them while looking at where you’re going; so you’re always precisely on track, but my guess is you’re just going to go super slowly, because that’s not how to row a boat.
Whereas you can imagine someone else, maybe someone that’s racing you, who is going to point the boat in pretty much the right direction — they’re not exactly sure it’s the right direction, and they might go a bit off course. And then they go, “Cool. I’m going to row hard for a minute, and then I’m going to stop and check I’m pointing in the right direction, and then I’m going to row hard for another minute.”
[1:37:56] The metaphor is trying to point at … the strategy, [which] is pretty clear: gather some information, make a decision with that information, stick to that decision for some period of time that you’ve planned in advance, and then reevaluate, gather some more information, and then make a new decision.
[1:35:58] … you [should] stick to some policy, which is like: “I’m going to look at a bunch of things, I’m going to actually seriously consider my options. And then, with all of the information I have, I’m going to make a decision. And I’m going to make the decision to do the thing that seems best for some fixed period of time. At the end of that fixed period of time, then I will consider other options.”
[1:47:43] … if you think expected value is a reasonable framework to use, … then I do actually want to say: I think having this kind of policy is actually the thing that seems best in expectation.
[1:41:21] … I think some people … they start doing a thing, and then they’re so worried about whether it’s the best, that the

...
Fermi Estimating How Long an Algorithm Takes
by SatvikBeri — 2024-08-10T01:34:43.540Z

This is a "live" Fermi estimate, where I expect to make mistakes and haven't done much editing.

If you're working on a simple mathematical program, you can attempt to estimate the number of floating point or integer operations, compare that to statistics on CPU latencies/throughput, and use that to get a rough estimate.

Let's try estimating how long it would take to run a simple primality test, a modified Sieve of Eratosthenes:

function modified_sieve(n)
known_primes = [2]
for i in 3:2:n
mods = [i % p for p in known_primes]
if all(mods .> 0)
push!(i, known_primes)
end
end
return known_primes
end

In this algorithm, we store a list of known prime numbers. We iterate through odd numbers, and check whether any of them are divisible by a known prime, by calculating the modulus against every known prime in our list. If it's not divisible by any of them, we add it to our list of primes.

There's a lot we could do to come up with a better algorithm, but what I'm interested in is estimating how fast an ideal execution of this algorithm could be. How long could we expect this to run, on a modern CPU?

I expect, looking at this, that the vast majority of the time is going to be spent on mods = [i % p for p in known_primes]. Calculating a modulus is something like 30-50 cycles according to the few google searches I found, and everything else should be significantly less than that.

The number of mods we need to calculate for any i is the number of known primes, which is roughly ilog(i). Let's say we're getting all the primes until 100,000. Then we should need to calculate about Σ99,999i=3ilog(i), which is about 200 million mods. Times 30-50 cycles is about .6-1 billion cycles. On a 3gHz processor, that should take about a 200-333 milliseconds.

What do we actually see?

using BenchmarkTools
@btime modified_sieve(100_000)
> 352.089 ms (240255 allocations: 2.12 GiB)

That...shouldn't actually happen. Actual programs usually take 3x-10x as long as my estimates, because there's a lot of overhead I'm not accounting for. This suggests that there's some optimization happening that I didn't expect. And looking at the generated code, I do see several references to simdloop. Ok then. That just happened to cancel out the overhead.

Let's try an algorithmic improvement. We don't actually need to check against all primes – we only have to go up to&nb...
What's so special about likelihoods?
by mfatt — 2024-08-10T01:07:58.592Z

This is a coin.

It might be biased.

This is Bayes’ theorem.

P(A|B)=P(B|A)×P(A)P(B)

Bayes’ theorem tells us how we ought to update our beliefs given evidence.

It involves the following components:

P(A|B), called the posterior; it is the probability of A given B. In the case of the coin this is the probability that the coin is biased given the result of an experiment (i.e., a sequence of flips).
P(B|A), called the likelihood, is the probability of B given A. For our coin example this would be the probability of some particular ratio of heads to tails, given that the coin is biased.
P(A) is called the prior. This is the probability that the coin is biased, before we consider any evidence.
P(B) is the marginal. This is probability of us getting some sequence of head and tails, before considering any evidence.

The overall shape of the theorem is this:

Posterior ∝ likelihood × prior.

If you were explain this to a high-school student, they might ask this naïve question:

Why should we bother to go through the process of calculating the likelihood and prior at all? Why can’t we just try and directly calculate the posterior? We have a formula for P(A|B), namely P(A∩B)P(B).

Maybe you'll say "That formula is fine but not useful in real life. It's usually tractable to go via conditional updates rather than the high school definition."

But if conditionals are easy to get, why not just go directly to the posterior? What's even the difference between A and B? Aren't they just symbols? We could easily rearrange the theorem to calculate P(B|A) as a function of P(A|B).

What is it that makes using strings of coin flips to calculate biases more natural or scientific?

Perhaps it is ease. If it is the case that for some reason calculating P(B|A) is easier, what makes it easier?

Perhaps it is usefulness. If likelihoods are what's worth publishing, not posteriors, why are they worthier?

How do you spot a likelihood in the wild?
Provably Safe AI: Worldview and Projects
by Ben Goldhaber — 2024-08-09T23:21:02.763Z

In September 2023, Max Tegmark and Steve Omohundro proposed "Provably Safe AI" as a strategy for AI Safety. In May 2024, a larger group delineated the broader concept of "Guaranteed Safe AI" which includes Provably Safe AI and other related strategies. In July, 2024, Ben Goldhaber and Steve discussed Provably Safe AI and its future possibilities, as summarized in this document.

Background

In June 2024, ex-OpenAI AI Safety Researcher Leopold Aschenbrenner wrote a 165-page document entitled "Situational Awareness, The Decade Ahead" summarizing AI timeline evidence and beliefs which are shared by many frontier AI researchers. He argued that human-level AI is likely by 2027 and will likely lead to superhuman AI in 2028 or 2029. "Transformative AI" was coined by Open Philanthropy to describe AI which can "precipitate a transition comparable to the agricultural or industrial revolution". There appears to be a significant probability that Transformative AI may be created by 2030. If this probability is, say, greater than 10%, then humanity must immediately begin to prepare for it.

The social changes and upheaval caused by Transformative AI are likely to be enormous. There will likely be many benefits but also many risks and dangers, perhaps even existential risks for humanity. Today's technological infrastructure is riddled with flaws and security holes. Power grids, cell service, and internet services have all been very vulnerable to accidents and attacks. Terrorists have attacked critical infrastructure as a political statement. Today's cybersecurity and physical security barely keeps human attackers at bay. When these groups obtain access to powerful cyberattack AI's, they will likely be able to cause enormous social damage and upheaval.

Humanity has known how to write provably correct and secure software since Alan Turing's 1949 paper. Unfortunately, proving program correctness requires mathematical sophistication and it is rare in current software development practice. Fortunately, modern deep learning systems are becoming proficient at proving mathematical theorems and generating provably correct code. When combined with techniques like "autoformalization," this should enable powerful AI to rapidly replace today's flawed and insecure codebase with optimized, secure, and provably correct replacements. Many researchers working in these areas bel...
All The Latest Human tFUS Studies
by sarahconstantin — 2024-08-09T22:20:04.561Z

from Peng, et al; stimulating the nucleus accumbens inhibits its activity

Transcranial focused ultrasound neuromodulation — altering the brain’s activity with low-intensity ultrasound — is really exciting .

It allows us to manipulate arbitrary regions of the brain without surgery, potentially replacing the (brain-damaging) electrode implants currently used for serious neurological conditions like epilepsy and Parkinson’s, and potentially also expanding applications of brain stimulation to milder conditions not worth the risks of brain surgery, like mental illness, addiction, or chronic pain.

The field is rapidly growing, and since I wrote my earlier post series there have been quite a few human studies published. Here’s a systematic overview of all the human studies published in 2024, by target brain region.

Headline Results

This year’s papers further confirm, to start with, that ultrasound does things to brain activity, if that was still in doubt, and that it is safe enough to run human experiments with (no adverse effects during experiments with small numbers of participants and brief exposures.)

There are notably inconsistent results in whether targeting ultrasound to a given brain area increases or decreases neural activity in that area, even in some cases when the same area is targeted with the same sonication parameters! We clearly need to get a better sense of what ultrasound even does.

Most studies don’t do the obvious (but admittedly expensive) thing of confirming a change in neural activity via a noninvasive measure like fMRI. Those that do, show different results (more activity in the targeted region, less activity in the targeted region, or neither) depending on which region is targeted; this tells us that “tFUS” as a class doesn’t have a globally consistent effect on targeted neural activity. Again, still more to learn.

However, despite the primitive state of our understanding of this modality, we do already seem to have some strikingly useful results. Ultrasound stimulation of the thalamus seems to be helpful for essential tremor, stimulation of the posterior insula seems to reduce pain sensitivity, and stimulation of the anterior medial prefrontal cortex seems to have quite strong effects on depression. These are before vs. after results without a control group, not randomized controlled studies, but I think they at least warrant followup.

I’m not a...