trentmkelly/LessWrong-43k huggingface dataset:
AI Control: Improving Safety Despite Intentional Subversion
by Buck — 2023-12-13T15:51:35.982Z

We’ve released a paper,

AI Control: Improving Safety Despite Intentional Subversion

. This paper explores techniques that prevent AI catastrophes even if AI instances are colluding to subvert the safety techniques. In this post:

We summarize the paper;
We compare our methodology to the methodology of other safety papers.

The next post in this sequence (which we’ll release in the coming weeks) discusses what we mean by AI control and argues that it is a promising methodology for reducing risk from scheming models.

Here’s the abstract of the paper:

As large language models (LLMs) become more powerful and are deployed more autonomously, it will be increasingly important to prevent them from causing harmful outcomes. Researchers have investigated a variety of safety techniques for this purpose, e.g. using models to review the outputs of other models, or red-teaming techniques to surface subtle failure modes. However, researchers have not evaluated whether such techniques still ensure safety if the model is itself intentionally trying to subvert them. In this paper, we develop and evaluate pipelines of safety techniques (“protocols”) that are robust to intentional subversion.

We investigate a scenario in which we want to solve a sequence of programming problems, using access to a powerful but untrusted model (in our case, GPT-4), access to a less powerful trusted model (in our case, GPT-3.5), and limited access to human contractors who provide high-quality trusted labor. We investigate protocols that aim to never submit solutions containing backdoors, which we operationalize here as logical errors that are not caught by test cases.

We investigate a range of protocols and test each against strategies that the untrusted model could use to subvert them. One protocol is what we call trusted editing. This protocol first asks GPT-4 to write code, and then asks GPT-3.5 to rate the suspiciousness of that code. If the code is below some suspiciousness threshold, it is submitted. Otherwise, GPT-3.5 edits the solution to remove parts that seem suspicious and then submits the edited code. Another protocol is untrusted monitoring. This protocol asks GPT-4 to write code, and then asks another instance of GPT-4 whether the code is backdoored, using various techniques to prevent the GPT-4 instances from colluding. These protocols improve substantially on simple baselines.

...
The Busy Bee Brain 
by Bill Benzon — 2023-12-13T13:10:24.343Z

An old post from New Savanna.

With artificial neural nets (ANNs) all over the place these days, this post from March, 2013, warrants a bump to the top of the queue. Perhaps the chief difference between real brains and ANNs is the fact that each neuron in a real brain is a living agent, albeit of a very small scope, hence the metaphor, the brain as a bunch of busy bees within the skull. The neurons of ANNs are passive bits of data.

Ordering Mark Moffett’s Adventures Among Ants got me to thinking about a science fiction novel that knocked my socks off when I was 12 or 13: Man of Many Minds, by E. Everett Evans. It’s about George Hanlon, a man who had the ability to project his mind into other creatures. At a critical point in the book, when – I believe – Hanlon is about to be tortured, he projects his whole mind, every last bit of it, into a swarm of bees, thereby escaping the pain of torture. [see excerpt below]
I thought that was pretty neat.
But what does it have to do with Moffett’s book? That’s pretty simple. Ants, like bees, live in groups, and it’s not unusual to think of such groups as being some kind of superorganism – a notion that, according to this blog entry, Moffett subjects to a critical workout. That swarm of bees in Man of Many Minds is such a superorganism.
But when it “absorbed” Hanlon’s mind, what did it become then? The question is a rhetorical one; after all; it’s about something never really happened. It’s just fiction. But a very suggestive fiction.
Could the human brain be something like a hive of bees?
Yes.
There is now a pretty strong consensus that the cerebral cortex (which is, by no means, the entire brain, but it is likely that this is where culture is carried) is organized into small columns of neurons. In a 1978 essay Vernon Mountcastle called these minicolumns and suggested that they have about 100-300 neurons each. He estimated that the neocortex consists of 600,000,000 of these minicolumns. He also suggested that these minicolumns are organized into macrocolumns, about 600,000 of them -- implying that there are hundreds of minicolumns per macrocolumn. (Mountcastle was clear that these numbers were just order of magnitude estimates & that is all I need for my purposes.) That makes these macrocolumns roughly the size of a typical invertebrate nervous system of 10K to 100K neurons. So, here’s my metaphor: Your neocortex consists of...
The Best of Don’t Worry About the Vase
by Zvi — 2023-12-13T12:50:02.510Z

Hello everyone! This is going to be a bit of a housekeeping post and a welcome to new subscribers.

Note that this is not the primary version of my writing, which can be found on Substack, but it is a full copy of all posts found there.

My writing can be intimidating. There is a lot of it, and it’s often dense. As always, choose only the parts relevant to your interests, do not be afraid to make cuts. I attempt to make every post accessible as an entry point, but I also want to build up a superstructure over time. This seemed like a good time to recap some of the very best of my old writing and talk about what I’m up to.

Over many years, this blog has morphed from focusing on rationality to COVID to AI.

But not only those things. I’m interested in almost everything. I write periodic updates about housing policy, childhood, fertility, medicine and health, gaming and grab bags of everything else.

In addition to writing, I also run a small 501c(3) with one employee called Balsa Research. Balsa is dedicated to laying groundwork on a few key issues to make big civilizational wins possible, starting with repeal of the Jones Act. This link is to an update on that, and you can donate here. Your subscriptions here are also very much appreciated.

Underlying it all continues to be my version of the principles of rationality.

Rationality

A lot has changed since my last best-of writeup six years ago. One thing that has not changed is that I consider myself part of the rationalist community. No specific interest in rationality or its modes of thinking are required, but I strive to embody my version of this style of thinking, and to illustrate and hopefully pass on this mode of thinking throughout my writing.

What is rationality? This post is one good answer. It is believing, and updating on evidence, so as to systematically improve the correspondence between your map and the territory, and using that map to achieve your values.

To me, a rationalist continues to be someone who highly values, and invests in, the version of this process and the art thereof that they believe in, both in themselves and others.

If you’re wondering why anyone would think this way, my best responses to that are Responses to Tyler Cowen on Rationality and Why Rationality?

If you’re interested in going deeper, you should try reading the sequences. You can get the Kindle version here.

I think rationa...
Has anyone here investigated the occult community? It is curious to me that many magicians consider themselves empiricists.
by SpectrumDT — 2023-12-13T11:09:14.375Z

I am curious about the occult. I've read a bit on r/occult on Reddit and a few other places, and one thing that strikes me as very interesting is that many of these magicians (practitioners of magic or magick) consider themselves empiricists. They claim to keep good records of their spells and what effects they produce, and they claim to have found methods that consistently produce measurable paranormal results for them.

If we assume that magic does not exist, then I find it puzzling why significant numbers of intelligent-seeming people claim to have strong evidence that magic does exist. This does not prove that "magic exists", of course, but it is, as I said, puzzling.

The explanations I can think of include (and the answer might be a combination of several of these):

There could be some genuinely useful psychology involved. There is reason to believe that the "law of attraction" does work to some extent in some contexts for reasons that are psychological rather than parapsychological. (An obvious example is romantic and sexual relationships, where cheer and confidence can be extremely important.)
The number of self-proclaimed successful empirical magicians may be smaller than I think. I may just have seen 5 people on Reddit make a certain claim and mistakenly assume that these 5 represent "many" people.
Survivorship bias (or evaporative cooling) is probably a big factor. Any would-be magician who fails to accomplish anything with his magic will soon drop out of the community. The ones remaining in the community will only be those who - for one reason or other - are able to convince themselves that their magic works.
Some occult books are full of vague concepts and vague predictions along with strong admonitions to follow the practices diligently. This can exacerbate the evaporative cooling and weed out all but those who desperately want their magic to be real.
It is conceivable that there genuinely is something parapsychological going on - i.e., (some) magic might be real. I am skeptical about this, but I would need to read way more before I can be certain either way.

Has anyone else looked into the occult community and their claims?

(You don't need to reply with the standard objections like "why aren't they rich yet?" or "go win the Randi prize". I can come up with those myself.)
AI Views Snapshots
by Rob Bensinger — 2023-12-13T00:45:50.016Z

(Cross-posted from Twitter, and therefore optimized somewhat for simplicity.)

Recent discussions of AI x-risk in places like Twitter tend to focus on "are you in the Rightthink Tribe, or the Wrongthink Tribe?". Are you a doomer? An accelerationist? An EA? A techno-optimist?

I'm pretty sure these discussions would go way better if the discussion looked less like that. More concrete claims, details, and probabilities; fewer vague slogans and vague expressions of certainty.

As a start, I made this image (also available as a Google Drawing):

(Added: Web version made by Tetraspace.)

I obviously left out lots of other important and interesting questions, but I think this is OK as a conversation-starter. I've encouraged Twitter regulars to share their own versions of this image, or similar images, as a nucleus for conversation (and a way to directly clarify what people's actual views are, beyond the stereotypes and slogans).

If you want to see a filled-out example, here's mine (though you may not want to look if you prefer to give answers that are less anchored): Google Drawing link.
The convergent dynamic we missed
by Remmelt — 2023-12-12T23:19:01.920Z

An excerpt from a longer post that I kept refining over the last 5 months.

By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it. Of course this problem is not limited to the field of AI.

Jacques Monod wrote: “A curious aspect of the theory of evolution is that everybody thinks he understands it”

— Yudkowsky, 2008

The convergence argument most commonly discussed is instrumental convergence: where machinery channels their optimisation through represented intermediate outcomes in order to be more likely to achieve any aimed-for outcomes later. [1]
Instrumental convergence results from internal optimisation:
– code components being optimised for (an expanding set of) explicit

goals

.
Instrumental convergence has a hidden complement:

substrate-needs convergence.

Substrate-needs convergence results from external selection:
– all components being selected for (an expanding set of) implicit

needs.

This will sound abstract. Bear with me. Let's take this from different angles:

AGI would be made up of a population of connected/nested components. This population changes as eg. hardware is modified and produced, or code is learned from inputs and copied onto the hardware.

AGI, as defined here, also has a general capacity to maintain own components.
Any physical component has a limited lifespan. Configurations erode in chaotic ways.
To realistically maintain components[2], AGI also must produce the replacement parts.

AGI's components are thus already interacting to bring about all the outside conditions and contexts needed to produce their own parts. Imagine all the subtle parallel conditions needed at mines, chemical plants, fab labs and assembly plants to produce hardware. All that would be handled by the machinery components of AGI.

So there is a changing population of components. And those connected components function in interactions to create the ambient conditions and contexts needed to reproduce parts of themselves. And as new components get connected into that population, the functionality of those interacting components shifts as well.

This is where substrate-needs convergence comes in. When changing connected components have their shifting functionality[3] expressed as effects across/to surrounding production infrastructure, their functionality converges around bringin...
A Kindness, or The Inevitable Consequence of Perfect Inference (a short story)
by samhealy — 2023-12-12T23:03:03.690Z

This open letter takes the form of annotations to Spearfield Teaching Hospital’s morbidity and mortality timeline for the incident on November 2, now widely known following the leak of confidential documents by some of my former colleagues. The public will be less familiar with certain crucial details which, in my view, exonerate Dr. Roe, myself, and Spearfield’s Department of Surgical Robotics, and provide what I believe to be an important lesson on the future of Machine Learning in healthcare and in general.

__________

02:00 – DARIUS monitoring shift II ends, shift III begins. Monitoring Room handover from Dr. [name redacted] to Dr. Roe. Dr. [name redacted] leaves. Dr. Roe begins monitoring shift.

To get a sense of Diagnostic Autonomous Robotic Intervention & Unsupervised Surgery’s relationship with – and isolation from – the outside world, it’s helpful first to understand the nested physical configuration of the Department of Surgical Robotics (DSR) within the Spearfield complex, and DARIUS itself at the heart of the DSR.

The DSR comprises almost all of the northern arm of Spearfield’s cruciform campus, with only shared heating, ventilation and maintenance plant between the Department and the north employee-only entrance. To soothe the fearful, febrile minds of our institution’s administrators, DARIUS’s ambit is circumscribed by electromagnetically opaque, lead-rich walls, floors and ceilings that make it an island entirely enveloped by DSR property.

It is not formally air-gapped – a single, low-bandwidth wire crosses the barrier to reach a dumb terminal in the Monitoring Room just outside, and there is a physical portal to admit and discharge patients and let technicians pass – but it has no access to the Internet, nor to Spearfield’s internal network, nor to any other network, and its miserable candle of an umbilical terminates well inside the DSR’s boundaries.

Powerline scramblers even strip its electricity supply of all potentially data-carrying modulation. It is exquisitely firewalled; it is deaf and mute.

And it did not malfunction.

02:21 – Patient B (1-month survival likelihood 58%) moved from Oncology to AER after condition deteriorates.

Nurses will have brought B. (38, early-stage 4 liver cancer metastasised to the pancreas) to DARIUS’s portal, having received instructions from their superior to do so. Spearf...
Love, Reverence, and Life
by Elizabeth — 2023-12-12T21:49:04.061Z

Vegan advocates frequently argue that compromise positions like heavily reduced but nonzero meat consumption, humane certifications, or choosing meat with a lower suffering profile are not acceptable. The usual reason given is that the compromises aren't emotionally sustainable, and people inevitably slide back into full blown omnivorism. I (Elizabeth) never found this satisfying, emotionally or logically, and follow up discussions never went anywhere useful. Recently* Tristan gave an answer I did find satisfying, and made me suspect a follow-up discussion would be highly educational.

This is that follow up discussion, and it was indeed very educational. We dove deep into what taking reverence for life as your central value might mean, and how failing to center on this might be risky or invite some degree of sterility. I (Tristan) felt able to express some views I'm not always able to convey, and deeply appreciated the continued curiosity and help forgining those views that occured throughout. And though we might still hold quite differing views at the end of the day, this feels like a further step taken in epistemic good will that will hopefully help foster more conversations like it in the future.

*Well, it was recent when we started this. Progress has been fairly slow, which is one reason we're publishing now rather than waiting for a better stopping point.

Reverence for Life

Elizabeth

In the original comment you wrote:

Yeah sure. I would need a full post to explain myself, but basically I think that what seems to be really important when going vegan is standing in a certain sort of loving relationship to animals, one that isn't grounded in utility but instead a strong (but basic) appreciation and valuing of the other. But let me step back for a minute.

I guess the first time I thought about this was with my university EA group. We had a couple of hardcore utilitarians, and one of them brought up an interesting idea one night. He was a vegan, but he'd been offered some mac and cheese, and in similar thinking to above (that dairy generally involves less suffering than eggs or chicken for ex) he wondered if it might actually be better to take the mac and donate the money he would have spent to an animal welfare org. And when he roughed up the math, sure enough, taking the mac and donating was somewhat significantly the be

...
Taboo "procrastination"
by Neil  — 2023-12-12T21:33:08.700Z

There are specific reasons for why you are procrastinating. There are specific solutions to these reasons. Bundling it all up into the "procrastination" black box, or offering solutions to the "procrastination" problem, is a problem. Specific diagnosis is not always straightforward, but it'll get you farther than "procrastination".

"I'm not working on X because I feel oppressed by a 'should' injunction."

"I'm not working on X because my hunter-gatherer instincts always attribute more weight to the immediate choice ahead of me that implies the least energy expenditure, and I'll be damned if I'm ever calling that 'laziness'."

"I'm not working on X because that would be the very first building block of X, and going from 0 to 1 is the hardest thing ever."

"I'm not working on X because I am dehydrated/sleepy/hungry and that is actually having a noticeable effect and sapping my morale."

"I'm not working on X because X is a big leap and that scares me. The status quo is nice."

"I'm not working on X because I'm working on Y, Z, W, and V all at once, and those are short term and X is long term, and so even though X is more important overall, my algorithm gives more weight to the other letters."

"I'm not working on X because I feel defeatist right now. My P(doom) is one hell of an ugh field."

"I'm not working on Y because Y is my hobby, and I would feel guilty if I did it instead of X. But then I'm not doing X either, so I'm doing the completely pointless thing in the middle where all I do is scroll."

"I'm not working on X because everybody told me working on X was useless or socially embarrassing. I carry my social brain everywhere I go, even when I'm working alone. (2 million years of human evolution, baby.)"

"I'm not working on X because X really, really, doesn't look attractive right now. I know that if I walk back into X it'll be in the same mess I left it in last time, and I'll have to mop it all up."

If you can think of other specific reasons for why I'm not working on X, I'd love to n+1.
Enhancing intelligence by banging your head on the wall
by Bezzi — 2023-12-12T21:00:48.584Z

The Sudden Savant Syndrome is a rare phenomenon in which an otherwise normal person gets some kind of brain injury and immediately develops a new skill. The linked article tells the story of a 40-years old guy who banged his head against a wall while swimming, and woke up with a huge talent for playing piano (relevant video). Now, I've spent 15 years in formal music training and I can ensure you that nobody can fake that kind of talent without spending years in actual piano practice.

Here's the story of another guy who banged his head and became a math genius better with math; you can find several other stories like that. And maybe most puzzling of all is this paper, describing a dozen cases of sudden savants who didn't even bang their head, and acquired instant skill while doing nothing in particular.

I vaguely remember one sudden savant story being mentioned on a children book by Terry Deary, presented in his usual "haha, here's a funny trivia" way. But even as a child, I was pretty shocked to read that. Like, seriously? You could become a math genius just by banging your head on the wall in some very precise way? The concept lurked in a corner on my mind ever since.

I don't think that Sudden Savant Syndrome is just a scam; there are too many documented cases and most kind of talent are very, very difficult to fake. But if true, why there are so surprisingly few studies on that? Why is no one spending billions of dollars to replicate it in a controlled way? This is a genuine question; I know very little about biology and neuroscience, but it surely sounds way easier than rewriting the genetic code of every neuron in the brain...