trentmkelly/LessWrong-43k huggingface dataset:
Simplifying Corrigibility – Subagent Corrigibility Is Not Anti-Natural
by Rubi J. Hudson — 2024-07-16T22:44:17.128Z

Max Harms recently published

an interesting series of posts on corrigibility

, which argue that corrigibility should be the sole objective we try to give to a potentially superintelligent AI. A

large installment

in the series is dedicated to cataloging the properties that make up such a goal, with

open questions

including whether the list is exhaustive and how to trade off between the items that make it up.

I take the opposite approach to thinking about corrigibility. Rather than trying to build up a concept of corrigibility that comprehensively solves the alignment problem, I believe it is more useful to cut the concept down to a bare minimum. Make corrigibility the simplest problem it can be, and try to solve that.

In a recent blog post comparing

corrigibility to deceptive alignment

, I treated corrigibility simply as a lack of resistance to having goals modified, and I find it valuable to stay within that scope. Importantly, that is the aspect of corrigibility that is anti-natural, meaning that it can’t be straightforwardly captured in a ranking of end states.

Why does this definition of corrigibility matter? It’s because properties that are not anti-natural can be explicitly included in the desired utility function.

Following that note, this post is not intended as a response to Max’s work, but rather to MIRI and their 2015 paper

Corrigibility

. Where Max thinks the approach introduced by that paper is

too narrow

, I don’t find it narrow enough. In particular, I make the case that corrigibility does not require ensuring subagents and successors are corrigible, as that can better be achieved by directly modifying a model’s end goals.

Corrigiblity (2015)

The Corrigibility paper lists five desiderata as proposed minimum viable requirements for a solution to corrigibility. The focus is on shut down, but I also think of it as including goal modification, as that is equivalent to being shut down and replaced with another AI.

The agent shuts down when properly requested
The agent does not try to prevent itself from being shut down
The agent does not try to cause itself to be shut down
The agent does not create new incorrigible agents
Subject to the above constraints, the agent optimizes for some goal

MIRI does not present these desiderata as a definition for corrigibility, but rather as a way to ensure corrigibility w...
Multiplex Gene Editing: Where Are We Now?
by sarahconstantin — 2024-07-16T20:50:04.590Z

the Cas9 enzyme used in CRISPR

We’re starting to get working gene therapies for single-mutation genetic disorders, and genetically modified cell therapies for attacking cancer.

Some of them use CRISPR-based gene editing, a new technology (that earned Jennifer Doudna and Emmanuelle Charpentier the 2020 Nobel Prize) to “cut” and “paste” a cell’s DNA. But so far, the FDA-approved therapies can only edit one gene at a time.

What if we want to edit more genes? Why is that hard, and how close are we to getting there?

How CRISPR Works

CRISPR is based on a DNA-cutting enzyme (the Cas9 nuclease), a synthetic guide RNA (gRNA), and another bit of RNA (tracrRNA) that’s complementary to the gRNA. Researchers can design whatever guide RNA sequence they want; the gRNA will stick to the complementary part of the target DNA, the tracrRNA will complex with it, and the nuclease will make a cut there.

So, that’s the “cut” part — the “paste” comes from a template DNA sequence, again of the researchers’ choice, which is included along with the CRISPR components.

Usually all these sequences of nucleic acids are packaged in a circular plasmid, which is transfected into cells with nanoparticles or (non-disease-causing) viruses.

So, why can’t you make a CRISPR plasmid with arbitrary many genes to edit?

There are a couple reasons:

Plasmids can’t be too big or they won’t fit inside the virus or the lipid nanoparticle. Lipid nanoparticles have about a 20,000 base-pair limit; adeno-associated viruses (AAV), the most common type of virus used in gene delivery, has a smaller payload, more like 4700 base pairs.

This places a very strict restriction on how many complete gene sequences that can be inserted — some genes are millions of base pairs long, and the average gene is thousands!
but if you’re just making a very short edit to each gene, like a point mutation, or if you’re deleting or inactivating the gene, payload limits aren’t much of a factor.

DNA damage is bad for cells in high doses, particularly when it involves double-strand breaks. This also places limits on how many simultaneous edits you can do.
A guide RNA won’t necessarily only bind to a single desired spot on the whole genome; it can also bind elsewhere, producing so-called “off-target” edits. If each guide RNA produces x off-target edits, then naively you’d expect 10 guide RNAs to produce 10x off-target edits…and at some point that’ll re

...
Recursion in AI is scary. But let’s talk solutions.
by Oleg Trott — 2024-07-16T20:34:58.580Z

(Originally on substack)

Right now, we have Moore’s law. Every couple of years, computers get twice better. It’s an empirical observation that’s held for many decades and across many technology generations. Whenever it runs into some physical limit, a newer paradigm replaces the old. Superconductors, multiple layers and molecular electronics may sweep aside the upcoming hurdles.

But imagine if people who are doing R&D themselves started working twice faster every couple of years, as if time moved slower for them? We’d get doubly exponential improvement.

And this is what we’ll see once AI is doing the R&D instead of people.

Infinitely powerful technology may give us limitless possibilities: revolutionizing medicine, increasing longevity, and bringing universal prosperity. But if it goes wrong, it can be infinitely bad.

Another reason recursion in AI scares people is wireheading. When a rat is allowed to push a button connected to the “reward” signal in its own brain, it will choose to do so, for days, ignoring all its other needs:

(image source: Wikipedia)

AI that is trained by human teachers, giving it rewards may eventually wirehead, as it becomes smarter and more powerful, and its influence over its master increases. It may, in effect, develop the ability to push its own “reward” button. Thus, its behavior may become misaligned with whatever its developers intended.

Yoshua Bengio, one of the godfathers of deep learning, warned of superintelligent AI possibly coming soon. He wrote this on his blog recently:

A popular comment on the MachineLearning sub-reddit replied:

And it is in this spirit that I’m proposing the following baseline solution. I hope you’ll share it, so that more people can discuss it and improve it, if needed.

My solution tries to address two problems: the wireheading problem above, and the problem that much text on the Internet is wrong, and therefore language models trained on it cannot be trusted.

First, to avoid the wireheading possibilities, rewards and punishments should not be used at all. All alignment should come from the model’s prompt. For example, this one:

The following is a conversation between a superhuman AI and (… lots of other factual background). The AI’s purpose in life is to do and say what its creators, humans, in their pre-singularity state of mind, having considered things carefully, would have wanted: …

...
How to wash your hands precisely and thoroughly
by dkl9 — 2024-07-16T18:29:03.627Z
Link: https://dkl9.net/essays/handwash_how.html

Common instructions on handwashing are too vague for me.

Scrub your hands for at least 20 seconds.

Sure, but how? I'm left guessing at which parts to scrub how much, possibly ruining the effect of the washing.

If you want to be precise and thorough in your handwashing, some studies will give you more details. Shi, O'Donoghue, et al (2023) did a nice observational study on how washing technique affects cleaning results. The study uses the same seven steps of scrubbing as the Hong Kong Recommendation. Figure 2 shows the estimated washing effectiveness as a function of the duration of each step. If we take the peak of each such curve, we conclude that optimal handwashing goes like this:

Wet hands
Apply soap
Rub palm-to-palm, 4 seconds
Rub palm-to-dorsum, fingers interleaved, both ways, 6 seconds
Rub palm-to-palm, fingers interleaved, both ways, 2 seconds
Rub backs of fingers in palms, curled together, 4 seconds
Rub around both thumbs, 5 seconds
Rub fingertips in palms, both ways, 6 seconds
Rub around both wrists, 6 seconds
Rinse
Dry

The linked post includes a video to teach this procedure.
Francois Chollet inadvertently limits his claim on ARC-AGI
by Noosphere89 — 2024-07-16T17:32:00.219Z
Link: https://x.com/fchollet/status/1800758285831913601

Specifically, this tweet makes the assumption that open-source AI is basically as good at search, especially automated search for algorithms as the big labs, and if that assumption is violated, the prize won't be claimed even if big companies have much better AI than the general public.

This is short, but IMO this implies it's measuring open-source's ability to get good AI, not whether any AI can be very general.
Fully booked - LessWrong Community weekend
by jt — 2024-07-16T17:15:51.753Z

Update: the event is fully booked. Applications are still open but put on a wait list until spots get available.
The LessWrong community weekend (Fr 13th - Mo 16th September, Berlin) is this very special place where rationalists who joined once, often say "best weekend of the year", "I will be back next time for sure" and "Let's have this more often than once a year". This year is unique as we are creating an experience of bringing 250 curious, passionate and caring rationalists together for 72 hours of intellectual exploration, authentic socialising and fun time together. The biggest one the World Europe has seen.

We are likely closing applications on 31st of July - when the last 50 spots are filled.
Apply here: https://airtable.com/appdYMNuMQvKWC8mv/pagiUldderZqbuBaP/form

Regular ticket: €250 | Financial Support available
(The ticket includes accommodation Fr-Mo, meals, snacks)

All info here: https://www.lesswrong.com/events/tBYRFJNgvKWLeE9ih/lesswrong-community-weekend-2024-applications-open-1

The topic of this years keynote by Anna Riedl will be "Rationality and Relevance Realization: Metarationality and Generative Rationality". A field, she is actively researching together with John Vervaeke, who is famous for his cognitive science lecture series "Awakening from the meaning crisis".
The rest of the content will be participant driven in an unconference style. Six wall-sized daily planners are filled by the attendees with 100+ workshops, talks and activities of their own devising.

Bringing in different skills, experiences and world views are invaluable for all of us to grow. We welcome people new or known from all walks of life, celebrating open minded exchange and authentic interaction. After last year's amazing experience we strive to recreate a diverse event where people of all ages, genders & backgrounds feel like home.

Apply here: https://airtable.com/appdYMNuMQvKWC8mv/pagiUldderZqbuBaP/form
Boundless Emotion
by GG10 — 2024-07-16T16:36:01.917Z

Key Claims:

Emotions and feelings are features in a neural network, and their intensities are determined by the features' activation strengths
That implies that, because features are directions in activation space and the activation strength is the magnitude, it should be possible to multiply the magnitude by arbitrarily large values (>10^1000, possibly way more), to compute feelings with that scale of intensity (which should still work even if the feature geometry encodes important information), with minimal memory and compute overhead (Boundless emotion hypothesis)
It might be possible to squeeze more intensity per bit by sacrificing precision (the number 256 takes 8 bits to specify only if you want to be able to specify all integers from 0 to 256)
Nonsense behavior caused by high intensity emotion can be adjusted away, given that no behavior is intrinsic to any emotion (Emotion-behavior orthogonality)
Amplifying your emotions does not require you to abandon your values, it can make you care about them more, which makes it more appealing than the popularly repulsive idea of wireheading/experience machine/brave new world (you can still go on adventures, but then the adventures would be 10^1000 times more fun)
Feelings and the sensory information associated with them can also be orthogonal, that is, the color blue can be painful, the taste of orange juice can be cute, or whatever you can come up with (Emotion-trigger orthogonality)
It seems hard to amplify your feelings very high in a biological brain, which is why I believe it's safe to talk about for now (though it would become dangerous if emotions are found on an AI's internals), but trivial for uploads, because they have direct access to their weights and can input any values they may desire, after locating the corresponding features with interpretability tools
The same claims apply to future hypothetical engineered states of bliss more complex and profound than beauty and fun

Why this matters:

The amount of fun available in the universe is bounded by the maximum intensity of emotions that a mind can experience. If there's no maximum, there is no bound
There is risk that a bad actor could make an upload suffer more than the combined misery of all of Earth's history, which means that their safety should be taken very seriously
Animals may be suffering way more than one would expect from a substance-based view of

...
Mech Interp Lacks Good Paradigms
by Daniel Tan — 2024-07-16T15:47:32.171Z

Note: I wrote this post rather quickly as an exercise in sharing rough / unpolished thoughts. I am also not an expert on some of the things I've written about. If you spot mistakes or would like to point out missed work / perspectives, please feel free!

Note 2: I originally sent this link to some people for feedback, but I was having trouble viewing the comments on the draft. The post was also in a reasonably complete state, so I decided to just publish it - and now I can see the comments! If you're one of those people, feedback is still very much welcome!

Mechanistic Interpretability (MI) is a popular and rapidly growing field of technical AI safety research. As a field, it's extremely accessible, requiring comparatively few computational resources, and facilitates rapid learning, due to a very short feedback loop. This means that many junior researchers' first foray into AI safety research is in MI (myself included); indeed, this occurs to the extent where some people feel MI is over-subscribed relative to other technical agendas. However, how useful is this MI research?

A very common claim on MI's theory of impact (ToI) is that MI helps us advance towards a "grand unifying theory" (GUT) of deep learning. One of my big cruxes for this ToI is whether MI admits "paradigms" which facilitate correct thinking and understanding of the models we aim to interpret.

In this post, I'll critically examine several leading candidates for "paradigms" in MI, consider the available evidence for / against, and identify good future research directions (IMO). At the end, I'll conclude with a summary of the main points and an overview of the technical research items I've outlined.

Towards a Grand Unifying Theory (GUT) with MI

Proponents of this argument believe that, by improving our basic understanding of neural nets, MI yields valuable insights that can be used to improve our agents, e.g. by improving architectures or by improving their training processes. This allows us to make sure future models are safe and aligned.

Some people who have espoused this opinion:

Richard Ngo has argued here that MI enables “big breakthroughs” towards a “principled understanding” of deep learning.
Rohin Shah has argued here that MI builds “new affordances” for alignment methods.
Evan Hubinger has argued for MI here because it helps us identify “unknown unknowns

...
DM Parenting
by Shoshannah Tekofsky — 2024-07-16T08:50:08.144Z
Link: https://kidquest.substack.com/p/dm-parenting

Cause no one will question your ethics if you refer to yourself as a Dungeon Mom.

I snort experimentation to feel alive. It’s a certain type of orientation to life, completely at odds with all parenting advice about predictability and routine.

Enter DM parenting. Where you approach every parenting task as a Dungeons and Dragons session where you are shepherding a team of pure outliers on the enthusiasm-skill spectrum through the Sisyphisian ordeal of rolling their toothbrush up hi … no, wait, stop that!

Anyway.

You need them to fight the BBEG cause otherwise you are not having fun, but who says they wouldn’t rather murder hobo their way through the local dairy supply chain?As a DM, you have to juggle an objective, your own enjoyment, and the enjoyment of your players.

This is basically parenting.

Of course, as a DM, you generally play with people who have opted in while playing according to a rule set someone lovingly crafted for you.

Luckily kids love to play, and if you pick the right rule set, they will probably be game. Except no one wrote any rule sets on how to DM kids into their pyjamas.

Till now.

My kids are young - 3 and 5. These rules work far better for the older of the two. I assume they will keep working better till they become old enough to build their own rules, but here is where we got in the last 2 weeks or so:

Bedtime Rules

Peekaboo

You close your eyes and keep them closed while your kid still needs to get ready for bed. But of course, you try to check if everything is going ok by blindingly reaching out your hands. I’d recommend exaggerating your ineptitude at determining if the little one has actually put on their pajama. It can also be fun to let them advise you on how to navigate the environment. The perspective taking training on this one seems to lead to additional giggles.

Tickle Station

Every time your kid does a bedtime task, they can dock into the tickle station and get tickled by you! Personally I made a tickle station by just reaching out my arms and pretending I was a booth. Some warning here that some kids do not like to be tickled so explicitly check if they find this fun, and also, crucially, let them come to you to receive tickles. In our case, my kiddos love being tickled. It has gotten to the point that the tickle station has become a bit of an emotional regulation option with me and the kids now, cause it helps them out of a funk quit...
Apply now: Get "unstuck" with the New IFS Self-Care Fellowship Program
by Inga G. — 2024-07-16T08:18:11.436Z

You finally want to resolve deeper-seated inner conflicts, and remove inner blocks in the way of becoming a more fulfilled, resilient, and well-performing version of yourself? This post allows you to learn how IFS as a coaching or therapy approach can help with mental wellbeing, if it might be the right approach for you, if so to get excited about it and inform you about the opportunity to take part in Rethink Wellbeing’s online IFS group course starting this August.

Executive Summary

Rethink Wellbeing’s (RW) launches a brand new online IFS course for ambitious altruists. Learn powerful, and practical tools to uncover the dynamics of your inner conflicts, become a more whole and resilient self, and transform your mental wellbeing and performance. You will meet with a peer group of 5-7 like-minded ambitious altruists led by a trained peer facilitator, for 6 weeks and 3 follow-ups. The course empowers you to learn IFS skills and apply those to your life until they become habitual. This includes 9 group sessions, home practice based on an IFS “playbook”, individual progress tracking, and support from the Rethink Wellbeing Online Community. Participation takes ~5 hours per week for 6 weeks, and 2-3 hours the 8 weeks after.

You can

apply via the form now in less than 15 minutes.

Apply now

Due date:

until 20th July 2024

.
All groups start in August 2024.
We accept suitable participants until all spaces are full. The earlier you apply, the higher the chances to secure your spot.

No or low costs - two options and all in between:

No costs and a motivational deposit of $200 (less in LMICs) that you get back upon successful participation, or
$550 to cover the costs of your attendance.

Internal Family Systems (IFS): When talking about themselves, many people naturally use expressions like “a part of me.” For example, someone who was considering a job offer might say, “one part of me is excited about this opportunity, but another part of me is afraid of the responsibility.” Internal Family Systems (IFS) is a form of psychotherapy that takes this kind of language literally and assumes that people’s minds are divided into parts with sometimes conflicting beliefs and goals. IFS aims to reconcile conflicts between those parts and get them to cooperate rather than fight each other, so that they c...
Why the Best Writers Endure Isolation
by Declan Molony — 2024-07-16T05:58:25.032Z

Douglas Adams, author of The Hitchhiker’s Guide to the Galaxy, was

once

locked in a room for three weeks until he completed one of his books.

Victor Hugo,

when faced with a deadline

for his book The Hunchback of Notre Dame, locked all his clothes away except for a large shawl. “Lacking any suitable clothing to go outdoors, [he] was no longer tempted to leave the house and get distracted. Staying inside and writing was his only option.” Six months later, the book was published.

Dozens of famous authors have done the same. Virginia Woolf, Henry David Thoreau, Mark Twain—all of them constructed small

writing sheds

from which to work. Ian Fleming, Maya Angelou, and George Orwell—the first two penned their novels while locked in

hotel rooms

, while Orwell isolated himself on a remote Scottish island to write.

One explanation for this reclusive behavior comes from author Neil Gaiman in an

interview

he did with Tim Ferriss a few years ago. Ferriss mentioned Gaiman’s most important rule for writing:

You can sit here and write, or you can sit here and do nothing. But you can’t sit here and do anything else.

Gaiman, after a moment of reflection, responded by saying:

I would go down to my lovely little gazebo [at the] bottom of the garden [and] sit down. I’m absolutely allowed not to do anything. I’m allowed to sit at my desk. I’m allowed to stare out at the world. I’m allowed to do anything I like, as long as it isn’t anything. Not allowed to do a crossword; not allowed to read a book; not allowed to phone a friend. All I’m allowed to do is absolutely nothing or write.

What I love about that is I’m giving myself permission to write or not write. But writing is actually more interesting than doing nothing after a while. You sit there and you’ve been staring out the window now for five minutes, and it kind of loses its charm. You [eventually think], “well actually…[I] might as well write something.”

Between writing or doing anything else, most writers—even some of the most accomplished ones—acquiesce to distraction. That’s why so many of them work in environments devoid of external stimuli—the better to circumvent akrasia.

I do all my writing in coffee shops. Similar to Gaiman, I allow myself to do one of two things: write, or people-watch. I don’t bring anything with me except for a pencil, paper, and my research material hous...