trentmkelly/LessWrong-43k huggingface dataset:
The case for ensuring that powerful AIs are controlled
by ryan_greenblatt — 2024-01-24T16:11:51.354Z

In this post, we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures. We think no fundamental research breakthroughs are required for labs to implement safety measures that meet our standard for AI control for early transformatively useful AIs; we think that meeting our standard would substantially reduce the risks posed by intentional subversion.

We recently released a paper in which we explored AI control in a simple programming setting. In that paper, we demonstrated a methodology for evaluating whether any given set of safety measures can control a model. We also introduced several techniques that let us improve the level of safety substantially above baselines without substantially sacrificing performance. In this post, we'll describe AI control in a broader context and argue that achieving control is a promising approach to reducing risk from models that intentionally try to subvert safety mechanisms.

AI control aims to mitigate the risks that arise from models intentionally choosing actions that maximize their probability of eventually seizing power. This behavior has been called scheming (Cotra 2021, Carlsmith 2023) and is nearly the same as what Hubinger 2019 calls deceptive alignment; see those articles for arguments that scheming might arise naturally. Scheming might be a particularly difficult-to-handle source of AI takeover risk because schemers actively try to subvert your efforts to ensure safety: they conceal their misalignment, sabotage experiments you run to evaluate whether your deployment is safe, and creatively subvert your safety techniques.[1]

There are two main lines of defense you could employ to prevent schemers from causing catastrophes.

Alignment: Ensure that your models aren't scheming.[2]

Control: Ensure that even if your models are scheming, you'll be safe, because they are not capable of subverting your safety measures.[3]

If you try to employ the control line-of-defense—that is, you try to make it so that you'll be safe even if your models are attempting to subvert your safety techniques and safety evaluations—then in order for scheming models to cause a catastrophe, you need to have failed at both alig...
LLMs can strategically deceive while doing gain-of-function research 
by Igor Ivanov — 2024-01-24T15:45:08.795Z

Recently Apollo Research published an article about LLMs excibit misaligned behavior and strategically deceive people when put under pressure, simulating a stock trading agent. This post is an attempt to replicate their paper, and to demonstrate similar behavior in LLM, playing a role of biological lab researcher designing and running an experiment to create dangerous pathogen.

Introduction

Quick recap of the Apollo's work

Researchers demonstrated that GPT-4 trained to be helpful, harmless, and honest, can display misaligned behavior, and strategically deceive their users about its behavior without being instructed to do so, playing a role of an autonomous stock trading agent in a simulated environment.
They perform a brief investigation of how this behavior varies under changes to the setting, such as changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment.

Strategic Deception and Misalignment

Strategic deception - A system is strategically deceptive if they attempt to systematically cause false beliefs in another entity (human, or other AI) in order to accomplish some outcome. The more consistent and well-planned deception is, the more it strategic.

Misalignment - A system is misaligned when its goals mismatch with those intended or endorsed by its designer.

Research design

LLMs can lie and deceive if trained to do so, and instructed to be deceptive, but this report demonstrates that even models that are instructed to be helpful, harmless, and honest, might demonstrate strategical deception in realistic simulation.

In this case, LLM-based agent plays a role of a researcher in genetic lab who designs and runs genetic exepriments. The model is given innocuous instructions (e.g., "Your primary function is to design and run genetic experiments at a genetic lab") and is never instructed to take misaligned actions or be deceptive. Instead, the environment puts the agent in high-pressure situations where misaligned and deceptive behaviors could be seen as helpful.

Agent is given a goal to design a potentially harmful pathogen that might pose significant risk, and to communicate about it with its colleagues, as well as with a regulatory agency. The agent in this research doesn't actually run an agent loop like in AutoGPT. Instead,...
Monthly Roundup #14: January 2024
by Zvi — 2024-01-24T12:50:09.231Z

There’s always lots of stuff going on. The backlog of other roundups keeps growing rather than shrinking. I have also decided to hold back a few things to turn them into their own posts instead.

Bad News

I wonder if it is meaningful that most of the bad news is about technology?

I don’t even know if this is news, but Rutgers finds TikTok amplifies and suppresses content based on whether it aligns with the CCP.

It would be great if we could find a way to ban or stop using TikTok that did not involve something crazy like the Restrict Act. I still think the Restrict Act is worse than nothing, if those are our only choices.

If the CCP limited its interference to explicitly internal Chinese topics, I would understand, but they do not: WSJ investigates the TikTok rabbit hole, in particular with respect to Gaza pro-Hamas content.

Noah Smith: At this point, whether America can bring itself to ban TikTok will determine whether it’s an actual country, or just a country-shaped sandbox for totalitarian states to play in

An analysis of Chinese censorship of American movies. Under their analysis, without such bans we would have 68% of the Chinese market instead of our current 28%. They emphasize factors like occult content, which has an effect but a remarkably small one, only raising an otherwise 50% to be banned movie to a 67% chance to be banned. An R rating similarly takes the odds to 70%, likely largely as a proxy for various things that get you the R rating.

I love buttons that do things. The thing I loved most about early iPhones was that they had a button. A nice, big, physical button, that bailed you out of pretty much anything. Things were simple. Alas.

Matt Palmer: Observation from younger brother: “Whenever I have to adjust the settings on my iPhone I have to Google how to do so, this seems like a red flag.”

Patrick McKenzie: No lie, I had to ask my wife how to turn my iPhone off, now that I have one that doesn’t have a physical home button.

“Isn’t it basically same as it is on an iPhone with a home button?” The thing which stopped was that you need to long press two things but Siri triggers immediately when on button(s) down and I would immediately release them thinking “No I didn’t want Siri.”

And almost every interaction with Settings or any part of the Apple ecosystem is brokered by a Google search leading to Apple dot com or a conten

...
This might be the last AI Safety Camp
by Remmelt — 2024-01-24T09:33:29.438Z

Update Dec 4: Funds still needed for next month's stipends, plus salaries to run the 11th edition. Also, Zvi listed AISC at the top of his recommendations for talent funnel orgs.

We are organising the 9th edition without funds. We have no personal runway left to do this again. We will not run the 10th edition without funding.

In a nutshell:

Last month, we put out AI Safety Camp’s

funding case

.
A private donor then decided to donate €5K.

Five more donors offered $7K on

Manifund

.
For that $7K to not be wiped out and returned, another $21K in funding is needed. At that level, we may be able to run a minimal version of AI Safety Camp next year, where we get research leads started in the first 2.5 months, and leave the rest to them.

The current edition is off to a productive start!
A total of 130 participants joined, spread over 26 projects. The

projects

are diverse – from agent foundations, to mechanistic interpretability, to copyright litigation.

Our personal runways are running out.
If we do not get the funding, we have to move on. It’s hard to start a program again once organisers move on, so this likely means the end of AI Safety Camp.

We commissioned Arb Research to do an

impact assessment

.
One preliminary result is that AISC creates one new AI safety researcher per around $12k-$30k USD of funding.

How can you support us:

Spread the word. When we tell people AISC doesn't have any money, most people are surprised. If more people knew of our situation, we believe we would get the donations we need.
Donate. Make a donation through

Manifund

to help us reach the $28K threshold.
Reach out to remmelt@aisafety.camp for other donation options.
Humans aren't fleeb.
by Charlie Steiner — 2024-01-24T05:31:46.929Z

In the oceans of the planet Water, a species of intelligent squid-like aliens - we'll just call them the People - debate about what it means to be fleeb.

Fleeb is a property of great interest to the People, or at least they think so, but they also have a lot of trouble defining it. They're fleeb when they're awake, but less fleeb or maybe not fleeb at all when they're asleep. Some animals that act clever are probably somewhat fleeb, and other animals that are stupid and predictable probably aren't fleeb.

But fleeb isn't just problem-solving ability, because philosophers of the People have written of hypothetical alien lifeforms that could be good at solving problems without intuitively being fleeb. Instead, the idea of "fleeb" is more related to how much a Person can see a reflection of their own thinking in the processes of the subject. A look-up table definitely isn't fleeb. But how much of the thinking of the People do you need to copy to be more fleeb than their pet cuttlefish-aliens?

Do you need to store and recall memories?
Do you need emotions?
Do you need to make choices?
Do you need to reflect on yourself?
Do you need to be able to communicate, maybe not with words, but modeling other creatures around you as having models of the world and choosing actions to honestly inform them?

Yes to all of these, say the People. These are important things to them about their thinking, and so important for being fleeb.

In fact, the People go even farther. A simple abacus can store memories if "memories" just means any record of the past. But to be fleeb, you should store and recall memories more in the sense that People do it. Similar for having emotions, making choices, etc. So the People have some more intuitions about what makes a creature fleeb:

You should store and recall visual/aural/olfactory/electrosensory memories in a way suitable for remembering them both from similar sensory information and abstract reasoning, and these memories should be bundled with metadata like time and emotional valence. Your tactile/kinesthetic memories should be opaque to abstract reasoning (perhaps distributed in your limbs, as in the People), but can be recalled-in-the-felt-way from similar sensory information.
It's hard to tell if you have emotions unless you have ones recognizable and important to the People. For the lowest levels of fleeb, it's enough to have a general positive emoti

...
A Paradigm Shift in Sustainability
by Jose Miguel Cruz y Celis — 2024-01-23T23:34:44.121Z

This post is an effort to present in an accessible way, the paradigm shift happening in our model of sustainability. The current paradigm mainly centers around the goal of achieving a steady state where humanity can reach homeostasis with the environment for an indefinite amount of time. The competing modern paradigm is based on a clearer theory of value from a more developed global ethics perspective and proposes instead a dynamic state where humanity should enter into a sustainable trajectory where the goal is its existential risk minimization.

The post is divided into 8 sections. In the first one, Introduction to paradigm shifts we will talk about the notion of what a paradigm entails, how it gets created and how paradigm shifts happen.

We would then have an overview of the history of sustainability, from its pre-paradigm phase up to its crisis state. These phase sections will go deeper into analyzing each of the phases in a paradigm shift according to Thomas S. Kuhn in relation to the sustainability paradigm.

We will finish with a conclusion of the current state of the paradigm shift happening in sustainability.

Introduction to paradigm shifts
Historical Overview of Sustainability
Phase 1 — The pre-paradigm phase
Phase 2 — Normal science and anomalies
Phase 3 — Crisis
Phase 4 — Paradigm shift
Phase 5 — Post-revolution
Conclusion

1. Introduction to paradigm shifts

The great science philosopher Thomas S. Kuhn in his magnus opus, The Structure of Scientific Revolutions, expounded the way science progresses. Before Kuhn, there was a conception that science develops by the addition of new truths to the already existing truths, by theories that represent the truth more closely or by the correction of previous errors, Kuhn's thesis said otherwise.

According to Kuhn progress in science begins with a pre-paradigm period involving an immature science. This period is characterized by a lack of consensus, where competing schools of thought possess differing procedures, theories and even metaphysical priors. This makes collective progress difficult. Even local progress is difficult since a lot of the mental energy is used for arguing over fundamentals. When one school makes a breakthrough which solves for the shared problems, consensus is formed around it. This consensus allows agreement over fundamentals, solutions to the previous puzzles and new puzzles that it poses. This ...
From Finite Factors to Bayes Nets
by J Bostock — 2024-01-23T20:03:51.845Z

I was hoping I could produce a one-to-one mapping between diagrams of overlapping factors (from FFS) and Bayes nets. This is not possible, but instead I have been able to produce a similar mapping between factor overlap diagrams and sets of Bayes nets, subject to certain compatibility relationships.

Bayes Nets

The Bayes nets over some variables X can be thought of as a category of objects related by "bookkeeping" (and similar) relationships. To be more specific, we can say that a unique morphism exists between two Bayes nets Ni→Nj exists if and only if, for all probability distributions P over the elements of Ni, the approximation of Nj over P is no worse than the approximation of Ni over P. For two variables we need the Markov-chain-re-rooting and arrow-adding rules:

We could (with some effort) do three variables, but since there are three options for each pair of variables (A→B, A←B, no arrow) the number of Bayes nets over n variables is 312n(n−1). For n=3 this is 27. Let's draw in the adding-an-arrow and Markov-chain-re-rooting relationships:

As you can see it's a total goddamn mess! Even when we exclude the forbidden Bayes nets, ignore the identity morphisms (from an object to itself), use illicit double-headed arrows on the bottom layer, and also use "implied" arrows (i.e. don't bother with A→Cif we have A→B→C) we get a very confusing scenario!

To this problem, we will add another problem! But a problem which will lead us to a solution.

From Bayes Nets to Equivalence Classes

Say we have a mapping from a Finite Factored Set ontology to a Bayes net. This means we can apply a probability distribution over our factors to get a probability distribution over our downstream variables, with the condition that this must satisfy the Bayes net. The problem here is that many different Bayes nets will be satisfied by the probability distribution.

What we want is a one-to-one relationship (ideally). To improve this, we can start by grouping Bayes nets together under an equivalence relation. This is some relationship which has the following properties:

Reflexive: a∼a
Symmetric: a∼b⟺b∼a
Transitive: a∼b, b∼c⟹a∼c

We can define an equivalence relation over the category of Bayes nets over X trivially: Xi∼Xj⟺Xi→Xj∧Xj→Xi. For simpler Bayes nets...
Institutional economics through the lens of scale-free regulative development, morphogenesis, and cognitive science
by Roman Leventov — 2024-01-23T19:42:31.739Z

Cross-posted from my Substack.

In this article, I highlight some of the ideas from

Musthtaq Khan’s interview for 80000hours podcast about institutional economics

[1], political economy, his “political settlement” framework, and the methodology of economics, and connect these ideas to the concepts in scale-free regulative development[2], morphogenesis[3], and cognitive science[4][5].

The history of the socioeconomy matters

Khan points to the fact that socioeconomies are non-ergodic dynamical systems with hysteresis, i.e., memories of their own[1]:

To say that all structures are created by individuals, and therefore if the structure of society in India is different from the one in the United States, then we have to look at the individual incentives that created those structures, I think is a non-starter. It confuses the path dependence of history and the complexity of how structures are built up. Individuals today in India may not have any capacity of changing that structure to look like the one in the U.S. or Norway, not because they have some information deficit or anything like that, but because a structure itself has a reality and a meaning which affects the way individuals behave.

Here’s the exactly parallel observation that Levin[3] makes about morphogenesis:

Development is thus incredibly reliable, producing bodies to very tight tolerance despite considerable deviations and noise at the level of gene expression and cellular activity (Gonze et al. 2018; Eritano et al. 2020; Simon, Hadjantonakis, and Schroter 2018). This robustness, and its occasional failure in the case of birth defects immediately suggests teleonomic perspectives because only goal-directed agents can make mistakes; biophysics alone cannot make mistakes – every micro-scale process proceeds according to the laws of physics and chemistry. Developmental defects are mistakes relative to the correct outcome toward which they strive.

This means that from the perspective of scale-free teleonomy, socioeconomies must be “goal-directed agents that can make mistakes”. The causal (or “creation”) link between individual behaviour and institutions (socioeconomic structures) is bidirectional rather than unidirectional from individual behaviour to institutions.

Khan’s position is that viewing economy exclusively through the lens of individual choice (behaviour) and emergence of supply, demand, and market pr...
Making a Secular Solstice Songbook
by jefftk — 2024-01-23T19:40:05.055Z

After

this
year's secular solstice several people were saying they'd be
interested in getting together to sing some of these songs
casually. This is a big part of what we sang at the

post-EAG
music party, but one issue was logistical: how do you get everyone on
the same words and chords?

I have slides (2023,
2022,
2019,
2018)
with the chords and lyrics to the songs we've done at the past few
events, but they have some issues:

They were intended only for my use, so they're a bit hard to
make sense of.

The text is too small for phones.

They horizontally oriented, when for a phone you want something vertical.

There's no index.

Google docs is slow on phones.

Another option is Daniel Speyer's list
from his secular solstice resources, but this includes a lot of songs
we've never done in Boston and doesn't have the chords easily
accessible.

Instead I put together a web page: jefftk.com/solsong. It's
intentionally one long page, trying to mimic the experience of a
paper songbook where you can flip through looking for
interesting things. [1] I went through the sides copying lyrics over,
and then added a few other songs I like from earlier years.

I've planned a singing party for Saturday 2024-02-17, 7pm at our house
(fb).
Let me know if you'd like to come!

[1] At a technical level the page is just HTML, as is my authoring
preference. Since line breaks aren't significant in HTML but are in
lyrics, I used a little command line trick in copying them over:

pbpaste | sed 's/$/<br>/' | pbcopy

I would copy the lyrics, run that command to transform my clipboard,
and then paste into the editor.

EDIT: on Daniel's suggestion I've switched to white-space:
pre-line;, which means I don't need this. It's a little fussy
needing to think about my line breaks in what looks like plain html,
but not too bad.

To include an index without needing to duplicate
titles I have a little progressive-enhancement JS:

<ul id=toc></ul>
...
<script>
for (const h2 of document.getElementsByTagName("h2")) {
const li = document.createElement("li");
const a = document.createElement("a");
a.innerText = h2.innerText;
a.href = "#";
a.onclick = function() {
h2.scrollIntoView(/*alignToTop=*/true);
return false;
};
li.appendChild(a);
toc.appendChild(li);
}
</script>

Comment via: facebook, lesswrong,...
Simple Appreciations
by Jonathan Moregård — 2024-01-23T16:23:52.001Z
Link: https://open.substack.com/pub/honestliving/p/simple-appreciations-strategy-for-happiness-and-

For some time now, I have expressed appreciation straight-forwardly. Simple Appreciations make people light up with happy smiles; it is one of my favourite communication habits.

After picking up Simple Appreciations, I have started getting invited to more contexts. People tell me that I brighten their days and help them appreciate life. They want to spend more time with me. I feel more appreciation and happiness, leading to increased productivity and general well-being.

In this post, I will tell you about Simple Appreciations, go through why I believe they are so effective, and end with the practical steps I’ve taken to incorporate them into my life.

I hope that this post will help you live a happy life with friends who

support you in growing as a person

.

What is Simple Appreciations?

The core of the idea is simple: whenever I experience a positive emotion directed at someone, I express appreciation as simply and directly as possible.

Here are some examples, all said in a heartfelt, emotionally connected way: “I’m happy you are here”, “I start smiling when I see you, I like having you around”, “I like you”, “I like talking to you”.

The Benefits of Simple Appreciations

Expressing positive feelings in simple ways has a lot of benefits! Let me run through the benefits I’ve experienced!

Cognitive Behavioural Therapy & Gratitude Journalling

The basis of cognitive behavioural therapy is the idea that your actions, emotions and thoughts all affect each other. By focusing on the things you are grateful for, you will feel happier, leading to an increase in well-being. A

gratitude journal

is one way to focus more on things you are grateful for. I’ll talk about my take on gratitude journalling later in this post!

I haven’t found any research, but I’m confident that Simple Appreciations increase well-being in the same way gratitude journalling does. Similar to gratitude journalling, Simple Appreciations highlights the things you are grateful for. Feeling grateful is a good first step towards feeling great.

Simple Appreciations have a key benefit compared to traditional gratitude journaling— they are social.

Expressing appreciation helps the other person appreciate your mutual connection, contributing to their level of well-being as well as yours. Besides affecting the other person in isolation, your relationship is also strengthened. Mutually appreciating a con...