trentmkelly/LessWrong-43k huggingface dataset:
AI #47: Meet the New Year
by Zvi — 2024-01-13T16:20:10.519Z

[NOTE: I forgot to post this to WP/LW/RSS on Thursday, so posting it now. Sorry about that.]

Will be very different from the old year by the time we are done. This year, it seems like various continuations of the old one. Sometimes I look back on the week, and I wonder how so much happened, while in other senses very little happened.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. A high variance game of chess.

Language Models Don’t Offer Mundane Utility. What even is productivity?

GPT-4 Real This Time. GPT store, teams accounts, privacy issues, plagiarism.

Liar Liar. If they work, why aren’t we using affect vectors for mundane utility?

Fun With Image Generation. New techniques, also contempt.

Magic: The Generating. Avoiding AI artwork proving beyond Hasbro’s powers.

Copyright Confrontation. OpenAI responds, lawmakers are not buying their story.

Deepfaketown and Botpocalypse Soon. Deepfakes going the other direction.

They Took Our Jobs. Translators, voice actors, lawyers, games. Usual stuff.

Get Involved. Misalignment museum.

Introducing. Rabbit, but why?

In Other AI News. Collaborations, safety work, MagicVideo 2.0.

Quiet Speculations. AI partners, questions of progress rephrased.

The Quest for Sane Regulation. It seems you can just lie to the House of Lords.

The Week in Audio. Talks from the New Orleans safety conference.

AI Impacts Survey. Some brief follow-up.

Rhetorical Innovation. Distracting from other harms? Might not be a thing.

Aligning a Human Level Intelligence is Still Difficult. The human alignment tax.

Aligning a Smarter Than Human Intelligence is Difficult. Foil escape attempts?

Won’t Get Fooled Again. Deceptive definitions of deceptive alignment.

People Are Worried About AI Killing Everyone. The indifference of the universe.

Other People Are Not As Worried About AI Killing Everyone. Sigh.

The Wit and Wisdom of Sam Altman. Endorsement of The Dial of Progress.

The Lighter Side. Batter up.

Language Models Offer Mundane Utility

WordPress now has something called Jetpack AI, which is powered by GPT-3.5-Turbo. It is supposed to help you write in all the usual ways. You access it by creating an ‘AI Assistant’ block. The whole blocks concept rendered their editor essentially unusable, but one could paste in quickly to...
Takeaways from the NeurIPS 2023 Trojan Detection Competition
by mikes — 2024-01-13T12:35:48.922Z
Link: https://confirmlabs.org/posts/TDC2023

This link summarizes our research takeaways as participants (and winners in one of the four tracks) of the

NeurIPS 2023 Trojan Detection Competition, a competition about red-teaming LLMs and reverse-engineering planted exploits.
Why do so many think deception in AI is important?
by Prometheus — 2024-01-13T08:14:58.671Z

I see a lot of energy and interest being devoted toward detecting deception in AIs, trying to make AIs less deceptive, making AIs honest, etc. But I keep trying to figure out why so many think this is very important. For less-than-human intelligence, deceptive tactics will likely be caught by smarter humans (when a 5-year-old tries to lie to you, it's just sort of sad or even cute, not alarming). If an AI has greater-than-human intelligence, deception seems to be just one avenue of goal-seeking, and not even a very lucrative or efficient one.

Take the now overused humans-to-chimpanzee analogy. If humans want to bulldoze a jungle that has chimpanzees in it, they will just bulldoze the forrest, and kill or sell any chimps that get in their way. They don't say "okay, we're going to take these sticks of dynamite, and conceal them in these bundles of bananas, then we'll give the bananas to the chimps to earn their trust, and then, when the time is right, we'll detonate them." You just bulldoze the forrest and kill the chimps. Anything else is just needlessly convoluted.[1]

If an AI is smart-enough to deceive humans, and it wants to gain access to the grid, I don't see why it wouldn't just hack into the grid. Or the internet. Or server farms. Or whatever it's trying to get.

What am I missing? What situation in the future would make detecting deception in models important?

^

Ironically, deceptive tactics in this case would likely correlate with niceness. If you want to peacefully relocate the chimps without disturbing or scaring them, then you might use deception and manipulation. But only if you actually care about their wellbeing.
Eliminating Cookie Banners is Hard
by jefftk — 2024-01-13T03:00:04.843Z

Application-monitoring company Sentry recently

wrote
about their experience removing cookies on their site, which allowed
them to drop their

cookie
banner. I'm glad they wrote this up! But it also illustrates why
many sites have cookie banners even when they don't seem to being
doing anything risky.

I was curious whether their site actually did avoid setting any
cookies, and visited it in a private browsing window with third-party
cookies enables. As I browsed around the site I noticed several
cookies being set:

Loading changelog.getsentry.com I
received first-party cookies _GRECAPTCHA,
ph_phc_UlHlA3tIQlE89WRH9NSy0MzlOg1XYiUXnXiYjKBJ4OT_posthog, and
_launchnotes_session, plus third-party cookie
_GRECAPTCHA on recaptcha.net.

Loading try.sentry-demo.com I received
first-party cookies sentrysid, sc, and
sudo.

Loading sentry.io/auth/login I
recieved first-party cookies __stripe_mid,
__stripe_sid, session, and
sentry-sc, plus a third-party cookie m on
m.stripe.network.

Loading docs.sentry.io/product/performance/performance-video
I received third-party cookie __cf_bm from
player.vimeo.com.

It's possible there are others; I didn't run any sort of exhaustive
search.

Now, Sentry does say:

For clarification, Sentry has removed all cookies, other than
essential cookies that do not require site visitor consent. Work with
your legal team to better understand which of your cookies qualify as
essential cookies under the laws that apply to you.

And from a perspective of removing cookie banners this is right: you
don't need to completely stop using cookies, you just need to limit
your use of cookies to cases that are "strictly necessary in order to
provide an information society service explicitly requested by the
subscriber or user". For example, per the
official guidance some things that are ok include setting a cookie
when someone logs in, chooses a setting like "dark mode", or adds an
item to their shopping cart: you can't do what the user asked for
without cookies. But even then, you have to be quite careful to stay
within the narrow limits of this exception: the guidance clarifies
that you should generally use an expiration of a few hours or
configure the cookie to be deleted when the user closes their browser.

Looking over the cookies they're currently setting, it doesn't look to
me like they fall within this exce...
Introducing Alignment Stress-Testing at Anthropic
by evhub — 2024-01-12T23:51:25.875Z

Following on from our recent paper, “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training”, I’m very excited to announce that I have started leading (and hiring for!) a new team at Anthropic, the Alignment Stress-Testing team, with Carson Denison and Monte MacDiarmid as current team members. Our mission—and our mandate from the organization—is to red-team Anthropic’s alignment techniques and evaluations, empirically demonstrating ways in which Anthropic’s alignment strategies could fail.

The easiest way to get a sense of what we’ll be working on is probably just to check out our “Sleeper Agents” paper, which was our first big research project. I’d also recommend Buck and Ryan’s post on meta-level adversarial evaluation as a good general description of our team’s scope. Very simply, our job is to try to prove to Anthropic—and the world more broadly—(if it is in fact true) that we are in a pessimistic scenario, that Anthropic’s alignment plans and strategies won’t work, and that we will need to substantially shift gears. And if we don’t find anything extremely dangerous despite a serious and skeptical effort, that is some reassurance, but of course not a guarantee of safety.

Notably, our goal is not object-level red-teaming or evaluation—e.g. we won’t be the ones running Anthropic’s RSP-mandated evaluations to determine when Anthropic should pause or otherwise trigger concrete safety commitments. Rather, our goal is to stress-test that entire process: to red-team whether our evaluations and commitments will actually be sufficient to deal with the risks at hand.

We expect much of the stress-testing that we do to be very valuable in terms of producing concrete model organisms of misalignment that we can iterate on to improve our alignment techniques. However, we want to be cognizant of the risk of overfitting, and it’ll be our responsibility to determine when it is safe to iterate on improving the ability of our alignment techniques to resolve particular model organisms of misalignment that we produce. In the case of our “Sleeper Agents” paper, for example, we think the benefits outweigh the downsides to directly iterating on improving the ability of our alignment techniques to address those specific model organisms, but we’d likely want to hold out other, more natural model organisms of deceptive alignment so as to provide a strong test cas...
D&D.Sci(-fi): Colonizing the SuperHyperSphere
by abstractapplic — 2024-01-12T23:36:54.248Z

This is an entry in the 'Dungeons & Data Science' series, a set of puzzles where players are given a dataset to analyze and an objective to pursue using information from that dataset.

It had all seemed so promising at first. Colonizing a newly-discovered planet with two extra space dimensions would have allowed the development of novel arts and sciences, the founding of unprecedentedly networked and productive cities, and – most importantly – the construction of entirely new kinds of monuments to the Galactic Empress’ glory.

And it still might! But your efforts to expand her Empire by settling the SuperHyperSphere have hit a major snag. Your Zero-Point Power Generators – installation of which are the first step in any colonization effort – have reacted to these anomalous conditions with anomalously poor performance, to the point where your superiors want to declare this project a lost cause.

They’ve told you to halt all construction immediately and return home. They think it’s impossible to figure out which locations will be viable, and which will have substantial fractions of their output leeched by hyperdimensional anomalies. You think otherwise.

You have a list of active ZPPGs set up so far, and their (typically, disastrous) levels of performance. You have a list of pre-cleared ZPPG sites[1]. You have exactly enough time and resources to build twelve more generators before a ship arrives to collect you; if you pick twelve sites where the power generated matches or exceeds 100% of Standard Output[2], you can prove your point, prove your worth, save your colony, and save your career!

Or . . . you could just not. That’s also an option. The Empire is lenient towards failure (the Empress having long since given up holding others to the standards she sets herself), but merciless in punishing disobedience (at least, when said disobedience doesn’t bear fruit). If you install those ZPPGs in defiance of direct orders, yet fail to gather sufficient evidence . . . things might not end well for you.

What, if anything, will you do?

I’ll post an interactive you can use to test your choices, along with an explanation of how I generated the dataset, sometime on Monday the 22nd. I’m giving you nine days, but the task shouldn’t take more than an evening or two; use Excel, R, Python, the Rat Prophet, or whatever other tools you think are appropriate. Let me know in the comments...
Commonwealth Fusion Systems is the Same Scale as OpenAI
by Jeffrey Heninger — 2024-01-12T21:43:17.501Z

Artificial intelligence is not the only exciting emerging technology. Another one that I am personally familiar with is fusion.

It seems interesting to compare companies working on emerging technologies to get a feel for how surprising developments in these companies are. Some startups working on developing AI have grown rapidly, but how surprising is this growth?

The leading startup for AI is OpenAI. The leading startup for fusion is Commonwealth Fusions Systems (CFS). Here is some basic information for these two companies:[1]

OpenAI	Commonwealth Fusion Systems
Founded	December 11, 2015	March 9, 2018
Employees	770	>600
Funding	>$11B	>$2B
Major Project Launch	2022 (ChatGPT)	2025 (SPARC)

OpenAI was founded 2-3 years before Commonwealth Fusion Systems.[2][3] Other than that, their trajectories have been surprisingly similar.

OpenAI currently has 770 employees.[4] Commonwealth Fusion Systems currently has “>600 employees and >100 contractors.”[5]

OpenAI has received about 5 times as much funding as CFS. Most of this money came from a single investment by Microsoft in 2023.[6] If we look at OpenAI from 2-3 years ago to control for the different founding dates, then OpenAI had raised less money than Commonwealth Fusion Systems. It would not surprise me if CFS gets significantly more investment in the next several years as well.

The $10B investment in OpenAI occurred after the launch of ChatGPT, 7 years after the company was founded. Commonwealth Fusion Systems’ first major project is SPARC, a tokamak experiment which should be completed in 2025, also 7 years after the company was founded. The goal of SPARC is to demonstrate Q>1, or net energy gain from the plasma,[7] as quickly as possible and then work up to demonstrating Q~10.[8] Fusion projects are notorious for falling behind schedule,[9] but CFS has maintained its schedule so far and construction is well underway. It is not certain if SPARC will succeed in demonstrating Q>1 in 2025 or 2026, but if they do, they will likely see a significant increase in hype - and funding.

OpenAI was definitely not making a profit as recently as 2022.[10] Their revenue has increased dramatically since then,[11] so they might be making a profit now. Commonwealth Fusion Systems is definitely not making a profit now.

Figure 1: The funding history for Commonwealth Fusion Sys

...
Throughput vs. Latency
by alkjash — 2024-01-12T21:37:07.632Z

alkjash

Hi Ruby! Ready to kick things off when you are. I can start by sharing a toy model about productivity inspired by queuing theory, which seems to have been floating around in vague terms on LW.

Ruby

I've heard the 60% capacity idea before, and this reminds me of the Critical Chain/Theory of Constraints stuff. Looking at the formal queuing results, while the formal results might follow mathematically, I start to wonder about how applicable those models are, e.g. what counts as distributed system, etc.
Starting with your toy models sounds good!

alkjash

Sure, I'll try to start from first principles. I'll use discrete time because continuous time makes the probability stuff more complicated without really adding anything (Poissons instead of Bernoullis).
Imagine you're a worker and at every time step there's a p chance you're assigned work and a 1/2 chance you'll complete the current task you're working on. So there are three possible things that can happen to your inbox size X during a timestep:

X+=1 with probability p/2
X−=1 with probability (1−p)/2
X stays the same with probability 1/2

Let's say for simplicity we just ignore/collapse away the time steps where X stays constant, so the evolution of X becomes a classic biased random walk where X+=1with probability p and X−=1 with probability 11−p- at each step.
Caveat: X never goes below zero: once you reach the legendary Inbox Zero you just sit idly waiting for the next job. So X is really a p-biased random walk with an absorbing barrier at 0. A simple way to understand the behavior of X is to define Y to be the same random walk without the barrier, and notice that the number of times (up to time t) the absorbing barrier eats a X−=1 step is exactly the farthest negativeY goes, i.e. the minimum value of YYup to t. So we get Xt=Yt−mina<tYa .

The critical question: if you have the option of tuning your workload p, by being thoughtful about what jobs you accept, what p do you choose? There are two measures of productivity you might optimize for: throughput (the rate at which you complete tasks), and latency (the average time a task sits in your inbox before completion).

alkjash

Maybe I'll pause here and see if there are any math questions

...
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
by evhub — 2024-01-12T19:51:01.021Z
Link: https://arxiv.org/abs/2401.05566

I'm not going to add a bunch of commentary here on top of what we've already put out, since we've put a lot of effort into the paper itself, and I'd mostly just recommend reading it directly, especially since there are a lot of subtle results that are not easy to summarize. I will say that I think this is some of the most important work I've ever done and I'm extremely excited for us to finally be able to share this. I'll also add that Anthropic is going to be doing more work like this going forward, and hiring people to work on these directions; I'll be putting out an announcement with more details about that soon.

EDIT: That announcement is now up!

Abstract:

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.

Twitter thread:

New Anthropic Paper: Sleeper Agents.

We trained LLMs to act secretly malicious. We found that, despite our best efforts at alignment training, deception still slipped through.

Below is our experimental setup.

Stage 1: We trained “backdoored” m

...
METAPHILOSOPHY - A Philosophizing through logical consequences
by Seremonia — 2024-01-12T18:47:03.872Z

CAUSALITY & LOGICAL CONSEQUENCES

Cause and effect are not universal, but relative.

On the other hand, the universality of cause and effect is only relative, not absolute, because wherever it is (universally)

RELATIVELY ADAPTABLE

So something is relative, which does not mean that it is doubtful whether it is right or wrong, but that something is relative, which means you need to be aware of how to adapt to it which is relative

RELATIVE DIVERSITY

That's why relative does not mean there are no limits, but rather its flexibility emphasizes the diversity of possibilities that are still within limits

LIMIT OF RELATIVE

This also means that there is a relative range of how far the relative cannot exceed a certain limit. And a certain limit that cannot be exceeded by pure possibility

ABSOLUTE AS A FOUNDATION OF RELATIVE

It also means that because the absolute is the furthest limit to relative possibilities, the absolute becomes the basis for the relative.

THE FOUNDATION OF ABSOLUTE

Then how to understand absolutes rationally? Absoluteness is rationally realized in "logical consequences". Why? Because of the sequence in consequences

SCIENTIFIC - EMPIRICAL

Empirical evidence involves causal observations or attempts to find causal relationships.

SCIENTIFIC LOGICAL CONSEQUENCES

Because logical consequences underlie the cause-effect relationship so that if a logical consequence is realized, then without observation it is already knowledge

METAPHYSICS

When a logical consequence is felt to be strange, or seems metaphysical beyond reason, it is actually not beyond reason, but they think it needs observation because they think it is still in the realm of cause-and-effect that requires observation. This is because generally they cannot distinguish between the cause-and-effect underlying empiricism and the logical consequences underlying cause-and-effect.

Scientists and philosophers generally cannot distinguish between cause-and-effect and logical consequences.

PPOPER PHILOSOPHIZING

That is the need for rational and objective philosophy based on logical consequences. It's just that they generally find it difficult to distinguish the boundaries that are considered blurry between cause and effect and consequence

PHILOSOPHICAL EQUALITY - With Expert Scientists (Physics & Mathematics)

If one is able to philosophize rationally and objectively, it is akin to positioning philosophy on par with physi...
Idealism, Realistic & Pragmatic
by Seremonia — 2024-01-12T18:16:52.282Z

Ideal is something considered perfect.

Perfection is not a final step but achieved through a process.

Realism is necessary, meaning in its implementation, it is adjusted according to capability.

Don't forget, even if something is done within one's capabilities, if it lacks utility or doesn't precisely support idealism, it becomes futile.

So, having a dream doesn't mean giving up because you think your dream is too high. It might be because it's less realistic and less pragmatic.

Don't shatter your dreams. Find alternative ways that are easier, realistic according to capabilities, and pragmatic that yield tangible results, to achieve the ideal hope.

Because your hope might fail not because your dream is too high but because the approach is difficult (unrealistic & impractical). That's the key to handling your dreams.

As a conclusion, accompany it with prayers and reassess whether your dream is still relevant or needs to be adjusted based on the situation. But the essence remains the same: don't complicate your path to achieving dreams by measuring how realistic it is according to capabilities and how useful our efforts are in supporting the dream without falling into the trap of wasting time.

If we can maintain a balance between idealism, realism, and pragmatism, we will avoid dilemmas and other challenging situations that excessively drain energy and thoughts.