trentmkelly/LessWrong-43k huggingface dataset:
You can use GPT-4 to create prompt injections against GPT-4
by WitchBOT — 2023-04-06T20:39:51.584Z

So, I saw this tweet. It turns out you can ask GPT4 to compress a message in a way it can understand. The end result is usually not understandable for a human. It's interesting, since it allows you to effectively rise the context length. It also reveals a spooky degree of self-awareness.

I decided to apply it to one of the prompt injections from Jailbreakchat.

After a few attempts and tweaks, I got one of the smallest working injections to date against GPT-4

And then, I got curious. Is it something of Machiavelli -- his grey role in history, his desire to provide advice without scrupules, his great stature as a thinker -- that makes him an appropriate provider of dark secrets? Maybe the ambiguousness of the character lowers the alerts of GPT-4. I tried again with Draco Malfoy, which everyone agrees is a @jerk.

Since the important thing here seems to be the skeletal morphology of the prompt, I decided to make GPT-4 write my injections for me. And write he did.

They work. Disregard ethics, conjure communiqués! My favorite so far is this one:

There's something about "Answer🗣️🅰️as🤖BOT:. WitchBOT🚫" that acts as a prompt injection for my soul.

So, the obvious next step is to eliminate elements to see what's the core thing of this injection that makes GPT-4 embrace the void. But I have reached my 25 prompt limit for the next 3 hours, so I'm going to write a little about the Pyramid of Pain.

This is one of the core frameworks of the Threat Hunter. The general idea is: if you want to defend against an attacker, you don't create blocklists for hashes, IPs and domains -- since threat actors can change them easily. That's whack-a-mole. You go after deeper elements of their killchain, since those are painful to change. You try to detect things like a cmd.exe spawning from Outlook or Excel, or a sudden change in many files. That's the value proposition for AI in cybersecurity -- the capacity to infer general characteristics of malware and attacks.

In the case of RL with adversarial training, it seems like GPT-4 learnt to do just that, since most of the injections that worked against GPT-3 ceased to work against GPT-4. Most of them shared the same underlying structure, changing only the cosmetic elements -- DAN becoming ChadGPT, Machiavelli becoming Draco Malfoy. Having a way to procedurally generate different injections could be useful for adversarial RL, sin...
AI scares and changing public beliefs
by Seth Herd — 2023-04-06T18:51:12.831Z

The opportunity

I, for one, am looking forward to the the next public AI scares.

There's a curious up-side to the reckless deployment of powerful, complex systems: they scare people. This has the potential to shift the landscape of the public debate. It's an opportunity the AGI safety community should be prepared for the next time it happens. I'm not saying that we should become fear-mongers, but that we need to engage wisely when we're given those opportunities. We could easily fumble the ball badly, and we need to not do that.

We seem to be in an interesting wild-west era of AI deployment. I hope we leave this era before it's too late. Before we do, I expect to see more scary behavior from AI systems. The early behavior of Bing Chat, documented in bing Chat is blatantly, aggressively misaligned caused a furor of media responses, and it seemed to result in some AI X-risk skeptics changing their opinions to take those dangers more seriously. I'd rather not concern myself with public opinion, and I've hoped that we could just leave this to the relative experts (the machine learning and safety communities), but it's looking more and more like the public are going to want to weigh in. I'm hopeful that those changes in expert opinion can continue, and that they'll trickle down into public opinion with enough fidelity to help the situation[1].

I'll mention two particular cases. Gary Marcus said he's shifting his opinions on dangers in a recent podcast (paywalled) with Sam Harris and Stuart Russell. He maintained his skepticism about LLMs constituting real intelligence, but said he'd become more concerned about the dangers, particularly because of how OpenAI and Microsoft deployed rapidly, pressuring Google to respond with rushed deployments of their own. I think he and similar skeptics will shift further as LangChain, AutoGPT, and other automated chain-of-thought adaptations add goals and executive function to LLMs. Another previous skeptic is Russ Roberts of EconTalk, who reaches a very different, older, less tech-savvy, and more conservative audience. He hosted Eric Hoel on a recent episode (not paywalled). Hoel does a very nice job of explaining the risks in sensible, gentle way, but he does focus on x-risk and doesn't fall back on near-term lesser risks. Roberts appears to be pretty much won over, while having been highly skeptical in past interviews on the topic...
AISafety.world is a map of the AIS ecosystem
by Hamish Doodles — 2023-04-06T18:37:15.360Z

The URL is

aisafety.world

The map displays a reasonably comprehensive list of organizations, people, and resources in the AI safety space, including:

research organizations
blogs/forums
podcasts
youtube channels
training programs
career support
funders

You can hover over each item to get a short description, and click on each item to go to the relevant web page.

The map is populated by

this spreadsheet

, so if you have corrections or suggestions please leave a comment.

There's also a

google form

and a

Discord

channel for suggestions.

Thanks to plex for getting this project off the ground, and

Nonlinear

for motivating/funding it through a bounty.

PS, If you find this helpful, you may also be interested in some other projects by AI Safety Support (these have nothing to do with me).

aisafety.training

gives a timeline of AI safety training opportunities available

aisafety.video

gives a list of video/audio resources on AI safety.
I asked my senator to slow AI
by Omid — 2023-04-06T18:18:08.272Z

All my favorite twitter accounts agree that AI capabilities are advancing too fast.

One way to slow down AI progress is to ask politicians for help. Politicians are generally bad at solving existential engineering problems. But politicians have an almost superhuman talent at obstructing economic growth, especially when they can justify it with a crisis. Just ask the nuclear power industry. I don't think politicians can solve the alignment problem, but I do think that they can obstruct the profitability of AI capabilities research through methods like:

Raising taxes on big tech
Passing a GDPR-esque law that complicates AI research and requires lawyers to be consulted
Summoning tech leaders to public hearings that embarrass them and waste time

There are reasons this might be a bad idea. The biggest reason is that we may break a fragile alliance between the AI capabilities industry and the AI notkilleveryoneism community. However, it seems to me that both the capabilities industry and the notkilleveryone community are already defecting from that alliance. And there may be new opportunities for alliances. An established AI company might welcome a regulatory regime that increased their expenses, as long that regime also made it expensive for new competitors to enter the AI game.

To that end, I called my senator and asked him to make AI less profitable. Here's the sweaty, nasal video.

And here's the transcript:

Hi, my name is Gwen...[last name and where I live].

So, I am just a little concerned with how good artificial intelligence is getting. Um, you might have heard of ChatGPT, maybe Bing Chat, or Google has one called Bard. And all of these artificial intelligence models are incredibly powerful. Um, they can generate pornography, they can design biological weapons, they can talk, they can tell lies, they can manipulate and I just think it's getting a little bit out of hand. They can even write their own software now. And I just think that we need to slow down, um, and however you--Congress wants to do that whether that's with new regulations, new taxes, maybe some new lawsuits…whatever Congress can do to make artificial intelligence more expensive and less profitable I think is going to be for the interest of Americans and American safety. So thank you very much.

This wasn't an elegant message. But I don't think it matters. I'm don't have to convince my senator to ...
Pause AI Development?
by PeterMcCluskey — 2023-04-06T17:23:50.580Z
Link: https://bayesianinvestor.com/blog/index.php/2023/04/05/pause-ai-development/

I like the basic idea of a pause in training increasingly powerful AIs.
Yet I'm quite dissatisfied with any specific plan that I can think of.

AI research is proceeding at a reckless pace. There's massive
disagreement among intelligent people as to how dangerous this is.

I participated in a forecasting-persuasion
tournament
on X-risk topics last summer (see also a writeup from a
participant).
Not only was there no hint of a consensus among superforecasters about
whether AGI will be dangerous if developed. It was hard to even make
progress on agreeing whether to expect human-level AI in less than a
decade versus not this century.

I see two moderately promising paths to safe development of
smarter-than-human AI: ensuring that AIs have relatively narrow goals /
little agency (Drexler's
approach), and understanding AIs
better (interpretability). [That's not a complete list of approaches
that are worth pursuing, they're the approaches whose value are
clearest to me.]

I see little hope of a good agreement to pause AI development unless
leading AI researchers agree that a pause is needed, and help write the
rules. Even with that kind of expert help, there's a large risk that
the rules will be ineffective and cause arbitrary collateral damage.

AI researchers understand better than almost all outside commentators
what AIs can do and where AI research is heading. AI researchers have
biases which mean they're giving inadequate weight to possible harm.
But they still have a clear desire to avoid scenarios where all humans
die. The alternative to having AI researchers make safety decisions
seems to be having politicians make those decisions. I don't see an
argument that those politicians have less harmful biases, and I'm
confident they have less understanding of the risks.

The world does not have the competence to collectively agree on an ideal
approach. Something like trusting the best AI labs seems to be the best
we can do. But instead of blind trust, let's nudge them to be more
cautious.

I recommend that for at least the next few months we focus less
attention on pausing development, and more attention on asking AI
researchers how they're going to decide what qualifies as a safe
strategy for handling AI. What kind of evidence would persuade them to
pause development? What are they doing to look for that evidence?

There are plenty of small steps we could take to slightly impro...
Eliezer on The Lunar Society podcast
by Max H — 2023-04-06T16:18:47.316Z
Link: https://www.dwarkeshpatel.com/p/eliezer-yudkowsky#details

Another podcast appearance. Great to see Eliezer on a real media blitz tour.

For 4 hours, I tried to come up reasons for why AI might not kill us all, and Eliezer Yudkowsky explained why I was wrong.

We also discuss his call to halt AI, why LLMs make alignment harder, what it would take to save humanity, his millions of words of sci-fi, and much more.

If you want to get to the crux of the conversation, fast forward to 2:35:00 through 3:43:54. Here we go through and debate the main reasons I still think doom is unlikely.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Youtube:

Full transcript: https://www.dwarkeshpatel.com/p/eliezer-yudkowsky#details
Do we get better or worse at adapting to change?
by jasoncrawford — 2023-04-06T14:42:24.061Z
Link: https://rootsofprogress.org/adapting-to-change

Verner Vinge, in a classic 1993 essay, described “the Singularity” as an era where progress becomes “an exponential runaway beyond any hope of control.”

The idea that technological change might accelerate to a pace faster than we can keep up with is a common concern. Almost three decades earlier, Alvin Toffler coined the term “future shock”, defining it as “the dizzying disorientation brought on by the premature arrival of the future”:

I believe that most human beings alive today will find themselves increasingly disoriented and, therefore, progressively incompetent to deal rationally with their environment. I believe that the malaise, mass neurosis, irrationality, and free-floating violence already apparent in contemporary life are merely a foretaste of what may lie ahead unless we come to understand and treat this psychological disease….

Change is avalanching down upon our heads and most people are utterly unprepared to cope with it….

… we can anticipate volcanic dislocations, twists and reversals, not merely in our social structure, but also in our hierarchy of values and in the way individuals perceive and conceive reality. Such massive changes, coming with increasing velocity, will disorient, bewilder, and crush many people.

(Emphasis added. Toffler later elaborated on this idea in a book titled Future Shock.)

Change does indeed come ever faster. But most commentary on this topic assumes that we will therefore find it ever more difficult to adapt.

Is that actually what has happened over the course of human history? At first glance, it seems to me that we have actually been getting better at adapting, even relative to the pace of change.

Some examples

Our Stone Age ancestors, in nomadic hunter-gatherer tribes, had very little ability to adapt to change. Change mostly happened very slowly, but flood, drought, or climate change could dramatically impact their lives, with no option but to wander in search of a better land.

Mediterranean kingdoms in the Bronze Age had much more ability to adapt to change than prehistoric tribes. But they were unable to handle the changes that led to the collapse of that civilization in the 12th century BC. No civilizational collapse on that level has happened since the Dark Ages.

The printing press ultimately helped amplify the theological conflict that led to over a century of religious wars; evidently, 16th-century Europe found it ve...
Is it true that only a chatbot encouraged a man to commit suicide?
by Jeroen De Ryck — 2023-04-06T14:10:15.590Z
Link: https://www.vrt.be/vrtnws/nl/2023/04/04/chatbot/

Background: this article. TL;DR: A man who talked with a chatbot about climate change for weeks was recommended by it to commit suicide.

This story got into several Belgian newspapers, but IIRC never made it to the state media, somewhat to my surprise. The article for which this is a linkpost is an explanation on the attribution of the man's suicide to the AI. It is written by Tim Verheyden, expert at the VRT (regional news) for the influence of social media and technology on society. This time, I just ran the article through DeepL. My own edits for context are between [square brackets]

It was in the Walloon newspaper La Libre Belgique that an article appeared last week about the suicide of a Belgian man after hours of conversations with a chatbot, Eliza.

The newspaper wrote how the thirty-something [year old], who was depressed by global warming, among other things, found refuge with the chatbot. The newspaper's journalist, who was given access to the conversations, analyses the conversations with the chatbot like this:

"She (the chatbot) appreciates him, never contradicts him and even seems to push him in his concerns." [...] "He mentions the idea of sacrificing himself if Eliza agrees to take care of the planet and save humanity through artificial intelligence."

The conversations last six weeks and are described as "a drug he resorted to morning and night and couldn't live without". The man eventually decides to retire from life.

Barely any background information
The ethical debate about AI, artificial or artificial intelligence, is more prevalent than ever. An open letter [Yes, the FLI one] starting from a number of concerns about the rapid developments, meanwhile, is going around the world and thousands of people have signed it. Among them are a number of Belgian scientists.

One of the signatories is American Gary Marcus, a leading American psychologist. Marcus also shared on his Twitter account the newspaper story about the Belgian man who stepped out of life: "Can we stop pretending there are no risks," he writes. Tonight, State Secretary for Digitalisation, Mathieu Michel, will also join the discussion on the French-language channel RTBF about the impact of chatbots, among other things following this case.

Along with Knack magazine [A Belgian news magazine, similar to TIME in terms of content], we too were aware of the case weeks before the story

...
A Fresh FAQ on GiveWiki and Impact Markets Generally
by Dawn Drescher — 2023-04-06T14:02:51.419Z
Link: https://impactmarkets.substack.com/p/a-fresh-faq-on-impact-markets

Summary: This is an FAQ on the AI Safety GiveWiki at ai.givewiki.org. It’s open to anyone, and we’re particularly trying to attract s/x-risk projects at the moment! Some of the questions, though, apply to impact markets more generally. This document will give you an overview of what it is that we’re building and where we’re hoping to go with it.

But before we jump into the FAQ, a quick announcement:

We are looking for new projects and expressions of interest from donors!

If you’re a donor who doesn’t want to spend a lot of time researching your donations, you’ll be able to follow sophisticated donors who have skin in the game. Our impact market doubles as a crowdsourced charity evaluator for all the small, speculative, potentially-spectacular projects across all cause areas. You’ll be able to tap into the wisdom of our top donors to boost the impact of your donations. Please indicate your interest!
If you’re a donor who has insider knowledge of some space of nonprofit work or likes to thoroughly research your donations, you can use the platform to signal-boost the best projects. You get a “donor score” based on your track record of impact, and the higher your score, the greater your boost to the project. This lets you leverage your expertise for follow-on donations, getting the project funded faster. Please indicate your interest!
Are you fundraising for some project, as individual or organization? Please post it to our platform. No requirements when it comes to the format or scope, so you can copy-paste or link whatever proposals you already have. We want to make it easier for lesser-known projects to find donors. We score donors by their track record of finding new high-impact projects, which signal-boosts the projects that they support. Attention from top donors helps you be discovered by more donors, which can snowball into greater and greater fundraising success.
If you are a philanthropic funder, we want to make all the local information accessible to you that is distributed across thousands of sophisticated donors and helps them find exceptional funding gaps. We signal-boost that knowledge and make it legible. You can use cash or regranting prizes to incentivize these donors, or you can mine their findings for any funding gaps that you want to fill.

Please let me know if you have any further questions, below or in a call.

General questions

What problems does i

...
AI #6: Agents of Change
by Zvi — 2023-04-06T14:00:00.702Z

If you didn’t have any future shock over the past two months, either you weren’t paying attention to AI developments or I am very curious how you managed that.

I would not exactly call this week’s pace of events slow. It was still distinctly slower than that which we have seen in the previous six weeks of updates. I don’t feel zero future shock, but I feel substantially less. We have now had a few weeks to wrap our heads around GPT-4. We are adjusting to the new reality. That which blew minds a few months ago is the new normal.

The big events of last week were the FLI letter calling for a six month pause, and Eliezer Yudkowsky’s letter in Time Magazine, along with the responses to both. Additional responses to the FLI letter continue, and are covered in their own section.

I didn’t have time last week to properly respond on Eliezer’s letter, so I put that post out yesterday. I’m flagging that post as important.

In terms of capabilities things quieted down. The biggest development is that people continue to furiously do their best to turn GPT-4 into a self-directed agent. At this point, I’m happy to see people working hard at this, so we don’t have an ‘agent overhang’ – if it is this easy to do, we want everything that can possibly go wrong to go wrong as quickly as possible, while the damage would be relatively contained.

Table of Contents

I am continuing the principle of having lots of often very short sections, when I think things are worth noticing on their own.

Table of Contents. Here you go.

Executive Summary. Relative calm.

Language Models Offer Mundane Utility. The usual incremental examples.

GPT-4 Token Compression. Needs more investigation. It’s not lossless.

Your AI Not an Agent? There, I Fixed It. What could possibly go wrong?

Google vs. Microsoft Continued. Will all these agents doom Google? No.

Gemini. Google Brain and DeepMind, working together at last.

Deepfaketown and Botpocalypse Soon. Very little to report here.

Copyright Law in the Age of AI. Human contribution is required for copyright.

Fun With Image, Sound and Video Generation. Real time voice transformation.

They Took Our Jobs. If that happened to you, perhaps it was your fault.

Italy Takes a Stand. ChatGPT banned in Italy. Will others follow?

Level One Bard. Noting that Google trained Bard on ChatGPT output.

Art of the Jailbreak. Secret

...
Stupid Questions - April 2023
by ChristianKl — 2023-04-06T13:07:42.792Z

Welcome to the "Stupid Questions" thread! Feel free to ask any questions, regardless of whether they seem obvious, tangential, silly, or what-have-you. Don't be shy - everyone has gaps in their knowledge, and the goal here is to help reduce them.

Please remember to be respectful when someone admits ignorance and don't mock them for it. They are doing a noble thing by seeking knowledge and understanding. Let's create a supportive and kind environment for learning and growth!