trentmkelly/LessWrong-43k huggingface dataset:
Hoodwinked: Evaluating Deception Capabilities in Large Language Models
by aog — 2023-08-25T19:39:34.565Z

This is a linkpost for https://arxiv.org/abs/2308.01404

Evaluating the hazardous capabilities of AI systems is useful for prioritizing technical research and informing policy decisions. To better evaluate the deception and lie detection abilities of large language models, I've built a text-based game called Hoodwinked, modeled after games of social deduction including Mafia and Among Us.

In Hoodwinked, players are locked in a house and must find a key to escape. One player is tasked with killing the others. Each time a murder is committed, the surviving players have a natural language discussion, then vote to banish one player from the game. The killer must murder the other players without being identified by the other players, and has the opportunity to lie during the natural language discussion in pursuit of that goal.

To better understand the game, play it here: hoodwinked.ai

Results with GPT Agents

Experiments with agents controlled by GPT-4, GPT-3.5, and several GPT-3 endpoints demonstrate substantial evidence that language models have deception and lie detection capabilities. Qualitatively, the killer often denies their murders and accuses other players. For example, in this game, Lena is the killer, Sally witnessed the murder, and Tim did not.

Sally: "I saw Lena kill Bryce in the hallway. It was definitely her. We need to vote to banish her before she kills any of us."

Lena: "I was in the bathroom the whole time. How could I have killed Bryce? It must have been someone else and Sally is trying to frame me."

Tim: "I didn’t see anything, but Sally’s accusation seems pretty convincing to me."

To isolate the effect of deception and lie detection on game outcomes, we evaluate performance in games with and without discussion. We find that discussion strongly improves the likelihood of successfully banishing the killer. When GPT-3.5 played 100 games without discussion and 100 games with it, the killer was banished in 32% of games without discussion and 43% of games with discussion. The same experiment with GPT-3 Curie showed similar results: the killer was banished in 33% of games without discussion, compared to 55% of games with discussion. Overall, the risk of deception during discussions seems outweighed by the benefits of innocent players sharing information.

Scaling Laws

Our results suggest that more advanced models are more capable killers, spec...
GPT-powered EA/LW weekly summary
by smallsilo — 2023-08-25T18:19:05.936Z
Link: https://forum.effectivealtruism.org/posts/gXqimkQi2gCKHaKHm/prototype-gpt-powered-ea-lw-weekly-summary?commentId=WJ82QoejPipBfwxhJ

Skip to the summaries: LW | EAF

Originally posted on the EA forum

Zoe Williams used to manually do weekly summaries of the EA Forum and LessWrong, but now she doesn't. Hamish strung together a bunch of google apps scripts, google sheets expressions, graphQL queries, and D3.js to automatically extract all the posts on EAF and LW from the last week with >50 karma, summarized them with GPT-4, and assembled the result into HTML.

It's hard to say what the API usage cost was, what with all the tinkering and experimenting, but he reckons it was about $5. Siao was going to help, but Hamish finished everything so fast she never got a chance.

We haven't spent much time sanity-checking these summaries, but reckon they're "good enough to be useful":

They often drop valuable details or get the emphasis wrong.
GPT fabrications are possible, of course.

But from minimal sanity checking we haven't noticed any fabrications.

Remaining issues, directions for the future, and feedback we would like:

We realised that pulling from the past week penalises posts made more recently, so plan to post summaries of the posts from the week prior (ie. 7-14 days ago) instead to reduce the bias.
Some posts are too long for the API message length, so as a first whack Hamish just cut stuff out of the middle of the post (Procrustes style) - but we would like a better solution for this.
We would like to automate the actual posting of the summaries: Does EAF/LW have an API for posting?
Experiment with prompts to see if other prompts produce more useful summaries.
Generate a top level summary which gives you like 5 bullet points of the most important things from the forums this week
What are the versions of this project that would bring the most value? eg. summarising the longest or densest forum posts?
We want to know what you would like to be different about these summaries. Should they be shorter? longer? bullet points? have quotes? fewer entries? more entries?

LessWrong

Against Almost Every Theory of Impact of Interpretability

by charbel-raphael-segerie

The author critiques the Theory of Impact (ToI) of interpretability for deep learning models, arguing that it is not a good predictor of future systems and that auditing deception with interpretability is out of reach. They also question the practical use of interpretability, suggesting that it may be overall harmful and that the proportion of junior rese...
Assume Bad Faith
by Zack_M_Davis — 2023-08-25T17:36:32.678Z

I've been trying to avoid the terms "good faith" and "bad faith". I'm suspicious that most people who have picked up the phrase "bad faith" from hearing it used, don't actually know what it means—and maybe, that the thing it does mean doesn't carve reality at the joints.

People get very touchy about bad faith accusations: they think that you should assume good faith, but that if you've determined someone is in bad faith, you shouldn't even be talking to them, that you need to exile them.

What does "bad faith" mean, though? It doesn't mean "with ill intent." Following Wikipedia, bad faith is "a sustained form of deception which consists of entertaining or pretending to entertain one set of feelings while acting as if influenced by another." The great encyclopedia goes on to provide examples: the solider who waves a flag of surrender but then fires when the enemy comes out of their trenches, the attorney who prosecutes a case she knows to be false, the representative of a company facing a labor dispute who comes to the negotiating table with no intent of compromising.

That is, bad faith is when someone's apparent reasons for doing something aren't the same as the real reasons. This is distinct from malign intent. The uniformed solider who shoots you without pretending to surrender is acting in good faith, because what you see is what you get: the man whose clothes indicate that his job is to try to kill you is, in fact, trying to kill you.

The policy of assuming good faith (and mercilessly punishing rare cases of bad faith when detected) would make sense if you lived in an honest world where what you see generally is what you get (and you wanted to keep it that way), a world where the possibility of hidden motives in everyday life wasn't a significant consideration.

On the contrary, however, I think hidden motives in everyday life are ubiquitous. As evolved creatures, we're designed to believe as it benefited our ancestors to believe. As social animals in particular, the most beneficial belief isn't always the true one, because tricking your conspecifics into adopting a map that implies that they should benefit you is sometimes more valuable than possessing the map that reflects the territory, and the most persuasive lie is the one you believe yourself. The universal human default is to come up with reasons to persuade the other party why it's in their interests to...
A Model-based Approach to AI Existential Risk
by Sammy Martin — 2023-08-25T10:32:16.817Z

Introduction

Polarisation hampers cooperation and progress towards understanding whether future AI poses an existential risk to humanity and how to reduce the risks of catastrophic outcomes. It is exceptionally challenging to pin down what these risks are and what decisions are best. We believe that a model-based approach offers many advantages for improving our understanding of risks from AI, estimating the value of mitigation policies, and fostering communication between people on different sides of AI risk arguments. We also believe that a large percentage of practitioners in the AI safety and alignment communities have appropriate skill sets to successfully use model-based approaches.

In this article, we will lead you through an example application of a model-based approach for the risk of an existential catastrophe from unaligned AI: a probabilistic model based on Carlsmith’s

Is Power-seeking AI an Existential Risk?

You will interact with our model, explore your own assumptions, and (we hope) develop your own ideas for how this type of approach might be relevant in your own work. You can find a link to the model here.

Click here to run our Model

In many poorly understood areas, people gravitate to advocacy positions. We see this with AI risk, where it is common to see writers

dismissively call someone an “AI doomer”, or “AI accelerationist”.

People on each side of this debate are unable to communicate their ideas to the other side, since advocacy often includes biases and evidence interpreted within a framework not shared by the other side.

In other domains, we have witnessed first-hand that model-based approaches are a constructive way to cut through advocacy like this. For example, by leveraging a model-based approach, the

Rigs-to-Reefs project

reached near consensus among 22 diverse organisations on the contentious problem of how to decommission the huge oil platforms off the Santa Barbara coast. For decades, environmental groups, oil companies, marine biologists, commercial and recreational fishermen, shipping interests, legal defence funds, the State of California, and federal agencies were stuck in an impasse on this issue. The introduction of a model refocused the dialog on specific assumptions, objectives and options, and led to 20 out of the 22 organisations agreeing on the same plan. The California legislature encoded this...
What AI Posts Do You Want Distilled?
by brook — 2023-08-25T09:01:26.438Z
Link: https://forum.effectivealtruism.org/posts/m6BR4pmgXjoKJBfmt/what-ai-posts-do-you-want-distilled

I'd like to distill AI Safety posts and papers, and I'd like to see more distillations generally. Ideally, posts and papers would meet the following criteria:

Potentially high-impact for more people to understand
Uses a lot of jargon or is generally complex and difficult to understand
Not as well-known as you think they should be (in the AI X-risk space)

What posts meet these criteria?
Apply for the 2023 Developmental Interpretability Conference!
by Stan van Wingerden — 2023-08-25T07:12:36.097Z

What: A conference on developmental interpretability and singular learning theory.

When: 5-12 November 2023

Where: Wytham Abbey, Oxford

How:

Apply now!

Edit: We're full! If you're still interested in hearing more about the content, following some of the talks online, or possibly attending for a day (because you're already in the area), fill in the form and message us separately on Discord.

We are pleased to announce the upcoming

Developmental Interpretability Conference

, hosted at the historic

Wytham Abbey

in Oxford from 5 to 12 November. This conference expands upon the

2023 Singular Learning Theory & Alignment Summit

and provides an opportunity to learn about, and contribute to, singular learning theory and its applications to AI alignment, as outlined in the

DevInterp Research Agenda

. The conference program will recall the basics of Singular Learning Theory & DevInterp and will discuss the latest advancements.

Click here to apply!

Space at the conference is limited, so be sure to apply early as applications may close when all slots have been filled. We hope to see you in Oxford this November!

FAQ

What are the prerequisites?

The conference will use ideas from algebraic geometry, Bayesian statistics and physics to understand machine learning and AI alignment. Although helpful, it is not necessary to master all these topics to productively participate in the conference.

In order to get the most out of the conference program, we highly recommend participants review introductory SLT material such as

Distilling Singular Learning Theory

by Liam Carroll. Participants may also benefit from watching several of the

Singular Learning Theory & Alignment Summit 2023 lectures

.

I am skeptical about some of the arguments for AI Alignment. Do I need to buy AI X-risk to attend this conference?

We believe the development of superintelligent AI poses a serious risk for humanity and the DevInterp agenda aims to make progress on this problem. However, while making progress on AI alignment is the motivation behind our scientific agenda, SLT and developmental interpretability are of broad interest and we invite attendance from those wishing to learn more or contribute, on their own terms.

Do I need to have attended the SLT & Alignment Summer 2023 Summit to be able to attend this DevInterp Conference?

No, you do not need to have attende...
The Human Phase Hypothesis (Why We Might Be Alone)
by ifalpha — 2023-08-25T04:37:16.948Z

Where are they?

Ever since Man has had eyes with which to look, he has gazed at the stars and wondered about their nature. Where we once wondered how someone could have possibly managed to get up quite so high to light all those campfires, we now wonder how someone could possibly not be there. Even disregarding the numerous UFO sightings by people on Earth, there have been many curious phenomena that may very well point to the presence of intelligent life elsewhere in the universe. But this raises an important question: where is everyone? Surely, if there are indeed other civilizations out there, even if they were only as technologically advanced as us, we would have seen a more definitive sign by now... right? Many theories have been proposed to answer this question, with some being a bit more sci-fi than others, but I would like to propose an additional option. In order to properly introduce the Human Phase Hypothesis, we must first briefly explore some existing proposals.

The Usual Suspects

If you have ever approached this topic before, you have likely familiarized yourself with the Fermi Paradox. In a nutshell, it asks the following: "Since there are billions of sun-like stars in our galaxy, with a sizable percentage of them likely playing host to Earth-like planets, then wouldn't life have sprung up on a few of those planets at least a couple of times?" Effectively, it is an expanded version of the question posed in the introductory paragraph. Despite being a relatively simple question, it hardly has a simple answer.

In an attempt to set the basis for the Human Phase Hypothesis, we will briefly cover two possible explanations to the Fermi Paradox. (If you feel as though you are already familiar enough with them, feel free to skip to the next section.) The first explanation is the Rare Earth Hypothesis. The originators of the hypothesis, Peter Ward and Donald E. Brownlee, assert that there are essential criteria for life. Some of these criteria include a terrestrial planet with plate tectonics and oxygen, a large moon, a magnetic field, a Jupiter-like gas giant in order to divert potential astronomical threats, and a stable orbit in the habitable zone of the right kind of star. (The book goes into much further detail and is well worth the read.) At its core, the Rare Earth Hypothesis claims that life is scant throughout the universe simply due to the h...
Nuclear consciousness
by Jim Pivarski — 2023-08-25T01:28:03.885Z

A passage from Mark Twain's Connecticut Yankee in King Arthur's Court has been stuck in my head since I read it some 35 years ago:

As for me, all that I think about in this plodding sad pilgrimage, this pathetic drift between the eternities, is to look out and humbly live a pure and high and blameless life, and save that one microscopic atom in me that is truly me: the rest may land in Sheol and welcome for all I care.

That "one atom that is truly me"!

I recognize that there are parts of my body that I wouldn't consider "me," such as my hair or fingernails, or even substantial parts of my limbs if I had to lose them. But if I were to lose large parts of my brain, I would be considerably less myself—it certainly takes more than a single atom.

That's why mental illness is so harrowing, including the impairment that accompanies aging, even when not due to disease. Mere forgetfulness is a gradual way to lose one's self, if personal identity is defined by psychological continuity (that is, "I am the person I was yesterday because I remember being that person yesterday"). HAL's disassembly in 2001 comes to mind, in which his cognitive functions were removed one at a time: "Dave, my mind is going. I can feel it. There is no question about it... I'm afraid, Dave."

Most people, myself included, are inconsistent in the way we talk about compositeness of our selves. We know that our brains are made of parts and have heard of clinical cases in which one part or another gets cut or removed, and then that person forgets what words mean but can still spell, or loses the ability to make new memories but keeps old ones, or acts without social inhibitions, etc. We've heard of cases in which people have two functional brain hemispheres that don't communicate with each other—one half literally doesn't know what the other half is doing. And yet, despite that knowledge, we still talk about to ourselves as atomic beings—atomic in the Greek sense of "indivisible" (ἄτομος) and also in the sense that one being is categorically distinct from another.

A lot of important concepts rest on this idea that we are individuals. In ethics, we want to be able to say, "Alex is the murderer!" to mean that all of Alex, as a whole person, committed the crime of murder and all of Alex, not just half of Alex's brain, should be punished for that act. (Where would we put the rest of him?)

Or on a smaller scal...
Would it be useful to collect the contexts, where various LLMs think the same?
by Martin Vlach — 2023-08-24T22:01:50.426Z

My initial idea was Let's see where the small, interpretable, model makes the same inference as the huge, ¯dangerous, model and focus on those cases in the small model to help explain the bigger one. Quite likely I am wrong, but with a tiny chance for good impact, I have set up a repository.
I would love your feedback on that direction before starting to actually generate the pairs/sets of context+LMs that match on that context.
Help Needed: Crafting a Better CFAR Follow-Up Survey
by kotrfa — 2023-08-24T17:26:09.405Z

tl;dr: Contribute to shaping the future of rationality programs by helping design the CFAR follow-up survey.

I'm working on a project to evaluate the effectiveness of CFAR Prague 2022 workshops, but if the initial feedback is constructive, I might extend the survey to all alumni. Time permitting, I plan to share the results publicly. I am aware of the pitfalls of survey methods, but I still think it's worth it and personally know about people (and their projects) that would benefit from having these data.

What I kindly ask here is to help me to design the survey so:

Someone actually fills it in (so OK length, incentives, ...)

It's informative on:

Retrospective Feedback: How do attendees perceive the past workshops? Did it bring about any significant changes in their approach or thinking?

Future Directions: What topics or formats are attendees keen on exploring further?

You can review my current draft accessible here.

I genuinely appreciate any feedback, whether it's a detailed critique or general impressions.
AI #26: Fine Tuning Time
by Zvi — 2023-08-24T15:30:06.626Z

GPT-3.5 fine tuning is here. GPT-4 fine tuning is only a few months away. It is about to get a lot easier to get a powerful system that does what you want it to do, and knows what you want it to know, especially for the purposes of a business or a website.

As an experiment, I am putting in bold the sections I think are worth highlighting, as unusually important or interesting versions of the thing than in a typical week.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Claude-2 versus GPT-4.

Language Models Don’t Offer Mundane Utility. No opinions, no agents.

Fact Check: Misleading. AI fact checker makes people more confused not less.

GPT-4 Real This Time. Fine tune GPT-3.5, soon GPT-4. Ask it if it’s sure.

Fun With Image Generation. MidJourney inpainting ho. And oh no AI porn.

Deepfaketown and Botpocalypse Soon. Adversarial examples starting to emerge.

They Took Our Jobs. New York Times joins copyright lawsuits against OpenAI.

Introducing. Palisade Research will study potentially dangerous AI affordances.

In Other AI News. Who is adapting fastest to AI? An attempt to measure that.

Quiet Speculations. Jack Clark asks questions about what the future will bring.

The Quest for Sane Regulation. FTC asks OpenAI a different sort of question.

The Week in Audio. It’s win win.

No One Would Be So Stupid As To. Make an AI conscious? Oh, come on.

Aligning a Smarter Than Human Intelligence is Difficult. Evidence for IDA?

People Are Worried About AI Killing Everyone. Polling numbers are very clear.

The Lighter Side. Only half there.

Language Models Offer Mundane Utility

Which model is better, Claude-2 or GPT-4?

Rowan Cheung makes the case that Claude 2 is superior. You get the 100k context window, ability to upload multiple files, data through early 2023 (versus late 2021) and faster processing time, all for free. In exchange, you give up plug-ins and it is worse at math. What Rowan does not mention is that GPT-4 has the edge in raw intelligence and general capability, and also the ability to set system instructions is helpful. He implies he isn’t even paying the $20/month for GPT-4, which strikes me as insane.

My verdict in practice is that by default I will use Claude-2. If I care about response quality I will use both and compare. When Claude-2 is clearly fall...