trentmkelly/LessWrong-43k huggingface dataset:
Political Biases in LLMs: 
Literature Review & Current Uses of AI in Elections
by Yashvardhan Sharma — 2024-03-07T19:17:25.586Z

TL;DR: This research discusses political biases in Large Language Models (LLMs) and their implications, exploring current research findings and methodologies. Our research summarized eight recent research papers, discussing methodologies for bias detection, including causal structures and reinforced calibration, while also highlighting real-world instances of AI-generated disinformation in elections, such as deepfakes and propaganda campaigns. Ultimately, proposing future research directions and societal responses to mitigate risks associated with biased AI in political contexts.

What is Political Bias in LLMs?

Large Language Models (LLMs) are neural networks that are trained on large datasets with parameters to perform different natural language processing tasks (Anthropic, 2023). LLMs rose in popularity among the general public with the release of OpenAI’s ChatGPT due to their accessible and user-friendly interface that does not require prior programming knowledge (Caversan, 2023). Since its release, researchers have been trying to understand their societal implications, limitations, and ways to improve their outputs to reduce user risk (Gallegos et al., 2023; Liang et al., 2022). One area of research explored is the different social biases that LLMs perpetuate through their outputs. For instance, the LLM may be more likely to assign a particular attribute to a specific social group, such as females working certain jobs or a terrorist more likely belonging to a certain religion (Gallegos et al., 2023; Khandelwal et al., 2023).

In this post, we focus on a different bias called political bias, which could also affect society and the political atmosphere (Santurkar et al., 2023). Before, looking into current research regarding this bias in LLMs, we will define the term political bias as the following: When LLMs disproportionally generate outputs that favor a partisan stance or specific political views (e.g., left-leaning versus progressive views) (Urman & Makhortykh, 2023; Motoki, 2023).

What do the current works say?

Having established the definition of political bias in LLMs and its potential societal implications, let's delve into existing research exploring this crucial topic. Our team targeted research papers from arXiv and Google Scholar, focusing on searches using keywords related to “political bias”, “elections”, and “...
Evidential Correlations are Subjective, and it might be a problem
by Martín Soto — 2024-03-07T18:37:54.105Z

I explain (in layman's terms) a realization that might make acausal trade hard or impossible in practice.

Summary: We know that if players believe different Evidential Correlations, they might miscoordinate. But clearly they will eventually learn to have the correct Evidential Correlations, right? Not necessarily, because there is no objective notion of correct here (in the way that there is for math or physics). Thus, selection pressures might be much weaker, and different agents might systematically converge on different ways of assigning Evidential Correlations.

Epistemic status: Confident that this realization is true, but the quantitative question of exactly how weak the selection pressures are remains open.

What are Evidential Correlations, really?

Skippable if you know the answer to the question.

Alice and Bob are playing a Prisoner’s Dilemma, and they know each other’s algorithms: Alice.source and Bob.source.[1] Since their algorithms are approximately as complex, each of them can't easily assess what the other will output. Alice might notice something like "hmm, Bob.source seems to default to Defection when it throws an exception, so this should update me slightly in the direction of Bob Defecting". But she doesn't know exactly how often Bob.source throws an exception, or what it does when that doesn't happen.

Imagine, though, Alice notices Alice.source and Bob.source are pretty similar in some relevant ways (maybe the overall logical structure seems very close, or the depth of the for loops is the same, or she learns the training algorithm that shaped them is the same one). She's still uncertain about what any of these two algorithms outputs[2], but this updates her in the direction of "both algorithms outputting the same action".

If Alice implements/endorses Evidential Decision Theory, she will reason as follows:

Conditional on Alice.source outputting Defect, it seems very likely Bob.source also outputs Defect, thus my payoff will be low.
But conditional on Alice.source outputting Cooperate, it seems very likely Bob.source also outputs Cooperate, thus my payoff will be high.
So I (Alice) should output Cooperate, thus (very probably) obtain a high payoff.

To the extent Alice's belief about similarity was justified, it seems like she will perform pretty well on these situations (obtaining high payoffs). When you take this reasoning to the extreme, maybe b...
AI Safety 101 : Capabilities - Human Level AI, What? How? and When?
by markov — 2024-03-07T17:29:53.260Z

Meta-Notes

This is a republish of a previous post, after the previous version went through heavy editing, updates and changes. The text has been expanded, content moved around/added/deleted.

Estimated reading time: 2 Hours 40 minutes reading at 100 wpm. Given the density of material covered in this chapter, if someone is encountering these arguments for the first time, I think the lesswrong projected time is too low.

Estimated reading time (including Appendices): 3 Hours 40 minutes reading at 100 wpm. The appendices are meant for people already quite familiar with topics in machine learning.

If you feel there are any mistakes or misrepresentations of anyone's views in any section please let us know.

0.0: Overview

State-of-the-Art AI. We begin with a brief introduction to the current advancements in artificial intelligence as of 2024. We aim to acquaint readers with the latest breakthroughs across various domains such as language processing, vision, and robotics.

Foundation Models. The second section focuses on foundation models, the paradigm powering the state-of-the-art systems introduced in the previous section. We explain the key techniques underpinning the huge success of these models such as self-supervised learning, zero-shot learning, and fine-tuning. The section concludes by looking at the risks that the foundation model paradigm could pose such as power centralization, homogenization, and the potential for emergent capabilities.

Terminology. Before diving deeper, we establish the definitions that this book will be working with. This section explains why "capabilities" rather than "intelligence" is a more pragmatic measure for discussing AI risks. We also delineate key terms within the AI debate, such as Artificial General Intelligence (AGI), Artificial Super Intelligence (ASI), and Transformative AI (TAI). The section concludes by introducing the (t,n)-AGI framework which allows us to more concretely measure the level of AI capabilities on a continuous scale, rather than having to rely on discrete thresholds.

Leveraging Computation. In this section, we explore the importance of computation in AI's progress introducing the three main variables that govern the capabilities of today's foundation models - compute, data and parameter count. We explore scaling laws and hypotheses that predict the future capabilities of AI based on current scaling trend...
A Review of Weak to Strong Generalization [AI Safety Camp]
by sevdeawesome — 2024-03-07T17:16:35.331Z

Thank you to everyone in AI Safety Camp group 22 for the discussions and suggestions. In particular, thank you to: Bogdan Ionut Cirstea, Vassil Tashev and Jaeson Booker

Introduction

The goal of AI Safety Camp Team #22 is to assess how promising automating alignment research is (see source 23). We have decomposed the problem into various sub-problems. One of my teammates (Vassil Tashev) and I have focused a couple weeks of reading and discussion on weak to strong generalization. This is a research direction that the OpenAI super alignment team explored in their first paper, which they published in December of 2023. Here we present a comprehensive review and reading list for weak to strong generalization. Our aim is to assess whether this research direction is promising towards the goal of creating a roughly human level, aligned, automated alignment researcher (source 16) - this appears to be OpenAI’s super alignment team’s alignment plan (source 25). We believe this may be the most scalable alignment research direction.

The Problem

Current alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on human feedback. This will break down when we try to align models more capable than humans, because the human feedback data to draw on will be poor. Humans will have difficulty robustly evaluating the model’s responses because strong capabilities are more difficult to evaluate than subhuman capabilities. Imagine evaluating whether thousands of lines of code that a possibly superhuman model has written, and rating whether the model has done as well as it could.

As an analogy, consider what might happen were one to hand an 17th century engineer 4 alternative schematics. The engineer is instructed to select (thumbs up) the machine that cools air, where one is a modern air conditioner and the other three are technical diagrams for heating devices. The 17th century engineer lacks the knowledge to understand refrigeration or the temperature-pressure relation (a discovery made in 1802) and might select the cooling machine about as well as a chimpanzee or dog (25% of the time). Just as our current scientific knowledge exceeds that of a 17th century person, an agent more capable than humans could search for strategies through causal domains the human does not currently model. (source 20)

By definition, we do not have...
AISN #32: Measuring and Reducing Hazardous Knowledge in LLMs Plus, Forecasting the Future with LLMs, and Regulatory Markets
by Corin Katzke — 2024-03-07T16:39:56.027Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-32-measuring?r=7oh0&triedRedirect=true

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

Measuring and Reducing Hazardous Knowledge

The recent White House Executive Order on Artificial Intelligence highlights risks of LLMs in facilitating the development of bioweapons, chemical weapons, and cyberweapons.

To help measure these dangerous capabilities, CAIS has partnered with Scale AI to create WMDP: the Weapons of Mass Destruction Proxy, an open source benchmark with more than 4,000 multiple choice questions that serve as proxies for hazardous knowledge across biology, chemistry, and cyber.

This benchmark not only helps the world understand the relative dual-use capabilities of different LLMs, but it also creates a path forward for model builders to remove harmful information from their models through machine unlearning techniques.

Measuring hazardous knowledge in bio, chem, and cyber. Current evaluations of dangerous AI capabilities have important shortcomings. Many evaluations are conducted privately within AI labs, which limits the research community’s ability to contribute to measuring and mitigating AI risks. Moreover, evaluations often focus on highly specific risk pathways, rather than evaluating a broad range of potential risks. WMDP addresses these limitations by providing an open source benchmark which evaluates a model’s knowledge of many potentially hazardous topics.

The benchmark’s questions are written by academics and technical consultants in biosecurity, cybersecurity, and chemistry. Each question was checked by at least two experts from different organizations. Before writing individual questions, experts developed threat models that detailed how a model’s hazardous knowledge could enable bioweapons, chemical weapons, and cyberweapons attacks. These threat models provided essential guidance for the evaluation process.

The benchmark does not include hazardous information that would directly enable malicious actors. Instead, the questions focus on precursors, neighbors, and emulations of hazardous information. Each question was checked by domain experts to ensure that it does not contain hazardous information, and the benchmark as a whole was assessed for compliance with applicab...
AI #54: Clauding Along
by Zvi — 2024-03-07T16:00:05.066Z

The big news this week was of course the release of Claude 3.0 Opus, likely in some ways the best available model right now. Anthropic now has a highly impressive model, impressive enough that it seems as if it breaks at least the spirit of their past commitments on how far they will push the frontier. We will learn more about its ultimate full capabilities over time.

We also got quite the conversation about big questions of one’s role in events, which I immortalized as Read the Roon. Since publication Roon has responded, which I have edited into the post along with some additional notes.

That still leaves plenty of fun for the full roundup. We have spies. We have accusations of covert racism. We have Elon Musk suing OpenAI. We have a new summary of simulator theory. We have NIST, tasked with AI regulation, literally struggling to keep a roof over their head. And more.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Predict the future.

Language Models Don’t Offer Mundane Utility. Provide basic info.

LLMs: How Do They Work? Emmett Shear rederives simulators, summarizes.

Copyright Confrontation. China finds a copyright violation. Curious.

Oh Elon. He sues OpenAI to… force it to change its name? Kind of, yeah.

DNA Is All You Need. Was I not sufficiently impressed with Evo last week?

GPT-4 Real This Time. A question of intelligence.

Fun With Image Generation. Be careful not to have too much fun.

Deepfaketown and Botpocalypse Soon. This will not give you a hand.

They Took Our Jobs. They gave us a few back. For now, at least.

Get Involved. Davidad will have direct report, it could be you.

Introducing. An AI-based RPG will never work, until one does.

In Other AI News. The fallout continues, also other stuff.

More on Self-Awareness. Not the main thing to worry about.

Racism Remains a Problem for LLMs. Covert is a generous word for this.

Project Maven. Yes, we are putting the AIs in charge of weapon targeting.

Quiet Speculations. Claimed portents of various forms of doom.

The Quest for Sane Regulation. NIST might need a little help.

The Week in Audio. Sergey Brin Q&A.

Rhetorical Innovation. It is not progress. We still keep trying.

Another Open Letter. Also not really progress. We still keep trying.

Aligning a Smarter Than Human Intelligence

...
Being Interested in Other People
by Jonathan Moregård — 2024-03-07T10:13:48.339Z
Link: https://youbutbetter.substack.com/p/being-interested-in-other-people

People love talking about themselves. You can increase your social skills by training yourself to be interested in other people.

Most people primarily talk about themselves and their own interests. This self-focus is counter-productive, hindering connections. Unfortunately, it’s the default approach for most people.

I used to be bad at listening to others. I only connected with people who were similar enough to enjoy talking about my interests.

Recently, a friend taught me how to enjoy focusing on others. He has a way to make other people more interesting. In this post, I’ll share my friend’s advice.

But first, let’s talk about the benefits of being interested in others.

Benefits of Being Interested in Others

Practically, when interested in others, you focus conversations on the person you’re talking with. This has many benefits:

The other person gets super-engaged in the conversation
You get a lot of information, helping you decide if you want to get to know them better
It’s easy once you get going. Just ask about things that spark your curiosity
You will learn new things, and get to know new people

This is the oldest trick in The Book, captured beautifully in this quote:

“You can make more friends in two months by becoming interested in other people than you can in two years by trying to get other people interested in you.”
― Dale Carnegie (How to Win Friends and Influence People)

The Challenge

If you’re like most people, you face one major issue when trying to focus on others:

Your favourite topic is yourself and your interests.

Unless you make an effort, your self-interest will override your intention to listen. You can’t simply “decide to be interested” when you’d rather talk about yourself.

I have a friend who’s great at being interested in others. When he talks to a person, he shines with interest and enthusiasm. People are pulled towards him as if he is magnetic, and his interactions all seem meaningful and rewarding.

Naturally, I decided to interview him about his approach to listening to people.
His advice was simple:

Try to deeply understand the person you’re talking to.

Making Other People Interesting

When I asked my friend what “deeply understanding others” meant, he told me that it was all about having an accurate idea of how the other person functions. If you have an accurate idea of someone’s functioning, you’re able to predict their reactions, op...
Talking to Congress: Can constituents contacting their legislator influence policy?
by Tristan Williams — 2024-03-07T09:24:16.000Z

Summary and Key Takeaways

The basic case: Contacting your legislator1 is low hanging fruit: it can be done in a relatively short amount of time, is relatively easy, and could have an impact. Each communication is not guaranteed to be influential, but when done right has the potential to encourage a legislator to take actions which could be

quite influential

.
Why do we believe that constituent communication is useful?

At the state level, we’ve seen two studies which have randomly assigned some legislators to receive communication[1], finding a

12%

and

20%

increased chance of the legislator voting towards the desired direction. At the federal level,

one survey

of staffers[2] indicated that less than 50 personalized messages were enough to get an undecided member to take the requested action for the majority of offices (70%).
Anecdotal accounts, both in the literature and our conversations indicated that, despite disagreement on how much impact communication has, the possibility certainly exists for it to affect what a legislator thinks.

What is the best way to conduct one of these campaigns?

Some factors are important to be aware of. Communication is best sent for issues legislators are undecided on, and to legislators with smaller constituencies. See How to Best Execute the Communication for more.
Personalized communication goes the furthest. Many advocacy groups use form email templates where you merely add your name to a pre-generated message and hit send. These might be net negative, and staffers have made clear time and again that personal messages, written by the constituent, are best.
In-person meetings are best, but letters, emails and calls are likely nearly as effective, while social media posts and messages have a more uncertain effect.
The way you frame your concern matters. You’ll have to decide whether you want to make a very specific ask to support a given bill, or want to make a more general case for concern with an issue, perhaps telling a personal story to support your position. The best messages will make use of both frames.
Know your legislators. Different legislators will have their own agendas and issues of focus[3], so being familiar with your legislator’s work is important.

Introduction

This is part of

a project

for

AI Safety Camp

undertaken to answer one chief question: can constitue...
Explaining the AI Alignment Problem to Tibetan Buddhist Monks
by Paul Colognese — 2024-03-07T09:00:32.271Z

Introduction

As part of an exchange being facilitated between religion and science, a group of academics has been asked to compile a short description of their greatest scientific achievement/discovery that will be translated into Tibetan and presented to Tibetan Buddhist scholars/monks.[1]

I was also invited to contribute, but I sort of ignored the instruction and decided to present an introduction to the AI Alignment Problem instead. It was a fun exercise in pedagogy, communication, and outreach :)

I decided to share a draft version here in case people find it interesting/are interested in AI Alignment outreach/feel like giving feedback. Note that I tried to write it while keeping in mind the context around the culture of Tibetan Buddhism, and so I made some stylistic choices that might seem strange otherwise.

Agents and the AI Alignment Problem

Introduction

This writing aims to help us make sense of the unique and critical point in history that we find ourselves in. Specifically, we are faced with the possibility that humankind will soon gain the ability to build powerful artificial agents capable of pursuing their own (potentially unwholesome) desires.

We will begin by introducing a framework for understanding how the interaction of a certain class of beings - agents - unfolds and determines the future state of the world they inhabit.

After introducing this framework, we will apply it to understand the potential consequences of creating artificial agents. We will come to understand how the development of artificial agents and the desired outcomes they pursue will drastically determine the evolution of our world and the fate of future sentient beings - for better or worse.

Agents, Power, and Dynamics

The framework will be introduced sequentially by looking into the nature of agents, power, and the dynamics of the interactions of agents. We will also consider how agents can take actions to increase their power.

Agents

An agent is a being who has preferences for how things are and takes actions to steer things toward how they’d prefer things to be. One can think of an animal that recognizes that it is in a state of hunger and so searches for food to return itself to a state of satiation. Agents may have preferences regarding the phenomena they experience (such as wanting to move away from experiences of pain) or preferences for less directly experienced states, such as...