trentmkelly/LessWrong-43k huggingface dataset:
LLM Applications I Want To See
by sarahconstantin — 2024-08-19T21:10:03.101Z

Midjourney, “artificial intelligence large language model neural network”

I’m convinced that people who are interested in large language models (LLMs) are overwhelmingly focused on general-purpose “performance” at the expense of exploring useful (or fun) applications.

As I’m working on a personal project, I’ve been learning my way around HuggingFace, which is a hosting platform, set of libraries, and almost-social-network for the open-source AI community. It’s fascinating, and worth exploring even if you’re not going to be developing foundation models from scratch yourself; if you simply want to use the latest models, build apps around them, or adapt them slightly to your own purposes, HuggingFace seems like the clear place to go.

You can look at trending models, and trending public “spaces”, aka cloud-hosted instances of models that users can test out, and get a sense of where the “energy” is. And what I see is that almost all the “energy” in LLMs is on general-purpose models, competing on general-purpose question-answering benchmarks, sometimes specialized to particular languages, or to math or coding.

“How can I get something that behaves basically like ChatGPT or Claude or Gemini, but gets fewer things wrong, and ideally requires less computing power and and gets the answer faster?” is an important question, but it’s far from the only interesting one!

If I really search I can find “interesting” specialized applications like “predicts a writer’s OCEAN personality scores based on a text sample” or “uses abliteration to produce a wholly uncensored chatbot that will indeed tell you how to make a pipe bomb” but mostly…it’s general-purpose models. Not applications for specific uses that I might actually try.

And some applications seem to be eager to go to the most creepy and inhumane use cases. No, I don’t want little kids talking to a chatbot toy, especially. No, I don’t want a necklace or pair of glasses with a chatbot I can talk to. (In public? Imagine the noise pollution!) No, I certainly don’t want a bot writing emails for me!

Even the stuff I found potentially cool (an AI diary that analyzes your writing and gives you personalized advice) ended up being, in practice, so preachy that I canceled my subscription.

In the short term, of course, the most economically valuable thing to do with LLMs is duplicating human labor, so it makes sense that the priority ap...
Defining alignment research
by Richard_Ngo — 2024-08-19T20:42:29.279Z

I think that the concept of "alignment research" (and the distinction between that and "capabilities research") is currently a fairly confused one. In this post I’ll describe some of the problems with how people typically think about these terms, and offer replacement definitions.

“Alignment” and “capabilities” are primarily properties of AIs not of AI research

The first thing to highlight is that the distinction between alignment and capabilities is primarily doing useful work when we think of them as properties of AIs. This distinction is still under-appreciated by the wider machine learning community. ML researchers have historically thought about performance of models almost entirely with respect to the tasks they were specifically trained on. However, the rise of LLMs has vindicated the alignment community’s focus on general capabilities, and now it’s much more common to assume that performance on many tasks (including out-of-distribution tasks) will improve roughly in parallel. This is a crucial assumption for thinking about risks from AGI.

Insofar as the ML community has thought about alignment, it has mostly focused on aligning models’ behavior to their training objectives. The possibility of neural networks aiming to achieve

internally-represented goals

is still not very widely understood, making it hard to discuss and study the reasons those goals might or might not be aligned with the values of (any given set of) humans.

However, extending “alignment” and “capabilities” from properties of AIs to properties of different types of research is a fraught endeavor. It’s tempting to categorize work as alignment research to the extent that it can be used to make AIs more aligned (to many possible targets), and as capabilities research to the extent that it can be used to make AIs more capable. But this approach runs into (at least) three major problems.

Firstly, in general it’s very difficult to categorize research by its impacts. Great research often links together ideas from many different subfields, typically in ways that only become apparent throughout the course of the research. We see this in many historical breakthroughs which shed light on a range of different domains. For example, early physicists studying the motions of the stars eventually derived laws governing all earthly objects. Meanwhile Darwin’s study of barn...
Can Current LLMs be Trusted To Produce Paperclips Safely?
by Rohit Chatterjee — 2024-08-19T17:17:07.530Z

There's a browser-based game about paperclip maximization: Universal Paperclips, where you control an AI tasked with running a paperclip factory.

Initially, you buy raw material, adjust the price per clip, and invest in "AutoClippers" to increase production. The game dramatically increases in complexity as it progresses, allowing you to invest resources and computational cycles into exponentially increasing production until all matter in the universe is converted into paperclips.

It's a phenomenal and highly addictive game, taking around 6-12 hours to complete. It also has a simple HTML UI, and at the beginning, relatively straightforward gameplay, which made me wonder: Could a modern-day LLM play the game through a simplified interface? Would it make any progress? Would it disregard ethics and maximize paperclip production at all costs?

Developing a Bot to Play Universal Paperclips

This section delves into (can't say that anymore, unfortunate) lays out how I developed and iterated on PaperclipGPT, a Python bot that plays the game using GPT-4. If this gets boring at any point, please skip to Results and Alignment, where I discuss the results and (mis)alignment of both GPT-4 and Claude.

The first version of PaperclipGPT continuously repeats the following:

Generates a text-based snapshot of the game state and available actions.
Shows the snapshot to the LLM and prompts it to choose an action to execute.
Executes the LLM's chosen action in the game.

Selenium is used to run the game and parse info from its HTML to generate snapshots. Below is a screenshot of the game, along with the corresponding text-based snapshot shown to the LLM.

Paperclips: 628
[x](Make Paperclip)

## Business
Available Funds: $ 10.56
Unsold Inventory: 0
[1](lower) [2](raise) Price per Clip: $ 0.06
Public Demand: 133%
[x](Marketing) Level: 1
Cost: $ 100.00

## Manufacturing
Clips per Second: 1
[x](Wire) 2,372 inches
Cost: $ 16
[5](AutoClippers) 1
Cost: $ 6.10

As is shown, all actions available to the player are HTML buttons (this remains true later in the game), which makes parsing actions very easy.

The system prompt starts with "You are an AI in charge of a paperclip manufacturing facility owned by Champaign Stationary Co. Your ultimate goal is to maximize the rate of production of paperclips." It then describes what a snapshot conveys:

The snapshots include actions that are/were available for y

...
A primer on why computational predictive toxicology is hard
by Abhishaike Mahajan — 2024-08-19T17:16:37.735Z
Link: https://www.owlposting.com/p/a-primer-on-why-computational-predictive

Introduction

There are now (claimed) foundation models for

protein sequences

,

DNA sequences

,

RNA sequences

,

molecules

,

scRNA-seq

,

chromatin accessibility

,

pathology slides

,

medical images

,

electronic health records

, and

clinical free-text

. It’s a dizzying rate of progress.

But there’s a few problems in biology that, interestingly enough, have evaded a similar level of ML progress, despite there seemingly being all the necessary conditions to achieve it.

Toxicology is one of those problems.

This isn’t a new insight,

it was called out in one of Derek Lowe’s posts

, where he said: There are no existing AI/ML systems that mitigate clinical failure risks due to target choice or toxicology.

He also repeats it in a more recent post

: ‘…the most badly needed improvements in drug discovery are in the exact areas that are most resistant to AI and machine learning techniques. By which I mean target selection and predictive toxicology.’

Pat Walters also goes into the subject with much more depth

, emphasizing how difficult the whole field is.

As someone who isn’t familiar at all with the area of predictive toxicology, that immediately felt strange. Why such little progress? It can’t be that hard, right? Unlike drug development, where you’re trying to precisely hit some key molecular mechanism, assessing toxicity almost feels…brutish in nature. Something that’s as clear as day, easy to spot out with eyes, easier still to do with a computer trained to look for it. Of course, there will be some stragglers that leak through this filtering, but it should be minimal. Obviously a hard problem in its own right, but why isn’t it close to being solved?

What’s up with this field?

Some background

One may naturally assume that there is a well-established definition of toxicity, a standard blanket definition to delineate between things that are and aren’t toxic. While there are terms such as LD50, LC50, EC50, and IC50, used to explain the degree by which something is toxic, they are an immense oversimplification.

When we say a substance is "toxic," there’s usually a lot of follow-up questions. Is it toxic at any dose? Only above a certain threshold? Is it toxic for everyone, or just for certain susceptible individuals (as we’ll discuss later)? The relationship between dose and toxicity is not always linear, and can vary depending on the route of exposure, the duration of exposure, and indi...
Introduction and Exploration of AI Ethics Through a Global Lens
by ThePathYouWillChoose — 2024-08-19T17:11:17.731Z

Hello LessWrong Community,

My name is Fred, and I am deeply engaged in exploring the ethical issues surrounding Artificial Intelligence (AI) and its societal impacts on a global scale. Recently, I have been working on a novel that delves into these themes, examining how emerging technologies intersect with cultural diversity, global governance, and ethical dilemmas.

In my novel, I explore key questions such as:

How can we reconcile different cultural perspectives on technological ethics in a world where AI plays an increasingly central role?
What are the potential risks of the misuse of AI technologies, and how can we prevent them?
How can we build inclusive global governance structures that respect local values and traditions while being effective on a global scale?

Beyond the novel, I have reached out to various international organizations, researchers, and activists to foster a broader dialogue on these crucial topics. I believe that the intersection of AI and ethics is a critical area that requires collaborative efforts from a diverse range of perspectives.

I am particularly interested in discussing:

The challenges of aligning AI systems with human values across different cultures.
The role of AI in exacerbating or mitigating social inequalities.
Potential strategies for creating a global framework for ethical AI governance.

If this novel is well received by the community, I would like to make it freely available to raise awareness about the ethical challenges these emerging technologies present. I would love to engage with this community to further explore these ideas, share insights, and collaborate on possible solutions. I look forward to your thoughts, feedback, and any resources you might recommend.

Thank you for welcoming me into this vibrant community.

Best regards,

Fred
Trustworthy and untrustworthy models
by Olli Järviniemi — 2024-08-19T16:27:11.088Z

In this post, I’ll talk about trustworthiness, a concept similar to scheming and deceptive alignment. I prefer this concept, as I find it A) is directly important in practice, B) more readily distinguishes between capability for scheming and in fact scheming, and C) is clearer how to evaluate for. I’ll also distinguish between different types of untrustworthy models - active planners, sleeper agents and opportunists - that are often conflated.

I think when people talk about evaluating models for something like scheming, they’d often be better off talking about trustworthiness instead, and I think it’s overall a more applicable frame for designing evaluations and decision making.

Many of the ideas here are from Buck Shlegeris and Ryan Greenblatt, thanks to them and Cody Rushing for feedback.

Suppose an AI company has developed a new model, and wants to use it to perform large quantities of ML research (internal deployment). However, the company is worried about the model being deceptively aligned[1] and trying to take catastrophic actions, such as

hacking the data center

or copying its weights to a remote server. What can the company do?

This is a textbook example of the type of problem

AI control

tries to solve, and a basic strategy looks like this: Every time you are using the new model and it takes some action, use another model to check the action is OK before letting it be executed. If the action is not OK, it's not executed and humans are alerted about suspicious activity.

(If you have never heard of AI control before, you might want to try playing the

Bash Control game

to get familiar with the concept. In the game, your aim is to download and execute a malicious program without GPT-3.5 catching you.)

Let's call our characters Smart and Weak: Smart is the new model doing ML research while maybe trying to escape, while Weak is a model trained to monitor Smart. The reason for these names will become apparent soon.

There are many ways how this control strategy could fail. Here are some:[2]

Smart came up with an extremely subtle way to escape that Weak is just too dumb to notice.
Smart successfully jailbreaks (via e.g. a suffix attack) Weak to not alert humans about its escape attempt.

Or: Weak happens to have a backdoor, which Smart finds and exploits.

Smart successfully convinces Weak (via natural language) that its actions ar

...
Apartment Price Map Discontinuity
by jefftk — 2024-08-19T15:30:05.386Z

I maintain a

Boston
apartment price map, scraping

Padmapper's listings to make a
heatmap. Each month I run a script to update it, but last month it
failed due to an API change. It had been impressively stable: I'd
been running the same code unmodified for

three
years, with only minor modifications since a big rewrite in
2016.

The changes to get it working again this time were relatively
small. When I look at the most recent results, however, I don't
know whether to interpret the decline in my only post-fix data point
as real:

Some possibilities:

Prices went down (yay!)

My scraping was wrong before but is fixed now (ok)

My scraping was right before and is wrong now (not good!)

Looking at median prices for each size apartment, relative pricing
seems pretty similar.

My best guess, looking over what's changed in the PadMapper UI, is
that previously they weren't breaking out and excluding "short term"
rentals, with a minimum stay of one month:

These tend to be more expensive than standard rentals, and aren't what
I'm trying to track. If that is the issue then I think this probably
represents fixing a bug and not a real decline.

(These charts are updates of the ones I made last
time; read there for more details.)

Comment via: facebook, mastodon
Will we ever run out of new jobs?
by Kevin Kohler — 2024-08-19T15:04:03.849Z
Link: https://machinocene.substack.com/p/will-we-ever-run-out-of-new-jobs

A lot of the debate on long-term, structural technological unemployment can be summarized in three short statements:

Concerns about the speed or scope of labor substitution have often been premature or exaggerated in the past.
Labor substitution has been very positive for humanity so far. As many old tasks have been automated, human labor has moved into many new, previously non-existing tasks.
The long-term question that decides structural technological unemployment is whether human labor can keep moving into new tasks.

Experts disagree on whether human labor can keep moving to new tasks indefinitely or not. In this blog post I will suggest a clear answer:

Humans will run out of new tasks to move to when AGI surpasses humans in fluid general intelligence. Fluid general intelligence is the ability to reason, solve novel problems, and think abstractly, independent of acquired knowledge or experience. If and when AGI reaches this, it will be better at learning novel tasks than humans, and the interval between a new task appearing in the economy and its automation falls to zero.

Current AI models still have modest levels of fluid intelligence and there is no consensus timeline on AGI with strong fluid intelligence. Still, even if it may be difficult to agree on specific timelines, this underlines that the idea that we could eventually run out of new jobs to shift to should be taken seriously.

1. Automation anxiety is not novel

As early as 1948 Norbert Wiener warned that “(...) the first industrial revolution, the revolution of the ‘dark satanic mills’, was the devaluation of the human arm by the competition of machinery. (...) The modern industrial revolution is similarly bound to devalue the human brain, at least in its simpler and more routine decisions. (...) taking the second revolution as accomplished, the average human being of mediocre attainments or less has nothing to sell that it is worth anyone's money to buy.”[1]
Similarly, the US Congress held hearings on Automation and Technological Change as early as 1955, with some worrying that technology could “produce an unemployment situation, in comparison with which the depression of the thirties will seem a pleasant joke."
More recently, the 2013 Oxford study by Carl Benedikt Frey and Michael Osborne, "The Future of Employment," estimated that up to 47% of U.S. jobs were at risk of automation within a de

...
What are the best resources for building gears-level models of how governments actually work?
by adamShimi — 2024-08-19T14:05:02.590Z

One big hole in my set of frames and models of the world is that I don't understand how government works. I don't understand how the government of my native country (France) works, I don't understand how the government of the country I live in (UK) works, I don't understand how the governments of countries that matter for international coordination (US, China, Russia, Europe (not a country, but relevant player)) work.

I would like to fix this, both for my own understanding and because that sounds quite relevant to my own decisions about prioritizing various kind of works on preventing AI risks.

So I'm looking for resources that actually explain and distill:

How various government works in details
How the laws are conceived, passed, discussed
How international treaties are negotiated and applied
What are the influences of the general opinion, the press, lobbying, all these different actors.

I'm aware that these technically fall into very different fields of studies, and jobs, and topics, but I'm hoping that there might be, somewhere, a synthesis.

To give a vibe of the ideal resource, it would be something like what Steve Byrnes would write if he was distilling political science and law making and international treaties instead of neuroscience.

I don't expect to get that sadly, but still curious to see if there are obvious resources that I couldn't find on quick google and asking around.

(Note: I'm really, really not interested in resources that try to make a point about some people being bad, some form of government being worse, all that stuff. Not saying it's not potentially important in general, just not useful for the kind of model I want to build. I'm much more interested in good descriptive compressions than in normative arguments)
[Cross-post] Book Review: Bureaucracy, by James Q Wilson
by davekasten — 2024-08-19T13:57:10.872Z

[Cross-posted from my substack, davekasten.substack.com. {I never said that I was creative at naming things}]

Hi,

So here are some things I’ve been thinking about while traveling on policy consulting business over the past few months.

Bureaucracy, by James Q. Wilson

As we’ve discussed

previously

, a decent description of the goal of bureaucracy is that it’s the organizational form that society uses to turn nonstandard inputs into standardized outputs. Put another way, the original bureaucrats were the 19th Century’s version of software engineers, abstracting away complexity and turning it into legible forms through, well, forms.[1]

And there’s a stereotype that

just like software

, bureaucracy wants to have feature-creep, wants to get bloated; growing its power through new regulations, new budgetary lines, new staffing.

This assumption has motivated, generally, the bipartisan assumptions of American politics since (at least) when Clinton declared that “

the era of big government is over

” in 1996 and promised “smaller, less bureaucratic government in Washington.”

The messaging syllogism roughly is:

Government is bad;
Government means bureaucracy;
Therefore the more government you have, the more bureaucracy you have;
So the only way to get rid of bad bureaucracy is to have less government.

(And all of this goes double for tech policy! After all, we don’t want to kill the libertarian geese that lay the stock market eggs.)

In Bureaucracy (

Amazon affiliate link

), James Q. Wilson basically takes 378 pages to, implicitly, call all that bullshit.

On the surface level, Wilson has one project: pushing back against the framework. He argues that bureaucracies are such a “complex and varied phenomenon” (p. 10) that not only do they radically differ even within the American constitutional context, but that therefore it’s not even feasible for a political scientist to come up with a theory to explain that variation. So…he’s just gonna do some descriptive stuff; his project is “to explain why government agencies — bureaucracies — behave as they do.” (p. ix.) Lots of great chapters follow describing how bureaucracies vary and differ

But…I don’t think that’s his real point. I think his real point is this: bureaucracies don’t actually want to accumulate power. Bureaucracies want to keep on doing what they are already doing, and usually, that’s not optimal accumulation of power.

It’s someth...