trentmkelly/LessWrong-43k huggingface dataset:
Prototyping Pluck Sensors
by jefftk — 2024-03-23T01:30:04.712Z

Harp guitars
are a neat weird instrument that never really took off:

You play it like a normal guitar, with six fretted strings, but there
are also some number of extra harp strings you can pluck.

I see four main downsides:

They're bulky and hard to transport.

You need to tune the strings to match the song.

You don't have many note choices.

Guitars are worse than mandolins.

We can fix all of these: an electronic harp mandolin! It wouldn't
need to be much bigger than a regular mandolin, it's electronic so
easy to switch between tunings, and narrow sensors can let us have two
axes to choose notes from instead of one.

I had previously been thinking
about simple switches, but it really would be nice to have
something velocity-sensitive.

Here's the idea for the pluck sensor:

The circle is a cheap little piezo
disc, the same kind I used in my stomp sensors. I was thinking of maybe
3D printing the angled bit, but was worried about durability. Instead
I went to the hardware store and wandered around until I found a 1/16"
aluminum offset angle:

The cross section is an 'L', 3/4" on one side and 1/2" on the other.
I used a hacksaw to cut it down to the width of one of my piezo discs,
and filed the rough bits and sharp corners. I epoxied the disc onto
the angle, and put extra epoxy on top to protect the solder joints:

Then I cut rectangles of sorbothane I had left over from a previous project into rectangles, and used
a cyanoacrylate
adhesive to attach them to the metal angle and the base plate:

After letting it dry for a day I hooked up some old 1/4" cables and
plugged it into an audio interface for testing:

Here's what I get for the two signals when plucking one of the two
teeth with a 0.6mm
pick:

This is really good news! There's a clear sharp peak on the channel
corresponding to the tooth I plucked, and a tiny signal on the tooth I
didn't pluck. The one I plucked is also nicely damped, so I think I
won't need to worry about distinguishing multiple plucks in short
succession vs continued vibration. You can see the 60hz hum of the
power grid as well, but it's nice and small relative to the signal.

One very interesting thing is I get clearly different signals when
plucking downward vs upward:

Another pices of good news! This lets it be bisonoric, like a diatonic
butto...
Dangers of Closed-Loop AI
by Gordon Seidoh Worley — 2024-03-22T23:52:22.010Z

In control theory, an open-loop (or non-feedback) system is one where inputs are independent of outputs. A closed-loop (or feedback) system is one where outputs are input back into the system.

In theory, open-loop systems exist. In reality, no system is truly open-loop because systems are embedded in the physical world where isolation of inputs from outputs cannot be guaranteed. Yet in practice we can build systems that are effectively open-loop by making them ignore weak and unexpected input signals.

Open-loop systems execute plans, but they definitionally can't change their plans based on the results of their actions. An open-loop system can be designed or trained to be good at achieving a goal, but it can't actually do any optimization itself. This ensures that some other system, like a human, must be in the loop to make it better at achieving its goals.

A closed-loop system has the potential to self-optimize because it can observe how effective its actions are and change its behavior based on those observations. For example, an open-loop paperclip-making-machine can't make itself better at making paperclips if it notices it's not producing as many paperclips as possible. A closed-loop paperclip-making-machine can, assuming its designed with circuits that allow it to respond to the feedback in a useful way.

AIs are control systems, and thus can be either open- or close-loop. I posit that open-loop AIs are less likely to pose an existential threat than closed-loop AIs. Why? Because open-loop AIs require someone to make them better, and that creates an opportunity for a human to apply judgement based on what they care about. For comparison, a nuclear dead hand device is potentially much more dangerous than a nuclear response system where a human must make the final decision to launch.

This suggests a simple policy to reduce existential risks from AI: restrict the creation of closed-loop AI. That is, restrict the right to produce AI that can modify its behavior (e.g. self-improve) without going through a training process with a human in the loop.

There are several obvious problems with this proposal:

No system is truly open-loop.
A closed-loop system can easily be created by combining 2 or more open-loop systems into a single system.
Systems may look like they are open-loop at one level of abstraction but really be closed-loop at another (e.g. an LLM that doesn't mod

...
Why The Insects Scream
by Bentham's Bulldog — 2024-03-22T19:47:26.302Z

Crossposted on my blog.

1 Beneath the giants’ heels

The giants ruled the world and

They reshaped it a lot

Nature warped beneath their hands

Vast palaces were built, but not

Much heed was paid to those below

Those trampled by their feet

They didn’t really care to know

What those tiny beings could be

One day some giant scientists performed some basic tests

On those tiny little creatures, who got crushed and squashed and bled

It turned out those little guys, by the word human, that were addressed

Could feel, and laugh, and love, and yet their blood was flowing red

A few especially caring giants wondered what to do

For these beings that they killed and maimed

Were, as they said, “just like me and you.”

And were screaming out in pain

They hoped and prayed that this would change

The way that things were done

That they could mitigate the insane

Destruction wrought upon the tiny screaming ones

They asked those advocating change for giants ill-treated

If they’d support reshaping

The practices done to these creatures

That they were mercilessly razing

But those progressive giants shook their head

And in their heels dug

For though these creatures became dead

Nothing should be done

And though the science did suggest

That they could feel all

Could morality need one acquiesce

To the demands of one so small

Meanwhile, many miles below, the little humans screamed

They wished that they’d have respite from

The giant torturous fiends

—A poem broadly inspired by

this article

.

2 Reasons to care

Soon, trillions of insects will starve.

This article is about the mistreatment of insects. I write this knowing that virtually no one cares about insects being mistreated, however badly. I am going to try very hard to get you to care about these creatures that matter, that cry out in agony by the trillions. Yet I recognize most of you will continue to be indifferent.

Caring about insects is treated as so manifestly absurd that people reject any view implying we should do it, however otherwise plausible. Bryan Caplan thinks we shouldn’t care about animals because if we do, we’d have to care about insects, and that’s clearly crazy (

his argument is wrong

).

And yet the evidence is mounting that

insects can feel pain

, just as non-insects can. The

most detailed report

on insect sentience concludes that the ones I discuss in this article, called black soldier flies, feel pain around 1.3% as intently as us. This means that a ...
What does "autodidact" mean?
by bhauth — 2024-03-22T18:37:50.516Z

Alice takes a chemistry class at a university. She gets a professor who basically just reads from the textbook during lectures. She reads the textbook on her own, talks to her classmates, and finds some relevant Wikipedia articles and youtube videos.

Bob studies chemistry on his own. He buys the textbook Alice uses because it's popular, reads it on his own, talks to other people interested in chemistry, and finds some relevant Wikipedia articles and youtube videos.

Bob is an autodidact. Alice is not.

OK, I understand that, but what's the key difference? What is the essence of autodidact-ness? Is it...

The mere involvement of a "legitimate" institution, even if it makes no real difference to the individual's learning experience?

Some essential difference in the experience that Alice and Bob have while learning?

Something different about the personal character of Alice and Bob?

I don't think there's a clear consensus, and I don't think it describes a clear distinction, and that's why I don't normally use the word "autodidact".
[Linkpost] Vague Verbiage in Forecasting
by trevor — 2024-03-22T18:05:53.902Z
Link: https://goodjudgment.com/vague-verbiage-forecasting/

“What does a ‘fair chance’ mean?”

It is a question posed to a diverse group of professionals—financial advisers, political analysts, investors, journalists—during one of Good Judgment Inc’s virtual workshops. The participants have joined the session from North America, the EU, and the Middle East. They are about to get intensive hands-on training to become better forecasters. Good Judgment’s Senior Vice President Marc Koehler, a Superforecaster and former diplomat, leads the workshop. He takes the participants back to 1961. The young President John F. Kennedy asks his Joint Chiefs of Staff whether a CIA plan to topple the Castro government in Cuba would be successful. They tell the president the plan has a “fair chance” of success.

The workshop participants are now asked to enter a value between 0 and 100—what do they think is the probability of success of a “fair chance”?

When they compare their numbers, the results are striking. Their answers range from 15% to 75% with the median value of 60%.

Figure 1. Meanings behind vague verbiage according to a Good Judgment poll. Source: Good Judgment.

It sure would be nice if we could get one of these with the numbers based on the actual results, rather than the subject's impressions of the numbers. You'd need a lot of data from a wide variety of people, and it would need to cover a pretty diverse variety of events.

The story of the 1961 Bay of Pigs invasion is recounted in Good Judgment co-founder Philip Tetlock’s Superforecasting: The Art and Science of Prediction (co-authored with Dan Gardner). The advisor who wrote the words “fair chance,” the story goes, later said what he had in mind was only a 25% chance of success. But like many of the participants in the Good Judgment workshop some 60 years later, President Kennedy took the phrase to imply a more positive assessment of success. By using vague verbiage instead of precise probabilities, the analysts failed to communicate their true evaluation to the president. The rest is history: The Bay of Pigs plan he approved ended in failure and loss of life.

Vague verbiage is pernicious in multiple ways.

1. Language is open to interpretations. Numbers are not.

According to research published in the Journal of Experimental Psychology, “maybe” ranges from 22% to 89%, meaning radically different things to different people under different circumstances. Survey research by Good Judgme

...
Wolf and Rabbit
by Richard Henage — 2024-03-22T17:20:19.265Z

Rabbit said to Wolf, "I shall make a new animal. It shall stand on two legs and run free across meadows and mountains."

Wolf said, "I will send large beasts to eat your new animal."

Rabbit said, "My animal shall band together and fight off even large beasts. They shall travel together to stay safe."

Wolf said, "I will weaken the bones of your animal over time, so that the old cannot walk, and are left behind."

Rabbit said, "They shall settle many homes close together and tame the plants to eat."

Wolf said, "I will send sickness that travels between their homes and kills them."

Rabbit said, "They shall learn from their parents how to stay safe from the sickness. They shall think of many things and work together to create them."

Wolf said, "I will send evil thoughts, and they will use their knowledge only to kill each other."

Rabbit said, "They shall speak wisdom and love each other and live in peace, making beautiful things."

Wolf said, "I will make their lives short and everything they love will wither and die."

Rabbit said, "They shall learn the secrets of nature, and make their lives longer than the crumbling of the world."

Wolf laughed and said, "Then you have created a God, and I will bow to it."

Rabbit agreed that this was fair and set to work.
AI Model Registries: A Regulatory Review
by Deric Cheng — 2024-03-22T16:04:15.295Z

This article is the third in a series of ~10 posts comprising a 2024 State of the AI Regulatory Landscape Review, conducted by the

Governance Recommendations Research Program

at

Convergence Analysis

. Each post will cover a specific domain of AI governance (e.g.

incident reporting

,

safety evals

, model registries, etc.). We’ll provide an overview of existing regulations, focusing on the US, EU, and China as the leading governmental bodies currently developing AI legislation. Additionally, we’ll discuss the relevant context behind each domain and conduct a short analysis.

This series is intended to be a primer for policymakers, researchers, and individuals seeking to develop a high-level overview of the current AI governance space. We’ll publish individual posts on our website and release a comprehensive report at the end of this series.

What are model registries? Why do they matter?

Note: The phrase “model registry” may also often be used to refer to a (typically)

private database of trained ML models

, often used as a version control system for developers to compare different training runs. This is a separate topic from model registries for AI governance.

Model registries, in the context of AI regulation, are centralized governance databases of AI models intended to track and monitor AI systems usually in real-world use. These registries typically mandate the submission of a new algorithm or AI model to a governmental body prior to public release.

Such registries will usually require basic information about each model, such as their purpose or primary functions, their computational size, and features of their underlying algorithms. In certain cases, they may request more detailed information, such as the model’s performance under particular benchmarks, a description of potential risks or hazards that could be caused by the model, or safety assessments designed to prove that the model will not cause harm.

Model registries allow governmental bodies to keep track of the AI industry, providing an overview of key models currently available to the public. Such registries also function as a foundational tool for AI governance - enabling future legislation targeted at specific AI models.

These registries adhere to the governance model of “algorithms as an entry point”, allowing governments to focus their regulations on individual alg...
Video and transcript of presentation on Scheming AIs
by Joe Carlsmith — 2024-03-22T15:52:03.311Z

(Cross-posted from my website.)

This is the video and transcript for a ~45-minutes talk I gave in February 2024 about my report “

Scheming AIs: Will AIs fake alignment during training in order to get power?

” (slides available here). See also

this podcast

for a more conversational overview of similar content.

Main talk

Okay, hi everybody. Thanks for coming. So this talk is about whether advanced AIs will fake alignment during training in order to get power later. This is a behavior that I'm calling scheming, it's also often called deceptive alignment.

Preliminaries

So I'm going to start with a few preliminaries.

As was mentioned, this talk is based on a public report called

Scheming AIs available on arXiv

. There's also an

audio version

on my podcast, Joe Carlsmith Audio, if you prefer that. So I encourage you to check that out. I'm going to try to cover many of the main points here, but the report is pretty long and so I'm not going to go into that much depth on any given point. So if you want more depth or you have burning objections or clarifications or you want to work on it, I encourage you to check out the full report.

Second, I'm going to assume familiarity with:

The basic argument for existential risk from misaligned AI. That is roughly the thought that advanced AI agents with goals that conflict with human goals would have instrumental incentives to seek power over humans, and potentially to disempower humanity entirely, an event I'll call takeover, AI takeover. If that story didn't sound familiar to you, I do have some other work on this topic which I would encourage you to check out. But some of the talk itself might make a little less sense, so I apologize for that.
I’m also going to assume familiarity with a basic picture of how contemporary machine learning works. So very roughly, imagine models with lots of parameters that are being updated via stochastic gradient descent (SGD), such that they perform better according to some feedback signal. And in particular, I'm often going to be imagining a default baseline training process that resembles somewhat what happens with current language models. Namely, a pre-training phase in which a model is trained on some combination of internet text and potentially other data. And then a fine-tuning phase, in which it's trained via some combination of maybe imitation learning

...
Benchmarking LLM Agents on Kaggle Competitions
by aog — 2024-03-22T13:09:04.757Z

tl;dr: I prompted ChatGPT to participate in a Kaggle data science competition. It successfully wrote scripts that trained models to predict housing prices, and ultimately outperformed 71% of human participants.

I'm not planning to build a benchmark using Kaggle competitions, but I think a well-executed version could be comparable to e.g. SWE-Bench. This is a proof-of-concept to inform others who might want to work on building that benchmark.

Motivation

Someone once said that benchmarking LLM agents on consequential real-world tasks could help the world understand and anticipate the advance of AI capabilities. Data science is a consequential real-world task that's also near and dear to my heart, as I spent a few years as a data scientist. So, could ChatGPT automate my old job?

To answer this question, I turned to Kaggle. Kaggle is a website that hosts online data science competitions. They provide instructions and a training dataset, and participants train models to make predictions on a test set. Based on test set performance, Kaggle ranks everyone on a public leaderboard.

Kaggle has a couple of great features for an LLM agent benchmark:

These competitions encapsulate a major part of the job of a data scientist. There are plenty of other parts of the job (gathering data, talking to human beings, making pretty slides to convince the executives to believe you, etc.). But training models to maximize performance on a given dataset is one of the key skills that separates data science from every other laptop job.
There are tons of competitions. (I can't find an exact number, but they have more than 300,000 datasets.) You could build a benchmark with thousands of Kaggle competitions without writing any new examples by hand.
Kaggle’s leaderboards provide a range of human performance estimates. The tails of the distribution likely include people who are cheating or not even trying, but most of the distribution is realistic human performance.

Arguments against building a benchmark using Kaggle competitions include:

It's somewhat obvious and someone else might do it. But, to the best of my knowledge, nobody has done it yet. You'd think benchmarking LLM agents on GitHub issues would be obvious, but again, few papers have done so.
Low ceiling of performance. I've already shown that ChatGPT can outperform 71% of human parti

...
American Acceleration vs Development
by Maxwell Tabarrok — 2024-03-22T13:01:50.365Z
Link: https://www.maximum-progress.com/p/american-acceleration-vs-development

Progress studies and metascience focus on reforms which will accelerate growth in the US. The US is a small portion of the world’s population and these few are already the world’s most prosperous. Development economics and effective altruism put more effort into

health and growth interventions

in the developing world.

There is enough

mistargeted philanthropy and state capacity

to increase the resources committed to both progress studies and EA, so we don’t have to pick just one. But there are always tradeoffs eventually and the comparison is interesting.

Global health interventions

and

catchup growth in the developing world

have precisely measured huge benefits. So what justifies the focus on getting the rich richer by accelerating American innovation when there are others who are far more in need?

Impact

The local benefits of accelerating growth in America are large. Tens of millions of people would directly benefit from cheaper rent and higher wages if we

deregulated housing

in American cities, for example. These large benefits go to people who already have lots of resources, though, so dollar-for-dollar they aren’t as impactful as cash transfers to the developing world. But that’s just the local benefits. Increasing American growth has massive positive externalities on the rest of the world.

Even small accelerations in frontier growth compounds into large amounts of wealth and resources. Profit seeking investment sends large quantities of this wealth to the developing world. More American growth means more of this investment. From 1982 to 2022 the amount of

direct investment from the US to other countries

increased by ~30x from 207 billion to 6.5 trillion. The amount of direct investment from the US to India increased 170x over the same period. This is mostly due to the fraction of foreign investment flowing from a larger pool of wealth but

investment outflows as a percent of GDP

also grew from .2 to 1.7%.

In addition to profit seeking investment, American growth is the largest source of funding for

philanthropy

and

international aid

. Frontier growth pays for

Open Philanthropy

, the

World Bank

, the

IMF

, and the

UN

. If these philanthropic contributions are a constant fraction of GDP, then increasing the US growth rate from 2% to 3% would double the amount that Americans commit to these efforts in 20 years.

Reforms which increase growth and innovation in the ...