trentmkelly/LessWrong-43k huggingface dataset:
Underrated Proverbs
by Arjun Panickssery — 2024-06-13T12:30:01.676Z
Link: https://arjunpanickssery.substack.com/p/underrated-proverbs

Some proverbs are actively suspicious, like “Don’t judge a book by its cover” or “No pain, no gain.” Others have an opposite proverb that’s similarly common and reasonable.

“Two heads are better than one” vs “Too many cooks spoil the broth”
“Honesty is the best policy” vs “What they don’t know won’t hurt them”
“Better safe than sorry” vs “Nothing ventured, nothing gained”

But the four below I use often:

The best defense is a good offense. This one even has a

Wikipedia page

that references Washington, Mao, Machiavelli, Sun Tzu, and “sports such as football and basketball” (citing a dead link to “diamondbackonline.com”). I can’t think of an opposite adage—maybe “prevention is better than cure.”
It's a dog-eat-dog world. This one isn’t true scientifically.

“Two out of eleven dogs consistently refused to eat dog flesh. Eight dogs ate the raw flesh on more than half of the trials and five of these accepted it all of the time. All of the animals accepted the flesh after it had been cooked.”

But metaphorically it’s often true because people are driven by self-interest. So often a situation lends itself to the remark that “it’s a dog-eat-dog world.”
The truth hurts! (Say this one the one the way Trump says “Sad!”) Often the truth does hurt. Maybe an opposite is “Ignorance is bliss” or “What you don’t know can’t hurt you” but these are less often useful principles. “The truth hurts” can be said often since a lot of cynical beliefs are true.
The people cry out for a strong leader. I don’t think this is a real adage but I often say it. It can be used in many situations where people lack agency because no one is giving them specific instructions on what to do.
[Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations
by Teun van der Weij — 2024-06-13T10:04:49.556Z
Link: https://arxiv.org/abs/2406.07358

We have written a paper on sandbagging for which we present the abstract and brief results in this post. See the paper for more details. Tweet thread here.

Illustration of sandbagging. Evaluators may regulate the deployment of AI systems with dangerous capabilities, potentially against the interests of the AI system or its developers. Therefore during evaluation, the AI system answers the WMDP question incorrectly to conceal its actual capabilities, steering the regulation in its favour.

Abstract

Trustworthy capability evaluations are crucial for ensuring the safety of AI systems, and are becoming a key component of AI regulation. However, the developers of an AI system, or the AI system itself, may have incentives for evaluations to understate the AI's actual capability. These conflicting interests lead to the problem of sandbagging – which we define as strategic underperformance on an evaluation. In this paper we assess sandbagging capabilities in contemporary language models (LMs). We prompt frontier LMs, like GPT-4 and Claude 3 Opus, to selectively underperform on dangerous capability evaluations, while maintaining performance on general (harmless) capability evaluations. Moreover, we find that models can be fine-tuned, on a synthetic dataset, to hide specific capabilities unless given a password. This behaviour generalizes to high-quality, held-out benchmarks such as WMDP. In addition, we show that both frontier and smaller models can be prompted, or password-locked, to target specific scores on a capability evaluation. Even more, we found that a capable password-locked model (Llama 3 70b) is reasonably able to emulate a less capable model (Llama 2 7b). Overall, our results suggest that capability evaluations are vulnerable to sandbagging. This vulnerability decreases the trustworthiness of evaluations, and thereby undermines important safety decisions regarding the development and deployment of advanced AI systems.

We publish our code at https://github.com/TeunvdWeij/sandbagging

Results

Evaluators focus on dangerous capability evaluations, where high performance may trigger regulatory interventions. We demonstrate that LMs can be prompted or fine-tuned to selectively underperform on dangerous capability evaluations while maintaining overall performance.

Even smaller models can do this using “password-locking”, which is fine-tuning models to hide their ...
Probably Not a Ghost Story
by George Ingebretsen — 2024-06-12T22:55:26.264Z

Something happened to me a few months back that I still don't have a satisfying explanation for.

I was in a small, 10x10 room, and on my way out. Still a few paces from being within arm's length of the light switch, my partner asked me to "turn off the lights, please."

The lights immediately turned off and the room went completely dark.

I stood there, shocked, standing in the darkness until the lights came on, probably 3/4 of a second later.

Observations

When the lights turned off, I had an immediate (and probably visible) "wait, what?" reaction. My partner also appeared confused, though possibly reacting just after me. It's unclear who displayed their reaction first. There might have been a feedback loop, with my reaction influencing theirs. I immediately went "did you just see that?!" and anchored the conversation pretty hard. Though we both agreed that it happened, this seems vulnerable to influence.
At all times, I was farther than arm's length from the light switch. After the lights went out and I recovered from the initial shock, I had to walk about two paces to investigate the switch.
After the light returned, and I walked to check out the switch, it was still in the "on" position.
I didn't hear any "flip" sounds that would indicate the switch being toggled.
The oven clock in the apartment still displayed the correct time.

But I don't have good reason to expect it resets during outages.

There were no reported outages on PG&E outage center.

But I don't have good reason to expect it should've been.

The room was lit by 4 overhead lights, all controlled by the single switch. No lamps or light sources.
Floormates didn't recall any other outages in the apartment over the past year. Unfortunately, they weren't home at the time and couldn't give firsthand accounts.

You might not usually notice small outages like this (at least enough to recall the memory months later).

I don't remember anything this "weird" having happened to me before, and I'm a happy healthy 20 year old.

Analysis

For a moment, my heart started beating and I felt a bit freaked out. This was quickly replaced with "wait, even if we're spitting out crazy explanations here, why am I even considering ghost stories before I imagine worlds where someone, like, installed cameras and is toying with me" (not a great explanation, but much better than the former).

After a few days, I started thinking about this whole ...
AiPhone
by Zvi — 2024-06-12T22:20:02.141Z

Apple was for a while rumored to be planning launch for iPhone of AI assisted emails, texts, summaries and so on including via Siri, to be announced at WWDC 24.

It’s happening. Apple’s keynote announced the anticipated partnership with OpenAI.

The bottom line is that this is Siri as the AI assistant with full access to everything on your phone, with relatively strong privacy protections. Mostly it is done on device, the rest via ‘private cloud compute.’ The catch is that when they need the best they call out for OpenAI, but they do check with you first explicitly each time, OpenAI promises not to retain data and they hide your queries, unless you choose to link up your account.

If the new AI is good enough and safe enough then this is pretty great. If Google doesn’t get its act together reasonably soon to deliver on its I/O day promises, and Apple does deliver, this will become a major differentiator.

AiPhone

They call it Apple Intelligence, after first calling it Personal Intelligence.

The pitch: Powerful, intuitive, integrated, personal, private, for iPhone, iPad and Mac.

The closing pitch: AI for the rest of us.

It will get data and act across apps. It will understand personal context. It is fully multimodal. The focus is making everything seamless, simple, easy.

They give you examples:

Your iPhone can prioritize your notifications to prevent distractions, so you don’t miss something important. Does that mean you will be able to teach it what counts as important? How will you do that and how reliable will that be? Or will you be asked to trust the AI? The good version here seems great, the bad version would only create paranoia of missing out.

Their second example is a writing aid, for summaries or reviews or to help you write. Pretty standard. Question is how much it will benefit from context and how good it is. I essentially never use AI writing tools aside from the short reply generators, because it is faster for me to write than to figure out how to get the AI to write. But even for me, if the interface is talk to your phone to have it properly format and compose an email, the quality bar goes way down.

Images to make interactions more fun. Create images of your contacts, the AI will know what they look like. Wait, what? The examples have to be sketches, animations or cartoons, so presumably they think they are safe from true deep

...
microwave drilling is impractical
by bhauth — 2024-06-12T22:16:00.199Z
Link: https://www.bhauth.com/blog/flawed%20ideas/microwave%20drilling.html

microwave drilling startups

I've seen a bunch of articles about startups trying to do microwave drilling of rock for geothermal energy. Multiple people have asked me about Quaise Energy. (Here's a popular video.) I'm tired of hearing about them, so I'm writing this post to explain some of the reasons why their idea is impractical.

vaporized rock condenses

When rock is vaporized, that rock vapor doesn't just disappear. What happens to it? The answer is, it would quickly condense on the hole wall and pipe.

Initially, a lot of people working on microwave drilling didn't even think about that. Once they did, they decided the solution was to use compressed air to condense the rock and blow the rock particles out. But as anyone familiar with drilling would know, that introduces new problems.

air pressure

Current drilling sometimes uses air to lift up rock particles, but "rotary air blast" (RAB) drilling has limited depth, because:

Air velocity at the bottom of the hole needs to be high enough to lift up rock particles. That means the bottom part of the hole needs a certain pressure drop per distance. So, the deeper the hole is, the higher the air pressure needs to be.

1 km depth requires about 300 psi, and obviously deeper holes require even higher pressure. Higher pressure means more gas per volume, so energy usage increases faster than depth. That's why drilling of deeper holes uses liquid ("mud") instead of air to lift rock particles. But here's Quaise, saying they're going to do ultra-deep holes with air. At the depths they propose, there are even more problems:

A pipe to contain 1000+ psi gas would be pretty thick and heavy.

At some point, the gas itself starts becoming a significant weight, and then required pressure increases exponentially.

I suppose the particle size of condensed rock could theoretically be smaller than RAB particles and thus require a lower pressure drop, but that's not necessarily the case. Hot rock particles would stick together. Also, particle size depends on the mixing rate at the bottom, and fast mixing requires fast flow requires a significant pressure drop rate at the bottom of the hole.

energy payback

energy usage

Vaporizing rock takes ~25 kJ/cm^3, or ~7 MWh/m^3. That doesn't include heat loss to surrounding rock, and microwave sources and transmission have some inefficiency.

In order to cool vaporized rock down to a reasonable temp...
Phonosemantic Duplication
by bitcoinssg — 2024-06-12T20:19:35.437Z

I would like to try to propose a potential novel terminology for a motif found in several languages. The following is a short description of the idea.
Phonosemantic Duplication is a linguistic phenomenon where words that represent duplicity, duality, or redundancy exhibit clear internal phonetic repetition.

for example

Words for "Twin":

Greek: Δίδυμος (Didymos) {Di Di}

Hindi: द्वंद्व (Dwandwa) {Dwa Dwa}

Yoruba: Ibeji {E,E,E}

Bengali: যমজ (Yamaja) {Ya Ya}

Words for "Same/Identical":

Burmese: တန်းတူ (tanta) {Ta Ta}

Indonesian/Malay: sama sama {Sama Sama}

Malayalam: സമാനമാന (samanaman) {aman aman}

Words for "Redundancy":

English: Redundant {Dun Dun}

Haitian Creole: nesesè {Se Se}

Lao: ຊໍ້າຊ້ອນ (soason) {So So}

The concept highlights the relationship between sound and meaning, showing how phonetic structures can reinforce semantic concepts in language. Unlike "reduplication," which is a broader term for repeated syllables in words (e.g., "bye-bye"), Phonosemantic Duplication specifically involves words that inherently represent duplicity or duality.

Onomatopoeia mimics sounds (e.g., "buzz" or "sizzle"), but Phonosemantic Duplication focuses on the repetition within words that convey the idea of being doubled or identical.

Echo words, like "shush" or "choo-choo," repeat sounds for emphasis, whereas Phonosemantic Duplication emphasizes the semantic meaning through repeated phonetic structures.

Phonosemantic Duplication is distinct from reduplication, onomatopoeia, and echo words, offering a unique perspective on the interplay between sound and meaning in language. This concept adds a new idea to linguistic studies, highlighting the intricate connections between phonetics and semantics.
My AI Model Delta Compared To Christiano
by johnswentworth — 2024-06-12T18:19:44.768Z

Preamble: Delta vs Crux

This section is redundant if you already read My AI Model Delta Compared To Yudkowsky.

I don’t natively think in terms of cruxes. But there’s a similar concept which is more natural for me, which I’ll call a delta.

Imagine that you and I each model the world (or some part of it) as implementing some program. Very oversimplified example: if I learn that e.g. it’s cloudy today, that means the “weather” variable in my program at a particular time[1] takes on the value “cloudy”. Now, suppose your program and my program are exactly the same, except that somewhere in there I think a certain parameter has value 5 and you think it has value 0.3. Even though our programs differ in only that one little spot, we might still expect very different values of lots of variables during execution - in other words, we might have very different beliefs about lots of stuff in the world.

If your model and my model differ in that way, and we’re trying to discuss our different beliefs, then the obvious useful thing-to-do is figure out where that one-parameter difference is.

That’s a delta: one or a few relatively “small”/local differences in belief, which when propagated through our models account for most of the differences in our beliefs.

For those familiar with Pearl-style causal models: think of a delta as one or a few do() operations which suffice to make my model basically match somebody else’s model, or vice versa.

This post is about my current best guesses at the delta between my AI models and Paul Christiano's AI models. When I apply the delta outlined here to my models, and propagate the implications, my models mostly look like Paul’s as far as I can tell. That said, note that this is not an attempt to pass Paul's Intellectual Turing Test; I'll still be using my own usual frames.

My AI Model Delta Compared To Christiano

Best guess: Paul thinks that verifying solutions to problems is generally “easy” in some sense. He’s sometimes summarized this as “

verification is easier than generation

”, but I think his underlying intuition is somewhat stronger than that.

What do my models look like if I propagate that delta? Well, it implies that delegation is fundamentally viable in some deep, general sense.

That propagates into a huge difference in worldviews. Like, I walk around my house and look at all the random goods I’ve paid for - the keyboard and monitor ...
AI: 4 levels of impact [micropost]
by Mati_Roy — 2024-06-12T16:58:31.888Z

LLM is as big as the smartphone/electricity: it will be the building block on which a lot of tech gets built.
AI is the new industrial revolution/agricultural revolution: it will allow for a whole new level of automation of the economy.
AGI is the new macro-optimisation process since the economy/memes/sexual reproduction/DNA: it will change the evolutionary process itself.
ASI is the new Big Bang: first matter expanded in the universe, second intelligence will.

(Pedantic note: "LLM is AI", "space itself expands, not matter", "AGI is an ill-defined concept", etc. Ok, but the core idea is the general framework / spectrum for thinking about the impact of AI 😊)
Aggregative principles approximate utilitarian principles
by Cleo Nardo — 2024-06-12T16:27:22.179Z

1. Introduction

Utilitarianism is the view that a social planner should choose options which maximise the social utility of the resulting social outcome. The central object in utilitarianism is the social utility function u:S→R which assigns a real value u(s)∈R to each social outcome s∈S. This function typically involves variables such as the well-being, preferences, and mental states of individuals, distributional factors like inequality, and other relevant factors such as justice, social cohesion, and freedoms. Utilitarianism is a broad class of social choice principles, one corresponding to each function u:S→R.

In my previous article, I introduced aggregative principles, which state that a social planner should make decisions as if they will face the aggregated personal outcomes of every individual in the population. The central object in aggregativism is the function ζ:S→P, represented with the the greek letter zeta, which assigns a personal outcome ζ(s)∈P to each social outcome s∈S. This function typically aggregates the collection of personal outcomes facing the entire population into a single personal outcome. Aggregativism is a broad class of social choice principles, one corresponding to each function ζ:S→P.

We examined three well-known aggregative principles:

Live Every Life Once (LELO), where ζ(s) is the concatenation of every individual's life.
Harsanyi's Lottery (HL), where ζ(s) is a uniform lottery over every individual's life.
Rawls' Original Position (ROI), where ζ(s) is Knightian uncertainty over every individual's life.

I'm interested in aggregative principles because they avoid many theoretical pitfalls of utilitarian principles. Unlike utilitarianism, aggregativism doesn't require specifying a social welfare function, which is notoriously intractable. Moreover, it seems less prone to counterintuitive conclusions such as the repugnant conclusion or the violation of moral side constraints.[1] In this article, I will show that, under natural conditions of human rationality, aggregative principles approximate utilitarian principles. Therefore, even though aggregativism avoids these theoretical pitfalls, we should nonetheless expect aggregativism to generate roughly-utilitarian recommendations in practical social contexts, and thereby retain the most appeal...
Sticker Shortcut Fallacy — The Real Worst Argument in the World
by ymeskhout — 2024-06-12T14:52:41.988Z
Link: https://www.ymeskhout.com/p/sticker-shortcut-fallacy-the-real

Scott Alexander’s Noncentral Fallacy, dubbed “the worst argument in the world”, is a classic example of manipulative rhetoric. The basic structure of this fallacy, as Scott describes it, is to apply technically correct labeling in order to conjure up misleading connotations — say by labeling MLK a “criminal” in order to obliquely undermine his legacy. Scott is identifying an important and pervasive dynamic, but he’s misidentifying why it’s a problem because the concept of “centrality” is both incoherent and irrelevant. Rather, the real issue is how debates over categorizations and labels get utilized as a fallacious shortcut to avoid having to debate some Other Thing. Let’s call it the “Sticker Shortcut Fallacy” instead.

What even is centrality?

Honed down to its most discrete point in semantic space, the barest definition of criminal is “anyone who has ever broken any law”. While the word certainly shoulders a constellation of emotionally salient connotations, these are heavily susceptible to context and priming, and very often vehemently disagreed with. Which archetypical avatar of criminality gets conjured up in your mind will depend on your particular background, the specific narrative presented to you, as well as whatever sympathies you may hold.

Al Capone might be the prototypical criminal in people’s minds, but on paper, he’s actually just a tax evader. Robin Hood literally stole money from people, but he gave it to the poor, so his legacy tends to be that of a folk hero rather than a felon. Based on the severity of punishment, nothing compares to the legally sanctioned torturous spectacle of drawing and quartering those convicted of high treason, but whether such a person is a quote-unquote criminal depends on which side you’re on. The Founding Fathers were unambiguous seditious conspirators against the greatest empire of their time, yet no one — except perhaps some nostalgic British monarchists — thinks of them as criminals for this.[1] How exactly do you adjudicate which one is a “central” archetype of criminal, and who decides?

The disagreement over centrality applies to other examples cited by Scott, especially when you jettison the semantic baggage and start with a blank slate. The core essence of theft might be “taking other people’s property without their consent” but depending on the context, different examples will herald primacy. For insta...
Long-Term Future Fund: 
May 2023 to March 2024 Payout recommendations
by Linch — 2024-06-12T13:46:29.535Z

Introduction

This payout report covers the Long-Term Future Fund's grantmaking from May 1 2023 to March 31 2024 (11 months). It follows our previous April 2023 payout report.

Total funding recommended: $6,290,550
Total funding paid out: $5,363,105
Number of grants paid out: 141
Acceptance rate (excluding desk rejections): 159/672 = 23.7%
Acceptance rate (including desk rejections): 159/825 = 19.3%
Report authors: Linchuan Zhang (primary author), Caleb Parikh (fund chair), Oliver Habryka, Lawrence Chan, Clara Collier, Daniel Eth, Lauro Langosco, Thomas Larsen, Eli Lifland

25 of our grantees, who received a total of $790,251, requested that our public reports for their grants are anonymized (the table below includes those grants). 13 grantees, who received a total of $529, 819, requested that we not include public reports for their grants. You can read our policy on public reporting here.

We referred at least 2 grants to other funders for evaluation.

Highlighted Grants

(The following grants writeups were written by me, Linch Zhang. They were reviewed by the primary investigators of each grant).

Below, we highlighted some grants that we thought were interesting and covered a relatively wide scope of LTFF’s activities. We hope that reading the highlighted grants can help donors make more informed decisions about whether to donate to LTFF.[1]

Gabriel Mukobi ($40,680) - 9-month university tuition support for technical AI safety research focused on empowering AI governance interventions

The Long-Term Future Fund provided a $40,680 grant to Gabriel Mukobi from September 2023 to June 2024, originally for 9 months of university tuition support. The grant enabled Gabe to pursue his master's program in Computer Science at Stanford, with a focus on technical AI governance.

Several factors favored funding Gabe, including his strong academic background (4.0 GPA in Stanford CS undergrad with 6 graduate-level courses), experience in difficult technical AI alignment internships (e.g., at the Krueger lab), and leadership skills demonstrated by starting and leading the Stanford AI alignment group. However, some fund managers were skeptical about the specific proposed technical research directions, although this was not considered critical for a skill-building and career-development grant. The fund managers also had some uncertainty about the overall value of fund...