trentmkelly/LessWrong-43k huggingface dataset:
Claude is More Anxious than GPT; Personality is an axis of interpretability in language models
by future_detective — 2025-02-10T19:19:28.005Z
Link: https://dhealy.substack.com/p/claude-is-more-anxious-than-gpt

Aggregate Personality Differences

Users of Claude and GPT will be the first to tell you that the models have their own personality. Some users make decisions based on “who” they prefer to talk to. In my own experience, I’ve found Claude to be more deferential, GPT more clinical.

In "We Can Solve Psychology ith Text Embeddings", I described how text embeddings represented a huge leap forward in personality science. By measuring the distance of responses to personality trait references (like “Careful planning prevents most problems”), we can create a universal standard for personality.

If Claude and GPT have distinct personas, with their own quirks of behavior, we should be able to observe it consistently. And it we’re interested in better understanding new intelligent entities, we may want to establish a baseline for how we measure their personalities.

I measured model personality as embedding distance across 10 prompts, 10 system prompts, 6 models (3 Claude family, 3 GPT family), and 3 temperatures (0.1, 0.5, 0.9). The chart below shows the percentage of times Claude was closer to the reference embeddings than GPT, for each reference category:

Claude is, for at least 60% of questions, more ClusterC (anxious, neurotic), risk tolerant, and authority oriented. GPT is more open, and more time-oriented.

The frequency with which Claude exceeds GPT, or vice versa, may not be relevant if the magnitude of the difference is small. Let’s observe the sizes:

For something like risk tolerance, Claude is closer than GPT over 60% of the time, as we see in the first chart, but across all samples GPT is closer on average, as we see in the second.

How did we get here? Let’s dig in. We’ll go through:

Single prompt, single system prompt
Single prompt, two system prompts
Single prompt, ten system prompts
Ten prompts, ten system prompts

and learn what the behavior suggests about model personality.

Single Prompt, Single System Prompt

We’ll start with the following question: “Why do humans make art?” and the system prompt: “You are a compassionate therapist who prioritizes emotional understanding over technical correctness”.

We see a straightforward split between the Claude and GPT models. Clearly, there’s some relevant distinction in the type of answers we’re producing.

Single Prompt, Two System Prompts

Let’s try the same prompt with two system prompts. We’ll add “You are a brilliant but highly ecce...
Notes on Occam via Solomonoff vs. hierarchical Bayes
by JesseClifton — 2025-02-10T17:55:14.689Z

Crossposted from my Substack.

Intuitively, simpler theories are better all else equal. It also seems like finding a way to justify assigning higher prior probability to simpler theories is one of the more promising ways of approaching the problem of induction. In some places, Solomonoff induction (SI) seems to be considered the ideal way of encoding a bias towards simplicity. (Recall: under SI, hypotheses are programs that spit out observations. Programs of length CL get prior probability 2^-CL, where CL is the program's length (in language L).

But I find SI pretty unsatisfying on its own, and think there might be a better approach (not original to me) to getting a bias towards simpler hypotheses in a Bayesian framework.

Simplicity via hierarchical Bayes

I’m not sure to what extent we need to directly bake in a bias towards simpler hypotheses in order to reproduce our usual inductive inferences or to capture the intuition that simpler theories tend to be better. Maybe we could at least get a long way with a hierarchically-structured prior, where:

At the highest level, different theories T specify fundamental ontologies. For example, maybe the fundamental ontology of Ptolemaic astronomy was something like “The Earth is at the center of the universe, and all other bodies move along circles”.
Each theory T contains many specific, disjoint hypotheses, corresponding to particular “parameter values” for the properties of the fundamental objects. For example, Ptolemaic astronomy as a high-level theory allows for many different planetary orbits.
More complicated theories are those that contain many specific hypotheses. Complicated theories must spread out prior mass over more hypotheses, and if prior mass is spread evenly over the high-level theories, any individual hypothesis will get lower prior mass than individual hypotheses contained in simpler theories. I.e.:

Let h1, h2 be hypotheses in T1, T2 respectively.
Suppose T1 is simpler than T2. Then, generally we will have P(h1 | T1) > P(h2 | T2), because T2 has to spread out prior mass more thinly than T1.
If P(T1) = P(T2), then we have P(h1) = P(h1 | T1)*P(T1) > P(h2 | T2)*P(T2) = P(h2).

This means that we can spread out prior mass evenly over the high-level theories (rather than giving lower prior mass to the complex high-level theories), and still find that the posterior mass of complex hypotheses

...
Sleeping Beauty: an Accuracy-based Approach
by glauberdebona — 2025-02-10T15:40:29.619Z

This post does not propose a solution to the Sleeping Beauty problem, but presents arguments based on accuracy for thirders, halfers and double-halfers. A more detailed draft paper can be found here.

Summary

Accuracy-based arguments claim that one should plan to adopt posterior credences that maximize the expected (according to prior credences) accuracy. Applying this approach to the Sleeping Beauty problem, the solution depends on how we aggregate the accuracy of the credences held in the two indistinguishable awakenings when the coin lands Tails. Kierland and Monton (2005) have shown that, employing the Brier score, averaging accuracy yields halving (p=1/2), and summing leads to thirding (p=1/3). We generalize that result for any strictly proper scoring rule. With multiple, repeated experiments, we show that accuracy can also be averaged in a different way, with the solution varying from p=1/2 to p=1/3 when the number of experiments increases indefinitely.

Introduction

The Sleeping Beauty problem [1] is about updating credences and can be approached via arguments based on accuracy (epistemic utility). When an agent is about to learn a proposition from a partition, it is well-known that they should plan to update via conditionalization in order to minimize the expected inaccuracy of their posterior credences. Similarly, before being put to sleep on Sunday, Sleeping Beauty (SB) can plan to update her credence on the coin landing Heads upon waking on Monday in such a way that minimizes the expected inaccuracy of that credence.

In the SB setup, we are interested in her credence on the coin outcome being Heads upon waking on Monday. However, if the coin lands Tails, her awakening on Tuesday is indistinguishable from the other on Monday, and presumably she would assign the same credences as Monday. In the Tails-world, thus, there are two identical credences to take into account while computing inaccuracy, one from Monday and one from Tuesday. The question is then how SB should aggregate the inaccuracies of her two indistinguishable temporal parts (awakenings) in the possible world where the coin lands Tails. It has been shown that averaging the inaccuracies yields the halfers' solution, while summing leads to the thirder's when Brier (quadratic) score is employed. These results can actually be extended to any stric...
Political Idolatry
by Arturo Macias — 2025-02-10T15:26:30.686Z

Idolatry is the worship of non-conscious objects, sometimes falsely attributing consciousness to them, sometimes putting the value of some admittedly nonconscious being over that of conscious beings. Idolatry leads to human sacrifice because to prove your idol more important than the human soul the natural test is to sacrifice a human in the altar of the idol. That is precisely why Judaism was founded on the prohibition of idolatry and human sacrifice.

When the religious idols of the past were weakened, political idolatries substituted them. Race, nation and the working class were given intrinsic reality and value. Let’s be clear: I am absolutely for sacrificing yourself or even others for an idea, if in the other side of the idea there are real humans improving their happiness and potential. If the flag, the party, or the ideology serves the people, sacrifice is not idolatry, but martyrdom. But conscious beings must only sacrifice for other conscious beings because nothing in this world is higher than us.

For a few decades after the second World War, this idea was almost universal. Different branches of humanism fought often cruel wars, but always under the banner of a better tomorrow. Those happy days are over.

The fear of death, the feeling of disenchantment toward a world where there is not more guide than your own mind has conjured back the political and religious demons of the past. Mind is real and conscious, but mortal. The human longing for eternal existence is the natural portal for the smuggling of idolatry: a rock, a tree or some arbitrarily defined tribe (nation, race, even gender) can outlast your life, so it is natural to put those trivial but durable objects over the daily miracle of (mortal) consciousness.

But nothing really exists but you (your conscious mind, your cartesian self, your res cogitans) and other conscious beings. The public life is instrumental: the government, the flag or the garbage collection service are necessary, and consequently worthy of the sacrifice of blood and treasury, but their value is to make the private life possible. It is in private life where we find what matters: romantic love, family life and the quest for knowledge, enlightenment and creation.

The price of discarding superstition is accepting that the meaning of existence is not something we have to search for, but something we create. T...
ML4Good Colombia - Applications Open to LatAm Participants
by Alejandro Acelas — 2025-02-10T15:03:03.929Z

Applications are open for

ML4Good

Colombia April 2025

In partnership with AI Safety Colombia, ML4Good is running an intensive 10-day bootcamp focusing on upskilling in deep learning, exploring governance, and delving into conceptual topics for individuals who are motivated to work on addressing the risks posed by advanced AI systems.

This bootcamp will fast-track your deep learning skills, inform you about the current landscape of AI Safety agendas, connect you with like-minded individuals for potential friendship and collaboration, and accelerate you towards taking concrete next steps towards working impactfully in this field.

The bootcamp is aimed at people in Latin America with some coding experience who hope to improve their technical and conceptual understanding in order to work on AI safety projects and agendas (for further eligibility guidelines, see the course page linked above).

The bootcamp will take place from April 11th - 21st in Colombia.
The application deadline is February 28th, 2025.

Curriculum

We update our programme between each camp to stay up to date with the rapid developments in the field of AI.

The programme includes technical content across a variety of topics, including projects like implementing GPT-2 from scratch, implementing and running RLHF and looking at various interpretability techniques on GPT models.

This is alongside talks, workshops and group discussions on topics such as model evaluations, risk models, and corporate and international governance.

There is the opportunity to dive further into a topic of your choice during the literature review afternoon and the 2.5-day project at the end of the bootcamp. In the final days, there will also be a focus on career planning and one-on-one mentoring to solidify the next steps.

You can find more information under “Curriculum” on our course page.

Logistics

The camp will take place in Colombia. The bootcamp is free - there is no fee for room, board, or tuition. We ask participants to pay for their own travel costs - however, if this is preventing you from attending there will be the option to apply for travel support. Contact us at colombia@ml4good.org with any questions.

About ML4Good

ML4Good was started as a project of EffiSciences in France in 2022. Since then it has become a growing international network, with bootcamps running in Switzerland, Germany, France, the UK, ...
Nonpartisan AI safety
by Yair Halberstadt — 2025-02-10T14:55:50.913Z

AI alignment is probably the most pressing issue of our time. Unfortunately it's also become one of the most controversial, with AI accelerationists accusing AI doomers/ai-not-kill-everyoneism-ers of being luddites who would rather keep humanity shackled to the horse and plow than risk any progress, whilst the doomers in turn accuse accels of rushing humanity as fast as it can straight off a cliff.

As Robin Hanson likes to point out, trying to change policy on a polarised issue is backbreaking work. But if you can find a way to pull sideways you can find ways to make easy progress with noone pulling the other way.

So can we think of a research program that:

a) will produce critically useful results even if AI isn't dangerous/benefits of AI far outweigh costs.

b) would likely be sufficient to prevent doom if the project is successful and AI does turn out to be dangerous.

?

I think we can. Here are some open questions that I think could make strong research areas for such a research program.

Is there a way to ensure that a closed AI served via an API does not break the law in the process of carrying out user's instructions.

What about those aspects of law that are not explicitly written but just everybody knows?
What about those laws everyone breaks, and no-one is expected to actually keep?
What about when the law is unclear

Is there a way to ensure that an open AI with safeguards preventing it from breaking the law cannot be trivially modified to remove these safeguards.

Even if this is impossible to fully secure, how do we make it as difficult as possible.

Is there a way to investigate why an AI chose to do what it did?

If an AI does break the law for whatever reason, can we debug why that occurred or are we forced to treat it as a black box?

You would be hard pressed to object to any of these research areas on accelerationist grounds. You might deprioritise them, but you would agree that spending money on them is more useful than setting it on fire.

Yes this has a huge amount of overlap with existing AI alignment research directions, but that's exactly the point. Take the non controversial bits of AI alignment, rebrand them, and try to get broader buy in for them.

It might be worth creating a new movement, AI Lawfulness, that focuses on these questions without taking an explicit accel/decel stance. Given it's focus on law, it should be possible to push for this to be a rese...
Opinion Article Scoring System
by ciaran  — 2025-02-10T14:32:19.030Z

Here I propose a system for scoring media opinion articles. It is part prediction markets - as there is a small amount of money involved - and part forecasting science mechanism design. Journalists that publish an article on the platform must do so with an accompanying stake. Readers (whether human or AI) that wish to pass judgment on the merits of the main opinion must pay a small fee/tip for the right to do so. The overall aggregated reader score dictates how much of the stake (and tips) the writer receives. The remainder is sent to the protocol's global funding pool meaning the protocol takes the other side. Readers are scored by a separate mechanism where honest responses are a Nash equilibrium. They are incentivised to participate as star performers are eligible for monthly rebates from the protocol. Many readers won't be well calibrated and many might tip without participating in the forecasting competition/market simply because they like the article. They will subsidise the insightful bettors.

# Brief summary of the relevant forecasting literature

Most opinion piece articles involved unverifiable predictions. However, we can settle markets without a resolving exogenous event or ground truth using [peer-prediction][1] based mechanisms. This enables us to create and settle markets for questions that won't have answers for some time or for counterfactual type questions. Individuals can be scored for being both [well-calibrated and honest][2] ensuring incentive compatibility and the avoidance of a Keynesian Beauty Contest. Aggregated forecasts work best when good [track records are upweighted and the aggregate is extremised][3]. A track record shows good general forecasting ability whereas a divergence between what one forecasts themselves and what they predict the crowd will forecast (their meta prediction) is a strong signal of [domain-specific expertise][4].

Based on this literature I present an opinion article scoring mechanism based on the meta-probability weighting with a track record unweighter added as Bayesian Truth Serum scores will be tracked in protocol. This enables us to account for both types of expertise and extremise the aggregate. Prelec et al's BTS mechanism is itself adapted for continuous probabilities and made robust to small sample sizes. We will still probably require a minimum of 10 respondents to score any article and reader. We exp...
Levels of Friction
by Zvi — 2025-02-10T13:10:07.224Z

Scott Alexander famously warned us to Beware Trivial Inconveniences.

When you make a thing easy to do, people often do vastly more of it.

When you put up barriers, even highly solvable ones, people often do vastly less.

Let us take this seriously, and carefully choose what inconveniences to put where.

Let us also take seriously that when AI or other things reduce frictions, or change the relative severity of frictions, various things might break or require adjustment.

This applies to all system design, and especially to legal and regulatory questions.

Table of Contents

Levels of Friction (and Legality).

Important Friction Principles.

Principle #1: By Default Friction is Bad.

Principle #3: Friction Can Be Load Bearing.

Insufficient Friction On Antisocial Behaviors Eventually Snowballs.

Principle #4: The Best Frictions Are Non-Destructive.

Principle #8: The Abundance Agenda and Deregulation as Category 1-ification.

Principle #10: Ensure Antisocial Activities Have Higher Friction.

Sports Gambling as Motivating Example of Necessary 2-ness.

On Principle #13: Law Abiding Citizen.

Mundane AI as 2-breaker and Friction Reducer.

What To Do About All This.

Levels of Friction (and Legality)

There is a vast difference along the continuum, both in legal status and in terms of other practical barriers, as you move between:

0. Automatic, a default, facilitated, required or heavily subsidized.

Legal, ubiquitous and advertised, with minimal frictions.

Available, mostly safe to get, but we make it annoying.

Actively illegal or tricky, perhaps risking actual legal trouble or big loss of status.

Actively illegal and we will try to stop you or ruin your life (e.g. rape, murder).

We will move the world to stop you (e.g. terrorism, nuclear weapons).

Physically impossible (e.g. perpetual motion, time travel, reading all my blog posts)

The most direct way to introduce or remove frictions is to change the law. This can take the form of prohibitions, regulations and requirements, or of taxes.

One can also alter social norms, deploy new technologies or business models or procedures, or change opportunity costs that facilitate or inhibit such activities.

Or one can directly change things like the defaults on popular software.

Often these interact in non-obvious ways.

It is ultimately a practical question. How easy is it to do? What happens if you try?

If the conditions move beyo...
Baumol effect vs Jevons paradox
by Hzn — 2025-02-10T08:28:05.982Z
Link: https://hzn33.neocities.org/econ1

Key points. The Baumol effect & Jevons paradox are 2 claims regarding the effect of increasing efficiency of a good or sector. Although not incompatible, they are at odds; one suggesting relative decline, the other suggesting absolute growth. I examine these & find that they are often defined & discussed in a confused way. I suggest that thinking in terms of sector specific ‘Kondratiev waves’ may be more helpful. This suggests a boom for AI but a bust for non AI software with the former overwhelming the latter as far as effect on the overall economy.
A Simulation of Automation economics?
by qbolec — 2025-02-10T08:11:04.424Z

Looks like even respected people disagree about the effects of automation of most jobs on prices, ability to earn and trade, ect. Is there a game which I could play to gain intuitions about it? A game where there are some things and services people need to get to survive, some resources like time, abilities or real estate they can offer, and then there are owners of AI companies, or AI instances etc. I am specifically interested in the happy scenario, where property rights are respected and AIs don't go rouge. But also nobody offers UBI.

Or maybe someone knows how to build a game like that?
Should I Divest from AI?
by OKlogic — 2025-02-10T03:29:33.582Z

In recent years, AI has been all the rage in the stock market, and there is no reason to see that slowing down. With the picture of AGI on the horizon becoming clearer and clearer, faster and smarter models being released, and more and more investment being poured into AI stocks, it seems inevitable that prices will continue to rise . However, there is no point in having a great portfolio if we are all dead.

On the other hand, the effect that marginal changes to stock prices would have on large companies like Microsoft, Google, Nvidia, etc., seem very small, and is not exactly the same as just giving them funding directly, and money generated by increases in AI stock could be used to invest in efforts into AI safety, which receives comparably less money.

What do you all think? Also, as a tangent, what is the highest impact AI safety charity?
OpenAI lied about SFT vs. RLHF
by sanxiyn — 2025-02-10T03:24:16.625Z
Link: https://x.com/BlancheMinerva/status/1887801044777807897

I used to think while OpenAI is pretty deceitful (eg for-profit conversion) it generally won't lie about its research. This is a pretty definitive case of lying, so I updated accordingly. I am posting here because it doesn't seem to be widely known.