trentmkelly/LessWrong-43k huggingface dataset:
Contagious Beliefs—Simulating Political Alignment
by James Stephen Brown — 2024-10-13T00:27:08.084Z
Link: https://nonzerosum.games/beliefs.html

Humans are social animals, and as such we are influenced by the beliefs of those around us. This simulation explores how beliefs can spread through a population, and how indirect relationships between beliefs can lead to unexpected correlations. The featured simulation only works in the

original post

. I recommend visiting to explore the ideas fully.
If you're interested in going deeper, you can also check out the detailed methodology.

STRANGE BED-FELLOWS

There are some strange ideological bed-fellows that emerge in the realm of human beliefs. Social scientists grapple with the strong correlation between

Christianity and gun ownership

when the “Prince of Peace” lived in a world without guns. Similarly there are other correlations between

atheism and globalisation

or pro-regulation leftists who are also pro-choice, and then we have the anti-vax movement infiltrating both the far-left and far-right of politics.

Does this all mean that people are just confused?

The simulation explores the network effects of belief transmission and runs on the principle that humans adopt beliefs that align with their pre-existing beliefs,

seeking cognitive coherence over cognitive dissonance

.

“A receiver of a belief either accepts the incoming belief or not based on the context of their own belief system (internal coherency)…”
- Rodriguez et al*

Each belief in this simulation has a valence with each other belief-with those sharing a positive valence being complementary ideas, and those with a negative valence being dissonant. The simulation doesn’t specifically model bias, but apparent bias is an emergent property of the system.

INSTRUCTIONS

The opening sample is simply my own intuitions about what logical relationship some religious and political beliefs have with one another on . I have purposefully left anything I do not see as directly connected as zero. You can edit these valence values or categories to reflect your own intuitions, or the issues important to you.

It’s a bit of a learning curve thinking about valences, as each belief here is actually a pair-the belief and its opposite. So, if you have have a single issue like “taxes” this will be interpretted as “Pro-Tax”/”Anti-Tax”. When relating this to another related factor like “Right Wing/”Left Wing” you are looking for one value to describe how aligned “Pro-Tax” and “Right-Wing” are, and also how aligned are “Anti-Tax” and “Left ...
Binary encoding as a simple explicit construction for superposition
by tailcalled — 2024-10-12T21:18:31.731Z

Superposition is the possibility of storing more than n features in an n-dimensional vector, by letting the features be slightly correlated with each other. It turns out that one can store exponentially many features in a given vector. The ability to store that many features in a single vector space is sometimes explained using the Johnson–Lindenstrauss lemma, but the lemma seems counterintuitive, so I came up with an alternative approach that I found simpler:

Suppose you have a set F with 2d elements and you want to embed it in a d-dimensional vector space. We label each element xi with integers i such that F={x0,x1,…,x2d−1}. You can write each integer i as a string of d bits, bd−1…b2b1b0. To improve symmetry, we translate a bit b from being 0 or 1 to being −1 or 1 by taking 2b−1. Join all the digits into a vector and normalize to get the embedding:

e(xi)=1√d[2bd−1−1,…,2b2−1,2b1−1,2b0−1]⊤

That is, we map each bit to a separate dimension, with a b=1 bit mapping to a positive value and a b=0 bit mapping to a negative value, and scale the embedding by 1√d to keep the embedding vector of unit length.

If we pick two random elements xa and xb of F, then an elementary argument shows that their dot product is well-approximated as following a normal distribution N(0,1√d).

In some ways this isn't quite as perfect as the Johnson-Lindenstrauss lemma since you could in principle be unlucky and get two elements that accidentally have a high similarity. After all, for a given element xi, there will be d elements xi⊕2k whose numbers merely differ from xi by a bitflip. However, it is straightforward to reduce the noise: just concatenate multiple embeddings based on different labels. If instead of using d dimensions, you use Rd dimensions, then you can pump down the noise to N(0,1√Rd).
How Should We Use Limited Time to Maximize Long-Term Impact?
by queelius — 2024-10-12T20:02:46.801Z

I've been reflecting on how researchers—particularly those with limited time or resources—can best contribute to influencing the long-term future.

Assumption: Individually, we may not have the resources to fund large projects, but we still want to make contributions that move the needle, however slightly.

I approach this question with epistemic humility—recognizing that predicting long-term impact is incredibly difficult—and with humility about our own limitations. I am personally guided by two key principles:

Asymmetry of suffering and joy: I believe that one conscious being enduring astronomical suffering (assuming that is a coherent concept) can potentially outweigh arbitrary numbers of others experiencing joy. This idea shapes where I think our focus should be.
Alignment challenge: Aligning AGI/ASI with any prescribed value system will be incredibly difficult. I do not subscribe to the view that human values are a reasonable target, given that our current systems already perpetuate (needlessly) enormous suffering (e.g., factory farming, extreme inequality).

Much has already been impressively written about these topics, and I’m particularly interested in mitigating s-risks, since reducing the chance of such suffering seems more urgent than pursuing joy-maximizing goals. However, many of us—whether due to health, time, or other constraints—are left wondering how to allocate our remaining time wisely.

My personal context: I have stage 4 cancer, so statistically, my time may be more limited than most. I’m a PhD student in Computer Science with a strong background in math (Masters). Given my situation, I’m especially interested in how those of us with limited time or resources can still make non-negligible contributions to reducing s-risks and positively shaping the long-term future.

What I’m asking for: How should we, as researchers or contributors with limited time, focus our efforts to maximize our positive impact? Are there specific areas of research or actions that seem particularly high-leverage for reducing future suffering on an astronomical scale?

Finally, I’m aware that this topic has been discussed extensively, and I’d be grateful for any links to relevant resources or existing discussions that cover this ground. My goal is not to waste anyone's time, as I’m not deeply well-read on the full body of literature. Any guidance, new or old, would be greatly app...
A Percentage Model of a Person
by Sable — 2024-10-12T17:55:07.560Z
Link: https://affablyevil.substack.com/p/a-percentage-model-of-a-person

The standard psychological questionnaire for depression doctors have given me is the PHQ-9. It names a symptom, and for each symptom it gives four possible responses in severity. The responses are worth points, the points are totaled, and the final score is supposed to be indicative of how severe a person’s depression is.

This is what it looks like:

According to Wikipedia, the final score indicates the following:

To be perfectly blunt, while I suppose this functions for doctors to get an idea of where their patient is at, it’s absolute garbage for giving the patient a sense of how depressed they are. Knowing how many times in the last two weeks you’ve felt ‘down, depressed, and hopeless’ isn’t super helpful when you’re trying to understand - or worse, explain to someone else - what it’s like in your head.

To that end I’ve been slowly developing my own sense of how depressed I am at any given time, and it’s based on a percentage scale.

The Scale

When trying to understand my own depression, I eventually came to the conclusion that, when depressed, I am not (in reality) the person I see myself as. I have a self-image, a conception of what I’m supposed to be like, and my depressed self isn’t it.

The key insight, however, was that there is a sense in which the person I am while depressed is a fraction of the person I identify as. My depression lessens me, but how much it lessens me is quantifiable.

Granted, it’s still self-reported so it’s not exactly an objective measurement, but neither is the PHQ-9.

If my depression lessens me, then I can get a feel for what percentage of myself I am at any given time - and that provides a reasonable sense for how depressed I currently am.

The scale runs from 0% to 100%, where 0% is ‘I am so utterly not myself that life does not feel worth living’ and 80% is ‘basically normal’. I can’t speak for others, but even when I’m not depressed at all I’m still not totally the person I want to be, which is why ‘normal’ is 80%. 100%, here, means something along the lines of ‘completely self-actualized, utterly myself in all the ways that matter’. I sincerely doubt that there are many (or plausibly any) people who spend all their time at 100%.

The scale in summary:

One of the most interesting parts of moving around this scale is that I’ve started to get a sense for what percentage certain feelings, behaviors, and cognitive abilities corre...
AI Compute governance: Verifying AI chip location
by Farhan — 2024-10-12T17:36:45.942Z

TL;DR: In this post I discuss a recently proposed on-chip compute governance mechanism, a delay based location verification mechanism. Then I demonstrate feasibility and applicability of the mechanism through a few exercises and identify a potential issue with frequent false positives. Finally I propose a potential solution for the identified issue.

Technical governance approaches will be critical for mitigating the risks posed by potentially misaligned AI systems. Even if the alignment problem is solved, technical governance methods will still likely be important for some time to limit misuse or the consequences of AI malfunctions.

Recent AI progress has largely been driven by increases in the amount of computing power used to train new models. Currently the compute used to train notable AI systems doubles every six months. Hence, governing compute could be an effective way to govern AI.

Figure 1: The amount of compute used to train notable AI models has doubled roughly every six months since 2010. From Epoch

On-chip Mechanism for Location Verification

A subset of compute governance, “on-chip governance mechanisms” could allow a regulator to perform verification or impose restrictions. In this article we explore location verification mechanisms specifically, delay based verification.

Speed of light based verification

In a recent report CNAS (Centre for a New American Security) proposed a location verification mechanism that relies on the hard limit of the speed of light (300 km/ms) and the lower bound on latency from existing communications infrastructure(I believe the maximum available is 200km/ms for pure fiber optic).

This approach has the following main design parameters.

The round trip distance (RTD) between subject chip and export control boundary (B) determines the maximum allowable round trip time (RTTmax), as calculated based on speed of light, for a verification handshake between the trusted server(S) and the subject chip(C).

RTTmax = RTD(C–>B)/300

Practical latency per distance (Lp) of communication infrastructure determines the maximum possible distance between trusted server and subject chip. The lowest bound on Lp is inverse of the speed of light in fibre optik (1ms/200km).

RTDmax(S–>C) = RTTmax / Lp

Substituting for RTTmax

RTDmax(S–>C) = RTD(C–>B) / (Lp*300)

Figure 2: Assuming speed of light, The distance from C to B determines t

...
Geoffrey Hinton on the Past, Present, and Future of AI
by Stephen McAleese — 2024-10-12T16:41:56.796Z

Introduction

Geoffrey Hinton is a famous AI researcher who is often referred to as the "godfather of AI" because of his foundational work on neural networks and deep learning from the 1980s until today. Arguably his most significant contribution to the field of AI was the introduction of the backpropagation algorithm for neural network training in 1986 which is widely used to train neural networks today. He was also involved in the development of dropout regularization, AlexNet, the RMSProp optimization algorithm, and the t-SNE visualization method. In 2018 he won the Turing award for his work on deep learning along with Yoshua Bengio and Yann LeCun. In 2024, Hinton won the Nobel Prize in Physics for his work on Boltzmann machines. He was also the PhD supervisor of Ilya Sutskever who co-founded OpenAI in 2016.

In 2023, Hinton left Google to retire and speak more freely about the risks of AI. In addition to speaking about the near-term risks of AI, Hinton has also thought carefully about the long-term consequences of AI such as whether AIs will become superintelligent and whether AI could become an existential risk to humanity.

This post starts with a brief history of AI from Hinton's perspective before describing how neural networks work, Hinton’s views on current AI models, and finally his thoughts on the future impact of AI.

A brief history of AI

"I could have said it then, but it wouldn't have convinced people. And what I could have said then is the only reason that neural networks weren't working really well in the 1980s was because the computers weren't fast enough and the datasets weren't big enough. But back in the 80s, the big issue was, could you expect a big neural network with lots of neurons in it, compute nodes, and connections between them, that learns by just changing the strengths of the connections? Could you expect that to just look at data and, with no kind of innate prior knowledge, learn how to do things? And people in mainstream AI thought that was completely ridiculous. It sounds a little ridiculous. It is a little ridiculous, but it works." - Geoffrey Hinton [1]

Today most leading AI models such as GPT-4 are artificial neural networks trained with gradient descent and backpropagation, which has become the dominant approach to building AI. However, this wasn't always the case. Hinton describes how historically there have been two primary sch...
I = W/T?
by HNX — 2024-10-12T15:15:36.806Z

What are some of the most deep, profound, meaningful, impactful; ideas, notions, concepts, viewpoints, and considerations, relative to the amount of time an average layman would need to wrap 'eir head around them? Chain of thoughts so far:

not all ideas are created equal
some ideas are clearly better than others
among the ideas, that are clearly better than others, there is a (+/-) clear order

the order of ideas, ordered by their respective impact, can further be enhanced, EA-wise, by considering the amount of time it would take to get acquainted with:

the idea itself
its own foundation
its practical implications

To anyone, who has been here long enough to tell, is there:

a set of all the candidates, pre-vetted or otherwise, for the (3), around here?
an order in which, at least some of, the elements within this set, are ranked by (2), or impact, or their overall contribution as per standards of rationalism and/or EA?
an order in which the list above would be refined, if time-to-learn was considered?

The sequences, essays, codex seem to be the prime candidates.

Are they, though?

Do they distill each individual idea down to its most fruitful essence, taking into consideration an average attention span of a new-blood, embarking upon LW/EA?

Or are they more of general thoughts and opinion pieces? Not to discard their overall importance, either. Just brainstorming the addition of the time variable into the equation. To reduce it to the barest possible terms, we could say:

I = W / T

Where 'I' is the (potential) impact of the idea itself, the 'W' is the wisdom, knowledge, understanding imparted by it, and 'T' is the amount of time required to 'grok' it.

Which ideas would rank the highest/lowest?

Realized after posting that T is not unary. A better equation would be:

I=W/(Tu∗Ti)

Where Tu is the time-to-understand and Ti is the time-to-implement.
AI research assistants competition 2024Q3: Tie between Elicit and You.com
by Elizabeth — 2024-10-12T15:10:05.417Z

Summary

I make a large part of my living performing literature reviews to answer scientific questions. For years AI has been unable to do anything to lower my research workload, but back in August I tried Perplexity, and it immediately provided value far beyond what I’d gotten from other tools. This wasn’t a fair comparison because I hadn’t tried any other AI research assistant in months, which is decades in AI time. In this post I right that wrong by running two test questions through every major tool, plus a smaller tool recommended in the comments of the last post

Spoilers: the result was a rough tie between You.com and Elicit. Each placed first on one task and was among top-3 in the other.

Tasks + Results

Tl;dr:

You.com had a small edge in searching for papers, followed by Elicit and Google Scholar. ChatGPT was absolute garbage.

Elicit, Perplexity, and You.com all surfaced the key piece of information when asked for analysis, with Elicit’s answer being the most concise. None of the other tools managed this.

You.com and Perplexity were tied for favorite UI, but I haven’t played with You.com very much.

You.com boasts a larger list of uses than Perplexity (which is narrowly focused on research), but I haven’t tried them out.

Finding papers on water gargling as an antiviral

I’m investigating gargling with water (salt or tap) as a potential antiviral. I asked each of the tools to find relevant papers for me.

ChatGPT was asked several versions of the question as I honed in on the right one to ask. Every other tool was asked “Please list 10 scientific papers examining gargling with water as a prophylactic for upper respiratory infections. Exclude nasal rinsing”. This is tricky because almost all studies on gargling salt water include nasal rinsing, and because saline is used as a control in many gargling studies.

Every tool correctly returned 10 results except for Elicit and Google Scholar, which by design will let you load papers indefinitely. In those cases I used the first 10 results.

Paper	Real, relevant results	Probably hallucinations	Notes
Perplexity- initial	?	The formatting was bad so I asked Perplexity to fix it
Perplexity- asked to format ^	4	2
ChatGPT 4o asking for “papers” without specifying “scientific”	0	unusable
ChatGPT 4o specifying “scientific papers” about gargling as a treatment	2	8
Cha

...
SAE features for refusal and sycophancy steering vectors
by neverix — 2024-10-12T14:54:48.022Z

TL;DR

Steering vectors provide evidence that linear directions in LLMs are interpretable. Since SAEs decompose linear directions, they should be able to interpret steering vectors.
We apply the gradient pursuit algorithm suggested by Smith et al to decompose steering vectors, and find that they contain many interpretable and promising-looking features. This builds off our prior work, which applies ITO and derivatives to steering vectors with less success.
Notably, we find several abstract features in Phi-3 Mini for refusal and steer with linear combinations of these features. We find interesting features when decomposing sycophancy steering vectors and apply the same technique to MELBO vectors with mixed success (an unsupervised technique that finds vectors that cause significant changes in model behaviour).

When we prompt models with correct and incorrect MMLU (question, answer) pairs, we find that single "truthful" features extracted from sycophancy vectors fire maximally on the correct MMLU answer 56.1% of the time (around Llama-2 13B level), not too far from Phi's 68.8% accuracy.

Prior work

In Sparse Feature Circuits, Marks et al. find features related to gender in a misgeneralizing classifier and ablate them. Their methods provide both insight about the way a classifier makes its decisions and a way to change its behavior: one of them, SHIFT, ablates features corresponding to gender to their mean activation and corrects the classifier’s generalization.

DeepMind’s SAE steering vector work looks at “wedding” and “anger” features. Their goal is to improve a steering vector (according to the original post’s metrics) by ablating interpretable and irrelevant features.

We want to follow both of their directions and find examples of interpretable SAE features which contain information about tasks performed and causally affect a model’s behavior.

We had some success applying inference-time Optimization (aka gradient pursuit) and its variants on simple in-context learning tasks in one of our previous posts. We applied Sparse Feature Circuits on the tasks and found interesting features (to be published to arXiv soon!). In this work, we want to explore the application of ITO and SAE encoding to previously studied...
Prices are Bounties
by Maxwell Tabarrok — 2024-10-12T14:51:40.689Z
Link: https://www.maximum-progress.com/p/prices-are-bounties

A man has just robbed a train. A Rock Island Rail car was held up in the desert of North Texas on route from Chicago carrying dozens of passengers and tens of thousands of dollars in a Wells Fargo express compartment. The men were slain and the women and children are kidnapped.

You’re the sheriff in charge of the investigation. In a case like this, tried and true procedure is to put a bounty on the man’s head. That gets everyone interested in looking and bringing this man to justice. An everyday 2-bit robber might get a bounty of a thousand or ten, but this is an emergency. You set the bounty at $200,000, dead or alive.

This bandit ain’t getting away easy.

With

two

major hurricanes

in the last couple of weeks, “price gouging” is in the news. Whether it’s

$10 for a gallon of gas

in North Carolina or

$2,000 flights

out of Tampa, charging high prices during an emergency is extremely unpopular.

The reasoning is intuitive: when someone is in a desperate situation, they might be willing to pay high prices for basic goods like gas or water, but charging them these high prices is taking advantage of their misfortune.

But imagine if you were the sheriff of Ashville, NC, and it was your job to get more gasoline and bring it into town. You might offer a bounty of $10 a gallon, dead or alive. That’s a lot more than the usual everyday bounty, but this is an emergency. Anyone who can get gas into western North Carolina should be rewarded because that’s where people need it most.

Prices aren't just a transfer between buyer and seller. They're also also a signal and incentive to the whole world economy to get more high-priced goods to the high-paying area; they're a bounty.

Whether you’re chasing train-gangs or gasoline the last thing you’d want if you were the sheriff is a cap on the bounty price you’re allowed to set.

High prices on essential goods during an emergency are WANTED posters, sent out across the entire world economy imploring everyone to pitch in and catch the culprit.

The difficulty that many people may have in paying these higher prices is a serious tragedy, and one that can be alleviated through prompt government response e.g by sending relief funds and shipping in supplies. But setting prices lower doesn’t mean everyone can access scarce and expensive essential goods. In an emergency, there simply aren't enough of them to go around.

The central problem remains unsol...