trentmkelly/LessWrong-43k huggingface dataset:
Paradigm Shifts—change everything... except almost everything
by James Stephen Brown — 2024-11-23T18:34:13.088Z
Link: https://nonzerosum.games/paradigmshifts.html

The following is a short supplement to an upcoming post on the implications for conscious significance (a framework for understanding the free will and determinism debate) but is a more general observation.

The world has undergone many paradigm shifts; where a profound truth has been revealed about the nature of the universe and our place within it. Individuals also go through their own personal paradigm shifts when they change their beliefs-which can be a frightening prospect. But I would argue it doesn’t need to be, because profound paradigm shifts seldom change as much as we expect.

This is because, if there is a significant practical benefit to behaviour in accordance with a fact about nature, cultural evolution will often find this behaviour before we discover the fact. A few examples without leaving the letter ‘ G’:

Gravity

Discovering gravity did not inform us about how we could move around by applying pressure with our limbs to the large gravitational body upon which we’re hurtling through space. We’d already worked out how to react to gravity without knowing exactly what it was (in fact,

we still don’t know exactly what it is

). It’s even feasible that we could have learned to fly while still maintaining a flat-earth perspective, and even with our incomplete understanding of gravity today, we are capable of space travel.

God is Dead

The 19th Century saw an increase in scientific discoveries such as Darwin’s theory of Evolution as well as increasingly secular forms of government born out of the Enlightenment. The associated atrophy of religious belief during this period in western philosophy was encapsulated in

Nietzsche’s phrase

…

“God is dead, God remains dead, and we have killed him”

… leading philosophers to grapple with

Dostoevsky’s assertion

that…

“If God is dead, all is permitted.”

Both Dostoevsky and Nietzsche independently assumed that God’s death leaves a moral vacuum.

In reality, a materialist worldview demands similar interpersonal ethics as a religious one. So, when belief declined, moral behaviour persisted, not by divine coincidence, but because many religious morals had differentially survived, primarily due to their facility for social cohesion.

Germs

From a modern perspective, the germ theory of disease seems a perfect counter-example of a profound truth that made a tremendous difference to everyday people; the imperative to wash one’s hands has i...
A Sober Look at Steering Vectors for LLMs
by Joschka Braun — 2024-11-23T17:30:00.745Z

We thank Madeline Brumley, Joe Kwon, David Chanin and Itamar Pres for their helpful feedback.

Introduction

Controlling LLM behavior through directly intervening on internal activations is an appealing idea. Various methods for controlling LLM behavior through activation steering have been proposed. Most steering methods add a 'steering vector' (SV) to the model’s activations at a given layer and token position during inference. This approach leverages the hypothesis that many human-interpretable 'concepts' like truthfulness, refusal, and sentiment are represented as directions in activation space. Steering interventions are appealing because they use much less data than fine-tuning and do not require changes to the model parameters. In principle, this makes them more efficient and easy to controlling properties of the generated text in a desired way.

However, steering methods face a variety of significant challenges that hinder their practical applicability, comparability and further improvement. In this blog post, we discuss recent work performed at KASL and UCL DARK investigating challenges to steering methods for LLMs, focusing on 3 key challenges:

Current steering methods have substantial limitations

When evaluated more thoroughly or in different settings, many steering methods turn out to be unreliable and often fail to generalize outside their specific training setup.
The steerability of different concepts varies significantly, with some proving resistant to steering.

Typically used performance metrics overestimate steering effectiveness

Most steering methods are trained and evaluated in artificial settings like Multiple Choice Question Answering instead of settings that reflect deployment, like free-form text generation or behavior on typical application tasks. This raises questions about whether steering methods are useful and reliable in practice.

Methods are not compared on the same benchmarks and metrics

Most steering methods are trained and evaluated on method-specific datasets, in which they show good performance. However, evaluations on datasets of prior work or common benchmarks are often missing. This makes it difficult to track progress and evaluate the strengths and weaknesses of new methods.

Overall, there is still much research to do to make steering methods a reliable and useful tool for controlling LLMs. We conclude with a disc...
Text Posts from the Kids Group: 2018
by jefftk — 2024-11-23T12:50:05.325Z

Another round of liberating kid posts from

Facebook. For reference,
in 2018 Lily turned 4 and Anna turned 3.

(Some of these were from me; some were from Julia. Ones saying "me"
could mean either of us. Ones from others are labeled.)

2018-01-12

Since Lily started listening to "Heidi" a lot, her language has started
sounding more nineteenth-century at times. My favorites:

"When Fani opens my present, she will cry with joy and surprise."

"I will crawl into bed, as happy as a lark." (This was accompanied by
crawling across the entire room toward the bed, which she believes is part of
crawling into bed.)

2018-01-15

"Anna, what would you like for your bedtime snack?"

"Snacks!"

"Would you like a cheese stick?"

"Snacks!"

"Would you like some toast?"

"Snacks!"

"How about some nuts?"

"Snacks!"

"Ok, but what kind of snacks would you like?"

"Snacks!"

"Could you be more specific?"

"Snacks!"

"How about an apple?"

[Shakes head no] "Yes. Cut. Pieces. Eat."

2018-01-20

Lily: [at the Goodwill, playing with a Barney stuffie] "I think this might be a Hobyah"

2018-01-21

At the end of Scuffy the Tugboat Lily says: "he sailed around the river
block!"

2018-01-23

Lily: why didn't you do what I wanted?

Me: because you didn't tell me what you wanted.

Lily: but whyyyy didn't the thought come into your head?

2018-01-27

Me: "Why do you keep taking Anna's [toy] baby's bottle? You have your own."

Lily: "[sobbing] But I don't like mine! My baby has a sippy cup not a bottle,
and it has ooorange juice! My baby isn't going to get enough proooteein!
[more sobbing]"

2018-02-03

Lily: "there are three people cuddling in bed"

Julia: "can you count them?"

Lily: "one, two, three, four"

Jeff: "who had you forgotten?"

Lily: "I forgot mama"

Jeff: "if one of us got out, how many people would be in the bed?"

Lily: "three people"

Jeff: "and if one of them for out?"

Lily: "two people"

Jeff: "and if another?"

Lily: "one person"

Jeff: "and another?"

Lily: "nobody"

Jeff: "and another?"

Lily: "nobody can get out of an empty bed!"

"if we have four people in a bed and two get out, how many are left?"

"two!"

"if we have five people in a bed and one gets out, how many are left?"

"three!"

"[Julia holds up five fingers, puts one down] how many are left?"

"[counting carelessly like she already knows the answ...
Reward Bases: A simple mechanism for adaptive acquisition of multiple reward type
by Bogdan Ionut Cirstea — 2024-11-23T12:45:01.067Z

Authors: Beren Millidge ,Yuhang Song, Armin Lak, Mark E. Walton, Rafal Bogacz.

Abstract:

Animals can adapt their preferences for different types of reward according to physiological state, such as hunger or thirst. To explain this ability, we employ a simple multi-objective reinforcement learning model that learns multiple values according to different reward dimensions such as food or water. We show that by weighting these learned values according to the current needs, behaviour may be flexibly adapted to present preferences. This model predicts that individual dopamine neurons should encode the errors associated with some reward dimensions more than with others. To provide a preliminary test of this prediction, we reanalysed a small dataset obtained from a single primate in an experiment which to our knowledge is the only published study where the responses of dopamine neurons to stimuli predicting distinct types of rewards were recorded. We observed that in addition to subjective economic value, dopamine neurons encode a gradient of reward dimensions; some neurons respond most to stimuli predicting food rewards while the others respond more to stimuli predicting fluids. We also proposed a possible implementation of the model in the basal ganglia network, and demonstrated how the striatal system can learn values in multiple dimensions, even when dopamine neurons encode mixtures of prediction error from different dimensions. Additionally, the model reproduces the instant generalisation to new physiological states seen in dopamine responses and in behaviour. Our results demonstrate how a simple neural circuit can flexibly guide behaviour according to animals’ needs.

Author summary:

Animals and humans can search for different resources depending on their needs. For example, when you are thirsty at work, you may go to a common room where hopefully coffee or water is available, while if you are hungry, you would rather go to a canteen. Such ability to seek different resources based on a physiological state is so fundamental to survival, that is present also in simple animals. This paper proposes how this ability could arise from a simple neural circuit that can be mapped on evolutionary older parts of the vertebrate brain, called the basal ganglia. The model suggests that this circuit learns the availability of different reward types, and then combines them accordi

...
On The Rationalist Megameetup
by Screwtape — 2024-11-23T09:08:26.897Z

Introduction

First, a bit of blatant advertising: The East Coast Rationalist Megameetup is the evening of December 13th through the morning of December 16th, in Brooklyn, New York City. It's a weekend long unconference timed to coincide with NYC Secular Solstice. Despite the name, people come from all over North America, some for their first rationalist meetup and some who've been going for years. If that sounds fun, you can find out more and buy tickets at

rationalistmegameetup.com

Second, a brief note on nomenclature and terminology.

This has in the past been called the East Coast Rationalist Megameetup, The East Coast Solstice Megameetup, the NYC Megameetup, and sometimes just The Megameetup. Calling it The Rationalist Megameetup seemed presumptuous, but other titles are varying amounts of inaccurate as well. Attendees have come from New York City, Massachusetts, Utah, California, and sometimes as far away as Germany.

I'm going to call it The Rationalist Megameetup in this essay, and I'm inclined to get more relaxed about call it The Rationalist Megameetup as a shorthand in the future. The LessWrong Community Weekend is held in Germany, Vibecamp is held in Maryland, and the Rationality Organizer's Conference was held in California. Effective Altruism is the only adjacent group that consistently is on top of naming their big conferences by city, probably because they’re the ones who run half a dozen or more of them every year.

When I say Rationalist Megameetup in this article, I'm talking about this particular tradition. The LessWrong Community Weekend is obviously a big gathering of people in the Rationalist community. The Rationalist Megameetup is obviously a Saturday and Sunday spent in the company of people who read LessWrong.

A brief history of the Rationalist Megameetup

The first Rationalist style Secular Solstice was held in New York in 2011.

As far as I can tell, the East Coast Rationalist Megameetup

actually predates

the

first solstice

by several months, originally a simple call for groups in nearby cities to carpool together, travel to NYC, sleep on the local NYC LessWronger’s couches, and spend a weekend all getting together and sharing skills. This went well enough that they ran it again the next year.

The first big Secular Solstice - held in a theatre, not a living room, with professional musicians instead of a couple f...
Have we seen any "ReLU instead of sigmoid-type improvements" recently
by KvmanThinking — 2024-11-23T03:51:52.984Z

I read this and, it said:

there are huge low hanging fruit that any AI or random person designing AI in their garage can find by just grasping in the dark a bit, to get huge improvements at accelerating speeds.

have we found anything like this? at all? have we seen any "weird tricks" discovered that make AI way more powerful for no reason?
A few questions about recent developments in EA
by Peter Berggren — 2024-11-23T02:36:25.728Z

Having recognized that I have asked these same questions repeatedly across a wide range of channels and have never gotten satisfying answers for them, I'm compiling them here so that they can be discussed by a wide range of people in an ongoing way.

Why has EV made many moves in the direction of decentralizing EA, rather than in the direction of centralizing it? In my non-expert assessment, there are pros and cons to each decision; what made EV think the balance turned out in a particular direction?
Why has Open Philanthropy decided not to invest in genetic engineering and reproductive technology, despite many notable figures (especially within the MIRI ecosystem) saying that this would be a good avenue to work in to improve the quality of AI safety research?
Why, as an organization aiming to ensure the health of a community that is majority male and includes many people of color, does the CEA Community Health team consist of seven white women, no men, and no people of color?
Has anyone considered possible perverse incentives that the aforementioned CEA Community Health team may experience, in that they may have incentives to exaggerate problems in the community to justify their own existence? If so, what makes CEA as a whole think that their continued existence is worth the cost?
Why do very few EA organizations do large mainstream fundraising campaigns outside the EA community, when the vast majority of outside charities do?
Why have so few people, both within EA and within popular discourse more broadly, drawn parallels between the "TESCREAL" conspiracy theory and antisemitic conspiracy theories?
Why do university EA groups appear, at least upon initial examination, to focus so much on recruiting, to the exclusion of training students and connecting them with interested people?
Why is there a pattern of EA organizations renaming themselves (e.g. Effective Altruism MIT renaming to Impact@MIT)? What were seen as the pros and cons, and why did these organizations decide that the pros outweighed the cons?
When they did rename, why did they choose to rename to relatively "boring" names that potentially aren't as good for SEO as one that more clearly references Effective Altruism?
Why aren't there more organizations within EA that are trying to be extremely hardcore and totalizing, to the level of religious orders, the Navy SEALs, the Manhattan Project, or even a really

...
Paraddictions: unreasonably compelling behaviors and their uses
by Michael Cohn — 2024-11-22T20:53:59.479Z

I've been thinking about something I will clumsily call paraddictions: desires that don't quite rise to the level of an addiction, but that have a disproportionate and hard-to-moderate influence over your behavior. Can they be used as a tool for motivation and behavior change?

epistemic status: Most of the ideas here are generally solid behavior change principles that I'm just applying to a specific type of situation. The larger thesis of "this kind of strategy works and might be generalizable" is one I have less evidence for, and experimenting with it could have significant downsides.

My story

In 2011 my life wasn't going well. I was doing a postdoc in which I felt cynical yet overwhelmed, and thanks to depression and academic brainwashing, I felt like I wasn't good for anything else. A lot of the time I was spending most of my workday taking naps and playing flash games[1], and the high point of my day was when the clock hit 19:00 and my daily turns in Kingdom of Loathing would refill.

KoL is a turn-based HTML MMO[2] and I was deeply into its strategy, seasonal events, and community. You only get a limited number of turns per day, and if you miss them, they're gone.[3] I was intensely motivated to play those turns and missing a day felt like it would be a disaster.

One day, I'm not even sure why, I said to myself, Michael, you are going to get three hours of actual work on the diabetes study done today, or you're not playing KoL tonight. Michael, to his credit, accepted the challenge.

If you deal with depression or ADHD, you might not be shocked to hear that getting myself to start work was incredibly painful, and that once I started it went pretty well. I made honest-to-god progress and played my turns with extra gusto. I renewed the vow the next day, and kept adapting and expanding it over the weeks and months that followed.

There were a few days when I got distracted and didn't hit my goals. Not getting to play felt so unfair and awful, I just wanted to throw a tantrum. And after each one I tried to make hella sure it never happened again.

My life is better now! I eventually drifted away from KoL. I still use other rewards and scores and systems to keep myself on track, but nothing has ever motivated me like those daily KoL turns did. I'm not sure how I'd have been able to start working my way out of my hole without a reward I wanted so ba...
Literacy Rates Haven't Fallen By 20% Since the Department of Education Was Created
by Maxwell Tabarrok — 2024-11-22T20:53:59.007Z
Link: https://www.maximum-progress.com/p/literacy-rates-havent-fallen-by-20

On the heels of Donald Trump’s election and his promises to end the Department of Education, you may have seen claims like these spreading around X.

Source

This claim is based on two datapoints. First, is the literacy rate of

around 99% in 1979

which was measured by the US Census. After the Department of Education was created in the same year, the census stopped measuring literacy in their surveys and it’s since been tracked by the National Center for Education Statistics (NCES). The tweet’s second number comes from a recent NCES result that shows that

around 16%

of sampled Americans are at or below level 1 English literacy.

The problem is that these claims compare two completely different standards of literacy. The census measure of illiteracy is defined as:

The inability to read and write a simple message in English or in any other language. Illiteracy, in this sense, should be clearly distinguished from *functional illiteracy,* a term used to refer to persons who were incapable of understanding the kinds of written instructions that are needed for carrying out basic functions or tasks.

-

Source

If you can write a few words or even just your name in any language, this census measure will count you as literate.

The more recent NCES data point is a measure of English functional literacy which they define as:

The ability to understand, evaluate, use and engage with written texts to participate in society, to achieve one’s goals, and to develop one’s knowledge and potential.

. . .

English literacy [means completing] tasks that require comparing and contrasting information, paraphrasing, or making low-level inferences.

-

Source

So the more recent data shows a lower literacy rate because you need more reading comprehension to count as literate in this data and you need to know English. No conclusions about how literacy has changed over time can be supported based on comparing these two data points.

There are long term assessments of literacy that we can compare over time. Scores on the

Long-Term Trend reading assessment

from the NCES have been essentially flat since 1971.

So the claim that literacy rates have fallen substantially since the Department of Education was founded is false.

You Don’t Have to Make Stuff up to Prove the Failure of the D.O.E

The real data on education is not as bad as collapsing literacy rates, but it is more than bad enough to merit removing or r...
Plausibly Factoring Conjectures 
by Quinn — 2024-11-22T20:11:56.479Z

Introduction

In software design, several key principles share a common foundation: the concept of factoring, or breaking complex problems into smaller, manageable parts. Three prominent examples are:

Unix philosophy: Design programs to do one thing well.
Black box abstraction: Isolate functionality to focus on core problems.
Interface-based programming: Build systems from well-defined, interconnected modules.

These principles, though known by different names, all rely on factoring to create efficient, reliable, and maintainable systems. Even when developers aren't sure how to factor a problem, they often intuitively recognize that it should be factored: this notion of factoring has a fundamental role in effective software design.

In this post, we consider how this idea of factoring applies to mathematical theorem proving. Similar to the Unix philosophy, there is a sort of factoring that takes place in mathematics: breaking down difficult conjectures into simpler lemmas. But just as in software, collaboration is crucial for solving these problems, with mathematicians often building upon one another’s work, even without full awareness of all the details involved. This is reminiscent of Milton Friedman’s Lesson of the Pencil: the wood chopper on one side of the world has nothing in common with the granite miner on another, and neither of them know how to make a pencil by themselves. Yet, their incentives fall such that pencils get manufactured as a global project.

Just as in software and in the manufacturing of a pencil, success relies on collaboration and breaking problems into manageable parts. To that end, we present a first attempt at an incentive structure to encourage collaborative theorem proving, inspired by a recent concept called Plausible Fiction.

Plausible Fiction

In a recent Topos Institute blog post, Spivak proposed a general theory of change, which he called Plausible Fiction. A work of plausible fiction has four requirements:

It begins with the world as it is now, including all relevant details and without distortion;
It ends with a future that the author desires;
It follows a trajectory that obeys natural laws and social dynamics; and
It is memetically fit for collaborators: they readily understand and can spread the idea.

The mechanism that makes plausible fiction work is gap-filling. To collaborate on a work of plausible fiction, one finds a gap in the sto...