trentmkelly/LessWrong-43k huggingface dataset:
Utilitarianism and the replaceability of desires and attachments
by MichaelStJules — 2024-07-27T01:57:42.419Z

Summary

Consider a pill that would cause a happy person with a fulfilling life to abandon their most important desires and cherished attachments, including goals, career and loved ones, but increase their lifetime subjective well-being. If what’s best for someone is just higher subjective well-being (including even higher lifetime preference/desire satisfaction), then it would be better for them to take the pill. However, it seems to me that if they prefer not to take such a pill, to honour their current specific desires and attachments, it could be worse for them to take it (more).
I anticipate some responses and reply to them:

People aren’t always right about what’s best for themselves. R: That’s true, but attitude manipulation is quite different from other cases, where individuals neglect, discount or otherwise misweigh attitudes they do or will have (more).
Deontological constraints against involuntary manipulation. R: Deontological constraints could oddly recommend not to do what’s better for someone on their behalf (more).
Indirect reasons count against involuntary attitude manipulation. R: Probably, but I also think it wouldn’t be better for them in many cases where it would increase their well-being (more).
We can’t compare someone’s welfare between such different attitudes. R: We wouldn’t then have reason either way about manipulation, or to prevent manipulation (more).
The thought experiment is too removed from reality. R: In fact, reprogramming artificial minds seems reasonably likely to be possible in the future, and regardless, if this manipulation would be worse for someone, views consistent with this could have important implications for cause prioritization (more).

This kind of attitude manipulation would be worse for someone on preference-affecting views, which are in favor of making preferences (or attitudes) satisfied, but neutral about making satisfied preferences (for their own sake). Such views are also person-affecting, and so neutral about making happy people or ensuring they come to exist (for their own sake). I expect such views to give relatively less priority to extinction risk reduction within the community (more).

Acknowledgements

Thanks to Lukas Gloor and Chi Nguyen for helpful feedback. Thanks to Teo Ajantaival, Magnus Vinding, Anthony DiGiovanni and Eleos Arete Citrini for helpful feedback on earlier related drafts...
Inspired by: Failures in Kindness
by X4vier — 2024-07-27T01:21:42.848Z

silentbob's post "Failures in Kindness" is excellent. I love the idea that sometimes, when we exaimine a situation in depth, the most "kind" course of action can be highly conterintuitive. A few other examples I'd like to offer:

Appreciative Kindness

Imagine you meet a friend-of-a-friend for the first time while attending a gathering at their home. "Hey, welcome! It's great to meet you - can I get you anything?" they ask. There's nothing you really want right now, and you don't want to take from them or cause inconvienience, so you say "I'm fine, thanks."
Some people might assume declining their offer is kind. After all, wouldn't it be inconsiderate to make them go to the effort to proivde you with something you don't even really want?
But declining in this way will likely be percieved as a minor rejection. From the other person's perspective, they can't know the difference between:

In all sincerity, you are totally comfortable already and there's nothing they can do for you right now.
There is something they could give you which you would enjoy, but you won't accept it becuase you don't want to initiate the early stages of a recipriocal relationship with them.

The geniunely kind thing to do in this case is to accept some kind of token gesture and show lots of grattitude for it. Even if you're not thirsty, ask for a cold glass of water and say "thanks so much!" with a smile.
This scales up to larger favours too. If a friend offers to spend their Saturday helping you move house - rejecting this due to feelings of guilt about taking too much from them, or anxiety about being endebted to them, can feel kind, but probably isn't. Most people we regularly interact with suffer little from material scarcity, but far too often suffer from a lack of feeling valued+appreciated+connected to others. So when someone offers a gift, the maximally kind option is almost always to enthusiastically accept it with exuberant grattitude.

Assertive Kindness

Say you're hanging out with a group and your friend is ordering takeaway for everyone. "Okay what should we order?" she asks the group (a failure of Computational Kindness). You're anxious about not wanting to impose your own preferences on everyone else, so you say you're fine with anything (and everyone else in the room does the same).
This leads to an akward, protracted standoff where the person doing the ordering refuses to take ...
My Experience Using Gamification

by Wyatt S — 2024-07-26T23:06:53.392Z

Introduction

During the summer of 2024, I used two different self-improvement gamification tools. I don’t know whether I will continue using them, use different ones, or stop using gamification at all. Something to keep in mind is that there are plenty of people who found these tools useful; My experience is just a single example.

Guild of the ROSE

The first tool I used was called Guild of the ROSE. Guild of the ROSE is a self-improvement program directed towards people interested in improving their rationality skills, like people in this community. It has a skill tree you advance through, a character sheet depicting different aspects of yourself that can be improved, and workshops that you can go through that help you build further on different skills. To get an idea of how much experience I've had with it, I've used it for about two months (with large breaks in between).

Likes

I like the rationality focus, because I am interested in finding out what is true. I also like the workshops, because they build a sense of community. Having small groups where you can talk to other people is a big motivator for me to visit the Guild of the ROSE.

Possible Improvements

One way the Guild of the ROSE could make itself better is by adding custom habits as a feature. This could implement the streak feature from the other self-improvement website I used, called Project Skill Tree. Some of these habits that wouldn’t be too demanding on your time would be getting a regular wake-up and bed time. Some others that would be more demanding would be exercise, meditation, and spaced repetition flashcards. Having habits that are performed continuously would provide that extra kick to help you do things you otherwise wouldn’t have. The feature could remind users to set safe, sustainable habits so that they would not become overwhelmed. There is one skill currently in the skill tree for trigger-action plans, and I think it could tie nicely into that. I think that Guild of the ROSE would be better if it had more customizability in what tasks are considered relevant. The “Autodidact” skill and the “Basic Life Enhancement” skill are good examples of this. Personally, I think the “Basic Life Enhancement” skill should be available from the beginning.

Why I don’t use it often anymore

When I first got into the Guild of the ROSE, completing tasks on the skill tree b...
How the AI safety technical landscape has changed in the last year, according to some practitioners
by tlevin — 2024-07-26T19:06:47.126Z

I asked the

Constellation

Slack channel how the technical AIS landscape has changed since I last spent substantial time in the Bay Area (September 2023), and I figured it would be useful to post this (with the permission of the contributors to either post with or without attribution). Curious if commenters agree or would propose additional changes!

This conversation has been lightly edited to preserve anonymity.

Me: One reason I wanted to spend a few weeks in Constellation was to sort of absorb-through-osmosis how the technical AI safety landscape has evolved since I last spent substantial time here in September 2023, but it seems more productive to just ask here "how has the technical AIS landscape evolved since September 2023?" and then have conversations armed with that knowledge. The flavor of this question is like, what are the technical directions and strategies people are most excited about, do we understand any major strategic considerations differently, etc -- interested both in your own updates and your perceptions of how the consensus has changed!

Zach Stein-Perlman: Control is on the rise

Anonymous 1: There are much better “model organisms” of various kinds of misalignment, e.g. the stuff Anthropic has published, some unpublished Redwood work, and many other things

Neel Nanda: Sparse Autoencoders are now a really big deal in mech interp and where a lot of the top teams are focused, and I think are very promising, but have yet to conclusively prove themselves at beating baselines in a fair fight on a real world task

Neel Nanda: Dangerous capability evals are now a major focus of labs, governments and other researchers, and there's clearer ways that technical work can directly feed into governance

(I think this was happening somewhat pre September, but feels much more prominent now)

Anonymous 2: Lots of people (particularly at labs/AISIs) are working on adversarial robustness against jailbreaks, in part because of RSP commitments/commercial motivations. I think there's more of this than there was in September.

Anonymous 1: Anthropic and GDM are both making IMO very sincere and reasonable efforts to plan for how they’ll make safety cases for powerful AI.

Anonymous 1: In general, there’s substantially more discussion of safety cases

Anonymous 2: Since September, a bunch of many-author scalable oversight papers have been published, e.g. this,...
A Visual Task that's Hard for GPT-4o, but Doable for Primary Schoolers
by Lennart Finke — 2024-07-26T17:51:28.202Z

Sure, transformers can get silver at the IMO. But before we address those two remaining problems that still stand between AlphaProof and gold, let's go a few steps back and see how transformers are doing at supposedly low-level math competitions. In particular, we will consider a problem aimed at grades 3 and 4, from the largest international math exam competition around (the Mathematical Kangaroo).

The problem is designed to test children's spatial thinking skills and rightfully held up as a good example in an interview with the competition's president Meike Akveld, a knot theorist at ETH. This article is where I came across the idea as well.

It goes like this:

In the following image, you can see an open picture book. There are holes in its cover. On the first page, there are pictures of objects. Which pictures are visible when the book is closed?

To remove some ambiguity, we also give the list of objects to choose from, so in this case "Ginger", "Kiwi", "Pineapple", "Pear", "Watermelon". OK then, GPT-4o, what do you think?

To determine which pictures will be visible when the book is closed, we need to align the holes on the cover with the pictures on the first page.

From the image:

1. The holes in the cover are located in the second and third columns from the left.
2. The pictures of objects on the first page — from left to right — are: pineapple, watermelon, ginger, kiwi, pear.

Thus, when the book is closed:
- The second column will show the first image: pineapple.
- The third column will show the second image: watermelon.

ANSWER: pineapple, watermelon

Hm, that's wrong! The answer is {"Pineapple", "Ginger", "Pear"}. Then again, there are some good ideas in the completion, namely the basic setup of the problem being correctly summarized and the pictures identified.
Now, it is not so surprising that vision transformers are bad at grids, so let us grant the error about the hole locations. But the model also doesn't understand the symmetry; that a hole further to the left reveals objects further to the right, which is what the task wants to test in children.
Investigating whether this failure mode repeats across different images and iterations (seemingly it does), I made a generator for these problems and sampled 50 of them, resulting in an accuracy of 0.12 for GPT-4o with Chain of Thought, where a random baseline gives 0.04. [The code is available here] and...
Index of rationalist groups in the Bay Area June 2025
by Lucie Philippon — 2024-07-26T16:32:25.337Z

[Last update: 2025-06-07]

The Bay Area rationalist community has an entry problem! Lots of listed groups are dead, the last centralized index disappeared, communication moved to private discord and slacks. This is bad, so we're making a new index, hopefully up to date and as complete as we can!

This is now being mirrored at BayRationality.com, which will hopefully get prettier formatting over time.

Communication

Discord: Bay Area Rationalists: https://discord.gg/EpG4xUVKtf

Email Group: BayAreaLessWrong: https://groups.google.com/g/bayarealesswrong

Local Meetup Groups

Taco Tuesday: by Austin Chen, founder emeritus of Manifold. Check his Manifold questions page for the next date!

Thursday Dinners in Berkeley: Advertised on the Discord server and Google group, alternating between a few restaurants on the northwest side of UC campus. Has an associated Signal group chat.

Lighthaven Sequences Reading Group: Currently Tuesday, hosted by the Lighthaven team. They advertise on LessWrong.

San Francisco LessWrong Meetup: Meets every first Saturday in San Francisco, hosted by Nate and Andrew.

Thai Temple Brunch, on irregular Sundays in South Berkeley. Advertised on Discord server and in some Signal groups.

Bay Area ACX Meetups: For the ACX everywhere meetups twice per year, and some other sporadic events.

The Bayesian Choir: Usually meets every other Sunday near North Oakland, and more often during the fall to rehearse for the Solstice. Choir Director and primary organizer is Blake Jones, and coordination happens on the linked Discord server.

Official policy on new singers is "Anyone who can pitch-match is welcome.", though joining tends to be suspended during the rehearsals for Solstice. See more detail on the introduction doc.

South Bay

David Friedman hosts irregular SSC/ACX meetups at his home in San Jose, tracked on a page of his personal website and typically also announced on the email group and Discord server. They are weekend events, starting early afternoon and running through dinner (provided) and into the evening. RSVPs are preferred for meal planning purposes.

Housing

To find spots in group houses, temporary or long term, you can use the Bay Area EA/Rationality Housing Board. The EA Houses spreadsheet also has some entries in the Bay.

It probably works best to ask people in the Bay if they know of housing opportunities, as lots of housing is provided peer-...
End Single Family Zoning by Overturning Euclid V Ambler
by Maxwell Tabarrok — 2024-07-26T14:08:45.046Z
Link: https://www.maximum-progress.com/p/ending-single-family-zoning-by-overturning

On 75 percent or more of the residential land in most major American cities

it is illegal

to build anything other than a detached single-family home.

95.8

percent of total residential land area in California is zoned as single-family-only, which is 30 percent of all land in the state. Restrictive zoning regulations such as these probably lower GDP per capita in the US by

8

-

36

%. That’s potentially tens of thousands of dollars per person.

Map of land use in San Jose, California

. Pink is single family only (94%)

The legal authority behind all of these zoning rules derives from a 1926 Supreme Court decision in

Village of Euclid v. Ambler Realty Co.

Ambler realty held 68 acres of land in the town of Euclid, Ohio. The town, wanting to avoid influence, immigration, and industry from nearby Cleveland, passed a restrictive zoning ordinance which prevented Ambler realty from building anything but single family homes on much of their land, though they weren’t attempting to build anything at the time of the case.

Ambler realty and their lawyer (

a prominent Georgist!

) argued that since this zoning ordinance severely restricted the possible uses for their property and its value, forcing the ordinance upon them without compensation was unconstitutional.

The constitutionality claims in this case are about the 14th and 5th amendment. The

5th amendment

to the United States Constitution states, among other things, that “private property [shall not] be taken for public use, without just compensation.” The part of the 14th amendment relevant to this case just applies the 5th to state and local governments.

There are two lines of argument in the case. First is whether the restrictions imposed by Euclid’s zoning ordinance constitute “taking” private property at all. If they are taking, then the 5th amendment would apply, e.g when the govt takes land via eminent domain, they need to compensate property owners. However, even government interventions that do take don’t always have to offer compensation. If the government, say, requires you to have an external staircase for fire egress, they don’t have to compensate you because it protects “health, safety, and welfare” which is a “

police powers

” carveout from the takings clause of the 5th amendment. The other line of argument in the case is that zoning ordinances, while they do take from property owners, do not require compensation because...
Common Uses of "Acceptance"
by Yi-Yang — 2024-07-26T11:18:30.719Z

Edit (2024-07-31): I changed the introduction.

If you have Googled “how to do acceptance” and you have found a lot of the explanations confusing or in conflict with one another, you have come to the right place. Learning about “acceptance” as a mental move was surprisingly challenging to me too, and I think it’s because of these reasons:

It is obvious that “acceptance” has different dictionary definitions (and hence mental moves), but when “acceptance” is used in more casual advice, they can be unclear. Take a look at these three examples: (a) “you should accept yourself”; (b) “you should accept your feelings”; and (c) “you should accept reality”. Ask me to differentiate these back in 2023 and I’ll probably give you some “vibey” answers that are probably half-correct. I’m usually fine with just going off from vibes, but they seem to bounce off from me this time.
More comprehensive advice do describe the actual mental moves at a concrete level, but they’re also all quite different from each other despite using the same word. In fact, I’ve found nine different types of “acceptance” related advice from ten different sources (e.g., Acceptance Commitment Therapy). A single standard for “acceptance” doesn’t exist.
Furthermore, a few of these sources incorporated new definitions (e.g., kindness) into “acceptance” that are not in fact dictionary definitions of “acceptance”. “Radical Acceptance” is one source that did that. And I got the sense that “acceptance” as a concept has been “stretched” pretty far.

Taken together, they turned my journey to practise “acceptance” into an amateur

disentanglement

exercise. My cursory search of “acceptance” turned into a much wider literature review, including literature around the use of words in relation to truth-seeking. And I’ve decided to put some of my findings down in this writing with this following structure:

“Acceptance” as seen in dictionaries
“Acceptance” as seen in various advice
Some thoughts and analysis on how “acceptance” is commonly used

Here’s a warning, this piece of writing can get tedious--I’ve written this for people who have found literature on “acceptance” confounding. Or if you’ve been wanting to practise “acceptance” but don’t know where to start, you can just jump straight to Section 2 and pick a type of advice that you resonate most with. Or if you’re just interested to learn more about my thoughts on “acc...
Universal Basic Income and Poverty
by Eliezer Yudkowsky — 2024-07-26T07:23:50.151Z

(Crossposted from Twitter)

I'm skeptical that Universal Basic Income can get rid of grinding poverty, since somehow humanity's 100-fold productivity increase (since the days of agriculture) didn't eliminate poverty.

Some of my friends reply, "What do you mean, poverty is still around? 'Poor' people today, in Western countries, have a lot to legitimately be miserable about, don't get me wrong; but they also have amounts of clothing and fabric that only rich merchants could afford a thousand years ago; they often own more than one pair of shoes; why, they even have cellphones, as not even an emperor of the olden days could have had at any price. They're relatively poor, sure, and they have a lot of things to be legitimately sad about. But in what sense is almost-anyone in a high-tech country 'poor' by the standards of a thousand years earlier? Maybe UBI works the same way; maybe some people are still comparing themselves to the Joneses, and consider themselves relatively poverty-stricken, and in fact have many things to be sad about; but their actual lives are much wealthier and better, such that poor people today would hardly recognize them. UBI is still worth doing, if that's the result; even if, afterwards, many people still self-identify as 'poor'."

Or to sum up their answer: "What do you mean, humanity's 100-fold productivity increase, since the days of agriculture, has managed not to eliminate poverty? What people a thousand years ago used to call 'poverty' has essentially disappeared in the high-tech countries. 'Poor' people no longer starve in winter when their farm's food storage runs out. There's still something we call 'poverty' but that's just because 'poverty' is a moving target, not because there's some real and puzzlingly persistent form of misery that resisted all economic growth, and would also resist redistribution via UBI."

And this is a sensible question; but let me try out a new answer to it.

Consider the imaginary society of Anoxistan, in which every citizen who can't afford better lives in a government-provided 1,000 square-meter apartment; which the government can afford to provide as a fallback, because building skyscrapers is legal in Anoxistan. Anoxistan has free high-quality food (not fast food made of mostly seed oils) available to every citizen, if anyone ever runs out of mone...