trentmkelly/LessWrong-43k huggingface dataset:
An unofficial "Highlights from the Sequences" tier list
by Orpheus16 — 2022-09-05T14:07:36.488Z

I previously wrote a summary of every "Highlights from the Sequences" post.

I also made a tier-list, ranking the posts based on how helpful or insightful I found them. I've included quotes from the posts in my top 5.

I'd be excited for others to share posts from The Sequences that they've found particularly helpful.

S+

Humans are not automatically strategic

Why do most of us, most of the time, choose to "pursue our goals" through routes that are far less effective than the routes we could find if we tried?

We do not automatically:

(a) Ask ourselves what we’re trying to achieve;
(b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;
(c) Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;
(d) Gather that information (e.g., by asking as how folks commonly achieve our goal, or similar goals, or by tallying which strategies have and haven’t worked for us in the past);
(e) Systematically test many different conjectures for how to achieve the goals, including methods that aren’t habitual for us, while tracking which ones do and don’t work;
(f) Focus most of the energy that *isn’t* going into systematic exploration, on the methods that work best;
(g) Make sure that our "goal" is really our goal, that we coherently want it and are not constrained by fears or by uncertainty as to whether it is worth the effort, and that we have thought through any questions and decisions in advance so they won't continually sap our energies;
(h) Use environmental cues and social contexts to bolster our motivation, so we can keep working effectively in the face of intermittent frustrations, or temptations based in hyperbolic discounting;

Lonely Dissent

Lonely dissent doesn’t feel like going to school dressed in black. It feels like going to school wearing a clown suit. That’s the difference between joining the rebellion and leaving the pack.

To be a scientific revolutionary, you’ve got to be the first person to contradict what everyone else you know is thinking. This is not the only route to scientific greatness; it is rare even among the great. No one can become a scientific revolutionary by trying to imitate revolutionariness.

You can only get there by pursuing the correct answer in all things, whether the correct answer is revolutionary or not. Bu

...
A Game About AI Alignment (& Meta-Ethics): What Are the Must Haves?
by JonathanErhardt — 2022-09-05T07:55:05.572Z

I'm working on a pc & mobile game about metaethics, ethics & AI alignment. (Our steam page + announcement teaser will be up in 1-2 weeks.) It's important to me that we nail the AI alignment part and give people a good idea about why AI alignment is hard and optimism shouldn't be our default position.

Some constraints that come with the medium:

We can't go into arguments that are too technical
We need to keep the exposition of these ideas short

What do you think of the following framing of why AI alignment is hard? Are we missing any crucial considerations? (This is merely our internal script and will later be turned into more digestible dialogues, mini games, etc.)

There are 2 ways alignment can pan out: Either the values are programmed into the system, or they are learned by the system.

1) Values are programmed into the system. But:

There is no agreement on which values.
All ethical theories we have use ontologies (valence, pain, phenomenology, rights, justice) that don’t easily translate into code.
All the ethical theories we have (including those on moral uncertainty) have *some* cases (often in the area of population ethics) where most people disagree with their results.
In practice we usually use acceptable proxy values when designing AI rewards (maximize link clicks instead of maximizing user value). But using proxy values is extremely risky for sufficiently powerful optimizers. They might find optimized world states which massively neglect our actual values.

2) Values are learned by the system. But:

There is much disagreement about the right dataset.
Even if we agree on a data set, it is unclear if the AI will generalize from the data set in an acceptable way.
Probably(?) the AI will find the most value in exploits. Exploits are cases where 1) the AI achieves its goals to a high degree, but 2) the AI has generalized poorly (by our standards). Examples: wireheading, goal modification, etc.

Other reasons for concern:

Ethical truths are probably different from empirical truths. An advanced AI may learn empirical truths on its own from enough data, but it seems unlikely that it will automatically converge on the ethical truth. Instead, it seems that any degree of intelligence can be combined with any kind of goal. (Orthogonality Thesis)
There are some instrumental goals that probably many sufficiently advanced intelligences will conve

...
[Exploratory] What does it mean that an experiment is high bit?
by Johannes C. Mayer — 2022-09-05T03:13:10.034Z

Disclaimer: This is an exploratory writing post. No checking for typos or other editing was done.

John Wentworth noticed that many people perfrom 1-bit, or low-bit experiments. Instead we we want to do experiments that give you as much information as possible. In this document I try to better understand what is going on.

An experiment is two things. A concrete set of steps that you perfrom, and a set of observations you record. Normally we start out with a set of questions (often just one) that we want the experiment to answer.

We can start out with a binary yes/no question, and then run the experiment. We would normally call this a one-bit experiment, but the experiment itself is not neccesairly 1-bit. We can collect many more observations that are not neccesary for answering the question. Normally we do.

E.g. when we train a neural network, then we get out a set of weights after training. This is our observation. To answer any specific question, we normally need to process our observations in specific ways. For example, we might want to know if a specific training setup produced sparse weights (most weights are zero). In that case we can process the weights to detect if they are sparse, given the training setup.

So normally we would call this a low bit experiment, that just tells if the weights are sparse. However, our observations from the experiments, i.e. the weights, can possibly be used to answer different questions as well.

So normally when we say that an experiment is low bit, we are talking about the question we are trying to answer.

We can think of an expeirment as a touple (Experimental Setup, Observations, Questions). Note that any experimental setup will have many possible observations that we can make, but usually we only record a tiny subset of these. The second entry in the touple is only containing these recorded observations.

An experiment can still be low bit, in terms that the experimental setup does not allow us to make many useful observations, or we do not record many useful observations, where useful refers to how many important questions we can answer. Also normally we want to ask questions that we can answer with an experiment, that give us a lot of useful information. E.g. if we can answer the quesion of how goals are strucutured in a neural network, then that would actually be really useful. For answering the question, we probably want t...
(Link) I'm Missing a Chunk of My Brain
by mukashi — 2022-09-05T02:10:04.912Z
Link: https://www.nytimes.com/2022/09/04/science/brain-language-research.html

I'm blown away by this story. This perfectly normal woman is missing a big chunk of the left hemisphere, where the functions associated with language are typically placed. I guess that the main takeaway could be:

"It seems that networks in the brain arrange in a particular way, but if you lose crucial brain regions as a baby - when the brain is still very plastic - these networks can reroute"
What's your Mission?
by GregorDeVillain — 2022-09-04T18:52:21.951Z

SOMETHING BIGGER THAN YOU

Your cause must be bigger than you.

It must become so important to you that if you fail or die in the service of that cause, it is okay, so long as it advances that cause.

How do you choose a good cause? Here are the main things to look for:

Does this cause accomplish something concrete? What?

How will you know when the cause is accomplished?

What are the concrete actions you can take to advance this cause?

If any of your answers are vague, you've chosen a bad cause.

Here's an example. Let's say your "cause" is a political party. Any political party; it doesn't matter which one.

For one thing, there is no point at which you can say, "We've won. Case closed. Pack your boots and go home, mission accomplished."

For another, because there is no concrete objective other than, "Get more people voted into office," it's one that stretches on forever, never freeing you - people get voted in, then they get voted out, and you've got to go do it all over again.

On the contrary, you may instead say, "I'm going to get XYZ law passed, because this law is good for the people." That is a good cause.

A purpose can be bigger than a cause. A purpose can be something like, "I will introduce hot pot restaurants to Western diners," because you think hot pot is absolutely delicious, and most of the Westerners you know who've tried hot pot think it's absolutely delicious, yet it's nearly impossible to find hot pot restaurants anywhere in the West.

To advance that purpose though, you will need a cause you believe in too, like, "I will open up the first hot pot restaurant in Leeds."

KNOWING WHAT'S IMPORTANT TO YOU

Beyond pure biological imperatives and emotional needs, you also have things that are important to you.

It might be that some teacher really turned you onto math when you were a child, when all your previous teachers had made math seem like the most horrifying subject there was. Now, you want to find a way to make math more appealing for school students too.

Or, it might be that you're mortified at the fact that people can lose limbs or be disfigured in fire or chemical or other accidents and there's nothing we can do about that except to offer them crude prosthetics, and you want to find a way to grow patients replacement parts using their own body cells.

Whatever it is, there's some way you want to change the world and make it bet...
EA, Veganism and Negative Animal Utilitarianism
by Yair Halberstadt — 2022-09-04T18:30:20.170Z

Veganism is on the face of it an extremely simple moral philosophy. If Animals have feelings, then eating animals is bad. Duh.

Effective Altruists tend to be people who actually take moral arguments at face value rather than ignoring them, and unsurprisingly many effective altruists are vegan.

But effective altruists also tend to be utilitarians, and as such are likely to want to minimise total animal suffering in aggregate, rather than taking the more deontologist view that you personally shouldn't cause suffering by eating animals.

So presumably vegan EAs assume that most farmed animals lives aren't worth living and so better not to eat farmed animals, even though this will cause them never to be raised in the first place.

It seems to me that most wild animals lives are just as terrible as those of farmed animals, especially for the vast majority of animals who give birth to thousands of young of which on average only 2 will ever reach adulthood. Their lives are presumably also not worth living.

As a utilitarian then, it should be far more important to wipe out as many animal habitats as possible rather than avoiding eating a relatively small number of animals by being a vegan.

I'm interested if anyone espouses such negative animal utilitarianism, or if anyone has any responses to it?
The ethics of reclining airplane seats
by braces — 2022-09-04T17:59:51.347Z

I enjoyed reading the replies to this tweet, since it's a lower stakes issue that has all the contours of broader ethical debates. Granted, what triggered the tweet was not low stakes:

There are passionately held beliefs on both sides (see replies to the original tweet for more). As with any argument, different principles lead to different conclusions. One principle with many adherents is that the rules follow directly from the design of the airplane:

But this could still screw over the person behind you. So maybe reclining is bad, based on the fact that it harms more than it helps?:

These views, by the way, are similar to what a travel industry analyst says to the NY Times: "Airplane etiquette is you only recline when necessary, and if you must recline, just put the seat back a little bit to get the comfort you need without encroaching too much on the person behind you."

Another principle: the person in back of you could have "property rights" over the area directly behind your unreclined seat:

But reclining may also be justified based on the consequences:

Many other variables. Long haul vs. short haul:

Dimmed lights:

Meals:

Height:

If the replies are at all representative, this issue is in a bad state where a significant share of people have opposing beliefs about what's right and when. So we should expect to see more conflicts between passionate passengers.

One potential solution is that the airlines try to coordinate everyone. An announcement could say "Our policy is that passengers should feel free to recline. Just check to make sure you do not spill the drink of the person behind you." This should douse the passions and lead to less conflict. A grumpy person being reclined on should feel less empowered; the loudspeaker announcement is common knowledge.

Another thing airlines could do is sell reclining and non-reclining tickets. Then everyone knows what they're getting---another way of making the policy more explicit.

This is not to say that either policy would be the best. Maybe an "asking equilibrium" is optimal, so that people can forge personalized agreements. "Is it alright if I recline?" "Sure, let me move my drink" or "Yes could I have a few minutes to finish eating?" or "I'm sorry but actually no, I'm extremely tall." I worry that this system would disadvantage nice people though.

This seems like an issue where kind and reasonable people could disagree. With ...
Russian Food for Petrov Day
by weft — 2022-09-04T17:57:55.770Z

For the past few years I've considered Russian food to be the traditional Petrov Day food for my own celebrations, but I've never managed to write a post about it at a reasonable time. Today, a few weeks before Petrov Day, I managed to think about it at a reasonable time and figured I'd rather do an 80/20 post about it, and hope people contribute more in the comments.

Why Russian Food?

It's good for celebrations to have specific food, such as turkey for Thanksgiving. They feel more special that way.

Most people don't eat Russian food on a regular basis, so it can be a "special" thing, in a way that e.g. hamburgers wouldn't be.

The biggest Petrov Day stories (Petrov and Arkhipov) are about Soviets during the Cold War, so it's thematic.
Many rationalists, such as myself, are of Russian / Eastern European descent so already have holiday memories around Russian food.

Easy Mostly Veg*an Russian Foods

Kvas, a beverage with tastes in between soda and beer, very lightly alcoholic, and available at your local Eastern European store.
Kasha aka buckwheat. Very easy to make. If you are a veg*n there are good fake chik'n bouillons available, and I strongly recommend cooking this with chicken / chik'n broth instead of plain water. Mushrooms and onions only if you're feeling fancy.
Veg*an Sausage. It's just a good easy veg*n protein option, and sausage is often paired with kasha.
Borscht, a vegetable soup with beets and cabbage, topped with (optional) sour cream. For an easy version, search for a recipe that uses canned beets. I'll put my dad's easy recipe in a comment below.
Black bread with butter and (optional) salmon roe.
Russian chocolates, individually wrapped. Available at your local Eastern European store. The Mishas (little bear) and chocolate covered jellies are generally well-liked.
Prototyping in C
by jefftk — 2022-09-04T17:50:01.565Z

Lately I've been writing some new ("

greenfield")
code in

C.
I feel a bit silly about this: isn't

Rust what you're supposed to use
if you want metal-level performance and are unconstrained by past
engineering decisions? Or

Go?
Something on the

JVM?
Even

C++? Why use
a 50 year-old language that is notorious for memory unsafety and

nasal
demons?

There are a few things about this situation that are a pretty good fit
for C:

I'm prototyping in a data-heavy situation, iterating on the
right way to solve various problems. Normally I would use Python for prototyping, and I am
using it a bunch here. In many cases, however, I need to run my code
quickly over a large amount of data and see how it works in practice,
and my Python implementations have generally been far too slow.

I'm implementing algorithms that are computationally
straight-forward: a Trie to count unique
fixed-length substrings (code),
or approximating the previous algorithm with multiple processes writing
to a big block of shared memory and accepting collisions (code).
If I needed library support or had a large amount of tricky logic I'd
use a different tool.

I'm processing very simple
formats: essentially just long strings of
[ACGT]*. This keeps the code rule
of two compliant. I wouldn't want to use C for any tricky parsing
unless it was sandboxed and I was very confident in the sandboxing
setup.

I'm working on this set of problems mostly by myself, so I
should choose the tooling where I'll be able to make progress most
quickly. It doesn't matter much right now if my code is a bit weird.
Once I get a better handle on how we're going to approach this problem
computationally it will likely make sense to rewrite in a modern
language, which will both be safer and more readable.

The biggest risk here is that prototype code will become production
code, there will always be something more urgent than a rewrite, and
at the core of our system we have a chunk of code in an unsafe
language that's poorly documented and confusing to read. This is
probably the strongest argument against starting anything in C, even
for prototyping. While I can't completely commit to ensuring this
doesn't happen, I'm going into this with my eyes open. And I will at
least commit to seriously documenting anything that's becoming
production code.

This is a bit of an unusual confluen...
Turn your flashcards into Art
by Heye Groß — 2022-09-04T17:31:08.528Z

Epistemic Status: this is just a fun idea I had, I will report back in the future if this increases my retention. Crosspost from heye.earth

Tl;dr: Use Midjourney/Dall-E 2/Stable Diffusion to generate visual mnemonics to increase retention.

Flashcards are powerful ways to learn, but they often use only the linguistic part of our brain. What if you could invoke the strong emotions that images entail for every piece of knowledge you want to learn, in a semi-automated way?

With Midyourney you can: just copy paste (please someone automate this for me) the front or back of any flashcard into the engine and you will get these beautiful images back. The following are a few examples from my Anki deck, more at my Midjourney account.

Consciousness

Exaptation

Emergence

Meta-Level

Probability Density

Reframing

System 2 Thinking

Mindfulness

Emotional

Utility Function

Value of Information

Systemic Desensitization

Cognitive

Environmental Design

Game Theory

End of Life

Backward Design

Action Theory
Let's Terraform West Texas
by blackstampede — 2022-09-04T16:24:15.151Z

[I'm half serious about this. I'm somewhat doubtful that it would work, but I like the idea. At the very least I think it would make a fantastic plot for a solar punk novel. If you want to rip it off for a story, go right ahead.]

I went to an ACX meetup yesterday and mentioned an idea that I’ve had off and on for a few years- terraforming the West Texas desert. They suggested I write something about it, so here is that something.

Land

West Texas has a huge amount of cheap land. I just went looking again and the first page of results were all between about $800 and $1700 per acre. Sorting by cheapest price per acre, I found several lots of 50 acres or so for 27-30k, which works out to about $550 an acre- and there are cheaper prices for larger chunks of land- there are multiple listings for 1000+ acre lots. The reason I mention this is to make it clear that the land out there is cheap and plentiful. The reason why it’s cheap and plentiful is because it’s damn near worthless- it’s desert. The water underneath the land is often brackish. The heat is unrelenting. The sun never stops shining.

Sunlight

West Texas is hot, dry and bright almost year around. Go to this website and select El Paso from the drop down- that region gets 5.87-7.42 peak sun hours every day, where “peak sun hours” is sunlight at maximum, unobstructed intensity- ~1000 watts of power per square meter. This is prime solar-panel land, but you can’t get that power to anywhere that people actually want to live. At a minimum, it would require hundreds of miles of power lines and infrastructure. If you wanted to use your electrical power locally, however, you have a near-endless supply. The only problem is storing it.

Storage

Conveniently, one of the other things that West Texas has in large quantities is brackish water, which coincides neatly with the cheapest land prices. You could store your electricity by pumping this water up into large reservoirs and retrieve it by allowing the water to flow back down to the aquifer through turbines. This is called pumped-storage hydroelectricity and it’s about 70-87% efficient- you lose 20-30% of your power, but once the water is pumped to the reservoir, there’s no loss of power over time- it can sit in a covered reservoir forever. Cool- but now what? You have endless electrical power and a method of storing it until you need it, but what’s the point?

Water

Th...
Help me find a good Hackathon subject 
by Charbel-Raphaël — 2022-09-04T08:40:30.115Z

Let's list the constraints:

Basically, people know how to code but don't really know ML
But at the same time, 3 days is enough time to learn a notion or understand and improve a template
If we don't want it to be a hard to evaluate, the evaluation metric must be clear
The topic must be related to AI safety, bonus if the topic is useful for ia safety
The topic must be difficult, and must be able to keep strong people busy even for 3 days
It must be possible to work on colab
bonus if the number of ways to approach the topic is large so that participants do not all implement the same thing
bonus if it's not too painful to organize

The goal of the hackathon is to find smart students who might be motivated to pursue in technical AI safety

I PayPal 50 euros for any idea I end up using in the hackathon