trentmkelly/LessWrong-43k huggingface dataset:
The Talk: a brief explanation of sexual dimorphism
by Malmesbury — 2023-09-18T16:23:56.073Z

Cross-posted from substack.

"Everything in the world is about sex, except sex. Sex is about clonal interference."
– Oscar Wilde (kind of)

As we all know, sexual reproduction is not about reproduction.

Reproduction is easy. If your goal is to fill the world with copies of your genes, all you need is a good DNA-polymerase to duplicate your genome, and then to divide into two copies of yourself. Asexual reproduction is just better in every way:

Sexual	Asexual
Build costly DNA-manipulating machinery that chops the DNA into pieces to produce gametes	Just copy yourself bro
Scout the perilous wild for a mate, and perform a complicated ceremony so your gametes fuse with each other	Just copy yourself bro
Only pass 50% of your genome on to the next generation	Just copy yourself bro
Differentiate into two types, making it twice as hard to find a matching gamete	Just copy yourself bro
Make all kinds of nonsense ornaments to satisfy the other sex's weird instincts	Just copy yourself bro

It's pretty clear that, on a direct one-v-one cage match, an asexual organism would have much better fitness than a similarly-shaped sexual organism. And yet, all the macroscopic species, including ourselves, do it. What gives?

Here is the secret: yes, sex is indeed bad for reproduction. It does not improve an individual's reproductive fitness. The reason it still took over the macroscopic world is that evolution does not simply select for reproductive fitness.[1]

Instead, the evolution of sexual dimorphism is a long sequence of strange traps, ratchets and outer-world eldritch cosmic forces that made it somehow inevitable. So let's talk about those things your parents never told you about.

SMBC

The birds, the bees, and the fission yeast

What bugs me is that, not only most people have absolutely no idea why sexual dimorphism exists, but they seem entirely fine with that. Our lives are punctuated with all sorts of frankly weird practices related to it, but the reasons we ended up there remain obscure even to many biologists.

So I figured I would write up a summary of some popular theories. This way, when time comes, you can explain to your children the long evolutionary trajectory that culminated in VR ChatGPT cat-girlfriends.

(Note 1: As always with evolutionary biology, everything in this article is subject to uncertainty, controversy and mystery. Always keep in mind the Golden Rules of biology: all ...
Where might I direct promising-to-me researchers to apply for alignment jobs/grants?
by abramdemski — 2023-09-18T16:20:03.452Z

It sometimes happens that people who I've talked to or worked with ask me where they should go for financial support for their research. I haven't developed a standard list of answers to this question. It seems to me like there are a lot of new orgs recently, and I'm losing track!

If you are looking for such applicants or know someone who is looking, consider replying as an answer (or sending me a PM if that makes more sense for whatever reason).
[Review] Move First, Think Later: Sense and Nonsense in Improving Your Chess
by Arjun Panickssery — 2023-09-18T15:10:16.454Z
Link: https://arjunpanickssery.substack.com/p/review-move-first-think-later-sense

The author is Dutch International Master Willy Hendriks (IM is the rank below Grandmaster; there are ~4000 IMs and ~2000 GMs). He takes aim at chess instructors like IM Jeremy Silman, whose popular books The Amateur’s Mind and How to Reassess Your Chess teach students a structure for thinking about chess positions:

Before you get carried away, let me remind you: DON'T look at individual moves! In fact, never calculate until you understand the basic components (imbalances) of the position.

The Amateur’s Mind

Imbalances include a space advantage on one side of the board, or two knights traded for two bishops, or an unsafe opposing king. Silman lists imbalances like these and tells you to use them to formulate plans which only then suggest individual moves.

Hendriks rejects this definitively: “No chess player thinks like this, no one has learned to play chess by thinking like this and even trainers and authors of chess books don’t think like this.” Instead, the verbal descriptions are mostly retroactive and follow the initial step of identifying good candidate moves from pattern recognition alone. He gives you some reasons to think this:

He gives some examples of positions where the best move is hard to find unless you’re familiar with the pattern. In the game below (Oleg Romanishin v Predrag Nikolic, Leningrad 1987) I marked the correct move for black with an arrow. It’s hard to find this move unless you’ve been exposed to similar maneuvers before but easy to fabricate yourself some position justifications (“Yes, I saw this bishop being a bit hampered by my pawns, and he has weakened his light squares … works nicely together with the rook’s semi-open file on f8”). It’s often difficult to get students to produce the move using verbal descriptions like these as a hint.
He cites Adrian de Groot’s research from Thought and choice in chess in which grandmasters and weaker players think out loud about positions. Hendriks argues against the top players displaying extreme verbal clarity. Instead, “de Groot saw no great differences regarding the process of decision-making: the grandmaster calculated not much more or deeper and didn’t seem to decide on his move in a fundamentally different way than the lesser player.”
He points out that common positional maxims are probably wrong and contradict each other in any case. An example is taken from Silman: “During my private lessons

...
Technical AI Safety Research Landscape [Slides]
by Magdalena Wache — 2023-09-18T13:56:04.418Z

I recently gave a technical AI safety research overview talk at EAGx Berlin. Many people told me they found the talk insightful, so I’m sharing the slides here as well. I edited them for clarity and conciseness, and added explanations.

Outline

This presentation contains

An overview of different research directions
Concrete examples for research in each category
Disagreements in the field

Intro

Overview

I’ll start with an overview of different categories of technical AI safety research.

The first category of research is what I would just call alignment, which is about making AIs robustly do what we want. Then there are various “meta” research directions such as automating alignment, governance, evaluations, threat modeling and deconfusion. And there is interpretability. Interpretability is probably not enough to build safe AI on its own, but it’s really helpful/probably necessary for various alignment proposals. Interpretability also helps with deconfusion.

I’m using clouds because the distinction between the categories often isn’t very clear.

Let’s take a closer look at the first cloud.

What exactly do I mean by alignment? What do we align with what?

In general, we want to make AIs do what we want, so we want to align “what we want” with “what the AI does”. That’s why it’s called alignment.

We can split this up into intent alignment (make the AI want what we want) and capability robustness (make it able to robustly do what it wants).

And we can split intent alignment up into outer alignment (find a function that captures what we want) and inner alignment (ensure that what the AI ends up wanting is the same as what’s specified in the function that we trained it on).

There are a few ways in which this slide is simplified: The outer/inner alignment split is

not necessarily the right frame

to look at things. Maybe “what the AI wants” isn’t even a meaningful concept. And many approaches don’t really fit into these categories. Also, this frame looks at making one AI do what we want, but we may end up in a

multipolar

scenario with many AIs.

Concrete Technical Research

In this section I’ll give some examples to give you a flavor of what kinds of research exists in this space. There is of course a lot more research.

Let’s start with outer alignment.

Outer alignment i...
The omnizoid - Heighn FDT Debate #5
by Heighn — 2023-09-18T11:54:25.081Z

A reaction to omnizoid's latest post in this debate.

omnizoid proposed a problem in an earlier post, which I steelmanned as follows:

there's an agent who can (almost) perfectly predict whether people will cut off their legs once they exist
this agent only creates people who he predicts will cut off their legs once they exist
existing with legs > existing without legs > not existing

My reaction to this problem was this:

FDT'ers would indeed cut off their legs: otherwise they wouldn't exist. omnizoid seems to believe that once you already exist, cutting off your legs is ridiculous. This is understandable, but ultimately false. The point is that your decision procedure doesn't make the decision just once. Your decision procedure also makes it in the predictor's head, when she is contemplating whether or not to create you. There, deciding not to cut off your legs will prevent the predictor from creating you.

omnizoid disagrees:

But you only make decisions after you exist. Of course, your decisions influence whether or not you exist but they don’t happen until after you exist. But once you exist, no matter how you act, there is a zero percent chance that you won’t exist.

But if the predictor implemented your decision procedure when making his prediction - and this is, I believe, what is usually meant by "predicting" in this context - then your decision is NOT only made after you exist: it is also made in the predictor's head before you even exist. That is the whole point of FDT: subjunctive dependence.

omnizoid and I also debated Wolfgang Schwarz's Procreation example. In omnizoid's latest post, he does mention the problem, but doesn't engage with my latest point - instead, he basically just reiterates his latest point. Therefore, I won't further discuss this part.

Unfortunately, omnizoid does continue his arguing-from-authority thing:

I think higher-order evidence of this kind is useful. If 99.9% of people who seriously study a topic agree about X, and you disagree about X, you should think your reasoning has gone wrong.

I actually agree this is a pretty good indication you are wrong about X. However, to make this particular claim, you should

give a definition for "seriously"
show the results and details of a study that demonstrates this supposed 99.9%

omnizoid has done neither, so his claim is pretty much meaningless. (If he had done both, I would still not reject FDT, a...
Ask for Feelings not Tunes
by jefftk — 2023-09-18T02:10:03.450Z

In putting on a contra dance, it's standard for the caller to pick the
dances and the band to pick the tunes. Since it's much more satisfying
when the tune fits the dance, the band and caller confer briefly
between each dance. For example, the caller might say they want
something "smooth and pretty" or "energetic with a chunky A-part".

Sometimes, however, callers fall into a trap of referencing a specific
tune: "Trip to Moscow, or something with a similar feel". They
remember the dance feeling especially good with that tune on some
prior occasion with another set of musicians, and they're hoping to
recreate that pairing. The problem is, while tunes do have inherent
qualities that affect how well they fit dances, there's also a huge
amount in how they're played. Consider a caller who really liked the
trancey interplay of the Free Raisins playing Ian Ball's Trip to Moscow:

(mp3
clip, full
track)

And then imagine they ask Kingfisher for it and get something more EDM
inspired:

(mp3
clip, full
track)

Or Last Exit and get a full up almost polka-feel:

(mp3
clip, full
track)

While the dance won't fall apart if they're hoping for one of these
and get a different one, it's probably also not going to be a great
fit for this spot in the program.

So:

Callers: instead of giving a tune as an
example, try thinking of what you liked about it and ask for that.

Bands: If a caller names a tune don't just go and play it; ask
them what they like about it in this context.

Comment via: facebook, mastodon
Three ways interpretability could be impactful
by Arthur Conmy — 2023-09-18T01:02:30.529Z

Epistemic status: somewhat confident that overall these arguments show interpretability could be impactful, but not confident on any individual point.

Background: I wrote a draft of this post

here

after reading

Charbel’s work

. Thanks to Charbel and several others’ feedback, the post was improved a lot: I would definitely recommend adversarial collaboration so thanks Charbel! I’m hoping to discuss interpretability’s impact further so posting now.

Three ways interpretability could be impactful

Just read the test and validation set parts if you have little time.

1. Test set interpretability: Finding alignment failures

What this is. We want to find alignment failures with interpretability that other forms of auditing do not find. Before deploying a model, we find inputs that seem to elicit new capabilities of the model. Either A) the model has correctly generalized to novel capabilities or B) the model is deceiving our evaluation process to produce convincing but false outputs. We then use our future interpretability tools to try to understand some of the computation the AI is doing to reach these new capabilities. We either become more confident that the AI has correctly generalized, or we’ve uncovered deceptive computation and we can retrain and/or provide robust evidence of misalignment to relevant actors.

Why this isn’t achievable with other methods. If a model can deceive our evaluation process it could be very difficult to observe this deception with mere behavorial evals. See

here

.

In my mind test set interpretability primarily targets a specific set of alignment failures, illustrated here:

Figure 1.

In the appendix I outline my reasoning behind (interpretability’s role in) Figure 1.

2. Validation set interpretability: A better science of alignment.

What this is (thanks to Neel). We have little ground truth on whether our models are misaligned now or how far methods such as RLHF will further scale. More generally, we understand little about how machine learning works, which limits our ability to reason about future systems. Interpretability could first and foremost actually provide evidence for what our alignment techniques are doing (e.g interpreting RLHF reward models) and secondly give us a better evidence base for reasoning about deep learning. I think that

Progress Measures for Grokking Via Mechanistic Interpretability

...
Show LW: Get a phone call if prediction markets predict nuclear war
by Lorenzo — 2023-09-17T22:25:21.206Z
Link: https://recursing.github.io/nukes/

hasrussialaunchednukes.com used to be a website "to help [the author] and a few of [their] friends get notified when Russia launches any nukes towards Ukraine, which would be a sign for many of us to leave the big cities we live in, and chill in the countryside for a few weeks." It used prediction markets for 2022, so it stopped working in 2023.

Some people asked me to make a version for 2023, so here it is at https://recursing.github.io/nukes.[1]

I manually curate the list of phone numbers, a bunch of which were sent to me by one of the people requesting this, you can ask for your number to be added using any of the links at the bottom of the website.

You can look at the incredibly hacky script I use to make phone calls here: it runs every 10 minutes on a Hetzner VPS I use for a bunch of things[2]

I don't think this is extremely reliable, I give an ~80% chance that it will work, and I'll do my best to prevent false alarms, but some people found it useful, and I was asked to share this here.

I would be curious to hear if you have any thoughts!

^

Why do it in mid-september, given that the year is almost over? I actually implemented this in February, but procrastinated until today to write this post. But it will be trivial to update it for 2024, once the year is over.

^

I use vonage instead of twilio to handle phone calls because someone else was implementing a similar thing with twilio, and we wanted to have redundancy
Microdooms averted by working on AI Safety
by Nikola Jurkovic — 2023-09-17T21:46:06.239Z
Link: https://forum.effectivealtruism.org/posts/DBDpnAhxvRWmfmtfv/microdooms-averted-by-working-on-ai-safety

Disclaimer: the models presented are extremely naive and simple, and assume that existential risk from AI is higher than 20%. Play around with the models using this (mostly GPT-4 generated) jupyter notebook.

1 microdoom = 1/1,000,000 probability of existential risk

Diminishing returns model

The model has the following assumptions:

Absolute Risk Reduction: There exists an absolute decrease in existential risk that could be achieved if the AI safety workforce were at an "ideal size." This absolute risk reduction is a parameter in the model.

Note that this is absolute reduction, not relative reduction. So, a 10% absolute reduction means going from 20% x-risk to 10% x-risk, or from 70% x-risk to 60% x-risk.

Current and Ideal Workforce Size: The model also takes into account the current size of the workforce and an "ideal" size (some size that would lead to a much higher decrease in existential risk than the current size), which is larger than the current size. These are both parameters in the model.
Diminishing Returns: The model assumes diminishing returns on adding more people to the AI safety effort. Specifically, the returns are modeled to increase logarithmically with the size of the workforce.

The goal is to estimate the expected decrease in existential risk that would result from adding one more person to the current AI safety workforce. By inputting the current size of the workforce, the ideal size, and the potential absolute risk reduction, the model gives the expected decrease.

If we run this with:

Current size = 350
Ideal size = 100,000
Absolute decrease (between 0 and ideal size) = 20%

we get that one additional career averts 49 microdooms. Because of diminishing returns, the impact from an additional career is very sensitive to how big the workforce currently is.

Graph of how much existential risk is reduced based on the size of the AI safety workforce under the diminishing returns model.

Pareto distribution model

We assume that the impact of professionals in the field follows a Pareto distribution, where 10% of the people account for 90% of the impact.

Model Parameters

Workforce Size: The total number of people currently working in AI safety.
Total Risk Reduction: The absolute decrease in existential risk that the AI safety workforce is currently achieving.

If we run this with:

Current size = 350
Absolute risk reduction (from current size) = 10%

We get that, i...
Eugenics Performed By A Blind, Idiot God

by Bentham's Bulldog — 2023-09-17T20:37:13.650Z

(Azathoth, Lovecraft’s famous blind, amoral, idiot God.)

Crosspost of this.

I’m hugely in favor of gene editing and other actions that would improve the welfare of future people. If we could perform genetic engineering that made future people smarter, happier, and less likely to get diseases, I’d be in favor of it. This assumption is controversial. Many people think there’s something immoral about changing the genes of humans, even in ways that are expected to improve their quality of life. They think that such an idea sounds too much like the clearly objectionable coercive eugenics of the Nazis, for instance.

But you know what would be almost as bad as eugenics performed by Nazis—eugenics performed by a totally random, amoral selector. This eugenicist wouldn’t have cruel ideas about Aryan superiority, for instance—instead, they have a bizarre fetishism of reproduction. This selector performs eugenics so that only those who reproduce a lot—and also help out their kin—are likely to pass on their genes.

Such a selector is wholly unconcerned with human welfare. It makes it so that humans can’t empathize with those far away, because warring with native tribes is effective. It makes it so that

men are naturally more aggressive than women

, committing 96% of homicides, all because in the evolutionary past it was beneficial—it enabled more efficient fighting, for instance.

In fact, this selector has

selected for individuals

who are likely to engage in “rape . . . infanticide, abuse of low-status individuals, and murdering all those people over there and taking their stuff.” It selects for individuals who reproduce frequently, rather than those who are good, moral, or even happy.

In fact, in some other species, things are even worse. Some species give birth to

hundreds of millions of eggs

, many of whom contain sentient beings almost all of whom die a

horrible painful death at a young age

. This selector makes it so that

male ducks have corkscrew penises

so that they can rape female ducks more efficiently.

This predictor has been operating for billions of years. Their amorality results in them designing both all sorts of horrifying, rapidly multiplying bacteria and viruses that kills lots of people and animals alike and various defense mechanisms. But after millions of years, it offers for you to take over their job. Rather than selecting for prolificness alone, you ca...
Actually, "personal attacks after object-level arguments" is a pretty good rule of epistemic conduct
by Max H — 2023-09-17T20:25:01.237Z

Background: this post is a response to a recent post by @Zack_M_Davis, which is itself a response to a comment on another post. I intentionally wrote this post in a way that tries to decontextualize it somewhat from the original comment and its author, but without being at least a bit familiar with all of the context, it's probably not very intelligible or interesting.

This post started as a comment on Zack's post, but I am spinning it out into own post because I think it has broader applicability and because I am interested in hearing thoughts and responses from the readers and upvoters of that post, moreso than from its author.

I originally responded to Zack's post with a comment here, but on further reflection, I want to strengthen and clarify some claims I alluded to in that comment.

I see Zack's post as making two main claims. I have medium confidence that both of the claims are false, and high confidence that they are not well-supported by the text in the post.

Claim one (paraphrased): "personal attacks (alt. negative character assessments) should only come after making object-level arguments" isn't actually a {good,useful,true} rule of epistemic conduct.

I see the primary justifications given for this claim in the text as (paraphrased):

The person claiming this is a rule should be able to explain why they think such a rule would systematically lead to more accurate beliefs ("maps that reflect the territory").
In fact, no such valid explanation is likely to exist, because following such a rule would not systematically lead to the formation of more accurate beliefs.

The issue with the first justification is that no one has actually claimed that the existence of such a rule is obvious or self-evident. Publicly holding a non-obvious belief does not obligate the holder to publicly justify that belief to the satisfaction of the author.

Perhaps a more charitable interpretation of the author's words ("I think [the claimant]... should be able to explain why such a rule systematically produces maps that reflect the territory...") is that the absence of a satisfactory explanation from the claimant is Bayesian evidence that no such explanation exists. But if that's what the author actually meant, then they should have said so more plainly, and acknowledged that this is often pretty weak evidence.

(Relevant background context: Zack has previously argued at great length that...