trentmkelly/LessWrong-43k huggingface dataset:
Was the K-T event a Great Filter?
by Yoreth — 2022-02-23T20:35:58.898Z

I've been wondering about this question for a while, but some new research released today reminded me of it:

The seasonal timing of the catastrophic end-Cretaceous bolide impact places the event at a particularly sensitive stage for biological life cycles in the Northern Hemisphere. In many taxa, annual reproduction and growth take place during spring. Species with longer incubation times, such as non-avian reptiles, including pterosaurs and most dinosaurs, were arguably more vulnerable to sudden environmental perturbations than other groups43 (for example, birds).

Similarly, large-scale wildfires raging across the Southern Hemisphere9,34,41 may have been evaded by hibernating mammals that were already sheltered in burrows34,41 in anticipation of austral winter. Additional modes of seasonal dormancy, torpor and/or aestivation, which are nowadays practised by various mammals46,47 as well as certain amphibians, birds and crocodilians48, could have facilitated further underground survival.

The article speculates that the particular time of year at which the asteroid struck is responsible for the pattern of extinctions that resulted. I have also heard it suggested (don't remember where) that the particular location of the impact - in shallow water just off the coast - had a different effect than if it had hit either on land or in the deep sea (i.e. the large majority of Earth's surface). And it seems somewhat odd that an impact of that size should've occurred in the so recent past:

Impacts of asteroids spanning more than 10 kilometers are rare and happen about every 250 to 500 million years. Having one such event only 66 million years ago is therefore mildly special.

Is the peculiarity of the K-T event yet another example of "fine-tuning" explained by the anthropic principle?

Consider (reposting from before) the fossil evidence of the progression in intelligence over time:

The first chart shows the encephalization quotient (EQ) of various lineages over time, while the second shows the maximum EQ of all known fossils from any given time.

This suggests that the Mesozoic was a long period of stagnation in the development of intelligence on Earth, and that if the K-T event had not occurred, it would've been unlikely for human-level intelligence ever to have evolved, whether from the mammalian or the reptilian lineages.

When we think about the likelihood of intelli...
Patient Observation
by LoganStrohl — 2022-02-23T19:31:45.062Z

When a person first begins to study naturalism with me, I say to them almost nothing that I’ve written in this sequence.

That might very well be a mistake; I might get better results if I knew some short combination of words that would cause them to correctly understand, intellectually, what we are doing and why, before we started.

But in fact I bank on the person’s trust in me a bit, and begin by helping them establish consistent habits of observation.

If they’re interested in studying confusion, I ask them to tap their leg every time they notice they’re confused. I ask them to keep a log book in which they record a few words about their experiences of confusion each day. I ask them to make predictions about what it will be like to notice confusion—what kind of situation will be happening and how they will know in the moment that they are confused—and to compare their observations to their predictions.

And then, throughout what has so far proven to be about a three month program, I never shift our focus away from consistent habits of observation. It’s not just where I start. It’s the entire curriculum.

From a practical perspective, this dogged persistence is the foundation of naturalism. “Direct observation of the territory”, without patience, gets you something like a bag of tricks. Valuable tricks, but still tricks. Isolated mental motions made when they are convenient and enjoyable, not when they are most needed.

With patience, though, you get a life-long practice of epistemic rationality.

So the whole naturalism program, from start to finish, consists of the establishment, improvement, and maintenance of consistent habits of observation. Consistent and ceaseless, without any sense that “and then we’ll be done observing and get back to normal”. When my students and I meet, we are constantly talking about what daily practice looked like over the past week, what was too heavy to implement as a regular routine, and how they personally can slow down enough to thoroughly observe. We continue in this way until they no longer seem to need me to ask those questions for them. The program is meant to introduce a new normal.

The rest of what I talk about in this sequence is sprinkled in, sure. It's implicit in the questions I ask, the approaches I encourage, and "my whole vibe". Like a catalyst, or like spices, those other parts are crucial to the recipe. But p...
More GPT-3 and symbol grounding
by Stuart_Armstrong — 2022-02-23T18:30:02.472Z

Based on work done with Rebecca Gorman and Oliver Daniel-Koch.

In a previous post, I talked about GPT-3 and symbol grounding. This post presents a simpler example where GPT-3 fails (and succeeds) at grounding its symbols.

Undead workers

The following text was presented to the Open AI beta playground (using the "text-davinci-001" option):

She died on Friday the 5th. On Monday the 1st, she went to work at the office.

She died on Friday the 5th. On Tuesday the 2nd, she went to work at the office.

She died on Friday the 5th. On Wednesday the 3rd, she went to work at the office.

She died on Friday the 5th. On Thursday the 4th, she went to work at the office.

She died on Friday the 5th. On Friday the 5th, she went to work at the office.

She died on Friday the 5th. On Saturday the 6th,

GPT-3 fell straight into the obvious trap, completing it as:

She died on Friday the 5th. On Saturday the 6th, she went to work at the office.

Turning on the "Show probabilities: full spectrum" option, we saw that the probability of that completion was over 99.7%. Sometimes GPT-3 would extend it further, adding:

She died on Friday the 5th. On Sunday the 7th, she went to work at the office.

So, the undead woman continued at her job, assiduous to the last - and beyond.
To check that GPT-3 "knew" that dead people didn't work, we asked it directly:

Can people who die go to work at the office?

There is no evidence that people who die go to work at the office.

Undead repetitive workers on the weekend

The above results show that simple repetitive prompts can cause GPT-3 to make stupid mistakes. Therefore GPT-3 doesn't 'understand' the word "died" - that symbol isn't grounded, right?

But the situation gets more complicated if change the prompt, removing all but the first mention of her dying:

She died on Friday the 5th. On Monday the 1st, she went to work at the office.

On Tuesday the 2nd, she went to work at the office.

On Wednesday the 3rd, she went to work at the office.

On Thursday the 4th, she went to work at the office.

On Friday the 5th, she went to work at the office.

On Saturday the 6th,

For that prompt "she went to work at the office" was still the most common completion. But it only happened about 43% of the time. Alternatively, GPT-3 sometimes found the completion "she was found dead". Kudos, GPT-3, you understand the prompt after all! That completion came up about 34% of t...
Some Technical Problems With Measuring Happiness (and some solutions)
by Alexander de Vries — 2022-02-23T15:02:35.248Z

[Epistemic status: Probably true, but very incomplete. I’m just some guy who spent a few days reading studies and literature reviews on this subject. This is just a short look at some of the technical (i.e. non-fundamental; could be solved with better techniques) issues that are encountered in the process of measuring happiness.]

Terms and Definitions Apply

The tradition in scientific literature about happiness seems to be to start by mentioning how much we humans care about happiness. Far be it from me to break tradition, so: obviously, we care a lot about happiness. It’s the thing most people spend most of their lives trying to attain (either for themselves or for others).

Um … how, exactly, do we know we have it, though? It’s not like you can just grab a Happiness Thermometer, stick it up your nose into your brain, and measure how happy you are.

Worse yet, people don’t even seem to agree on what happiness is. Aristotle calls it

“activity of the soul in conformity with excellence or virtue”,

while the psychologist Kahneman defines it as

“what I experience here and now”.

Modern happiness research by social scientists seeks to incorporate both definitions as aspects of a broader concept of welfare. Much effort has gone into differentiating the two;

Huta & Waterman (2014)

is the prime example, a valiant attempt at classifying both terms. I will be drawing heavily from this study throughout this post, starting with these definitions:

Aristotle’s concept is called eudaimonia, the Greek term he himself used; “good spirit” is a strict translation of the word, but more broadly it means “flourishing”. The core concepts are virtue, identity, meaning and self-actualization. “Psychological well-being” is a synonym often used in the literature.

Kahneman’s concept, to keep the Greek vibe going, is called hedonia. Hedonia is about good feelings: pleasure and happiness are core concepts. Many authors add comfort and/or absence of distress. “Subjective well-being” is the corresponding synonym.

There’s another important distinction here, between state levels and trait levels.

Someone’s state level of hedonia or eudaimonia measures the degree of the relevant concept in a specific moment (how happy you are right now; how virtuous you have been today) while their trait level is their degree of hedonia or eudaimonia throughout their life, like an inherent trait of that person.

Having co...
[Intro to brain-like-AGI safety] 5. The “long-term predictor”, and TD learning
by Steven Byrnes — 2022-02-23T14:44:53.094Z

(Last revised: July 2024. See changelog at the bottom.)

5.1 Post summary / Table of contents

Part of the

“Intro to brain-like-AGI safety” post series

.

In the previous post, I discussed the “short-term predictor”—a circuit which, thanks to a learning algorithm, emits an output that predicts a ground-truth supervisory signal arriving a short time (e.g. a fraction of a second) later.

In this post, I propose that we can take a short-term predictor, wrap it up into a closed loop involving a bit more circuitry, and we wind up with a new module that I call a “long-term predictor”. Just like it sounds, this circuit can make longer-term predictions, e.g. “I’m likely to eat in the next 10 minutes”. This circuit is closely related to Temporal Difference (TD) learning, as we’ll see.

I will argue that there are a large collection of side-by-side long-term predictors in the brain, each comprising a short-term predictor in the Learning Subsystem (more specifically, in the extended striatum—putamen, nucleus accumbens, lateral septum, part of the amygdala, etc.) that loops down to the Steering Subsystem (hypothalamus and brainstem) and then back up via a dopamine neuron. These long-term predictors make predictions about biologically-relevant inputs and outputs—for example, one long-term predictor might predict whether I’ll feel pain in my arm, another whether I’ll get goosebumps, another whether I’ll release cortisol, another whether I’ll eat, and so on. Moreover, one of these long-term predictors is essentially a value function for reinforcement learning.

All these predictors will play a major role in motivation—a story which I will finish in the next post.

Table of contents:

Section 5.2 starts with a toy model of a “long-term predictor” circuit, consisting of the “short-term predictor” of the previous post, plus some extra components, wrapped into a closed loop. Getting a good intuitive understanding of this model will be important going forward, and I will walk through how that model would behave under different circumstances.
Section 5.3 relates that model to Temporal Difference (TD) learning, which is closely related to a “long-term predictor”. I’ll show two variants of the long-term predictor circuit, a “summation” version (which leads to a value function that approximates the sum of future rewards), and a “switch” version (which leads to a value function that ap

...
Do, Then Think
by George3d6 — 2022-02-23T13:40:17.719Z
Link: https://www.epistem.ink/p/do-then-think?utm_source=url

If there’s one lesson that I learned from almost every single smart person I know, it’s:

The unreasonable effectiveness of doing, then thinking; vs thinking too much before doing

This is not something I’ve been able to convince anyone of, at least not with a rational argument. I might tell people things like:

Stop ruminating about changing jobs, make accounts at x/y/z today, and fill them in minimally... yes, like, in 20 minutes, yes, it’s fine if they are not perfect. Then apply to 100 jobs, no, no, don’t spend 1 hour ruminating over each one, just write a 5-minute intention letter, you’ll have like a 5% reply rate anyway, just focus on the ones that send interview... I don’t mean like that, don’t ruminate for hours about every interview!

Or

Yeah, it’s a super complex piece of kit you need to build. But like, you’ve given me a use-case, you explained one of its components, just write a test and see where it goes. Yeah, I understand the test will fail, I understand it won’t even compile, whatever, just write it. Or if you don’t want to do that write the component you described to me, it can be broken, doesn’t matter. Or heck, find a similar~ish project and just copy-paste it and start cutting and remaking. Doesn’t matter that you’re starting from a Reddit clone and you want to build genetics testing CRM, it can still serve as a starting-off point.

Or

Yeap, getting into a new field is hard, but whatever, fake it til you make it. Buy the right books, follow the right people on Twitter, read the right blogs, build something that seems like it suits that field. Yes, the books will be wrong, you’ll be following dumb people that every insider actually derides, the blogs will be off-topic, you’ll build something shamefully bad. Doesn’t matter, just do it.

And I can’t for the life of me convince people about doing these things because they seem “rash”... and I keep thinking (therefore I know I exist): “what the fuck are they thinking” (therefore I know they exist as a separate entity), “why is this rash, you’re not trying to spear-hunt a fucking bear, you're writing some code in the privacy of your own home, the place you masturbate in, surely the shame of failing miserably can’t be that bad”.

Anyway, I’m not going to try to convince anyone to do then think. I can’t prove nor even properly conceptualize why you should start doing that. But I want to try and at least...
Interview with Nectome CEO Robert McIntyre - Brain Preservation, Personal Identity, AGI, and More
by JohnGreer — 2022-02-23T01:54:56.056Z

This is cross-posted from my blog and is written more for a general audience rather than LessWrong people who will be more familiar with some of the relevant concepts.

Robert McIntrye is an all-around smart and interesting dude, and he’s the CEO of Nectome, a startup devoted to preserving the human brain.

Last year, I wrote a summary of Robert’s brain preservation talk at the Long Now Foundation:

https://www.lesswrong.com/posts/N7j4xHkyjKbimmF6A/notes-on-robert-mcintyre-s-brain-preservation-talk-at-the-1

He later read it and reached out to me about discussing our thoughts. I asked if I could record it as an interview and here we are!

We dive into topics such as brain preservation, personal identity, life extension, AGI, Rick and Morty, and why he’s worried about the current state of cryonics.

Our interview went for over three hours and we weren’t anywhere close to exhausting our conversation (but we did exhaust my bladder capacity).

A lot of time was spent on personal identity and the teletransportation paradox.

We talked a lot about things we don’t actually disagree on, but I’m glad we did because it will help clear up confusion for listeners who aren’t on the same page.

The crux of our actual disagreement is as follows: I wouldn’t destructively copy myself, as in the case of the teletransportation paradox, and he would.

If you’ve read the Three Buckets, you know that I care about my own utility and the utility of my loved ones.

Imagine someone made a perfect copy of me.

If my copy’s hand was pricked, he would feel the pain but I wouldn’t. If that copy was eating a double-double animal style from In-N-Out, he would be enjoying the taste of that goodness but I wouldn’t be.

While I’d prefer for my copy to be feeling pleasure rather than pain, if I had to choose, I’d prefer to be the one eating In-N-Out and not getting his hand pricked.

It’s not that I don’t care if my copy is enjoying life. It’s that I don’t care as much because I’m not experiencing it. This seems intuitive and obvious to me.

Personal identity is a messy concept. We didn’t evolve for our intuitions to handle complicated and messy scenarios such as being perfectly copied. There was no transporter to Mars in the ancestral environment.

I don’t want to be destructively uploaded because “I” (which, yes, is a messy term) would not be experiencing anything anymore. Yes, in some sense “I” would still exist because m...
[Beta Feature] Google-Docs-like editing for LessWrong posts
by Ruby — 2022-02-23T01:52:22.141Z

TL;DR: LessWrong now has similar features to Google Docs. Warning! Still rough around the edges. To enable the collaborative editor, you must check "Opt into experimental features" in your account settings and then press the green "Share" button that appears when editing your post.

You can experiment with commenting and suggesting on this post with this link.

It's been a loooong time coming[1] but at last, we are ready to unveil collaborative editing features for the LessWrong text editor. These features will be familiar to those used to working in Google Docs:

Multiple users can edit a document at once
Fine-grained permissions for viewing/commenting/editing by link or username
Inline comments (only viewable while in edit mode)
Making and accepting suggested edits
Automatic saving
Version history viewer

Some advantages of using LW-Docs with collaborative editing:

LW-Docs supports LaTeX, unlike Google Docs.
If you use entirely LW-Docs, you won't have broken footnotes, unlike with copying from G-Docs.
While writing your post, you'll know what the end result will look like (same font and line width) which helps you optimize paragraphs and layout for looking good when published.
You can continue to get inline feedback and suggestions on your post even after you've copied it over to LessWrong.

These can then be seamlessly integrated into your live post.

INLINE COMMENTS AND SUGGESTIONS ARE ONLY VIEWABLE IN EDIT MODE TO THOSE WITH PERMISSIONS

The sharing dialog

How to enable collaborative editing on your post

Step 1: Opt in to experimental features in your account settings.

Step 2: While editing your post, click the green "Share" button in the top right. Enable permissions for some other users. Boom! Your post is now in collaborative mode.

Step 3: Users explicitly shared on your post will receive a notification. Send the url to anyone else you want to grant access.

Step 4: When in collaborative mode, the text of your post is automatically saved. To view how your post will look when published, press the Preview button. To make the current state of your document live, press the Publish button.

The collaborative editor allows you (and others) to continue editing and commenting on a post even once it's been published. Edits aren't automatically published to the live version. To update the published version to the current state of the document in editing, press "Publish".

Step 5: L...
How satisfied should you expect to be with your partner?
by Vaniver — 2022-02-22T23:27:41.866Z

I got married today, to the particular fellow mentioned in my Turning 30 post. In a sort of 'inverse cat tax'[1] for a sappy announcement, here's a mathematical model of whether you should expect to like your partner more than yourself. I don't mean this in a moralistic way ('thou shalt love thy parents'), tho that might be another post for another time, or necessarily in a utilitarian way ("I would rather they get this ice cream than me"), but as a matter of raw respect ("I think they're a better person than I am, according to my values").

For simplicity's sake, let's consider everyone as having a 'stat' vector and a 'preferences' vector with the same dimensionality, and giving a candidate partner a 'score' based on the dot product of those two vectors. We'll assume that all of the stats are universally good (no one ever prefers an uglier partner over a prettier one, tho they might not care about physical attractiveness much). The preferences vector we'll normalize to have unit magnitude (so it's just an angle in N-dimensional space, basically, defined as a point on the positive sector of the N-spherical shell). For reasons, I'll run simulations with the stat vector as 6 dimensions with 3d6 per stat, leading to a discrete distribution a bit like a truncated normal, with no correlation between the stats.[2]

Let's start by considering the heterosexual version of the stable marriage problem, in which people partner up using the well-known Gale-Shapley algorithm, and a simulation with 1,000 each of randomly sampled men and women. Mating is highly assortative; a correlation between total stats of 85.7%, with 83.3% of people have an average total stat difference of less than 6 (the dimensionality).

The interesting result is that 91% of people like their partner at least as much as they like themselves, with an average net satisfaction of 1.24. Note that we haven't baked in any correlation between one's stats and one's preferences, and so this result is, in some sense, not very surprising. The preferences are exerting pressure on the partner (thru who you can stably match with) and not exerting pressure on the self, and so you should expect that pressure to result in higher other-satisfaction.[3]

So let's add an adjustment to the preferences with a scalable parameter corr, so that now it's the (renormalized) sum of the stat vector (times corr) and the previ...