trentmkelly/LessWrong-43k huggingface dataset:
Don't Influence the Influencers!
by lhc — 2021-12-19T09:02:18.556Z

This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the

Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program

.

TL;DR: AGI is likely to turn out unsafe. One likely way that can happen is that it fools us into thinking it is safe. If we can make sure to look for models that are ineffective at "bad" things (so it can't deceive us) and effective at "good" things (so it is useful), and importantly, do that prior to the models reaching a point-of-no-return of capability, we can avert catastrophe. Which spaces of algorithms do we look in? What do they look like? Can we characterize them? We don't know yet. But we have a very concrete point in such a space: an "LCDT agent". Its details are simple and we'll look at it.

Format note: The original post is already pretty well-written and I urge you to check it out. In trying to summarize an already well summarized post (to any alignment researcher anyway), I've aimed lower: catering to a dense set of possible attention-investments. This (ie. the linked dynalist in the first section) is an experiment, and hopefully more fun and clarifying than annoying, but I haven't had the time to incorporate much feedback to guarantee this. I hope you'll enjoy it anyway.

Epistemic status: I'd say this post suffers from: deadline rushedness, low feedback, some abstract speculation, and of course, trying to reason about things that don't exist yet using frameworks that I barely trust. It benefits from: trying really hard to not steamroll over concerns, being honest about flailing, being prudent with rigor, a few discussions with the authors of the original post, and its main intent being clarification of what someone said rather than making claims of its own.

Summary

Here's the link to a dynalist page.

Click on a bullet to expand or collapse it. Try a more BFS-ish exploration than a DFS one. If you've used something like Roam, it's similar.

The rest of the post assumes you've clicked through to the summary above and finished reading it!

We're Not Really Doing Decision Theory

If you're used to reading about Newcomb-like problems and comparisons of various kinds of correct ways to think about the influence of your action, LCDT is obviously just a lie. And as we all know, it's silly to program a lie into your AGI; no consequentialist will keep it around any more than i...
Should I delay having children to take advantage of polygenic screening?
by Mati_Roy — 2021-12-19T06:39:25.071Z

I would like to know whether I should delay having children to take advantage of polygenic screening technology. I imagine this could be valuable for other aspiring parents to know as well, and would (probably) have positive externality for aspiring parents to know this. Might also be useful info for grant-makers in the space.

Related: Welcome Polygenically Screened Babies

EtA: If you're interested in helping with related research in any way or just interested in being kept up-to-date with related research (at a low frequency / high-value), you can PM me.
Falsifying Ethical Values
by lsusr — 2021-12-19T01:44:52.366Z

There are three branches of philosophy: natural philosophy, moral philosophy and metaphysical philosophy. Colloquially, we refer to natural philosophy as "science", "moral philosophy as "ethics" and metaphysical philosophy as "philosophy". There's also math "in fact, it would not be a bad definition of math to call it the study of terms that have precise meanings."

Science is a worthwhile endeavor. Metaphysical philosophy tends to be worthless because it makes no falsifiable predictions about reality.

Losing touch with reality is boring.

―Write Surprisingly About Reality

What about ethics? Western philosophy tends to take it for granted that ethics is consequentialist. Consequentialist ethics breaks ethics down into three components.

A model of the world.

An axiomatic value function dependent on the world.

A strategy for optimizing the value function.

Modelling the world is science. Optimizing a value function is math. Science and math are ethically trivial because consequentialist value functions are orthogonal to science and math. Value functions can be anything, which is why consequentialist moral paradigms tend to produce paperclip maximizers.

Consequentialist moral frameworks reduce ethics to axiomatic value functions (which could be anything because they are are unfalsifiable) and optimization strategies (which are ethically trivial). The optimization strategies belong in the realms of science and math. The axiomatic value functions of a consequentialist moral framework are declared by fiat. They are upstream of reality, like mathematical axioms.

It is fine for mathematical axioms to be upstream of reality because mathematical axioms don't make statements about reality. The Axiom of Choice isn't a statement about reality. It's a statement about Mathland. The Axiom of Choice isn't even a statement about all of Mathland. The Axiom of Choice is applies only to the region of Mathland where the Axiom of Choice applies. Two plus two equals four in the real numbers R but two plus two equals zero in the Z4 cyclic group.

It's fine for mathematical axioms to be unfalsifiable because math itself is upstream of reality. You cannot falsify mathematics by experiment (except in the subjective Bayesian sense).

Ethics must be falsifiable because of reducto ad absurdum. If ethics wasn't falsifiable then there would be no truth values to statements like "genocide is bad", ...
Elevated Air Purifier Cubes
by jefftk — 2021-12-19T00:50:03.162Z

I've been

a

big

proponent of

box fan air purifier cubes, but
they are bulky and fragile. If your ceiling is 8ft+,
elevating them is an attractive option:

I made two of these for my dad's house this afternoon. The goal is
for them to be out-of-the-way, and easy to turn on any time he has
visitors.

The only heavy part is the box fan, since the rest of the cube is
filters and tape, so the goal is to make sure the fan is well
supported and not going to tip over. I used an angle bracket and a
short length of wood to make a shelf under each fan, and then taped
the fan to the shelf and wall. The filters are also taped to the
wall. When taking them down I'll probably need to use a small amount
of spackle for the holes, and touch up the paint if the tape takes
some paint off, which it probably will. If I wanted to avoid having
to tape anything to the wall I could have used two angle-bracket
shelves each.

With an 8' ceiling you would want to have the cube flush against the
ceiling, which gets you a tight-but-probably-ok 6'3" of clearance and
three sides for filters. If your ceiling is taller, like these ones
were, you can have a fourth filter on top while still maintaining
reasonable clearance.

Comment via: facebook
Exploring Decision Theories With Counterfactuals and Dynamic Agent Self-Pointers
by JoshuaOSHickman — 2021-12-18T21:50:13.751Z

This is a follow-up to A Possible Resolution To Spurious Counterfactuals, which was addressing a technical problem in self-proof. See the original post for the suggestion provided for a solution to the 5 and 10 problem as described in the Embedded Agency paper.

I suggested, at the end of that post, that the crux of the solution is that we don't want exactly a self-proof, we want explicit counterfactual reasoning. This actually allows the proofs (for the simple examples, but the principle should extend much further), and avoids accidentally inferring actions were valuable just because they were taken by a prudent agent.

Here I want to make the case that, when we expand the usage of explicit agent pointers to allow self-modification (in the form of precommitments), we can talk meaningfully about many different decision theories. This framing doesn't decide your decision theory for you, it's only meant to help with counterfactual reasoning and Lobian obstacles in proof construction. The pseudocode presentation reminds us that programs can, in fact, point to themselves, and that this can be a suitable way to at least partially embed agents in their reasoning.

This is where we left off:

Situation :: Nonce
Agent :: Agent -> Situation -> Choice
Universe :: Agent -> Reward

CounterfactuallyDoing(PriorAgent, CounterfactualSituation, Choice) :=
NewAgent := \Agent, Situation ->
if Situation == CounterfactualSituation:
return Choice
else:
return PriorAgent(NewAgent, Situation)
return NewAgent

A(Self, Situation == OnlySituation) :=
Spend some time searching for proofs of sentences of the form
“[U(CounterfactuallyDoing(Self, OnlySituation, 5)) = x] & [U(CounterfactuallyDoing(Self, OnlySituation, 10)) = y]” for x,y ∈{0,5,10}.
if a proof is found with x > y:
return 5
else:
return 10

U(Actor) :=
if Actor(Actor, OnlySituation) = 10:
return 10
if Actor(Actor, OnlySituation) = 5:
return 5

I'd like to propose an extension to the model (which is more like a toy phrasing of the idea). The core idea is that, with the counterfactual agents, we ought to introduce the idea of replacing the Universe's pointer to the Agent with one of those counterfactual agents. In other words, we'd like to be able to precommit: self-modify to resolve specific future decisions. I'm going to ignore the necessary practical complications, such as any form o...
Occupational Infohazards
by jessicata — 2021-12-18T20:56:47.978Z

[content warning: discussion of severe mental health problems and terrifying thought experiments]

This is a follow-up to my recent post discussing my experience at and around MIRI and CFAR. It is in part a response to criticism of the post, especially Scott Alexander's comment, which claimed to offer important information I'd left out about what actually caused my mental health problems, specifically that they were caused by Michael Vassar. Before Scott's comment, the post was above +200; at the time of writing it's at +61 and Scott's comment is at +382. So it seems like people felt like Scott's comment discredited me and was more valuable than the original post. People including Eliezer Yudkowsky said it was a large oversight, to the point of being misleading, for my post not to include this information. If I apply the principle of charity to these comments and reactions to them, I infer that people think that the actual causes of my psychosis are important.

I hope that at least some people who expressed concern about the causes of people's psychoses will act on that concern by, among other things, reading and thinking about witness accounts like this one.

Summary of core claims

Since many people won't read the whole post, and to make the rest of the post easier to read, I'll summarize the core claims:

As a MIRI employee I was coerced into a frame where I was extremely powerful and likely to by-default cause immense damage with this power, and therefore potentially responsible for astronomical amounts of harm. I was discouraged from engaging with people who had criticisms of this frame, and had reason to fear for my life if I published some criticisms of it. Because of this and other important contributing factors, I took this frame more seriously than I ought to have and eventually developed psychotic delusions of, among other things, starting World War 3 and creating hell. Later, I discovered that others in similar situations killed themselves and that there were distributed attempts to cover up the causes of their deaths.

In more detail:

Multiple people in the communities I am describing have died of suicide in the past few years. Many others have worked to conceal the circumstances of their deaths due to infohazard concerns. I am concerned that in my case as well, people will not really investigate the circu

...
Laurie Anderson talks
by Dagon — 2021-12-18T20:47:01.484Z
Link: https://www.youtube.com/playlist?list=PLtxVM47qfVNCAPdgEesjlfIQX6rptwXDp

Not rationalist per-se, and much more abstract than legible (it is part of a Poetry lecture series), but Laurie Anderson has long been one of my favorite artists, and this is likely of interest (or of enjoyment or some positive experience) to many people here. 10 minutes into the first lecture, she states "We'll start out with two questions. The first question is 'Why do anything?' and the second question is 'How do you know what's good?'"
Disentangling Perspectives On Strategy-Stealing in AI Safety
by shawnghu — 2021-12-18T20:13:08.882Z

This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program.

Additional thanks to Ameya Prabhu and Callum McDougall for their thoughts and feedback on this post.

Introduction

I’ve seen that in various posts people will make an offhanded reference to “strategy-stealing” or “the strategy-stealing assumption” without very clearly defining what they mean by “strategy-stealing”. Part of the trouble with this is that these posts are often of wildly different flavors, and it’s often not clear what their connection to each other is, or perhaps it’s unclear under what conditions it might be feasible to “steal the strategy” of an AI, or sometimes it’s unclear where there is any notion of doing any stealing of strategies at any point.

Spoiler upfront, part of the reason for this is that the term comes from a relatively esoteric game-theoretic concept that basically never applies in a direct sense to the scenarios in which it is name-dropped. In this post, I’ll try to explain the common use of this seemingly-inappropriate term and why people are using it. In order to do this, I’ll first attempt to develop a more general framework for thinking about strategy-stealing, and then I’ll try to extend the intuition that develops to bridge the gap between a couple of seemingly highly disparate posts. Along the way I’ll attempt to clarify the connection to a few other ideas in alignment, and clarify what implications the strategy-stealing framework might have for the direction of other alignment research. I’ll declare up front that I think that in an AI safety context, the strategy-stealing framework is best thought of as a tool for managing certain intuitions about competition; rhetorically the goal of my post is to distill out those aspects of “game-theoretic” strategy-stealing that are broadly applicable to alignment so that we can mostly throw out the rest.

As an aside, I think that due to the (very relatable) confusion I’ve seen elsewhere, strategy-stealing is not actually a good name for the concept in an AI safety context, but the term is already in use and I don’t have any ideas for any better ones, so I’ll continue to just use the term strategy-stealing even in contexts where no strategies are ever stolen. My hope is that I do a good enough job of unifying the ex...
The end of Victorian culture, part II: people and history
by David Hugh-Jones — 2021-12-18T18:55:24.585Z
Link: https://wyclif.substack.com/p/the-end-of-victorian-culture-part-149

The last post was about deep structural reasons for the end of the Victorian cultural system. Here, I’ll discuss the unpredictable role of people and events.

In every society, some people get listened to and copied. We listen to the rich and powerful, because we want to be like them. Other people — artists, scientists, intellectuals — become influential by having interesting things to say. Being listened to and copied gives these people power to sway the rest of society. They set expectations and norms, by what they say and by their example.

The cultural elites of the nineteenth century were conscious of their power and had a strict sense of duty in this regard. They saw themselves as missionaries to their own societies. They wanted to pass their values down the social scale, by preaching and example. At the start of the period, this was often literally preaching (Victorian Britons consumed sermons the way we watch box sets); later cultural elites were more secular, but still wanted to spread “sweetness and light”, as Matthew Arnold put it. This was a key part of the Victorians’ highly moralized idea of progress.

At the turn of the twentieth century, these attitudes began to shift. A new generation lost its missionary zeal in different ways.

They lost faith in the moral superiority of bourgeois values.
They became increasingly sceptical and fearful of “mass man”.
Art and “high culture” turned inward upon itself in order to progress, seeking freedom from its social role.

Bourgeois values

The phrase itself explains the change. Marxists saw Christianity as the ideology of an unjust social system. This especially made sense on the European continent, where reactionary elites were hand-in-glove with established churches. Trotsky put it pungently in Their Morals and Ours:

Whoever does not care to return to Moses, Christ, or Mohammed... must acknowledge that morality is a product of social development; that there is nothing immutable about it; that it serves social interests; that these interests are contradictory; that morality more than any other form of ideology has a class character…. A revolutionary Marxist cannot begin to approach his historical mission without having broken morally from bourgeois public opinion and its agencies in the proletariat.... Whoever fawns before precepts established by the enemy will never vanquish that enemy!

Arthur Koestler described the ef...
How I became a person who wakes up early
by Alex_Altair — 2021-12-18T18:41:45.732Z

This is an extended anecdotal description of a life change that honestly still surprises me; I went from the classic internet nerd who stays up until 3 and sleeps until noon, to a person who reliably goes to bed before 10pm and wakes up at 6:30am every morning. This didn't happen overnight (pun intended); it happened slowly, increasingly earlier and with increasing reliability, over the last *checks notes* ten years (although most of the shift is in the last four years).

This is not intended as a how-to guide for others, and probably won't help anyone with serious sleep issues.

I don't have a particular stance on whether waking up early is objectively good. But I've always had a personal aesthetic appreciation of it, and I'm not entirely sure why. I like the idea of being awake for more of the sunlight hours in the day, and especially of waking up just before the sun rises. I also like the idea of being so thrilled to be alive that you want to just cast the blankets off you and jump out of bed as soon as you realize you're awake. Which is the extreme opposite of how I would previously feel upon being woken up by my alarm.

Aging

It's entirely possible that the entire explanation here is "your natural circadian rhythm shifted as you got older" (e.g. ~20 to ~30 years old). That certainly happened to some extent, especially in the earlier phase. But I also notice that every night, I do want to stay up and keep doing what I'm doing. I'm pretty sure I could, because I never really feel super tired. And if I'm socializing, it is trivially easy to stay up, and I have to use executive function override to go to bed.

A regular social commitment

The chronologically first and possibly most significant structural change was scheduling with friends to regularly go to the climbing gym twice a week. I had gone with them a few times on weekends, and it didn't take me long to discover that I absolutely loved climbing. They then invited me to their preexisting regular climbing sessions in the mornings before work.

Climbing nights and evenings is obviously more convenient, but it also means that the gym is really full. I never liked climbing when it was busy, because there's a much higher risk of people running into each other on the wall; I could never really relax. You also have to sometimes wait for your turn. In the morning it's practically empty, so you can climb unrestricted.

She...
[Extended Deadline: Jan 23rd] Announcing the PIBBSS Summer Research Fellowship
by Nora_Ammann — 2021-12-18T16:56:47.442Z

What can we learn from biological and social systems about designing beneficial AI systems?

We are pleased to introduce the PIBBSS summer research fellowship.

Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS) aims to facilitate knowledge transfer with the goal of building human-aligned AI systems.

In Summer 2022, the PIBBSS fellowship will bring together researchers from fields studying complex and intelligent behavior in natural and social systems, such as :

Evolutionary biology
Ecology
Cognitive science
Neuroscience
Linguistics
Sociology
Political economy
Legal theory
Statistical mechanics
Media studies
And more.

Over the course of 10-12 weeks, fellows will work on a project at the intersection of their own field and AI safety, under the mentorship of experienced AI alignment researchers.

For a more extensive introduction, see Introducing the Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS) Fellowship or visit

our website

.

The program is organized in partnership with the

Swiss Existential Risk Initiative

, the

Stanford Existential Risks Initiative

, and the

Center for Theoretical Study

at Charles University. We are generously supported by a grant from the Long-Term Future Fund.

We’d be grateful if you could help us pass on the word to people who might be interested in applying.

Application deadline: Jan 16th, 2022
Learn more and apply

here

.
Universal counterargument against “badness of death” is wrong
by avturchin — 2021-12-18T16:02:00.043Z

All the usually offered counterarguments to the idea that death is evil boil down to one universal counterargument:

If there is no death, then there will be a bad phenomenon X (overpopulation, cessation of progress, eternal dictatorship, boredom), therefore, death is necessary.

However, this universal counterargument does not work:

The counter-argument does not tell us that death is not evil. He tells us that, from the point of view of altruism, there are situations when death is necessary, for example, it is supposedly needed to combat overpopulation.
The counter-argument does not deny the evil of death. It talks about the collision of the value "not to die" and some other value Y (progress, the good of mankind), and then argues that the second value is more important.
If there is a way to achieve the value of Y without people dying, then death is not needed.

For example, if at the birth, each person is granted so many resources that it will be possible to provide her with resources for entire endless life, then overpopulation will cease to be a problem. Overpopulation is only a problem because of the clash between the idea of ​​finite resources and the prospect of exponential proliferation.

The counterargument also does not work because it is not central. The negative X phenomenon can persist even if death continues. Lack of progress, eternal dictatorship, overpopulation and degradation are possible in societies where people have short life expectancy.