trentmkelly/LessWrong-43k huggingface dataset:
Do insects' lives matter?
by Carn — 2022-09-30T15:38:55.482Z

When it comes to this issue, it makes more sense to think of another topic, frogs. Frogs eat insects all the time. This leads to a moral dilemma that will help us answer our question. Do the lives of insects matter more than the hunger of a frog?

Situation 1 ~ No

This is the answer most people would give, so I'm going to look at the implications first. We should allow frogs to continue eating insects, because their existence is more important than the insects'. If you are willing to accept this position, then you necessarily must allow the idea of higher and lower beings into your moral system. There is no way of justifying allowing one kind of being to murder another without saying that some beings get higher priority over others. This kind of reasoning, the kind that most people agree on, leads to other dilemmas. How are higher beings determined? How much more is the frog valued over a fly? Is the annoyance of a human worth more than an insect's life? Insects carry diseases, so should all harmful insects be killed if it saves one human life? However you answer these questions are up to you, but I would say most people would agree that insects barely matter if at all.

This also leads to one of the more controversial questions. Are all humans equal? With the concept of higher and lower beings, this isn't something you can take for granted. Although no one wants to be in the position to judge a potato farmer, for instance, against Donald Trump, for instance, you now would have to go out of your way to prove that all humans are equal, or at least, they should be treated as if they are equal.

I am actually going to attempt to answer this right now. I would say that all humans should be treated equally because of human potential. Jimmy Carter, for instance, was, not a potato, but a peanut farmer who became president with a little luck and a little trickery. I mean, his slogan was "My name is Jimmy Carter, and I'm running for President." How in the world did that convince anybody? Also, Adolf Hitler, for instance, was a homeless painter who ended up getting all of Europe embroiled in a war that cost millions of lives. Since anyone could become anybody given the right circumstances, all humans have this potential in common. This creates a sort of fungibility of humans. Yes, you have to now include some value of the future or of potential in your moral system, but this...
Another Loft Bed
by jefftk — 2022-09-30T14:50:02.838Z

Anna also wants a loft bed. I was thinking I'd build

one for Lily first, see how it goes,
then build one for Anna, but it would be easier to get the materials
for both together so I'm thinking about Anna's now. Her room is a bit
bigger, so the loft would be in a corner. For the two wall sides and
the wall-beam junctions I can use the same design as last time, but
what should the free-standing corner look like?

It needs some kind of support, which could be a post, a triangle back
to the wall below, or a cable to the rafter above. I talked with Anna
about what she wanted, and she likes the idea of a post.

There are a lot of ways you could connect the post to the beams and
the beams to each other, but here's what I'm thinking:

This is cutting away 2/3 of the top of the post, so both beams can
rest on the post. To improve the connection between the beams, I'm
thinking of notching the edges of one to create a tongue that will fit
into a notch in the middle of the other. I don't know much about
joinery, though, so it's possible I'm planning something that either
doesn't work very well or is too hard to construct? This will be the
first part I do, before cutting things to length, so that if I screw
up I have enough left to try again.
Eli's review of "Is power-seeking AI an existential risk?"
by elifland — 2022-09-30T12:21:19.465Z
Link: https://docs.google.com/document/d/1nCqjJXydfPQGRTKT71jQn8yXi4FSHnVSdfZbhQUMa1I

See also Joe Carlsmith's report "Is power-seeking AI an existential risk" and previous reviews. I'll excerpt 2 portions of my review below.

Thinking about reaching goal states rather than avoiding catastrophe

I have a serious complaint around the framing of how the risk is decomposed, which may systematically bias Joe, reviewers and readers toward lower estimates of existential risk than is warranted. It’s similar to

Nate Soares’ concern

regarding the risk being decomposed conjunctively when it might be more natural to think of it disjunctively, but I’ll express it in my own words.

In one sentence, my concern is that the framing of the report and decomposition is more like “avoid existential catastrophe” than “achieve a state where existential catastrophe is extremely unlikely and we are fulfilling humanity’s potential”, and this will bias readers toward lower estimates.

When playing a strategy board game, it’s common to think in terms of “

win conditions

”: states of the game that you could get to where you’d feel pretty confident about your chances to win, even if they’re intermediate. This is often something like a set of cards that puts you in a very strong position to accumulate a lot of points. I claim that we should often be thinking about AI futures more like “win conditions” in board games and less like avoiding a negative outcome.

What are the win conditions we should be thinking about? The ultimate goal state is something like some number of aligned APS-AIs filling the universe with value and bringing the chance to essentially 0 that misaligned APS-AIs will be created and ruin the equilibrium. When reasoning about the probability that we will avoid existential risk from APS-AI, we should ultimately be chaining to this goal state in some sense. Similar to in board games, it might make sense to aim for intermediate stable-ish states which we feel pretty good about and think about the probability we can reach those: a possible example might be coordinating to get the major players to slow down on AI scaling, utilizing aligned non-APS-AIs to police potential PS-misaligned APS-AIs from popping up, and doing tons of alignment research via whole brain emulation and/or non-APS AIs (thanks to Joe for verbal discussion on this point). But either way we should often be thinking about win conditions and goal states, rather than avoiding a negative outcome.

In co

...
Truth-Seeking: Reason vs. Intuition
by sakraf — 2022-09-30T12:12:07.286Z

A model of approaches to truth-seeking
Two kinds of approaches to truth-seeking that people tend to have. Most average people in western society seem to be in the middle between the two. Rationalists aim to be on the side of reason. I am trying hard to be, too.

The intuition-based system can lead to more possible worldviews as it doesn't have the aim to produce worldviews that are isomorphic with one objective reality. In the following, I am listing exemplary, stereotypical views of the intuition-based approach to illustrate my points.

Both systems are a meta-system of the other one in the sense that they can ‘explain away’ the truth and importance of the conclusions of the other one.

Both are internally consistent.

Systems	Reason	Intuition
Definition of truth	Truth is when the map reflects the territory. (i.e. correspondence of your system of claims with reality)	Truth is that which feels true.
Importance is that which feels important.
Good is that which feels good. (etc.)
Axioms and tools of truth-seeking	Reason, Logic, Empirical Observation, Language	Emotions
Explanation for the other system’s inferiority	Feeling is flawed for truth-seeking as our feelings are shaped by Darwinian, evolutionary pressure to maximize fitness, not to maximize map-territory correspondence.	Thinking is flawed for truth-seeking as it feels false to use the methods of empirical observation, and logical deduction/induction if they are not accompanied by feelings of truth/falseness. These days, science is completely detached from feeling and thus does not produce any truths.
Free will?	Malformed question	There is free will.
Concepts of meaning, beauty, good	Meaning, beauty, and good are subjective, human concepts that you’ll fail to explicitly locate on the territory and that you’ll thus fail to map.	There is meaning, beauty, and good in the world.
Explanation for the other system sometimes being successful	Sometimes but rarely, what people feel like is true actually turns out to be true because of good instincts, chance, etc. but if so, often for the wrong reasons (at least explicitly)	Because many people have been raised to value the methods of science, empiricism, and logic, science sometimes produces truth as its findings will feel true to people. If so, they will feel it to be true but for the wrong reasons (at least explicitly).
Utility of the systems	The conclusions of science produce real-world outcome

...
(Structural) Stability of Coupled Optimizers
by Paul Bricman — 2022-09-30T11:28:35.698Z

This post is part of my hypothesis subspace sequence, a living collection of proposals I'm exploring at Refine. Preceded by an interlude.

Thanks Adam Shimi, @artaxerxes, Herbert Jaeger, Tamsin Leake, and Ze Shen for discussion informing this post.

Intro

In the previous post, I explored a grammar of alignment proposals based on (what I later learned could be referred to as) coupled dynamical systems. To help you get a sense of this view, one family of proposals tackles the alignment problem by having human researchers optimize a trainer2 (e.g. oversight leagues) which optimizes a trainer (e.g. SGD across model space employing empirical risk as heuristic) which optimizes a model (e.g. LLM) which optimizes a world (e.g. text). Other families have different numbers of optimization levels (e.g. True Name partially obviates the need for trainer2), integrate feedback loops (e.g. human amplification), or extend the optimization stack in some other way. In this, the focus moves from what to point an AI at, to how to move towards the associated trainer state (e.g. using heuristics across trainer space and above, similar to empirical risk as heuristic across model space). A shift from targets to processes for reaching said targets.

From this level of abstraction, one can identify issues which systematically plague entire families of proposals. For instance, there's still a bounded human at the top guiding downstream optimization (e.g. on trainer2). In the current post, I aim to explore how two notions from dynamical systems apply at different levels of optimization, and how those might help alleviate systematic issues. The two notions are:

Stability. A region of the state space of a dynamical system is deemed stable if slight perturbations to objects in that region don't result in the object permanently leaving the region. For instance, a tiny asteroid which crashes into Earth won't cause Earth to leave its orbit around the Sun, because Earth's orbit is stable. In contrast, the smallest gust of wind blowing on a pencil balanced on its tip will cause it to leave that region of state space behind, as it's not stable. Note: While stability is mathematically defined only for such regions of state space, I'll stretch the term "stable" for the purpose of this post to also describe the dynamical system which in turn contains reachable stable solutions. This reading of stabili

...
Rank the following based on likelihood to nullify AI-risk
by Aorou — 2022-09-30T11:15:58.171Z

Rank the following based on the likelihood to nullify AI risk (whether by achieving alignment, stopping AI development, or another way)
If you think you have better solutions to AI risk than I came up with, please add them to your ranking.

[1]

Give EY[2] $10M
Give EY $100M
Give EY $1B
Give EY $10B
Give EY $100B
Give EY $1T

Give Organization[3] $1B
Give Organization $10B
Give Organization $100B
Give Organization $1T

Achieve widespread agreement[4] on AI risk, by 2025
Achieve widespread agreement on AI risk, by 2030
Achieve widespread agreement on AI risk, by 2040

Convince Major Personalities[5] that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2025
Convince Major Personalities* that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2030
Convince Major Personalities that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2040

Convince the top STEM students/talents[6] that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2025
Convince the top STEM students/talents that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2030
Convince the top STEM+CS students/talents that AI risk is real and achieving aligned AI is way harder than achieving AI, by 2040

Achieve widespread agreement that P(Doom|AI) is more likely than not, by 2025
Achieve widespread agreement that P(Doom|AI) is more likely than not, by 2030
Achieve widespread agreement that P(Doom|AI) is more likely than not, by 2040

Convince Major Personalities that P(Doom|AI) is more likely than not, by 2025
Convince Major Personalities that P(Doom|AI) is more likely than not, by 2030
Convince Major Personalities that P(Doom|AI) is more likely than not, by 2040

Convince the top STEM students/talents that P(Doom|AI) is more likely than not, by 2025
Convince the top STEM students/talents that P(Doom|AI) is more likely than not, by 2030
Convince the top STEM students/talents that P(Doom|AI) is more likely than not, by 2040

[Solution I’m not thinking of #1]
[Solution I’m not thinking of #2]
.
.
&nbs

...
The Importance-Avoidance Effect
by David Hartsough — 2022-09-30T10:09:14.238Z

My current working theory is that one of the worst things someone with a big project can do is to make that project feel extremely important. (This is doubly worse for someone with ADHD.) The more important the project, the more daunting it becomes, and the less it gets worked on.

When a project becomes ultra important, it tends to get in its own way.

The worst scenario is when someone believes that the project is their "one and only" "personal mission". ("This is my magnum opus." "This is my big project." "This is the one to shape my career." etc, etc, etc.)

For people who struggle with the importance-avoidance effect, they tend to follow this pattern:

The more important they make my project out to be, the less they get it done, and the more they procrastinate on it.

Maybe you're the sort of person who tends to think about what the world's most important problems are and wants to contribute to an important cause. (Cough, cough, EA.) Maybe you have a large project or writing in mind, awaiting action.

Effects

See if any of this sound familiar:

The more emphasized and extreme the value/importance of a project becomes, ...

the more likely we are to procrastinate

the less likely we are to prioritize it

the less we get done

the more it gets in its own way

the more motivation we believe we need

the more likely we are to expect perfection

the more time, attention, dedication, and effort we believe is required

the more overwhelming, difficult, and daunting it feels

the more severe, complicated, and complex it appears

the less clear the vision becomes

the less clarity we have on what to do and how to do it

the less capable and confident we feel

the less we believe we can "get it right" the first time

the more afraid and anxious we get

the more negative emotions we associate with it

the more likely we are to avoid and neglect it

Expectation vs Reality

We like to think that: the more we emphasize the importance (and value) of something, the more motivated we will become. Yet often the truth is: the more we emphasize the importance (and value) of something, the more likely we are to avoid it.

Definition

The Importance-Avoidance Effect is a cognitive bias that causes people to procrastinate and avoid prioritizing something perceived as extremely important.

Side tangent about naming

On the streets, this cognitive bias effect also goes by the nickname, "The Fainting Goat Effec...
History of the theory of well-being
by David Hartsough — 2022-09-30T10:08:32.918Z

An outline of the theories, definitions, and operationalizations of well-being and flourishing throughout the history of positive psychology.

Introduction

Since the dawn of the branch of positive psychology, psychologists have been theorizing definitions and operationalizations of human well-being or "flourishing" as it later became known. This review discusses seven distinguished measurements of well-being and flourishing, examining how they differ and what they have in common.

At a glance, what is the whole history of "flourishing" as a field of study? What is the history of its progress?

Why this matters to EA

Put simply: well-being is often regarded as the end goal and outcome of virtually all humanistic altruism.

80000 Hours defines social impact as:

"Social impact" or "making a difference" is (tentatively) about promoting total expected wellbeing — considered impartially, over the long term — without sacrificing anything that might be of comparable moral importance.

So the goal of many EAs is to promote wellbeing.

Further down that article, they also say:

Why "wellbeing"? We understand wellbeing as an inclusive notion, meaning anything that makes people better off. We take this to encompass at least promoting happiness, health, and the ability for people to live the life they want.
[...] What wellbeing consists of more precisely is a controversial question [...]

They choose to take a philosophical approach to understanding wellbeing. But if there's a field of science that is actively trying to participate in defining "what wellbeing consists of more precisely", then we should pay attention and maybe even help out. (I believe this field of positive psychology could use a lot of help.)

If we want to be more precise about our "inclusive notion" of wellbeing, then maybe we should try "defining what's all encompassed in wellbeing that we should promote".

If we believe that "wellbeing should be a central focus in efforts to make a difference", then maybe we should take a shot at giving that an operationalization, model, framework, or theory.

... Or maybe it's not feasible... maybe it should just go back to philosophy. We'll see.

Acknowledgement

All names listed here are only useful in aiding our learning and our memories. The reality is that no single individual is solely responsible for the ideas and breakthroughs mentioned here. Every person collaborate...
Media Publication Governed by Prediction Markets
by joao_abrantes — 2022-09-30T10:01:49.398Z
Link: https://joao-abrantes.com/writing-monks

The post describes a tool that allows communities to own a decentralised media publication (a shared Twitter account in this case). In contrast to other shared media publications like subreddits, LessWrong or Hacker News, this tool sorts content using Prediction Markets. Any community member can bet on the number of likes each content would have if published on Twitter. The estimated number of likes is used to sort the content.

Members of the community are rewarded according to their contribution. They can contribute by submitting content that gets published and by betting accurately on the markets and being good at knowing which content will perform well on Twitter. The tokens received represent ownership of the Twitter account and can become valuable if someone wants to buy the account, or buy "sponsored tweets" to be posted on the account.

The post goes into the advantages of using Prediction Markets for content curation when compared to Upvotes/Downvotes (spoiler alert: a big advantage is resistance to manipulation).

I am the author of the post and I would love LessWrong to have a Decentralised Twitter Account. If you find this interesting please join our discord.
If it seems intuitive, you're not building intuition
by myutin — 2022-09-30T07:46:09.384Z

Epistemic status: I'm describing a problem I have, which probably has an existing solution.

1.

Let's say you're reading an article, and there's one sentence in particular that doesn't stick out to you. The sentence seems intuitive, so your eyes slide over it and continue down the page.

This happens when your intuition matches with the sentence. The sentence follows some subconscious prediction. It does not surprise you. You extract no evidence from the sentence, and you don't update your beliefs.

The sentence is a missed opportunity. Its presence narrows the possibilities for the territory (unless you really did predict those words in that order). Some update should happen.

The immediate territory here is, which words the author wrote. But this territory is readily mapped, hard to compress, and not applicable elsewhere. There are more interesting territories. What words would you write, if you had to continue the article? What is the space of true statements on the subject? What sentences best explain the topic to other people?

This might seem like a minor nitpick. We're humans, not AIXI. We have finite spoons. We have better things to do than questioning our intuitions all day. But this can sometimes be a bigger issue.

2.

Habits are practiced actions. They reflect some long-term optimization[1], and cost no spoons. Similarly, intuitions are practiced thought patterns. They consist of cached thoughts, both object- and meta-level. If they reside in System 1, then they cost no inferential steps.

Intuitions are powerful tools. They reflect a long period of updating. We should make sure those updates are in the correct direction.

It can be hard to gauge your intuitions. If you're actively learning a subject, you will have plenty of thoughts cached in System 2. This makes you overestimate the extent of your intuition, and consequently, underestimate the amount you need to update.

3.

Back to that article. If you skim over a sentence, you miss an opportunity to strengthen your intuitions. Maybe that's ok; your intuitions are already strong enough, and you're directing effort in a different direction. But if you skim over every sentence, you might as well not read the article. You should have some policy, which sentences to skim, and which to update on. This policy should favor updates when you're actively learning the subject.

It's the next day, and you're in class discus...