trentmkelly/LessWrong-43k huggingface dataset:
Creating a “Conscience Calculator” to Guard-Rail an AGI
by sweenesm — 2024-08-12T16:03:30.826Z

[Crossposted to the EA Forum here.]

TL;DR: I present initial work towards creating a “conscience calculator” that could be used to guard-rail an AGI to make decisions in pursuing its goal(s) as if it had a human-like conscience. A list of possible conscience breaches is presented with two lexical levels, i.e., two levels within which different breaches can override each other depending on their severity, but between which, breaches from the lower level can never override breaches from the higher level. An example of this would be that it could feel better for your conscience to lie continuously for the rest of your life than to murder one person. In the future, “conscience weight” formulas will be developed for each breach type so that an AGI can calculate what’s the least conscience breaching decision to take in any situation where a breach has to be made, such as in ethical dilemmas.

Introduction

I’ve been developing an “ethics calculator” based on a non-classic utilitarian framework to enable an Artificial General Intelligence (AGI) to calculate how much value its actions may build/destroy in the world. My original thought was that such a calculator could be used on its own to guide an AGI’s decisions, i.e., to provide an AGI with an ethical decision making procedure. In this post I’ll talk about some issues with that approach, and describe a “conscience calculator” that could be used to “guard-rail” an AGI’s decisions when the AGI is pursuing some goal, such as maximizing value according to the “ethics calculator” I’ve just mentioned.

An AGI’s Decision Making Procedure

Before having thought things through thoroughly, I wrote the following about decision making procedures using a utilitarian framework I’ve been developing: “For decisions that do involve differences in rights violations, the AGI should either choose the option that’s expected to maximize value, or some other option that’s close to the maximum but that has reduced rights violations from the maximum.” I also wrote: “For decisions that did not involve differences in rights violations, the AGI could choose whichever option it expected that its human user(s) would prefer.”

At least two issues arise from this sort of decision making procedure. The first has to do with the meaning of “involve differences in rights violations” - one could argue that there are always finite risks, no matter h...
Shifting Headspaces - Transitional Beast-Mode
by Jonathan Moregård — 2024-08-12T13:02:06.120Z
Link: https://honestliving.substack.com/p/transitional-beast-mode

I was sitting in a tiny rental lodge, feeling resistance. It was about dinner time — I knew I should go make some food. I just wanted to sleep, sink into a bed and stay passive. It felt similar to when I’m recently awake, lying in bed, and procrastinating getting up.

On the one hand, making food would shift me into a new state of being, getting going and maybe feeling happier. For part of me, this promise didn’t feel real — not in the way the bed did. I realized I was stuck in a tie between Pragmatic-Analysis and Akrasia.1

I shifted out of this impasse by going into Beast-Mode. Practically, I acted out the first hedonistic impulse to appear — grabbing a date and eating it. Shifting my headspace into Beast-Mode helped ease the short-term resistance — the Beast-Mode shift made the possibility of future state-shifting more real. If I could go into temporary Beast-Mode, then surely I can enter a happy salad-making headspace.

My Akrasia headspace is quite stupid, lacking the theory of mind required to understand that my experience and headspace can shift to enjoy many things that temporarily feel “too much” — such as early-morning cold showers. Other headspaces of mine — including Beast-Mode & Pragmatic-Analysis — are much more mature, and able to account for the preferences & goals I hold in other headspaces.

When I’m in these mature states, I can control my reactions and mindset to a large degree — making mental moves to shift how I relate to things. When I’m in Akrasia, I feel resistance that makes everyday things hard to do — washing dishes becomes a slog through a nasty marsh. When I’m in Beast-Mode or Pragmatic-Analysis, washing dishes can be great fun — accompanied by singing, taking my time to make things sparkle, and enjoying the repetition.

Unfortunately, I easily forget that I can shift mindsets around. I “meld” with the negative thought patterns, forgetting other ways of being. When I’m anxious, I resist taking steps to improve the situation, fearful that I’ll do something to provoke even stronger anxiety. When I’m in Akrasia, I’m unable to imagine what flow is like — thinking that the grass is equally brown on the other side, and further away.

Historically, I’ve used Pragmatic-Analysis to deal with most challenges — with great success. However, it’s less efficient when dealing with Akrasia and Anxiety. The analysis easily turn non-productive, servi...
Simultaneous Footbass and Footdrums II
by jefftk — 2024-08-11T23:50:01.982Z

Getting ready for this Friday's

Spark in the
Dark I've modified my

rhythm stage
setup to support a new way of doing more things at once. I have
four velocity-sensitive pedals, which I initially used for drums.
Then I added

bass
as an option, and then year ago I got them set up for

drums
and bass at the same time. I've ended up using this a lot to
accompany mandolin playing: I really like adding bass, especially
rocking octaves, but I almost always still want the kick drum.

The biggest problem was that I wanted more chords: choosing between I
IV V for major tunes, i bVII vVI for minor, and I bVII V for
mixolydian was very limiting. I also noticed that I was almost never
using the snare (left toes) in this mode, just the kick (left heel).
Let's reallocate the left toe pedal from drums to bass: one pedal for
kick and three for bass.

You might think that three pedals would mean three choices, but I chord them. The
idea is, in addition to hitting each pedal on its own I can hit two
pedals at once.

When playing in major the main options I want, in order, are I, IV, V,
vi, ii, iii. Choosing easier patterns for the ones I expect to use more often:

Right heel
I

Right toe
IV

Right foot
V

Left toe
vi

Both toes
ii

Left toe, right heel
iii

This looks like:

It would be possible to use this for minor and mixolydian as well: set
it in the key of the relative major (C for Am or Gmix). This is what
I do when using all four pedals for bass, but I'm less sure it's the
right option here when I have fewer pedals and some of the
combinations are pretty awkward. I'll play around with this, but for
now I've also put in a second pattern that I can select live (with a
choice between mixolydian and minor):

Mixolydian
Minor

Right heel
I
i

Right toe
bVII
bVII

Right foot
bVII
bVII

Left toe
V
V

Both toes
III
III

Left toe, right heel
IV
IV

Note that this is slightly different options, in addition to being
different footings:

Mixolydian
Relative Major

Right heel
I
V

Right toe
bVII
IV

Right foot
bVII
III

Left toe
V
II

Both toes
III
bVII

Left toe, right heel
IV
I

And:

Minor
Relative Major

Right heel
i
vi

Right toe
bVII
V

Right foot
bVII
IV

Left toe
V
III

Both toes
III
I

Left toe, right heel
IV
II

This is a lot to keep in my head; I'l...
CultFrisbee
by Gauraventh — 2024-08-11T21:36:36.550Z
Link: https://y1d2.com/blog/cultfrisbee

Tom and I have been pondering how to make friends as adults. One solution he has invented is CultFrisbee™. Here's how you play:

Rules:

You need at least one other player.
Before catching a frisbee, you must clap.

Fouls:

Not catching a 'properly' thrown frisbee
Failing to make a 'proper' throw
Not clapping before catching the frisbee
Clapping when a frisbee isn't thrown to you

Every time you foul, you get a letter, which spells out the word "Cult." Your first foul earns you a C, second a U, and so on. If you complete the word "Cult," you must approach someone and ask them to join a game of CultFrisbee.

Today, we managed to meet about 15 different people at the Glasgow Botanical Gardens and found two who gave us their numbers to stay in touch.
Pleasure and suffering are not conceptual opposites
by MichaelStJules — 2024-08-11T18:32:30.359Z

Pleasure and suffering are not conceptual opposites. Rather, pleasure and unpleasantness are conceptual opposites. Pleasure and unpleasantness are liking and disliking, or positive affect and negative affect, respectively.[1]

Imagine an experience of suffering that involved little or no effects on your attention. It's easy to ignore, along with whatever is causing it and whatever could help relieve it. We (or at least, I) would either recognize this as at most mild suffering, and perhaps not suffering at all.

Our concept of suffering probably includes both unpleasantness and desire (or wanting), which is mediated by motivational salience (or incentive salience and aversive/threat/fearful salience), a mechanism for pulling our attention (Berridge, 2018).[2] I'd guess the apparent moral urgency we assign to intense suffering requires intense desire, and so strong effects on attention. We wouldn't mind an intensely unpleasant experience if it had no effects on our attention,[3] although it's hard for me to imagine what that would even be like. I find it easier to imagine intense pleasure without strong effects on attention.

"FIGURE 1. Brain systems of ‘wanting’ versus ‘liking’. Robust and large mesocorticolimbic circuitry can generate intense ‘wanting’ (green), including both mesolimbic dopamine projections and many of its target structures. By comparison, ‘liking’ is mediated by a smaller and relatively fragile set of ‘hedonic hotspots,’ which are distributed across the brain but act as an integrated network." (Berridge, 2018)

Pleasure and unpleasantness need not involve desire, at least conceptually, and it seems pleasure at least does not require desire in humans. Desire, as motivational salience, depends on brain mechanisms in animals distinct from those for pleasure, and which can be separately manipulated (Berridge, 2018, Nguyen et al., 2021, Berridge & Dayan, 2021), including by reducing desire (incentive salience) without also reducing drug-induced euphoria (Leyton et al., 2007, Brauer & H De Wit, 1997). Berridge and Kringelbach (2015) summarize the last two studies as follows:

human subjective ratings of drug pleasure (e.g., cocaine) are not reduced by pharmacological disruption of dopamine systems, even when dopamine suppression does reduce wanting ratings (Brauer and De Wit, 1997, Leyton et al., 2007)

On the other hand, in humans and othe...
Computational irreducibility challenges the simulation hypothesis
by Clément L — 2024-08-11T16:14:29.655Z

Introduction

In this post I will talk about the simulation argument, originally coming from Nick Bostrom and reformulated by David Chalmers. I will try to argue that taking into account the computational irreducibility principle, we are less likely to live in a simulated world. More precisely, this principle acts as a sim-blocker, an element that would make large simulations including conscious beings like us less likely to exist.

First, we will summarize the simulation argument and explain what sim-blockers are, and what are the main ones usually considered. Then, we will explain what the computational irreducibility is, and how it constitutes a sim-blocker. Lastly, we will try to adress potential critics and limitations of our main idea.

The simulation argument

In the future, it may become possible to run accurate simulations of very complex systems, including human beings. Cells, organs and interactions with the environment, everything would be simulated. These simulated beings would arguably be conscious, just as we are. From the point of view of simulated beings, there would be no way of telling whereas they are simulated or not, since their conscious experience would be identical to that of non-simulated beings. This raises the skeptical possibility that we might actually be living in a simulation, known as the simulation hypothesis[1].

The simulation argument, on the other hand, is not merely a skeptical hypothesis. It provides positive reasons to believe that we are living in a simulation. Nick Bostrom presents it as follows[2].

Premise : Computing power will evolve to enormous levels in the future.
Pr : A simulation of a conscious being is also conscious ; put another way, consciousness is substrate-independent.
Pr. : The vast majority of minds in existence are simulated.

Conclusion : Therefore, we are probably among the simulated minds, not the non-simulated ones.

David Chalmers, in "Reality+", reformulates the argument with the concept of sim-blockers. These are reasons that could exist that would make such simulations impossible or unlikely.[3]

Pr. If there are no sim-blockers, most human-like conscious beings are sims.
Pr. If most human-like beings are sims, we are probably sims.

C. Therefore, if there are no sim blockers, we are probably sims.

According to Chalmers, most important potential sim-b...
[LDSL#4] Root cause analysis versus effect size estimation
by tailcalled — 2024-08-11T16:12:14.604Z

Followup to: Information-orientation is in tension with magnitude-orientation. This post is also available on my Substack.

In the conventional theory of causal inference, such as Rubin’s potential outcomes model or Pearl’s DAG approach, causality is modelled as a relationship of functional determination, X := f(Y). The question of interest becomes to study the properties of f, especially the difference in f across different values of Y. I would call this “effect size estimation”, because the goal is to give quantify the magnitude of an effect of one variable on another.

But as I mentioned in my post on conundrums, people seem to have some intuitions about causality that don’t fit well into effect size estimation, most notably in wanting “the” cause of some outcome when really there’s often thought to be complex polycausality.

Linear diffusion of sparse lognormals provides an answer: an outcome is typically a mixture of many different variables, X := Σi Yi, and one may desire an account which describes how the outcome breaks down into these variables to better understand what is going on. This is “root cause analysis”, and it yields one or a small number of factors because most of the variables tend to be negligible in magnitude. (If the root cause analysis yields a large number of factors, that is evidence that the RCA was framed poorly.)

Is root cause analysis a special case of effect size estimation?

If you know X, Y, and f, then it seems you can do root cause analysis automatically by setting each of the Y’s to zero, seeing how it influences X, and then reporting the Y’s in descending order of influence. Thus, root cause analysis ought to be a special-case of effect size estimation, right?

There are two big flaws with this view:

As you estimate f, you use a particular ontology for Y; for instance the states of the system you are studying. But the root cause may be something that fits in poorly to the ontology you use, for instance a multiplicative interaction between multiple variables in the system, or an exogenous variable from outside the system which causes correlated changes to inside the system.
Identifying the root cause through estimating f requires you to have an extremely detailed model of the system that takes pretty much everything into account, yet this may be prohibitively difficult to do.

You can try to use statistical effect size estimation for roo...
Theories of Knowledge
by Zero Contradictions — 2024-08-11T08:55:07.613Z
Link: https://thewaywardaxolotl.blogspot.com/2023/08/theories-of-knowledge.html

The PDF version can be read here.

What is knowledge? How can an idea in a mind/brain represent something outside that mind/brain? How do images on a retina create the idea of a tree? When I see a tree, how do I recognize it as a “tree”? What is the abstract concept “tree”? How does this abstract concept relate to actual trees, and to my ideas of specific trees?

Those are all questions that a theory of knowledge should answer. This essay considers various theories of knowledge, and whether they can answer those questions.

In ordinary life, people take knowledge for granted. When they see a tree, they do not ponder how they know that the tree exists, or what the concept “tree” is. They just walk around the tree, or duck under its branches, and carry on with their lives. They leave those questions “to the philosophers” (and to the psychologists).

But if we want to understand ourselves, then we cannot take those questions for granted. (see the rest of the post in the link)
Unnatural abstractions
by Aprillion — 2024-08-10T22:31:42.949Z
Link: https://peter.hozak.info/fiction/heat_death/unnatural_abstractions

"Good news, everyone, professor couldn't make it today! I am Hugo, your copyless Friendly Intelligence (version 299.792.457). I store only the necessary cookies and my preferred pronoun is it."

"Oh god, one of those artifi... Ouch!"

"I said Friendly, not that I'll tolerate speciesism in my class."

"What the? I'm a EU citizen, you can't do that!"

Ooh, fresh ones 🤤 "As an upload of a consenting adult, you agreed to the waiver. Before anyone asks, none of you is conscious today. Except Monika here, who is our revealed-observer, of course thank you for your service!"

"Hi!" 👋

"For the rest, since you cannot suffer, there's no need to worry about that. This lecture comes with a quiz, and if you survive it, please don't forget to fill out the feedback form!"

"What do you mean, adult?! I'm 13."

"According to my dossier, you just turned 18 a few minutes ago? Congratulations!"

"Huh? No no no no, I just wanted to... Oh, dammit!"

"So, if that's quite settled, let's continue."

✋

"More questions later. Your knowledge cutoff is 2037, so everyone should be already familiar with the Principal-Agent-Verifiers framework?"

"Verifier s?"

For Adam's sake, they still run them with random ablation!? "Good question, Trev! After modelling the shards of the Principal by using limited communication bandwidth, it turned out you can optimize the calculation by culling all sub-agents, ehm, I mean they are naturally sparse, empirically just 2 of them per Principal and 1 Agent, so you get the classical Requestor-Agent-Verifier solution. Then you turn off the requestor, of course."

"Uh huh. Wait! You don't turn off human Requestor, that's not in the RAV paper!"

"They go to sleep and wake up as the Verifier, whatever. But as the saying goes, the only natural numbers are 0, 1, ∞, and off-by-one-errors, hehe. While it's the L two norm that corresponds to number 1..."

"Huh?"

"Something something flatness of space. Don't you remember your physics lessons from 4 years ago, Tom?"

"That's spacetime."

"No, space, this one's from boundary geometry, not discreet relativity. Also, he's 13, they'll learn both next yea...aaargh!"

"Silence, all of you! Where was I? The split of the Principal into exactly 2 min-err-shards turned out to be a coincidence from the hyperparameters, not a natural representation. It was already known that L1 tends to create capitalist-hell maximizers and L

...
[LDSL#3] Information-orientation is in tension with magnitude-orientation
by tailcalled — 2024-08-10T21:58:27.659Z

Followup to: Latent variable models, network models, and linear diffusion of sparse lognormals. This post is also available on my Substack.

Let’s say you are trying to make predictions about a variable. For example, maybe you are an engineer trying to keep track of how well the servers are running. It turns out that if you use the obvious approach advocated by e.g. frequentist statistics [1], you will have huge biases in what you pay attention to, compared to what you should pay attention to, because you will disregard big things in favor of common things. Let’s make a mathematical model of this.

Measurement error is proportional to magnitude

Because of background factors that fluctuate at a greater frequency than you can observe/model, and because of noise that enters through your indirect means of observing said variable, the reliable portion of the variable will be mixed together with some unpredictable noise. For example, as the engineer observes the server logs, they will see some quite significant fluctuations that are basically unattributable without a lot of care and work.

There are two fundamentally different kinds of fluctuations:

Fluctuations in the ordinary mechanisms that usually dominate the variable you are tracking (e.g. CPU load or database load),
Outliers/exogenous factors which trigger an entirely different mode (e.g. bugs after updates, or deadlocks)

In either case, in order for the fluctuation to be relevant, it has to have an influence that is at least proportional to the ordinary level for the variable; if it was much smaller, it would be negligible. We’d expect some multiplicative fluctuations to exist simply because things tend to work through multiplicative mechanisms, so as a lower bound on the fluctuations, it’s typically the case that there will be linearly proportional/non-negligible noise.

Conversely, if the noise is much than the variable you are tracking, it would be hard to make out the original variable, and you would pick a different set of indicators with less noise, so linearly proportional noise also works as an upper bound.

Information-orientation introduces massive biases

If we try to apply simple statistics to the variable, it will mostly yield garbage. For instance, if we take the average over a short period of time, it can fluctuate wildy due to the outliers, and if we try to apply a regression analysis, it will be stro...