# Intrinsic properties and Eliezer's metaethics

#### Abstract

I give an account for why some properties seem intrinsic while others seem extrinsic. In light of this account, the property of moral goodness seems intrinsic in one way and extrinsic in another. Most properties do not suffer from this ambiguity. I suggest that this is why many people find Eliezer's [metaethics](https://wiki.lesswrong.com/wiki/Metaethics_sequence) to be confusing.

#### Section 1: Intuitions of intrinsicness  

What makes a particular property seem more or less _intrinsic_, as opposed to _extrinsic_?

Consider the following three properties that a physical object _X_ might have:

1.  The property of having the shape of a regular triangular. (I'll call this property "∆-ness" or "being ∆-shaped", for short.)
2.  The property of being hard, in the sense of resisting deformation.
3.  The property of being a key that can open a particular lock _L_ (or _L_-opening-ness).

To me, intuitively, ∆-ness seems entirely intrinsic, and hardness seems somewhat less intrinsic, but still very intrinsic. However, the property of opening a particular lock seems very extrinsic. (If the notion of "intrinsic" seems meaningless to you, please keep reading. I believe that I ground these intuitions in something meaningful below.)

When I query my intuition on these examples, it elaborates as follows:

(1) If an object _X_ is ∆-shaped, then _X_ is ∆-shaped _independently of any consideration of anything else_. Object _X_ could manifest its ∆-ness even in perfect isolation, in a universe that contained no other objects. In that sense, being ∆-shaped is intrinsic to _X_.

(2) If an object _X_ is hard, then that fact does have a whiff of extrinsicness about it. After all, _X_'s being hard is typically apparent only in an interaction between _X_ and some other object _Y_, such as in a forceful collision after which the parts of _X_ are still in nearly the same arrangement.

Nonetheless, _X_'s hardness still feels to me to be primarily "in" _X_. Yes, something else has to be brought onto the scene for _X_'s hardness to do anything. That is, _X_'s hardness can be detected only with the help of some "test object" _Y_ (to bounce off of _X_, for example). Nonetheless, the hardness detected is intrinsic to _X_. It is not, for example, primarily a fact about the system consisting of _X_ and the test object _Y_ together.

(3) Being an _L_-opening key (where _L_ is a particular lock), on the other hand, feels very extrinsic to me. A thought experiment that pumps this intuition for me is this: Imagine a molten blob _K_ of metal shifting through a range of key-shapes. The vast majority of such shapes do not open _L_. Now suppose that, in the course of these metamorphoses, _K_ happens to pass through a shape that does open _L_. Just for that instant, _K_ takes on the property of _L_-opening-ness. Nonetheless, and here is the point, an observer without detailed knowledge of _L_ _in particular_ wouldn't notice anything special about that instant.

Contrast this with the other two properties: An observer of three dots moving in space might notice when those three dots happen to fall into the configuration of a regular triangle. And an observer of an object passing through different conditions of hardness might notice when the object has become particularly hard. The observer can use a generic test object _Y_ to check the hardness of _X_. The observer doesn't need anything _in particular_ to notice that _X_ has become hard.

But all that is just an elaboration of my intuitions. What is really going on here? I think that the answer sheds light on how people understand Eliezer's metaethics.

#### Section 2: Is goodness intrinsic?  

I was led to this line of thinking while trying to understand why Eliezer's metaethics is [consistently confusing](/lw/435/what_is_eliezer_yudkowskys_metaethical_theory/).

The notion of an _L_-opening key has been my personal go-to analogy for thinking about how _goodness_ (of a state of affairs) can be objective, as opposed to subjective. The analogy works like this: We are like locks, and states of affairs are like keys. Roughly, a state is _good_ when it engages our moral sensibilities so that, upon reflection, we favor that state. Speaking metaphorically, a state is good just when it has the right shape to "open" us. (Here, "us" means normal human beings as we are in the actual world.) Being of the right shape to open a particular lock is an objective fact about a key. Analogously, being good is an objective fact about a state of affairs.

Objective in what sense? In this important sense, at least: The property of _being L-opening_ picks out a particular point in key-shape space^1^. This space contains a point for every possible key-shape, even if no existing key has that shape. So we can say that a hypothetical key is "of an _L_-opening shape" even if the key is assumed to exist in a world that has no locks of type _L_. Analogously, a state can still be called good even if it is in a counterfactual world containing no agents who share our moral sensibilities.

But the discussion in Section 1 made "being _L_-opening" seem, while objective, very extrinsic, and not primarily about the key _K_ itself. The analogy between "_L_-opening-ness" and goodness seems to work against Eliezer's purposes. It suggests that goodness is extrinsic, rather than intrinsic. For, one cannot properly call a key "opening" in general. One can only say that a key "opens this or that particular lock". But the analogous claim about goodness sounds like relativism: "There's no objective fact of the matter about whether a state of affairs is _good_. There's just an objective fact of the matter about whether it is good _to you_."

This, I suppose, is why some people think that Eliezer's metaethics is just warmed-over relativism, despite [his protestations](/lw/t3/the_bedrock_of_morality_arbitrary/).

#### Section 3: Seeing intrinsicness in simulations  

I think that we can account for the intuitions of intrinsicness in Section 1 by looking at them from the perspective _simulations_. Moreover, this account will explain why some of us (including perhaps Eliezer) judge goodness to be intrinsic.

The main idea is this: In our minds, a property _P_, among other things, "points to" the test for its presence. In particular, _P_ evokes whatever would be involved in detecting the presence of _P_. Whether I consider a property _P_ to be intrinsic depends on how I would test for the presence of _P —_ NOT, however, on how I would test for _P_ "in the real world", but rather on  how I would test for _P_ **_in a simulation_ _that I'm observing from the outside_**.

Here is how this plays out in the cases above.

(1) In the case of being ∆-shaped, consider a simulation (on a computer, or in your mind's eye) consisting of three points connected by straight lines to make a triangle _X_ floating in space. The points move around, and the straight lines stretch and change direction to keep the points connected. The simulation itself just keeps track of where the points and lines are. Nonetheless, when _X_ becomes ∆-shaped, I notice this "directly", from outside the simulation. Nothing else within the simulation needs to react to the ∆-ness. Indeed, nothing else needs to be there at all, aside from the points and lines. The ∆-shape detector is in _me_, outside the simulation. To make the ∆-ness of an object _X_ manifest, the simulation needs to contain only the object _X_ itself.

In summary: A property will feel extremely intrinsic to _X_ when my detecting the property requires only this: "Simulate just _X_."

(2) For the case of hardness, imagine a computer simulation that models matter and its motions as they follow from the laws of physics and my exogenous manipulations. The simulation keeps track of only fundamental forces, individual molecules, and their positions and momenta. But I can see on the computer display what the resulting clumps of matter look like. In particular, there is a clump _X_ of matter in the simulation, and I can ask myself whether _X_ is hard.

Now, on the one hand, I am not _myself_ a hardness detector that can just look at _X_ and see its hardness. In that sense, hardness is different from ∆-ness, which I can just look at and see. In this case, I need to build a hardness detector. Moreover, I need to build the detector _inside the simulation_. I need some other thing _Y_ in the simulation to bounce off of _X_ to see whether _X_ is hard. Then I, outside the simulation, can say, "Yup, the way _Y_ bounced off of _X_ indicates that _X_ is hard." (The simulation itself isn't generating statements like "_X_ is hard", any more than the 3-points-and-lines simulation above was generating statements about whether the configuration was a regular triangle.)

On the other hand, crucially, I can detect hardness with practically _anything at all_ in addition to _X_ in the simulation. I can take practically any old chunk of molecules and bounce it off of _X_ with sufficient force.

A property of an object _X_ still feels intrinsic when detecting the property requires only this: "Simulate just _X_ \+ practically any other arbitrary thing."

Indeed, perhaps I need only an arbitrarily small "epsilon" chunk of additional stuff inside the simulation. Given such a chunk, I can run the simulation to knock the chunk against _X_, perhaps from various directions. Then I can assess the results to conclude whether _X_ is hard. The sense of intrinsicness comes, perhaps, from "taking the limit as epsilon goes to 0", seeing the hardness there the whole time, and interpreting this as saying that the hardness is "within" _X_ itself.

In summary: A property will feel very intrinsic to _X_ when its detection requires only this: "Simulate just _X_ \+ epsilon."

(3) In this light, _L_-opening keys differ crucially from ∆-shaped things and from hard things.

An _L_-opening key differs from an ∆-shaped object because I myself do not encode lock _L_. Whereas I can look at a regular triangle and see its ∆-ness from outside the simulation, I cannot do the same (let's suppose) for keys of the right shape to open lock _L_. So I cannot simulate a key _K_ alone and see its _L_-opening-ness.

Moreover, I cannot add something merely arbitrary to the simulation to check _K_ for _L_-opening-ness.  I need to build something very precise and complicated inside the simulation: an instance of the lock _L_. Then I can insert _K_ in the lock and observe whether it opens.

I need, not just _K_, and not just _K_ \+ epsilon: I need to simulate _K_ \+ _something complicated in particular_.

#### **Section 4: Back to goodness**  

So how does goodness as a property fit into this story?

There is an important sense in which goodness is more like being ∆-shaped than it is like being _L_-opening. Namely, goodness of a state of affairs is something that I can assess myself from outside a simulation of that state. I don't need to simulate anything else to see it. Putting it another way, goodness is like _L_-opening would be if I happened myself to encode lock _L_. If that were the case, then, as soon as I saw _K_ take on the right shape inside the simulation, that shape could "click" with me outside of the simulation.

That is why goodness seems to have the same ultimate kind of intrinsicness that ∆-ness has and which being _L_-opening lacks. We don't encode locks, but we do encode morality.

### Footnote

1\. Or, rather, a small region in key-shape space, since a lock will accept keys that vary slightly in shape.