trentmkelly/LessWrong-43k huggingface dataset:
The mathematical universe: the map that is the territory
by ata — 2010-03-26T09:26:15.631Z

This post is for people who are not familiar with the Level IV Multiverse/Ultimate Ensemble/Mathematical Universe Hypothesis, people who are not convinced that there’s any reason to believe it, and people to whom it appears believable or useful but not satisfactory as an actual explanation for anything.

I’ve found that while it’s fairly easy to understand what this idea asserts, it is more difficult to get to the point where it actually seems convincing and intuitively correct, until you independently invent it for yourself. Doing so can be fun, but for those who want to skip that part, I’ve tried to write this post as a kind of intuition pump (of the variety, I hope, that deserves the non-derogatory use of that term) with the goal of leading you along the same line of thinking that I followed, but in a few minutes rather than a few years.

Once upon a time, I was reading some Wikipedia articles on physics, clicking links aimlessly, when I happened upon a page then titled “Ultimate Ensemble”. It described a multiverse of all internally-consistent mathematical structures, thereby allegedly explaining our own universe — it’s mathematically possible, so it exists along with every other possible structure.

Now, I was certainly interested in the question it was attempting to answer. It’s one that most young aspiring deep thinkers (and many very successful deep thinkers) end up at eventually: why is there a universe at all? A friend of mine calls himself an agnostic because, he says, “Who created God?” and “What caused the Big Bang?” are the same question. Of course, they’re not quite the same, but the fundamental point is valid: although nothing happened “before” the Big Bang (as a more naïve version of this query might ask), saying that it caused the universe to exist still requires us to explain what brought about the laws and circumstances allowing the Big Bang to happen. There are some hypotheses that try to explain this universe in terms of a more general multiverse, but all of them seemed to lead to another question: “Okay, fine, then what caused that to be the case?”

The Ultimate Ensemble, although interesting, looked like yet another one of those non-explanations to me. “Alright, so every mathematical structure ‘exists&rsquo...
Maximise Expected Utility, not Expected Perception of Utility
by JGWeissman — 2010-03-26T04:39:17.551Z

Suppose we are building an agent, and we have a particular utility function U over states of the universe that we want the agent to optimize for. So we program into this agent a function CalculateUtility that computes the value of U given its current knowledge. Then we can program it to make decisions by searching through its available actions for the one that maximizes its expectation for its result of running CalculateUtility. But wait, how will an agent with this programming behave?

Suppose the agent has the opportunity (option A) to arrange to falsely believe the universe is in a state that is worth utility uFA but this action really leads to a different state worth utility uTA, and a competing opportunity (option B) to actually achieve a state of the universe that has utility uB, with uTA < uB < uFA. Then the agent will expect that if it takes option A that its CalculateUtility function will return uFA, and if it takes option B that its CalculateUtility function will return uB. uFA > uB, so the agent takes option A, and achieves a states of the universe with utility uTA which is worse than the utility uB it could have achieved if it had taken option B. This agent is not a very effective optimization process1. It would rather falsely believe that it has achieved its goals than actually achieve its goals. This sort of problem2 is known as wireheading.

Let us back up a step, and instead program our agent to make decisions by searching through its available actions for the one whose expected results maximizes its current calculation of CalculateUtility. Then, the agent would calculate that option A gives it expected utility uTA and option B gives it expected utility uB. uB > uTA, so it chooses option B and actually optimizes the universe. That is much better.

So, if you care about states of the universe, and not just your personal experience of maximizing your utility function, you should make choices that maximize your expected utility, not choices that maximize your expectation of perceived utility.

1. We might have expected this to work, because we built our agent to have beliefs that correspond to the actual state of the world.

2. A similar problem occurs if the agent has the opportunity to modify its CalculateUtility function, so it returns large values for states of the universe that would have occurred anyways (or any state of th...
Newcomb's problem happened to me
by Academian — 2010-03-25T20:53:55.266Z

Okay, maybe not me, but someone I know, and that's what the title would be if he wrote it. Newcomb's problem and Kavka's toxin puzzle are more than just curiosities. Like a lot of thought experiments, they approximately happen. They make the issues with causal decision theory relevant, not only to designing artificial intelligence, but to our everyday lives as well.

Yet somehow it isn't mainstream knowledge that these are more than merely abstract linguistic issues, as evidenced by this comment thread (please no Karma sniping of the comments, they are a valuable record). Scenarios involving brain scanning, decision simulation, etc., can establish their validy and future relevance, but not that they are already commonplace. I want to provide an already-happened, real-life account that captures the Newcomb essence.

So let's say my friend is named Joe. In his account, Joe is very much in love with this girl named Omega… er… Kate, and he wants to get married. Kate is somewhat traditional, and won't marry him unless he proposes, not only in the sense of explicitly asking her, but also expressing certainty that he will never try to leave her if they do marry.

At this point, many of you could easily make up a simple conclusion to this post. As such, I want to convey the actual account, in which Joe's beliefs are roughly schematized as follows:

if he proposes sincerely, she is effectively sure to believe it.

if he proposes insincerely, she will 50% likely believe it.

if she believes his proposal, she will 80% likely say yes.

if she doesn't believe his proposal, she will surely say no, but will not be significantly upset in comparison to the significance of marriage.

if they marry, Joe will 90% likely be happy, and will 10% likely be unhappy.

He roughly values the happy and unhappy outcomes oppositely:

being happily married to Kate: 125 megautilons

being unhapily married to Kate: -125 megautilons.

So what should he do? What should this real person have actually done?1 Well, as in Newcomb, these beliefs and utilities present an interesting and quantifiable problem…

EU(marriage) = 90%·125 - 10%·125 = 100,

EU(sincere proposal) = 80%·100 = 80, and

EU(insincere proposal) = 50%·80%·100 = 40.

No surpris...
Over-encapsulation
by PhilGoetz — 2010-03-25T17:58:56.809Z

Take a look at "Role of Layer 6 of V2 Visual Cortex in Object-Recognition Memory", Science 3 July 2009:
Vol. 325. no. 5936, pp. 87 - 89. The article has some good points, but I'm going to pick on some of its tests.

The experimenters believed they could enhance object-recognition memory (ORM) by using a lentivirus to insert a gene into area V2 of visual cortex. They tested the ORM of rats by putting an object in a field with a rat, and then putting either the same object ("old"), or a different object ("new"), in the field 30, 45, or 60 minutes later. The standard assumption is that rats spend more time investigating unfamiliar than familiar objects.

They chose this test: For each condition, measure the difference in mean time spent investigating the old object vs. the new object. If the latter is more than the former, and the difference is statistically-significant, conclude that the rats recognized the old object.

Figure 1 Graph A (below the article summary cutoff) shows how much time normal rats spent investigating an object. Here it is in HTML table form: How much time the rats spent exploring old and new objects:

Minutes after first exposure
30
45
60

Old
8
12
14

New
17
28
14

The black bars (new) are significantly longer than the white bars (old) after 30 and 45 minutes, but not after 60 minutes. Therefore, the normal rats recognized the old objects after 30 and 45 minutes, but not after 60 minutes.

Figure 3 Graph D (also below the article-summary cutoff) shows how much time different types of rats spent exploring old and new objects. The "RGS" group is rats given the gene therapy, but in parietal cortex rather than in V2.

Here it is in HTML form: How much time the rats spent exploring old and new objects, by rat type:

Rat type
Normal (after 45 min)
Parietal RGS (after 60 min)

Old
10
11

New
27
12

Parietal RGS rats displayed no difference in time spent exploring old and new objects after 60 minutes; therefore, this gene therapy to parietal cortex does not improve ORM.

To recap:

We conclude that rats no longer recognize an old object if they spend about the same time investigating it as investigating a new object.

Normal rats spend the same time investigating old and new objects 60 minutes after first exposure to the old object.

Parietal RGS rats also spend the same time investigating old and new

...
SIA won't doom you
by Stuart_Armstrong — 2010-03-25T17:43:06.467Z

Katja Grace has just presented an ingenious model, claiming that SIA combined with the great filter generates its own variant of the doomsday argument. Robin echoed this on Overcoming Bias. We met soon after Katja had come up with the model, and I signed up to it, saying that I could see no flaw in the argument.

Unfortunately, I erred. The argument does not work in the form presented.

First of all, there is the issue of time dependence. We are not just a human level civilization drifting through the void in blissful ignorance about our position in the universe. We know (approximately) the age of our galaxy, and the time elapsed since the big bang.

How is this relevant? It is relevant because all arguments about the great filter are time-dependent. Imagine we had just reached consciousness and human-level civilization, by some fluke, two thousand years after the creation of our galaxy, by an evolutionary process that took two thousand years. We see no aliens around us. In this situation, we have no reason to suspect any great filter; if we asked ourselves "are we likely to be the first civilization to reach this stage?" then the answer is probably yes. No evidence for a filter.

Imagine, instead, that we had reached consciousness a trillion years into the life of our galaxy, again via an evolutionary process that took two thousand years, and we see no aliens or traces of aliens. Then the evidence for a filter is overwhelming; something must have stopped all those previous likely civilizations from emerging into the galactic plane.

So neither of these civilizations can be included in our reference class (indeed, the second one can only exist if we ourselves are filtered!). So the correct reference class to use is not "the class of all potential civilizations in our galaxy that have reached our level of technological advancement and seen no aliens", but "the class of all potential civilizations in our galaxy that have reached our level of technological advancement at around the same time as us and seen no aliens". Indeed, SIA, once we update on the present, cannot tell us anything about the future.

But there's more. Let us lay aside, for the moment, the issue of time dependence. Let us instead consider the diagrams in Katja's post as if the vertical axis were time: all potential civilizations start at the same point, and progress at the same rate. Is there still a rol...
More thoughts on assertions
by Scott Alexander — 2010-03-25T01:39:12.829Z

Response to: The "show, don't tell" nature of argument

Morendil says not to trust simple assertions. He's right, for the certain class of simple assertions he's talking about. But in order to see why, let's look at different types of assertions and see how useful it is to believe them.

Summary:
- Hearing an assertion can be strong evidence if you know nothing else about the proposition in question.
- Hearing an assertion is not useful evidence if you already have a reasonable estimate of how many people do or don't believe the proposition.
- An assertion by a leading authority is stronger than an assertion by someone else.
- An assertion plus an assertion that there is evidence makes no factual difference, but is a valuable signal.

Unsupported assertions about non-controversial topics

Consider my assertion: "The Wikipedia featured article today is on Uriel Sebree". Even if you haven't checked Wikipedia today and have no evidence on this topic, you're likely to believe me. Why would I be lying?
This can be nicely modeled in Bayesian terms - you start with a prior evenly distributed across Wikipedia topics, the probability of me saying this conditional on it being false is pretty low, and the probability of me saying it conditional on it being true is pretty high. So noting that I said it nicely concentrates probability mass in the worlds where it's true. You're totally justified in believing it. The key here is that you have no reason to believe there's a large group of people who go around talking about Uriel Sebree being on Wikipedia regardless of whether or not he really is.
Unsupported assertions about controversial topics

The example given in Morendil's post is that some races are biologically less intelligent than others. Let's say you have no knowledge of this whatsoever. You're so naive you don't even realize it might be controversial. In this case, someone who asserts "some races are biologically less intelligent than others" is no less believable than someone who asserts "some races have slightly different frequencies of pancreatic cancer than others." You'd accept the second as the sort of boring but reliable biological fact that no one is particularly prone to lie about, and you'd do the same with the first.
Now let's say you're familiar with controversies in sociology and genetics, you already know that some people believe some races are biologica...
SIA on other minds
by KatjaGrace — 2010-03-25T01:09:15.000Z

Another interesting implication if the self indication assumption (SIA) is right is that solipsism is much less likely correct than you previously thought, and relatedly the problem of other minds is less problematic.

Solipsists think they are unjustified in believing in a world external to their minds, as one only ever knows one’s own mind and there is no obvious reason the patterns in it should be driven by something else (curiously, holding such a position does not entirely dissuade people from trying to convince others of it). This can then be debated on grounds of whether a single mind imagining the world is more or less complex than a world causing such a mind to imagine a world.

The problem of other minds is that even if you believe in the outside world that you can see, you can’t see other minds. Most of the evidence for them is by analogy to yourself, which is only one ambiguous data point (should I infer that all humans are probably conscious? All things? All girls? All rooms at night time?).

SIA says many minds are more likely than one, given that you exist. Imagine you are wondering whether this is World 1, with a single mind among billions of zombies, or World 2, with billions of conscious minds. If you start off roughly uncertain, updating on your own conscious existence with SIA shifts the probability of world 2 to billions of times the probability of world 1.

Similarly for solipsism. Other minds probably exist. From this you may conclude the world around them does too, or just that your vat isn’t the only one.
The Spotlight
by Alicorn — 2010-03-24T23:43:52.719Z

Sequence index: Living Luminously
Previously in sequence: Lights, Camera, Action
Next in sequence: Highlights and Shadows
Inspecting thoughts is easier and more accurate if they aren't in your head. Look at them in another form from the outside, like they belonged to someone else.

You may find your understanding of this post significantly improved if you read the fourth story from Seven Shiny Stories.
One problem with introspection is that the conclusions you draw about your thoughts are themselves thoughts. Thoughts, of course, can change or disappear before you can extract information about yourself from them. If a flash of unreasonable anger crosses my mind, this might stick around long enough to make me lash out, but then vanish before I discover how unreasonable it was. If thoughts weren't slippery like this, luminosity wouldn't be much of a project. So of course, if you're serious about luminosity, you need a way to pin down your thoughts into a concrete format that will hold still.
You have to pry your thoughts out of your brain.
Writing is the obvious way to do this - for me, anyway. You don't have to publicize what you extract, so it doesn't have to be aesthetic or skillful, just serviceable for your own reference. The key is to get it down in a form that you can look at without having to continue to introspect. Whether this means sketching or scribing or singing, dump your brain out into the environment and have a peek. It's easy to fool yourself into thinking that a given idea makes sense; it's harder to fool someone else. Writing down an idea automatically engages the mechanisms we use to communicate to others, helping you hold your self-analysis to a higher standard.
To turn your thoughts into non-thoughts, use labels to represent them. Put them in reference classes, so that you can notice when the same quale, habit of inference, or thread of cognition repeats. That way, you can detect patterns: "Hey, the last time I felt like this, I said something I really regretted; I'd better watch it." If you can tell when something has happened twice, you can tell when it hasn't - and new moods or dispositions are potentially very important. They mean that you or something around you has changed, and that could be a valuable resource or a tricky hazard.
Your labels can map onto traditio...
An empirical test of anthropic principle / great filter reasoning
by James_Miller — 2010-03-24T18:44:43.628Z

If our civilization doesn’t collapse then in 50 to 1000 years humanity will almost certainly start colonizing the galaxy. This seems inconsistent with the fact that there are a huge number of other planets in our galaxy but we have not yet found evidence of extraterrestrial life. Drawing from this paradox, Robin Hanson writes that there should be some great filter “between death and expanding lasting life, and humanity faces the ominous question: how far along this filter are we?”

Katja Grace reasons that this filter probably lies in our future and so we are likely doomed. (Hanson agrees with Grace’s argument.) Please read Grace’s post before reading the rest of this post.

Small groups of humans have been in situations similar to that currently faced by our entire species. To see this imagine you live in a prehistoric hunter gatherer tribe of 200 people. Your tribe lives on a large, mostly uninhabited island. Your grandparents, along with a few other people came over to the island about 30 years ago. Since arriving on the island no one in your tribe has encountered any other humans. You figure that if your civilization doesn’t get wiped out then in 100 or so years your people will multiply in number and spread throughout the island. If your civilization does so spread, then any new immigrants to the island would quickly encounter your tribe.

Why, you wonder, have you not seen other groups of humans on your island? You figure that it’s either because your tribe was the first to arrive on the island, or because your island is prone to extinction disasters that periodically wipes out all its human inhabitance. Using anthropic reasoning similar to that used by Katja Grace you postulate the existence of four types of islands:

1) Islands that are easy to reach and are not prone to disaster.
2) Islands that are hard to reach and are not prone to disaster.
3) Islands that are easy to reach and are prone to disaster.
4) Islands that are hard to reach and are prone to disaster.

You conclude that your island almost certainly isn’t type (1) because if it were you would have almost certainly seen other humans on the island. You also figure that on type (3) islands lots of civilizations will start and then be...
The "show, don't tell" nature of argument
by Morendil — 2010-03-24T17:38:09.105Z

Consider a statement of the form "based on knowledge and common sense and estimating the probabilities of alternative hypotheses, I believe that X". Or, perhaps, "based on Bayesian reasoning I have come to the conclusion X". How much, when an interlocutor of yours uses such a statement, should it affect your credence in X?

I claim that the answer is "negligibly", in most cases. (I discuss exceptions below.)

The most appropriate way of conducting an argument is in most cases to simply state your belief X. If you are going to adduce evidence and argument in support of your belief X, simply state these. If you've derived probability estimates, simply provide them, perhaps with the method of derivation. The "meta" observation by itself has no place in an argument.

(Isn't that obvious, you might ask? Not to everyone, as this statement form turns out to be actually used in discussions here. So, in the interest of contributing even a little to a possible theory of argumentation, I develop some further observations on this pattern.)

The assertion "my opinions on X are soundly arrived at" has an equivalent structure to saying "I am a truthful person": it cannot be verified by an interlocutor, other than (in the latter case) by looking at what statements you utter and independently verifying they are truthful, or (in the first) by looking at what arguments and evidence you adduce in support of your opinions on topic X, and independently assessing whether the evidence is credible and the arguments are individually sound.

Claiming to be a truthful person doesn't mean much. I know parents who are constantly telling their kids "oh, you shouldn't lie, it's bad", but who actually lie to their kids quite often. My own approach has been to tell my kids the truth (and to answer pretty much all their questions), while almost never mentioning "truth" as a moral topic. Empirically, I observe that my kids are growing to be truthful and trustworthy persons, more so than many of e.g. my neighbours' kids.

Claiming that you use sound methods in arriving at a conclusion of which you want to convince your interlocutor similarly has little content. It is the sort of thing that can only be assessed by looking at your justifications for belief. The claim may (if stated in an authoritative tone) actually manage to nudge your interlocutor a little in the direction of accepting your conclusions. On...