trentmkelly/LessWrong-43k huggingface dataset:
Boltzmann Brains, Simulations and self refuting hypothesis 
by Donald Hobson — 2018-11-26T19:09:42.641Z

Let's suppose, for the purposes of this post, that our best model of dark energy is such that an exponentially vast number of Boltzman brains will exist in the far future. The idea that we may be in an ancestor simulation is similar in its self refuting nature but slightly vaguer, as it depends on the likely goals of future societies.

What do I mean when I say that these arguments are self refuting? I mean that accepting the conclusion seems to give a good reason to reject the premise. Once you actually accept that you are a Boltzmann brain, all your reasoning about the nature of dark energy becomes random noise. There is no reason to think that you have the slightest clue about how the universe works. We seem to be getting evidence that all our evidence is nonsense, including the evidence that told us that. The same holds for the simulation hypothesis, unless you conjecture that all civilizations make ancestor simulations almost exclusively.

What's actually going on here. We have three hypotheses.

1) No Boltzmann brains, the magic dark energy fairy stops them being created somehow. (Universe A)

2) Boltzmann brains exist, And I am not one. (Universe B)

3) I am a Boltzmann brain. (Universe B)

As all these hypothesis fit the data, we have to tell them apart on priors, and anthropic decision theory, with the confusion coming from not having decided on an anthropic theory to use, but ad-lib-ing it with intuition.

SIA Selects from all possible observers, and so tells you that 3) is by far the most likely.

SSA, with an Ocamian Prior says that Universe B is slightly more likely, because it takes fewer bits to specify. However most of the observers in Universe B are Boltzmann brains seeing random gibberish. The observation of any kind of pattern gives an overwhelming update towards option 1).

If we choose to minimize the sum of both the amount of info needed to describe the universe, and the amount needed to specify your place within it, then we find that Universe B is simpler to describe, and it is far easier to describe the position of an evolved life-form near the beginning of time, than to locate a Boltzmann brain around 1010100 years in. An AIXI that is simulating the rest of the universe, with patching rules to match its actions up to the world, will act as if it believes option 2).
Quantum Mechanics, Nothing to do with Consciousness
by Donald Hobson — 2018-11-26T18:59:19.220Z

Epistemic status: A quick rejection of the quantum consciousness woo. If you have already read the sequences, there's nothing new in here. If your new to the site, or need a single page to point people to, here it is.

Real Quantum mechanics looks like pages of abstract maths, after which you have deduced the results of a physics experiment. Given how hard the maths is, most of the systems that we use quantum mechanics to predict are quite simple. One common experiment is to take a glass tube full of a particular element and run lots of electricity through it. The element will produce coloured light, like sodium producing orange, or neon producing red. So take a prism and split that light to see what colours are being produced. Quantum physisists will do lots of tricky maths about how the electrons move between energy levels to work out what colour different elements will produce.

There have been no quantum mechanics experiments that show consciousness to have any relevance to particle physics. The laws of physics do not say what is or is not conscious, in much the same way that they don't say what is or is not a work of art. Of course, consciousness is a property of human brains, and human brains, like everything else in the universe, are made of electrons and quarks playing by quantum laws. The point is that human brains are not singled out for special treatment, they get the same rules as everything else.

For the writers among you, think of a word processor feature that takes some text, and turns it into ALL CAPS. You can put a great novel into this feature if you want. The point is that the rule its self acts the same way whether or not it's given great literature. You can't use the rule to tell what is great literature, you have to read it and decide yourself. Consciousness, like literature, is a high level view that's hard to pin down precisely, and is largely a matter of how we choose to define it. Quantum mechanics is a simple, mechanistic rule.

Yes I know that some of you are thinking of the double slit experiment. You make a screen with two slits, shine light through and get an interference pattern. Put a detector at one slit, attach a dial to the detector, and have a scientist watching the dial so they can see which slit the photon went through, and the interference pattern disappears. Perhaps, thought some...
Status model
by Bucky — 2018-11-26T15:05:12.105Z

[Epistemic status: My best guess.]

Following a conversation on a previous post, I decided to do some research into the community’s thoughts on status. The community talks about status a lot and I spent a few happy hours sifting through everything I could find.

I came up with a status model based on the posts and comments which I read:

Essentially, status-related mental adaptations are executed which lead to certain status behaviours. These behaviours determine our social standing which, at least in the ancestral environment, tended to affect out fitness.

I don’t think there’s anything ground breaking here (a similar model would probably apply to any adaptation execution vs fitness maximising effect) but I haven’t seen it sketched out specifically for status before.

The word "status" gets used to refer to items on all 4 levels and this can lead to confusion where two people are referring to different levels.

For instance, what is status and how can I measure it? One way is to look directly at row 2 (status rating): who respects whom and how much? Maybe I could give everyone a questionnaire to rate each other’s status. Or I can look at row 3 (status behaviours): who acts like they have high status? Or row 1 (status benefits): who has the most social control etc.? Whoever gets the most benefits probably has the most status.

Each option has its advantages and disadvantages e.g. accuracy, ease of assessment but it is important to know which level is being referred to.

Probably the most commonly debated issue on this topic is whether status is zero-sum.

If we consider status in row 2 then status is probably going to be relatively zero-sum, although you can maybe get around this a bit by splitting into smaller sub-cultures.

If we consider row 1, insofar as the status rating determines the benefits, they are close to zero-sum. However, the benefits are also controlled by things other than status (how good are we at getting food, how well coordinated are we as a group?) and so are able to be positive sum.

Row 4 is where it gets really interesting - our adaptations which implement status behaviour are not zero-sum. We can feel more self-esteem without increasing our status rating (see That Other Kind Of Status). These mental adaptations are the things which we care about on a gut level and give plenty of scope for positive sum behaviours (e.g. give praise).

I don'...
Humans Consulting HCH
by paulfchristiano — 2018-11-25T23:18:55.247Z

(See also: strong HCH.)

Consider a human Hugh who has access to a question-answering machine. Suppose the machine answers question Q by perfectly imitating how Hugh would answer question Q, if Hugh had access to the question-answering machine.

That is, Hugh is able to consult a copy of Hugh, who is able to consult a copy of Hugh, who is able to consult a copy of Hugh…

Let’s call this process HCH, for “Humans Consulting HCH.”

I’ve talked about many variants of this process before, but I find it easier to think about with a nice handle. (Credit to Eliezer for proposing using a recursive acronym.)

HCH is easy to specify very precisely. For now, I think that HCH is our best way to precisely specify “a human’s enlightened judgment.” It’s got plenty of problems, but for now I don’t know anything better.

Elaborations

We can define realizable variants of this inaccessible ideal:

For a particular prediction algorithm P, define HCHᴾ as:
“P’s prediction of what a human would say after consulting HCHᴾ”
For a reinforcement learning algorithm A, define max-HCHᴬ as:
“A’s output when maximizing the evaluation of a human after consulting max-HCHᴬ”
For a given market structure and participants, define HCHᵐᵃʳᵏᵉᵗ as:
“the market’s prediction of what a human will say after consulting HCHᵐᵃʳᵏᵉᵗ”

Note that e.g. HCHᴾ is totally different from “P’s prediction of HCH.” HCHᴾ will generally make worse predictions, but it is easier to implement.

Hope

The best case is that HCHᴾ, max-HCHᴬ, and HCHᵐᵃʳᵏᵉᵗ are:

As capable as the underlying predictor, reinforcement learner, or market participants.
Aligned with the enlightened judgment of the human, e.g. as evaluated by HCH.

(At least when the human is suitably prudent and wise.)

It is clear from the definitions that these systems can’t be any more capable than the underlying predictor/learner/market. I honestly don’t know whether we should expect them to match the underlying capabilities. My intuition is that max-HCHᴬ probably can, but that HCHᴾ and HCHᵐᵃʳᵏᵉᵗ probably can’t.

It is similarly unclear whether the system continues to reflect the human’s judgment. In some sense this is in tension with the desire to be capable — the more guarded the human, the less capable the system but the more likely it is to reflect their interests. The question is whether a prudent human can achieve both goals.

This was originally posted here on 29th January 2016.

Tomorrow's AI...
Approval-directed bootstrapping
by paulfchristiano — 2018-11-25T23:18:47.542Z

Approval-directed behavior works best when the overseer is very smart. Where can we find a smart overseer?

One approach is bootstrapping. By thinking for a long time, a weak agent can oversee an agent (slightly) smarter than itself. Now we have a slightly smarter agent, who can oversee an agent which is (slightly) smarter still. This process can go on, until the intelligence of the resulting agent is limited by technology rather than by the capability of the overseer. At this point we have reached the limits of our technology.

This may sound exotic, but we can implement it in a surprisingly straightforward way.

Suppose that we evaluate Hugh’s approval by predicting what Hugh would say if we asked him; the rating of action a is what Hugh would say if, instead of taking action a, we asked Hugh, “How do you rate action a?”

Now we get bootstrapping almost for free. In the process of evaluating a proposed action, Hugh can consult Arthur. This new instance of Arthur will, in turn, be overseen by Hugh—and in this new role Hugh can, in turn, be assisted by Arthur. In principle we have defined the entire infinite regress before Arthur takes his first action.

We can even learn this function by examples — no elaborate definitions necessary. Each time Arthur proposes an action, we actually ask Hugh to evaluate the action with some probability, and we use our observations to train a model for Hugh’s judgments.

In practice, Arthur might not be such a useful assistant until he has acquired some training data. As Arthur acquires training data, the Hugh+Arthur system becomes more intelligent, and so Arthur acquires training data from a more intelligent overseer. The bootstrapping unfolds over time as Arthur adjusts to increasingly powerful overseers.

This was originally posted here on 21st December 2014.

Tomorrow's AI Alignment Forum sequences will take a break, and tomorrow's post will be Issue #34 of the Alignment Newsletter.

The next post in this sequence is 'Humans consulting HCH', also released today.
Values Weren't Complex, Once.
by Davidmanheim — 2018-11-25T09:17:02.207Z

The central argument of this post is that human values are only complex because all the obvious constraints and goals are easily fulfilled. The resulting post-optimization world is deeply confusing, and leads to noise as the primary driver of human values. This has worrying implications for any kind of world-optimizing. (This isn't a particularly new idea, but I am taking it a bit farther and/or in a different direction than this post by Scott Alexander, and I think it is worth making clear, given the previously noted connection to value alignment and effective altruism.)

First, it seems clear that formerly simple human values are now complex. "Help and protect relatives, babies, and friends" as a way to ensure group fitness and survival is mostly accomplished, so we find complex ethical dilemmas about the relative values of different behavior. "Don't hurt other people" as a tool for ensuring reciprocity has turned into compassion for humanity, animals, and perhaps other forms of suffering. These are more complex than they could possibly have been expressed in the ancestral environment, given restricted resources. It's worth looking at what changed, and how.

In the ancestral environment, humans had three basic desires; they wanted food, fighting, and fornication. Food is now relatively abundant, leading to people's complex preferences about exactly which flavors they like most. These differ because the base drive for food is overoptimizing. Fighting was competition between people for resources - and since we all have plenty, this turns into status-seeking in ways that aren't particularly meaningful outside of human social competition. The varieties of signalling and counter-signalling are the result. And fornication was originally for procreation, but we're adaptation executioners, not fitness maximizers, so we've short-cutted that with birth control and pornography, leading to an explosion in seeking sexual variety and individual kinks.

Past the point where maximizing the function has a meaningful impact on the intended result, we see the tails come apart. The goal seeking of human nature, however, needs to find some direction to push the optimization process. The implication from this is that humanity finds diverging goals because they are past the point where the basic desires run out. As Randall Munroe...
Library of things
by UnplannedCauliflower — 2018-11-25T08:57:44.991Z

Hi, people!

Now that I've found this place and before I'm on the mailing list I can share the idea I mentioned during the most recent dojo

A library of things like ladders, waffle irons, drills, projectors, hair trimmers - items which are not all the time in use (or even rarely in use), but it's still useful to have access to them. In order to make it as simple as possible I was thinking about creating a document somewhere decently protected from uninvited people, e.g. a file in the mailing list's folder, where the items, their owners and current location would be listed.

Other information that will be useful: how to contact the owner, rules for renting (use only at owner's place, rent for a couple of days, rent long-term and return on request, rent long-term and you can pass to other members of the group etc.), notes (e.g. this ladder is only 1m high; waffle iron comes with additional plates for sandwich making)

Using: find the item you need, contact the owner, arrange the details between the two of you and update the item's location in the file if it'll stay at the new place for longer than a few days

General rules: take good care of the item you borrow. Don't request money for lending unless it actually costs you to have other people use the item. If you break something, be prepared to pay for it. Keep the file up-to-date

(Oh, and if the name is unknown to you, it's Asia writing)
A culture of exploitation?
by Bae's Theorem — 2018-11-24T22:00:09.189Z

Thoughts in the car:

I just had a discussion with a coworker who bemoans the lack of respect and responsibility of the newer generations. While I do think these concerns are typically overstated due to availability bias, there could be some truth to them.

I hypothesize that a significant contributing factor is our ever-expanding population; our increasing ability to discard one social group and easily form a new one.
One of our primary motivations for being kind to others is the hope that others will be kind to us. (see the thought experiment 'Prisoner's Dilemma')
Yes, most of us have empathy as well, but this can vary by degree and situation; the most reliable motivations are usually the selfish ones.
If we are able to exploit one social group, then move on to a new group, thereby avoiding the consequences of our actions, this encourages exploitation ("exploitation" is defined here as reaping the benefits of a social group while not respecting others or taking responsibility for your part in the group).
Side note: A lack of consequences for our actions is also an explanation for why internet communications can so easily turn toxic.

Before we can falsify this, we would have to first establish quantifiable data to represent our culture of exploitation.
Then we could demonstrate a similar environment (large choice of social groups) where such a culture of exploitation (among relatively equal members) doesn't occur.
Or, we could find a culture where there ISN'T a large choice of social groups, where exploitation (among relatively equal members) occurs regularly.
Finally, we could identify a different factor that better explains the data.
Fixed Point Discussion
by Scott Garrabrant — 2018-11-24T20:53:39.545Z

Warning: This post contains some important spoilers for Topological Fixed Point Exercises, Diagonalization Fixed Point Exercises, and Iteration Fixed Point Exercises. If you plan to even try the exercises, reading this post will significantly reduce the value you can get from doing them.

Core Ideas

A fixed point of a function f is an input x such that f(x)=x. Fixed point theorems show that various types of functions must have fixed points, and sometimes give methods for finding those fixed points.

Fixed point theorems come in three flavors: Topological, Diagonal, and Iterative. (I sometimes refer to them by central examples as Brouwer, Lawvere, and Banach, respectively.)

Topological fixed points are non constructive. If f is continuous, f(0)>0, and f(1)<1, then we know f must have some fixed point between 0 and 1, since f(x) must somewhere transition from being greater than x to being less than x. This does not tell us where it happens. This can be especially troublesome when there are multiple fixed points, and there is no principled way to choose between them.

Diagonal fixed points are constructed with a weird trick where you feed a code for a function into that function itself. Given a function f, if you can construct a function g, which on input x, interprets x as a function, runs x on itself, and then runs f on the result (i.e. g(x):=f(x(x)).), then g(g) is a fixed point of f because g(g)=f(g(g)). This is not just an example; everything in the cluster looks like this. It is a weird trick, but it is actually very important.

Iterative fixed points can be found through iteration. For example, if f(x)=−x2, then starting with any x value, iterating f forever will converge to the unique fixed point x=0.

(There is a fourth cluster in number theory discussed here, but I am leaving it out, since it does not seem relevant to AI, and because I am not sure whether to put it by itself or to tack it onto the topological cluster.)

Topological Fixed Points

Examples of topological fixed point theorems include Sperner's lemma, the Brouwer fixed point theorem, the Kakutani fixed point theorem, the intermediate value theorem, and the Poincaré-Miranda theorem.

Brouwer is the central example of the cluster. Brouwer states that any continuous function f from a compact convex set to itself has a fixed point.
Sperner's Lemma is a discrete analogue which is used in one

... ,,
Four factors that moderate the intensity of emotions
by Ruby — 2018-11-24T20:40:12.139Z

Epistemic status: influenced by my study of emotions over the years, but primarily based on personal observation. Presented in the spirit of "here's a model, judge for yourself whether you think it's any good."

If you’re short on time or just skimming, I suggest skipping to the description of Closeness-of-the-counterfactual. It’s the factor I think is least recognized and the one I most wanted to write about.

What makes some emotions stronger than others? Why are some the faintest whispers, easily missed and others roaring, crashing storms which threaten to consume us?

The obvious answer is that emotions vary in intensity in proportion to the magnitude of what they’re about. Things which are a little bit good or a little bit bad evoke weak pleasant or aversive feelings, while things which are amazingly good or terribly bad provoke strong feeling. However, I assert that this magnitude is only one factor among several and is insufficient on its own to explain what causes strong or weak emotions.

In this post, I list the factors which are salient to me: magnitude plus three others. I do not think that they are especially surprising or profound, but I claim paying attention to them allows us to be better appreciate the mechanistic and lawful operation of emotions. This appreciation is practical in that it lets us better recognize and remedy common emotional pathologies. [See Section 2: Problematic manipulations of the factors].

Section 1: The Factors

Magnitude [of the stimulus]
Attention
Closeness-of-the-counterfactual
Actionability

Magnitude [of the stimulus]

This factor is the most obvious and least profound of all of the factors. However, it is illuminating to note just how insufficient it is to drive an emotion in the absence of the other factors.

Taking it as an assumption that all emotions are about something in the world [1], the strength of the emotion generally scales with magnitude of the “goodness” or “badness” which caused it. One feels stronger grief when their house burns down than when they dropped their cookie in the dirt. Making thousands from Bitcoin feels better than finding a twenty dollar bill on the street.

Attention

Obvious and yet still underappreciated.

Any given person is aware of thousands of situations, circumstances, and facts that could evoke just about any feeling. Meditate on y...