trentmkelly/LessWrong-43k huggingface dataset:
The dark arts: Examples from the Harris-Adams conversation
by Stabilizer — 2017-07-20T23:42:03.562Z

Recently, James_Miller posted a conversation between Sam Harris and Scott Adams about Donald Trump. James_Miller titled it "a model rationalist disagreement". While I agree that the tone in which the conversation was conducted was helpful, I think Scott Adams is a top practitioner of the Dark Arts. Indeed, he often prides himself on his persuasion ability. To me, he is very far from a model for a rationalist, and he is the kind of figure we rationalists should know how to fight against.

Here are some techniques that Adams uses:

Changing the subject: (a) Harris says Trump is unethical and cites the example of Trump gate-crashing a charity event to falsely get credit for himself. Adams responds by saying that others are equally bad—that all politicians do morally dubious things. When Harris points out that Obama would never do such a thing, Adams says Trump is a very public figure and hence people have lots of dirt on him. (b) When Harris points out that almost all climate scientists agree that climate change is happening and that it is wrong for Trump to have called climate change a hoax, Adams changes the subject to how it is unclear what economic policies one ought to pursue if climate change is true.

Motte-and-bailey: When Harris points out that the Trump University scandal and Trump's response to it means Trump is unethical, Adams says that Trump was not responsible for the university because it was only a licensing deal. Then Harris points out that Trump is unethical because he shortchanged his contractors. Adams says that that’s what happens with big construction projects. Harris tries to argue that it’s the entirety of Trump’s behavior that makes it clear that he is unethical—i.e., Trump University, his non-payment to contractors, his charity gate-crashing, and so on. At this points Adams says we ought to stop expecting ethical behavior from our Presidents. This is a classic motte-and-bailey defense. Try to defend an indefensible position (the bailey) for a while, but then once it becomes untenable to defend it, then go to the motte (something much more defensible).

Euphemisation: (a) When Harris tells Adams that Trump lies constantly and has a dangerous disregard for the truth, Adams says, I agree that Trump doesn’t pass fact checks. Indeed, throughout the conversation Adams never refers to Trump

...
Be my neighbor
by KatjaGrace — 2017-07-20T20:16:57.000Z

(Tldr: If you want a bunch of neighbors like me in Berkeley, this is a good opportunity.)

My own ideal living situation would probably involve a lot of private units containing people who get on well together, around some kind of central place that they could go when they were in the market for seeing other people. And pragmatically, with a lot of shared amenities: someone mowing everyone’s lawn at once, someone cooking for whoever wants it, one amazing washing machines replacing five shoddy washing machines etc. I liked living in a dorm, though I would amend some things.

My impression is that other people also want this kind of thing often, but that very little of it exists for adults. Possibly for good reasons, but I’m not sure.

One relatively close institution is the group house. And while I had mixed experiences in these prior to moving to the Bay, I have loved the slightly modified version, the ‘rationalist group house’. This is like the group house, but with members selected from a social scene whose people are especially likely to be interesting and weird in compatible ways to me.

Two years ago I decided to try to make an extra good (for me) rationalist house, by inviting a list of people I especially wanted to live with to start a new one. This led (via some more complicated algorithmic and experimental shenanigans) to The Bailey, which has indeed been pretty great. It is the sort of place where there is always someone interesting to talk to about something interesting—from gossip to AI to this week’s ambitious world-bettering plans. And where people are busy, but will often stop and eat together, whether it be a bulk-stocked MealSquare, or some delicious and elaborate home-cooked thing. Where people will talk for an hour about how communication works before they lose their temper with you for messing it up. Where the dog is cute and the internal currency is confusing, and the whiteboards are many.

Another nearby institution—a step up in scale from the rationalist house—would be the rationalist neighborhood. And that is what we are starting to try now around The Bailey (on Ward St, Berkeley), to my great excitement.

SSC says more:

SSC is part of a wider movement of philosophy enthusiasts, transhumanists, effective altruists, etc which has somehow ended up with the simultaneously boring and arrogant moniker of “the rationalist

...
What Value Subagents?
by Gordon Seidoh Worley — 2017-07-20T19:19:08.957Z
Link: https://mapandterritory.org/what-value-subagents-868b3b3fc076

Among rationalist thinkers the subagent theory of psyche is popular: CFAR teaches a version of internal family systems (IFS), Max Harms writes AI fiction from the point of view of a subagent, Brienne describes herself as metaphorically made up of multiple people, Alicorn seems to have been the first rationalist to explicitly suggest the idea, the notion goes back to our intellectual forebearers, and nearly everyone seems to use a subagent interpretation of dual process theory at the least. Yet although I’ve previously explored multi-part theories of psyche, I don’t think of these parts as subagents. Instead, as I’ve hinted at, I think of the psyche primarily as a unified optimization process with inconsistent preferences that does not decompose into subagents. I didn’t always think this way, though, and have at times thought of myself and others as made up of different configurations of subagents, so I want to explore some of how I came to give up subagent theories and see what value they may still hold.

Stepping back to start, what do rationalists mean by subagents of the psyche? “Agent” has roughly the economic sense of a process that makes decisions, but we should understand “decision” to include any kind of behavior, including thoughts, so in the context of theories of psyche “agent” is a rough synonym for “person” or “mind”. A subagent, then, is an agent operating inside of another agent, forming a system of agents that externally present as a single agent. Explaining how subagents work in IFS should make this clearer.

IFS is a psychoanalytic model that, to my reading, originates from thinking of the Freudian id, ego, and superego like three homunculi competing to control the mind, but replaces the id, ego, and superego with groups of subagents serving exile, firefighter, and manager roles, respectively. These roles differer in both subtle and obvious ways from their inspirational forms, but the basic idea is that exiled subagents are those that would create undesired thoughts and behaviors, managers are subagents that work to keep undesired thoughts and behaviors from occurring, and firefighters are subagents that work to mitigate suffering when exiled subagents gain enough strength to influence a person’s behavior. The psyche can then be thought of l...
Change Is Bad
by Zvi — 2017-07-20T00:48:24.000Z

Epistemic Status: Public service reminder (I want to be able to link to this in the future)

Leads to: Choices are Bad, Choices Are Really Bad, Complexity Is Bad

Almost all changes are bad.

People forget that. They say they want change. They say things like:

At the end of the day, I want to see change come about, whatever it would take. Whatever it would take to see change come about, I would welcome. – Chumbawamba, Be With You

What they actually want one of those rare, carefully chosen good, friendly changes. They do exist within change space.

Change space, like mind space, is deep and wide. Friendly change space isn’t quite to change space what friendly mind space is to mind space, but before you apply any filters of common sense, it’s remarkably close.

The more optimized things currently are, the less likely any given change is to be good.

The more time people have had to optimize other things around the current state of the thing you are trying to change, the less likely any given change is to be good.

The more effort people have put into optimizing other things, based on the thing you are looking to change, the less likely any given change is to be good. You could break a lot of things.

When you break those things, you cause harm. Since people hate losses more than they love gains, even a net improvement can make a person or group feel worse off.

If you do break things, often they stay broken. It is usually a lot harder to build or repair something than it is to break that thing.

The faster and bigger you make changes, the more other things you are likely to break, and the more critically you will break them. At a minimum, even when your change is strictly for the better, those things then must change to adapt. In many cases, they are broken entirely, beyond repair, and this goes on to break other things.

Modern life is highly optimized. It’s far from perfect. Often it is optimizing for the wrong things. Nevertheless, it is highly, highly optimized. We certainly do not lack for hill climbers. In some ways it is too optimized!

If I currently have a highly optimized package of things, such as a house or apartment (which is implicitly a collection of things tied to my location), often I will have put a lot of work and sunk costs into the current package of goods I’ve assembled, so breaking up that package or adding new things to it is pret...
Sleeping Beauty Problem Can Be Explained by Perspective Disagreement (II)
by Xianda_GAO_duplicate0.5321505782395719 — 2017-07-19T23:00:40.027Z

This is the second part of my argument. It mainly involves a counter example to SIA and Thirdism.

Different part of the argument can be found here: I, II, III, IV.

The 81-Day Experiment(81D):

There is a circular corridor connected to 81 rooms with identical doors. At the beginning all rooms have blue walls. A random number R is generated between 1 and 81. Then a painter randomly selects R rooms and paint them red. Beauty would be put into a drug induced sleep lasting 81 day, spending one day in each room. An experimenter would wake her up if the room she currently sleeps in is red and let her sleep through the day if the room is blue. Her memory of each awakening would be wiped at the end of the day. Each time after beauty wakes up she is allowed to exit her room and open some other doors in the corridor to check the colour of those rooms. Now suppose one day after opening 8 random doors she sees 2 red rooms and 6 blue rooms. How should beauty estimate the total number of red rooms(R).

For halfers, waking up in a red room does not give beauty any more information except that R>0. Randomly opening 8 doors means she took a simple random sample of size 8 from a population of 80. In the sample 2 rooms (1/4) are red. Therefore the total number of red rooms(R) can be easily estimated as 1/4 of the 80 rooms plus her own room, 21 in total.

For thirders, beauty's own room is treated differently.As SIA states, finding herself awake is as if she chose a random room from the 81 rooms and find out it is red. Therefore her room and the other 8 rooms she checked are all in the same sample. This means she has a simple random sample of size 9 from a population of 81. 3 out of 9 rooms in the sample (1/3) are red. The total number of red rooms can be easily estimated as a third of the 81 rooms, 27 in total.

If a bayesian analysis is performed R=21 and R=27 would also be the case with highest credence according to halfers and thirders respectively. It is worth mentioning if an outside Selector randomly chooses 9 rooms and check them, and it just so happens those 9 are the same 9 rooms beauty saw (her own room plus the 8 randomly chosen rooms), the Selector would estimate R=27 and has the highest credence for R=27. Because he and the beauty has the exact same information about the rooms their answer would not change even if they are allowed to ...
Sleeping Beauty Problem Can Be Explained by Perspective Disagreement (I)
by Xianda_GAO_duplicate0.5321505782395719 — 2017-07-19T22:11:06.795Z

EDIT: It seems this is not getting much attention except for part II with the counter example to SIA. Let me try to intrigue you in to reading this. It provides a systematic argument for double halving in the sleeping beauty problem, if my reasoning is correct and perspective disagreement is indeed the key, we can rejects Doomsday Argument, and disagree with the Presumptuous Philosopher also reject the Simulation Argument.My argument is quite long but the concept is not hard to understand.

Different part of the argument can be found here: I, II, III, IV.

First thing I want to say is that I do not have a mathematics or philosophy degree. I come from an engineering background. So please forgive me when I inevitably messed up some concept. Another thing I want to mention is that English is not my first language. If you think there is any part that is poorly described please point them out. I would try my best to explain what I meant. That being said, I believe I found a good explanation for the SBP.

My main argument is that in case of the sleeping beauty problem, agents free to communicate thus having identical information can still rightfully have different credence to the same proposition. This disagreement is purely caused by the difference in their perspective. And due to this perspective disagreement, SIA and SSA are both wrong because they are answering the question from an outsider's perspective which provides different answer from beauty's. I concluded that the correct answer should be double-halving.

My argument involves three thought experiments. Here I am breaking it into several parts to facilitate easier discussion. The complete argument can also be found at www.sleepingbeautyproblem.com. However do note it is quite lengthy and not very well written due to my language skills.

First experiment:Duplicating Beauty (DB)

Beauty falls asleep as usual. The experimenter tosses a fair coin before she wakes up. If the coin landed on T then a perfect copy of beauty will be produced. The copy is precise enough that she cannot tell if herself is old or new. If the coin landed on H then no copy will be made . The beauty(ies) will then be randomly put into two identical rooms. At this point another person, let's call him the Selector, randomly chooses one of the two rooms and enters. Suppose he saw a beauty in the chosen room. What ...
Looking for ideas about Epistemology related topics
by Onemorenickname — 2017-07-19T18:56:05.007Z

Notes :

"Epistemy" refers to the second meaning of epistemology : "A particular theory of knowledge".

I'm more interested in ideas to further the thoughts exposed here than exposing them.

Good Experiments

The point of "Priors are useless" is that if you update after enough experiments, you tend to the truth distribution regardless of your initial prior distribution (assuming its codomain doesn't include 0 and 1, or at least that it doesn't assign 1 to a non-truth and 0 to a truth). However, "enough experiments" is magic :

The pure quantitative aspect : you might not have time to do these experiments in your lifetime.

Having independent experiments is not defined. Knowing which experiments are pairwise independent embeds higher-level knowledge that could easily be used to derive truths directly. If we try to prove a mathematical theorem, comparing the pairwise success probability correlations of different approaches would give much more insights and results than trying to prove it as usual.

We don't need pairwise independence. For instance, assuming we assume P=/=NP because we couldn't prove it, we assume so because we expect all used techniques not to be all correlated together. However, this expectation is ether wrong (Small list of fairly accepted conjectures that were later disproved), or stems from higher-order knowledge (knowledge about knowledge). Infinite regress.

Good Priors

However, conversely, having a good prior distribution is magic too. You can have a prior distribution affecting 1 to truths, and 0 to non-truths. So you might want the additional requirement that the prior distribution has to be computable. But there are two problems :

There aren't many known computable prior distribution. Occam's razor (in term of Kolmogorov complexity in a given language) is one. But fails miserably in most interesting situations. Think of poker, or a simplified version thereof : A+K+Q. If someone bets, the simplest explanation is that he has good cards. Most interesting situations where we want to apply bayesianism are from human interactions (we managed to do hard sciences before bayesianism, and we still have troubles with social sciences). As such, failing to take into account bluff is a big epistemic fault for a prior distribution.

Evaluating the efficiency of a given prior distribution will be done over the course of several experiments,

...
Book Reviews
by Torello — 2017-07-18T14:19:19.301Z

Hello,

I wanted to share some book reviews that I hope would be useful to readers of this site.

If you have any suggestions about relate reading, or suggestions about how to improve my reviews, please leave comments.

Homo Deus by Yuval Noah Harari, 2017

Sapiens: A Brief History of Humankind by Yuval Noah Harari (2015)

Superintelligence: Paths, Dangers, Strategies by Nick Bostrom (2014)

Surfaces and Essences – Analogy as the Fuel and Fire of Thinking by Douglas Hofstadter and Emmanuel Sanders (2013)

What Technology Wants by Kevin Kelly (2010)

A Skeptic’s Guide to the Mind by Robert A. Burton, 2013

Inside Jokes: Using Humor to Reverse-Engineer the Mind by Matthew Hurley, Daniel Dennett, and Reginald Adams, Jr., 2013
On the computational feasibility of forecasting using gamblers
by Vanessa Kosoy — 2017-07-18T14:00:00.000Z

This is an informal note about an idea which I consider important, but which I'm not planning to start working on immediately (first I want to develop the theory of delegation some more in the realizable setting). Putting it up here as a reminder, a point to bear in mind, and in case someone else wants to pick it up.

Forecasting using gamblers (a.k.a. logical induction generalized outside of logic, a.k.a. dominant forecasting)* can be regarded as a generalization of Bayesian inference, however it seem qualitatively less computationally tractable than Bayesian inference. Bayesian inference for a modest number of efficiently computable hypotheses (i.e. hypotheses that allow computing the conditional probability distribution of the next observation efficiently; this is a stronger requirement than efficient samplability) is efficiently computable. On the other hand, computing a dominant forecaster seems intractable even for a modest number of efficiently computable gamblers, since it amounts to finding a fixed point which is a problem that is often PPAD-equivalent.

However, it occurred to me that there is a fairly large class of gamblers for which the problem is tractable, in particular it probably includes gamblers for incomplete models. A gambler is a continuous function G:ΔOω→C(Oω) (we will ignore dependence on past forecasts since it is immaterial for our purpose). For now, pretend that ΔOω is a finite dimensional space (indeed, when O is finite, the simplest way to get a computable approximation is truncating the space to finite dimension via replacing Oω by On; nevertheless it would be nice to get the functional analysis right later). Then, C(Oω) is its cotangent space and therefore G is a covector field = section of the cotangent bundle = differential form of rank 1. It is thus natural to consider the special case when G is exact i.e. equal to the differential of some function g:ΔOω→R. For convenience, we will take a minus sign, i.e. G=−dg. Moreover, we can consider the further special case when g is convex. The resulting class of gamblers is closed under linear combinations, so dominating a countable set of "convex gamblers" reduces to dominating a single convex gambler.

For convex differentiable g, G=−dg and any μ∈ΔOω and x∈Oω, we have

G(μ,x)−Eμ[G(μ)]=Eμ[dg(μ)]−Eδx[dg(μ)]≤g(μ)−g(δx)

It follows that for any μ∗∈argming

suppμ∗⊆argmaxG(μ∗)

Thus, we reduced the

...
Open thread, Jul. 17 - Jul. 23, 2017
by MrMind — 2017-07-17T08:15:02.502Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting
Current thoughts on Paul Christano's research agenda
by jessicata — 2017-07-16T21:08:47.000Z

This post summarizes my thoughts on Paul Christiano's agenda in general and ALBA in particular.

(note: at the time of writing, I am not employed at MIRI)

(in general, opinions expressed here are strong and weakly held)

AI alignment research as strategy

Roughly, AI alignment is the problem of using a system of humans and computers to do a good thing. AI alignment research will tend to look at the higher levels of what the system of humans and computers is doing. Thus, it is strategy for doing good things with humans and computers.

"Doing good things with humans and computers" is a broad class. Computing machines have been around a long time, and systems of humans involving rules computed by humans have been around much longer. Looking at AI alignment as strategy will bring in intuitions from domains like history, law, economics, and political philosophy. I think these intuitions are useful for bringing AI strategy into near mode.

Paul Christiano's agenda as strategy

Paul Christiano's agenda "goes for the throat" in ways that the other agendas, such as the agent foundations agenda, do not. Thus, it yields an actual strategy, rather than a list of research questions about strategy. I will now analyze Paul Christiano's research agenda as strategy. Here I caricature Paul Christiano's strategic assumptions:

There are 2 phases to agentic activity: an influence-maximizing phase, and a value-fulfillment phase.

Any strategy can be decomposed into (a) the expansion strategy (the strategy for the influence-maximizing phase) and (b) the payload (the strategy for the value-fulfillment phase), which can vary independently in principle.

The power of an agent after the influence-maximizing phase is a function of its starting power and the effectiveness of its expansion strategy.

An outcome is good if the vast majority of power after the influence-maximizing phase is owned by agents whose payload is good.

Almost all powerful actors start with good values, i.e. they intend to select a good payload.

Selecting a good payload is not hard (e.g. specifying a social arrangement that would reflect and arrive at good moral conclusions)

Powerful actors can't coordinate with each other; they must expand independently.

Powerful actors with bad values exist.

Good powerful actors can anticipate and copy bad powerful actors' expansion strategies while retaining the same payload.

T

...