trentmkelly/LessWrong-43k huggingface dataset:
Mixed-Strategy Ratifiability Implies CDT=EDT
by abramdemski — 2017-10-31T05:56:05.395Z

(Cross-posted from IAFF.)

I provide conditions under which CDT=EDT in Bayes-net causal models.

Previously, I discussed conditions under which LICDT=LIEDT. That case was fairly difficult to analyse, although it looks fairly difficult to get LICDT and LIEDT to differ. It’s much easier to analyze the case of CDT and EDT ignoring logical uncertainty.

As I argued in that post, it seems to me that a lot of informal reasoning about the differences between CDT and EDT doesn’t actually give the same problem representation to both decision theories. One can easily imagine handing a causal model to CDT and a joint probability distribution to EDT, without checking that the probability distribution could possibly be consistent with the causal model. Representing problems in Bayes nets seems like a good choice for comparing the behavior of CDT and EDT. CDT takes the network to encode causal information, while EDT ignores that and just uses the probability distribution encoded by the network.

It’s easy to see that CDT=EDT if all the causal parents of an agent’s decision are observed. CDT makes decisions by first cutting the links to parents, and then conditioning on alternative actions. EDT conditions on the alternatives without cutting links. So, EDT differs from CDT insofar as actions provide evidence about causal parents. If all parents are known, then it’s not possible for CDT and EDT to differ.

So, any argument for CDT over EDT or vice versa must rely on the possibility of unobserved parents.

The most obvious parents to any decision node are the observations themselves. These are, of course, observed. But, it’s possible that there are other significant causal parents which can’t be observed so easily. For example, to recover the usual results in the classical thought experiments, it’s common to add a node representing “the agent’s abstract algorithm” which is a parent to the agent and any simulations of the agent. This abstract algorithm node captures the correlation which allows EDT to cooperate in the prisoner’s dilemma and one-box in Newcomb, for example.

Here, I argue that sufficient introspection still implies that CDT=EDT. Essentially, the agent may not have direct access to all its causal parents, but if it has enough self-knowledge (unlike the setup in Smoking Lesion Steelman), the same screening-off phenomenon occurs. This is somewhat like saying that the output of ...
Suggestion: Implement a Change My View Feature
by Chris_Leong — 2017-10-31T00:46:58.753Z

I know that the devs are pretty busy at the moment, and won't be able to do anything like this for a long time, but if I don't post this now, I'll probably forget it.

Change My View: is an extremely popular sub on Reddit and the responses tend to be pretty good. However, there are some questions that rationalists might have that can't be asked there due to lack of knowledge. Just off the top of my head, these might include:

CMV: I believe that we should two box in Newcomb's problem
CMV: I believe that hypothetical problems tend to be too abstract to be worth discussing
CMV: I believe that philosophy does not have much value for discovering the truth

Just as in Change My View, you would be expected to elaborate on your view.

The best way to implement this would probably be to have a special cmv tag. These posts would then have a special option to allow the OP to award a delta when their view was changed (we could also copy this suggestion of big and little deltas for big and little changes respectively). I think that deltas would be good as they provide incentive for people to say something that might change a persons view and they also encourage a person to acknowledge when their view has changed.

We could consider importing some of the rules or norms of change my view for these threads, ie:

Top level responses should challenge at least one aspect of the view or ask for a clarification
You must be open to changing the view if the evidence points that direction
You must personally hold the view and not be playing devil's advocate or neutral
The person who wants their view changed should be active in the thread
No soapboxing

I'll admit that someone could create a Change My View thread already, but I believe that implementing this feature would make people more aware of this and nudge them towards actually doing putting their beliefs on the line. It would also allow people to filter these threads if they felt that there were too many of them.

So, as I said at the start, this is intended as a long term suggestion only, but it seems like it would be worth implementing eventually.
Open thread, October 30 - November 5, 2017
by Elo — 2017-10-30T23:37:03.960Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top-level comments on this article" and "Make this post available under..." before submitting.
Leave beliefs that don't constrain experience alone
by zlrth — 2017-10-30T04:03:34.677Z

Epistemic status: This is the first time I've expressed these thoughts. I've thought for a long time that people do their jobs well, and are numbskulls in every other area of life. Here I say that it's OK to be a numbskull.

I read Raising the Sanity Waterline some time ago. I thought, "These are great points! I've needed them!" I made arguments that used those points a few times.

When I listened to the Bayesian Conspiracy&#x27;s episode on it I thought, "How did BC get this article so wrong? RtSW isn't about making oblique attacks on religion by teaching people things like Occam's Razor!"

It is about that!

I think I took these sentences and made a different conclusion:

Behind every exciting, dramatic failure, there is a more important story about a larger and less dramatic failure that made the first failure possible.

The general sanity waterline is currently really ridiculously low. Even in the highest halls of science.

I wrongly took this article to mean (actually, this is what I believe):

You can be a Nobel prize winning scientist and believe in God. The sanity waterline is so low that having many, many irrational beliefs won't impact your ability to contribute to the world and be happy. Why is that? One hypothesis is if you have just a few beliefs that pay rent you're doing better than most. Just as the difference between rationalist and rationalist-adjacent people is small, the difference between an imagined highly-effective scientist and a highly-effective scientist who believes in God is small. In general, the effect of adding another belief that pays rent is logarithmic.

If this bothers you, your options are limited. It's not effective to relieve people of their beliefs in God, because the low sanity waterline predicts that they'll have many other low-water beliefs. And beliefs don't have to pay rent to stick around.

But you can embark on a more ambitious project: raise the sanity waterline. Do this by making a more rational world. The world which is so effective-and-rational that a belief in god impedes your progress is a rational world, indeed.

I don't think that Nobel prize-winning scientists don't understand "epistomology 101." That scientist needs to have epistomology 101 pay rent! Not that you asked, but the broken coupling of professional achievement...
Frequently Asked Questions for Central Banks Undershooting Their Inflation Target
by Eliezer Yudkowsky — 2017-10-29T23:36:22.256Z

If you are a central bank undershooting your inflation target, please read this first before posting a question about how to create more inflation.

Q. Help! I keep undershooting my 2% inflation target!

A. It sounds like you need to create more money.

Q. I tried that, and it didn't work!

A. How much money did you create?

Q. Five dollars.

A. Okay, now we know that five dollars wasn't enough money. You need to create more.

Q. More than five dollars?

A. Right.

Q. I've heard that it's very bad when governments create lots of money just so they can buy things. Prices skyrocket, and soon nobody wants to hold on to the money anymore! That's why we have stern and independent central banks, to prevent too much money from being created.

A. Yes, that's a big problem, all right! That problem is the opposite of the problem you actually have. It's definitely true that if prices start going up faster than you want, you should stop creating money and maybe destroy some of the money you already made. But that is not the problem you have right now. Your current problem is that there's too little money flowing through the economy, meaning, not enough money to drive all the buying of real goods and labor that could be exchanged if your economy had more money.

You know how hyperinflation happens because there isn't enough real stuff and so the tons of new money are just competing with other money to buy a limited amount of real stuff? One way of looking at your 2% inflation target is that you're supposed to create just enough money flow that it's a little above what's required to animate all the trades your economy can make. If there's a little more money flow than the minimum required to animate all potential trades, that money is competing just a little with other money to buy stuff. Producing, say, around 2% inflation. That's why your government gave you an inflation target that was low, but not super-low and not zero. Right now your economy doesn't have that much inflation, because there isn't enough money flowing and your economy is failing to make all the trades it could make. That's why 'undershooting your inflation target' is associated with a country that feels sad and listless, with all the factories and shops still there but people not having enough money to buy things from them, becaus...
In defence of epistemic modesty
by Thrasymachus — 2017-10-29T20:00:31.366Z

This piece defends a strong form of epistemic modesty: that, in most cases, one should pay scarcely any attention to what you find the most persuasive view on an issue, hewing instead to an idealized consensus of experts. I start by better pinning down exactly what is meant by ‘epistemic modesty’, go on to offer a variety of reasons that motivate it, and reply to some common objections. Along the way, I show common traps people being inappropriately modest fall into. I conclude that modesty is a superior epistemic strategy, and ought to be more widely used - particularly in the EA/rationalist communities.

[gdoc]

Provocation

I argue for this:

In virtually all cases, the credence you hold for any given belief should be dominated by the balance of credences held by your epistemic peers and superiors. One’s own convictions should weigh no more heavily in the balance than that of one other epistemic peer.

Introductions and clarifications

A favourable motivating case

Suppose your mother thinks they can make some easy money day trading blue-chip stocks, and plans to kick off tomorrow shorting Google on the stock market, as they’re sure it’s headed for a crash. You might want to dissuade her in a variety of ways.

You might appeal to an outside view:

Mum, when you make this short you’re going to be betting against some hedge fund, quant, or whatever else. They have loads of advantages: relevant background, better information, lots of data and computers, and so on. Do you really think you’re odds on to win this bet?

Or appeal to some reference class:

Mum, I’m pretty sure the research says that people trying to day-trade stocks tend not to make much money at all. Although you might hear some big successes on the internet, you don’t hear about everyone else who went bust. So why should you think you are likely to be one of these remarkable successes?

Or just cite disagreement:

Look Mum: Dad, sister, the grandparents and I all think this is a really bad idea. Please don’t do it!

Instead of directly challenging the object level claim (i.e. “Google isn’t overvalued, because X”). These considerations attempt to situate the cogniser within some population, and from characteristics of this population infer the likelihood of this cogniser getting things right.

Call the practice of using these techniques considerations epistemic modesty. We can distinguish two components:

‘In theory’ modest

...
Leaders of Men
by Zvi — 2017-10-29T18:10:00.369Z

Related to (Eliezer Yudkowsky): Inadequacy and Modesty

Epistemic Status: Confident. No sports knowledge required.

In 2005, Willie Randolph became manager of the New York Mets.

In his first five games as manager, all of which he lost, Willie made more decisions wrong than I thought possible. If he needed to change pitchers, he waited. Other times he changed pitchers for no reason. Starting lineups made zero sense. Position players bunted. And so on. He cost us at least one of those games. My friend Seth and called for Willie’s head.

He would go on to an excellent 97 win season in 2006, come in second in manager-of-the-year voting, get a contract extension, and only get fired after wearing out our starting pitchers so much that we experienced one of the most epic late season collapses in baseball history in 2007, followed by a horrible 2008.

Willie’s in-game decisions did not improve. If anything, they got worse.

Despite this, we came to understand why Willie got and kept his job.

Willie Randolph was a leader of men.

Players liked Willie. They wanted to play for him, work hard for him, be the best they could be. They put the team first. He created a positive clubhouse atmosphere. He inspired good performances, spotted ways players could improve.

That is what counts.

Do bad in-game decisions cost games? Absolutely. But not that many games. Maybe they lose you 4 a year out of 162.

If the lineup makes your players unhappy, that costs a lot more. If your pitchers lose motivation or have their rhythms disrupted, that matters more than getting high leverage for your best reliever. Maybe bunting inspires team unity. The reason we hate bunting so much isn’t because it’s a huge mistake. It’s an obvious mistake. A pure mistake. An arithmetic error.

Plenty of people could get those technical decisions right. I could do it.

What most of us can’t do is lead men. Leading men is what counts. That’s the real job, but it comes with these other tasks.

Sometimes these other tasks land in good hands, at other times they land in terrible hands. Those who do the little things right do succeed more, but you can still win championships without them. If you can lead men.

Other sports follow the pattern. Why do football teams employ Andy Reid, who could not manage a two-minute drill if his life depended on it? Why do highly successful basketball players refuse to cooperate with their teammates or pra...
Brief comment on featured
by Ben Pace — 2017-10-29T17:12:51.320Z

Someone asked me a meta question in the object level of this site (i.e. the frontpage), so I'm copying my reply here instead of continuing discussion there.

Me (on Chapter 1 of Inadequate Equilibria):

I've promoted this to Featured because it's a great piece of writing, and also because this book seems to be about some of the fundamental epistemological disagreements in our broad communities. I really hope this is the place we can communicate clearly and successfully together on these topics.

Lucretious:

What are the grounds for relying on individual judgment to promote posts to Featured? Isn't it more reasonable to rely on community opinion, as expressed in the epistocratic karma system?

(I'm aware that this is a tangent, but the point seemed germane given that the reasoning behind this decision may itself reflect the "fundamental epistemological disagreements" which you refer to.)

Me:

Yup, it's a tangent and best suited for Meta, but seems like a fine question, and I'll respond to it here for now [edit: well, now I've moved it to Meta].

My current plan is to have karma do a lot of work in sorting, but for the moderators (Sunshine Regiment) to also have a lot of say on what goes to Featured. I often think of karma as the System 1 of the site, and the mods as the System 2 - while S1 makes most decisions and sets up basic incentive gradients, S2 tries to give useful input on important decisions, make plans and route around obstacles. I expect the S1 to be imperfect - as an example, I imagine some posts will be undervalued that dont' contain many new ideas but summarise a great deal of previous discussion well, so that people can read a single post instead of 5 posts and 3 long comment threads. Recently I Featured one or two posts in part for being no longer than they needed to be, which I want authors to know is great, but I also don't know that the karma system is incentivising as much.

A related key point is that only the S2 can be easily queried for its reasons - if you ask the karma system why it promoted something, it can't tell you, and so authors have to speculate on why their posts got into Featured. I'm very pro transparency.

In general while I think the karma system is very useful and I take guidance from it a lot, I want the key decisions on the site not to be made by an algorithm whose pa...
Immodesty, Deceit and Motivated Reasoning
by  — 2017-10-29T10:34:16.320Z

It seems that there is another reason for immodesty apart from Inadequacy. If you are walking with someone who looks at the $20 bill from a far and obviously dismisses it as fake, while you are considering going to pick it up, then you might want to think why she is doing this. It may be she is trying to convince you not to go and pick it up, so she has a shot at it later. So Deceit.

If you see a scientist, hired by the people that run the subway, walking close past the $20. When asked they may say, " I thought it is likely to be fake due to an blog post I read by Eliezer". But sub-conciously they might be factoring in the disruption to the flow of people it would cause if they bent down to investigate the $20 dollars, so the "Eliezer Post" was not their true reason. So Motivated Reasoning.

These are small scale examples of reasons to distrust the apparent and reported beliefs of others. They can also apply to society at large. Advertising and Lobbying can shift public opinion away from the truth. An example of this is sugar lobbying , So you shouldn't necessarily trust majoritarianism if people are being swayed by experts and those experts have non-truth seeking incentive structures.

Taken to the extreme this can cause paranoia. It is worth trying to look at the experts and seeing how big the incentives are for believing what they do. To take a non-real world example, if all the experts studying sugar are employed by sugar manufacturers and would lose their jobs if sugar became significantly less popular, then they have strong incentives to say sugar is not dangerous (whether through deceit or motivated reasoning). If all the experts on sugar are generic food health scientists who had tenure and could easily switch to studying some other food stuff if sugar went down in popularity, then they don't have huge incentives to say sugar is better than it is (apart from things like protecting the epistemology).

This doesn't mean you are any more right about the things you believe. You also probably have non-truth seeking incentive structures in your life. If epistemic rationality is important to you, it is probably best to try and minimise those structures you get enmeshed in.
Inadequacy and Modesty
by Eliezer Yudkowsky — 2017-10-28T21:51:01.339Z

The following is the beginning of Inadequate Equilibria, a new sequence/book on a generalization of the notion of efficient markets, and on this notion's implications for practical decision-making and epistemic rationality.

This is a book about two incompatible views on the age-old question: “When should I think that I may be able to do something unusually well?”

These two viewpoints tend to give wildly different, nearly cognitively nonoverlapping analyses of questions like:

My doctor says I need to eat less and exercise, but a lot of educated-sounding economics bloggers are talking about this thing called the “Shangri-La Diet.” They’re saying that in order to lose weight, all you need to do is consume large quantities of flavorless, high-calorie foods at particular times of day; and they claim some amazing results with this diet. Could they really know better than my doctor? Would I be able to tell if they did?

My day job is in artificial intelligence and decision theory. And I recall the dark days before 2015, when there was plenty of effort and attention going into advancing the state of the art in AI capabilities, but almost none going into AI alignment: better understanding AI designs and goals that can safely scale with capabilities. Though interest in the alignment problem has since increased quite a bit, it still makes sense to ask whether at the time I should have inferred from the lack of academic activity that there was no productive work to be done here; since if there were reachable fruits, wouldn’t academics be taking them?

Should I try my hand at becoming an entrepreneur? Whether or not it should be difficult to spot promising ideas in a scientific field, it certainly can’t be easy to think up a profitable idea for a new startup. Will I be able to find any good ideas that aren’t already taken?

The effective altruism community is a network of philanthropists and researchers that try to find the very best ways to benefit others per dollar, in full generality. Where should effective altruism organizations like GiveWell expect to find low-hanging fruit—neglected interventions ripe with potential? Where should they look to find things that our civilization isn’t already doing about as well as can be done?

When I think about problems like these, I use what feels ...