trentmkelly/LessWrong-43k huggingface dataset:
Smoking lesion as a counterexample to CDT
by Stuart_Armstrong — 2012-10-26T12:08:27.851Z

I stumbled upon this paper by Andy Egan and thought that its main result should be shared. We have the Newcomb problem as counterexample to CDT, but that can be dismissed as being speculative or science-fictiony. In this paper, Andy Egan constructs a smoking lesion counterexample to CDT, and makes the fascinating claim that one can construct counterexamples to CDT by starting from any counterexample to EDT and modifying it systematically.

The "smoking lesion" counterexample to EDT goes like this:

There is a rare gene (G) that both causes people to smoke (S) and causes cancer (C). Susan mildly prefers to smoke than not to - should she do so?

EDT implies that she should not smoke (since the likely outcome in a world where she doesn't smoke is better than the likely outcome in a world where she does). CDT correctly allows her to smoke: she shouldn't care about the information revealed by her preferences.

But we can modify this problem to become a counterexample to CDT, as follows:

There is a rare gene (G) that both causes people to smoke (S) and makes smokers vulnerable to cancer (C). Susan mildly prefers to smoke than not to - should she do so?

Here EDT correctly tells her not to smoke. CDT refuses to use her possible decision as evidence that she has the gene and tells her to smoke. But this makes her very likely to get cancer, as she is very likely to have the gene given that she smokes.

The idea behind this new example is that EDT runs into paradoxes whenever there is a common cause (G) of both some action (S) and some undesirable consequence (C). We then take that problem and modify it so that there is a common cause G of both some action (S) and of a causal relationship between that action and the undesirable consequence (S→C). This is then often a paradox of CDT.

It isn't perfect match - for instance if the gene G were common, then CDT would say not to smoke in the modified smoker's lesion. But it still seems that most EDT paradoxes can be adapted to become paradoxes of CDT.
The Problem With Rational Wiki
by  — 2012-10-26T11:31:51.542Z

Related to: RationalWiki's take on LW, David Gerard's Comments, Vladimir_M's comments, Public Drafts

I wanted to bring more attention to this argument because I've ran into related discussion several times in the comment section and because it demonstrates a failure mode that LessWrong may find itself vulnerable to.

Since it has been cited as a source especially on the reputation LessWrong may or may not have elsewhere I think readers should be aware Rational Wiki has a certain reputation here as well. I'm not talking about the object level disagreements such as cryonics, existential risk, many-worlds interpretation and artificial intelligence because we have some reasonable disagreement on those here as well. Even its cheeky tone while not helping its stated goals can be amusing. I'm somewhat less forgiving about their casual approach to epistemology and their vulnerability to cargo cult science, as long as it is peer reviewed cargo cult science.

While factually it is as about as accurate as Wikipedia, it is very selective about the facts that it is interested in. For example what would you expect from a site calling itself "Rational Wiki" to have on its page about charity. Do you expect information on how much good charity actually does? What kinds of charities do not do what they say on the label? How to avoid getting misled? The ethics of charity? The psychology, sociology or economics of charity?

I'm sorry to disappoint you but the article consists of some haphazardly arranged facts and stats on how much members of some religions give or are supposed to give to charity, a dig against Christianity and a non-sequitur unfavourable comparison of the US to Sweden. Contrast this with what you can find on the topic on sites like LessWrong or 80, 000 Hours. Basically the material presented is what a slightly left of centre atheist needs to win an internet debate. As is much of the rest of the site.

Indeed some entries have a clear ideological bias that is quite startling to behold on a "rational wiki" and it has been noted by some.

Now to avoid any misunderstandings there are good articles, a few LWers are contributors to the rational wiki and there is certainly nothing wrong with being a left of centre atheist! Nearly everyone on this site is an atheist, and people who identify as left wing politically form a large majority here. The tribal markers and its politica...
Prediction market sequence requested
by  — 2012-10-26T10:59:11.445Z

Related to: Eliezer's Sequences and Mainstream Academia, Intellectual insularity and productivity, I Stand by the Sequences, Why don't people like markets?

Looking at some of the more recent arguments against them showing up in discussions I've been quite disappointed, they seem betray a sort of lack of background knowledge or opinions built up from a bottom line of "markets are baaad therefore prediction markets are baaad". The casual arguments for them are lacking as well. I will say the same of other discussions on economic, since it is apparently suddenly too mind-killing or too political to talk about markets and similar things at all. We didn't use to have tribal alerts flying up in our brains discussing such matters.

The Overcoming Bias community started with an assumption of certain kinds of background knowledge, this included economics and things like game theory. In the early days of LessWrong/Overcoming Bias Eliezer did a whole sequnece on filling in people on Quantum mechanics which despite his claims to the contrary doesn't seem that vital (if still important).
We now have a different demographic that we used to. Not only that, we now have young people basically using the sequences as their primary source for education on matters of human rationality, quite different from the autodidacts exploring the literature on their own terms who where common in previous years. We've recognized this to a certain extent. We wrote a series of introductory sequences and articles to fill in such background knowledge explicitly such as Yvain's recent one on Game Theory. Also part of the reason we now have a norm of more citations that EY originally did is to give study and research aids to people. Indeed I think adding comments to old articles featuring more citations or editing those in would be wise so as to avoid misconceptions.

I think we need several sequences on economics, and a good one to start would be one systematically investigating prediction markets. To a certain extent just reading Robin Hanson's relevant posts on this topic would do much the same, but unfortunately we don't have an organized series of sequences by him (beyond the tags he uses on his articles). I still hope Karmakaiser or someone else will one day undertake a project of writing up summary articles that organize links to RH's posts into sequences so new members will...
[SEQ RERUN] The Nature of Logic
by MinibearRex — 2012-10-26T05:50:01.799Z

Today's post, The Nature of Logic was originally published on 15 November 2008. A summary (taken from the LW wiki):

What logic actually does is preserve truth in a model. It says that if all of the premises are true, then this conclusion is indeed true. But that's not all that minds do. There's an awful lot else that you need, before you start actually getting anything like intelligence.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Selling Nonapples, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
If we live in a simulation, what does that imply?
by JoshuaFox — 2012-10-25T21:27:44.749Z

If we live in a simulation, what does that imply about the world of our simulators and our relationship to them? [1]

Here are some proposals, often mutually contradictory, none stated with anything near certainty.

1. The simulators are much like us, or at least are our post-human descendants.

Drawing on some of the key points in Bostrom's Simulation Argument:

Today, we often simulate our human ancestors' lives, e.g., Civilization. Our descendants will likely want to simulate their own ancestors, namely us, and they may have much-improved simulation technology which support sentience. So, our simulators are likely to be our (post-)human descendants.

2. Our world is smaller than we think.

Robin Hanson has said that computational power will be dedicated to running only a small part of the simulation at low resolution, including the part which we are in. Other parts of the simulation will be run at a lower resolution. Everything outside our vicinity, e.g., outside our solar system, will be calculated planetarium-style, and not from the level of particle physics.

(I wonder what it would be like if we are in the low-res part of the simulation.)

3. The world is likely to end soon.

There is no a priori reason for an base-level (unsimulated) universe to flicker out of existence. In fact, it would merely add complexity to the laws of physics for time to suddenly end with no particular cause.

But a simulator may decide that they have learned all they wanted to from their simulation; or that acausal trade has been completed; or that they are bored with the game; and that continuing the simulation is not worth the computational cost.

The previous point was that the world is spatially smaller than we think. This point is that the world is temporally smaller than we hope.

4. We are living in a particularly interesting part of our universe.

The small part of the universe which the simulators would choose to focus on is the part which is interesting or entertaining to them. Today's video games are mostly about war, fighting, or various other challenges to be overcome. Some, like the Sims, are about everyday life, but even in those, the players want to see something interesting.

So, you are likely to be playing a pivotal role in our (simulated) world. Moreover, if you want to continue to be simulated, do what you can to make a differ...
A follow-up probability question: Data samples with different priors
by PhilGoetz — 2012-10-25T20:07:01.498Z

(Rewritten entirely after seeing pragmatist's answer.)

In this post, helpful people including DanielLC gave me the multiply-odds-ratios method for combining probability estimates given by independent experts with a constant prior, with many comments about what to do when they aren't independent. (DanielLC's method turns out to be identical to summing up the bits of information for and against the hypothesis, which is what I'd expected to be correct.)

I ran into problems applying this, because sometimes the prior isn't constant across samples. Right now I'm combining different sources of information to choose the correct transcription start site for a gene. These bacterial genes typically have from 1 to 20 possible start sites. The prior is 1 / (number of possible sites).

Suppose I want to figure out the correct likelihood multiplier for the information that a start site overlaps the stop of the previous gene, which I will call property Q. Assume this multiplier, lm, is constant, regardless of the prior. This is reasonable, since we always factor out the prior. Some function of the prior gives me the posterior probability that a site s is the correct start (Q(s) is true), given that O(s). That's P(Q(s) | prior=1/numStarts, O(s)).

Suppose I look just at those cases where numStarts = 4, I find that P(Q(s) | numStarts=4, O(s)) = .9.

9:1 / 1:3 = 27:1

Or I can look at the cases where numStarts=2, and find that in these cases, P(Q(s) | numStarts=2, O(s)) = .95:

19:1 / 1:1 = 19:1

I want to take one pass through the data and come up with a single likelihood multiplier, rather than binning all the data into different groups by numStarts. I think I can just compute it as

(sum of numerator : sum of denominator) over all cases s_i where O(s_i) is true, where

numerator = (numStarts_i-1) * Q(s_i)

denominator = (1-Q(s_i))

Is this correct?
Ambitious utilitarians must concern themselves with death
by Mitchell_Porter — 2012-10-25T10:41:41.269Z

And I don't mean that they must concern themselves with death in the sense of ending death, or removing its sting through mental backups, or delaying it to the later ages of the universe; or in the sense of working to decrease the probability of extinction risks and other forms of megadeath; or even in the sense of saving as many lives as possible, as efficiently as possible. All of that is legitimate and interesting. But I mean something far more down to earth.

First, let me specify more precisely who I am talking about. I mean people who are trying to maximize the general welfare; who are trying to achieve the greatest good for the greatest number; who are trying to do the best thing possible with their lives. When someone like that makes decisions, they are implicitly choosing among possible futures in a very radical way. They may be making judgments about whether a future with millions or billions of extra lives is better than some alternative. Whether anyone is ever in a position to make that much of a difference is another matter; but we can think of it like voting. You are at least making a statement about which sort of future you think you prefer, and then you do what you can, and that either makes a difference or it doesn't.

It seems to me that the discussions about the value of life among utilitarians are rather superficial. The typical notion is that we should maximize net pleasure and minimize net pain. Already that poses the question of whether a life of dull persistent happiness is better or worse than a life of extreme highs and lows. A more sophisticated notion is that we should just aspire to maximize "utility", where perhaps we don't even know what utility is yet. Certainly the CEV philosophy is that we don't yet know what utility really is for human beings. It would be interesting to see people who took that agnosticism to heart, people whose life-strategy amounted to (1) discovering true utility as soon as possible (2) living according to interim heuristics whose uncertainty is recognized, but which are adopted out of the necessity of having some sort of personal decision procedure.

So what I'm going to say pertains to (2). You may, if you wish, hold to the idea that the nature of true utility, like true friendliness, won't be known until the true workings of the human mind are known. What follows is something you should think on in order to...
Cake, or death!
by Stuart_Armstrong — 2012-10-25T10:33:26.545Z

Here we'll look at the famous cake or death problem teasered in the Value loading/learning post.

Imagine you have an agent that is uncertain about its values and designed to "learn" proper values. A formula for this process is that the agent must pick an action a equal to:

argmaxa∈A Σw∈W p(w|e,a) Σu∈U u(w)p(C(u)|w)

Let's decompose this a little, shall we? A is the set of actions, so argmax of a in A simply means that we are looking for an action a that maximises the rest of the expression. W is the set of all possible worlds, and e is the evidence that the agent has seen before. Hence p(w|e,a) is the probability of existing in a particular world, given that the agent has seen evidence e and will do action a. This is summed over each possible world in W.

And what value do we sum over in each world? Σu∈U u(w)p(C(u)|w). Here U is the set of (normalised) utility functions the agent is considering. In value loading, we don't program the agent with the correct utility function from the beginning; instead we imbue it with some sort of learning algorithm (generally with feedback) so that it can deduce for itself the correct utility function. The expression p(C(u)|w) expresses the probability that the utility u is correct in the world w. For instance, it might cover statements "it's 99% certain that 'murder is bad' is the correct morality, given that I live in a world where every programmer I ask tells me that murder is bad".

The C term is the correctness of the utility function, given whatever system of value learning we're using (note that some moral realists would insist that we don't need a C, that p(u|w) makes sense directly, that we can deduce ought from is). All the subtlety of the value learning is encoded in the various p(C(u)|w): this determines how the agent learns moral values.

So the whole formula can be described as:

For each possible world and each possible utility function, figure out the utility of that world. Weigh that by the probability that that utility is correct is that world, and by the probability of that world. Then choose the action that maximises the weighted sum of this across all utility functions and worlds.

Naive cake or death

In the initial formulation of value loading, p(C(u)|w) (probability of the correctness of u ...
Final cause is epistemologically primary, but efficient cause is metaphysically primary
by hankx7787 — 2012-10-25T06:02:07.611Z

(or, while final cause can be best for your map, efficient cause is the primary out in the territory)

Describing a phenomenon in terms of final cause is often the most useful and effective way to explain the given phenomenon for one's purposes. For example if you want to know why a plane flies or why a computer program operates the way it does, it's because it was designed that way. A squirrel climbs trees because it wants to eat nuts, it wants to eat nuts because it wants to live, it wants to live and reproduce because evolution designed it that way. Evolution designs organisms a certain way because it wants to maximize genetic fitness. A person acts a certain way because they desire the expected outcome.

It's virtually never a good answer to explain a plane's behavior in terms of the atomic and subatomic interactions which ultimately account for all the efficient causes behind the plane's behavior (except possibly in extremely advanced military fighter or space shuttle research laboratories or something).

However, in every case of final cause we observe, science at some point over the last two and a half millennia has found corresponding efficient causes. And, more importantly than finding that these efficient causes correspond with final causes, science has found that the efficient causes are *primary*. Without legs, the squirrel won't climb a tree, no matter how much it wants the nuts. If you take away the necessary brain function, the free will disappears. Without reproducing species and the rest of evolutionary mechanics discovered by science, evolution won't go on evolving things.

But, I would not go on to say that final cause, freewill, experience, and so on are illusory and "all that exists is efficient cause". When someone describes behavior in terms of final causes, or describes experience or free will, in the terms and meanings they are using, all of those things certainly do exist. You could no more deny final cause than to deny efficient cause - because ultimately, the final causes we observe and talk about, we have found they DO have corresponding efficient causes.

It's just important to remember that while the final cause is often epistemologically primary, so to speak, the efficient cause is metaphysically primary.

(this is just another way of trying to help dissolve the general classes of reductionism, freewill/determinism, and qualia i...
[SEQ RERUN] Selling Nonapples
by MinibearRex — 2012-10-25T04:58:40.129Z

Today's post, Selling Nonapples was originally published on 13 November 2008. A summary (taken from the LW wiki):

In most cases, if you say that something isn't working, then you have to specify a new thing that you think could work. You can't just say that you have to not do what you have been doing. If you observe that selling apples isn't working out for you financially, you can't just decide to sell nonapples.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Weighted Majority Algorithm, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Melbourne, practical rationality
by matt — 2012-10-24T23:11:21.338Z

Discussion article for the meetup : Melbourne, practical rationality

WHEN:
02 November 2012 07:00:00PM (+1100)

WHERE:
55 walsh st west melbourne 3003 australia

Practical rationality. This meetup repeats on the 1st Friday of each month and is distinct from our social meetup on the 3rd Friday of each month.

Discussion: http://groups.google.com/group/melbourne-less-wrong

All welcome from 6:30pm. Call the phone number on the door and I'll let you in.

Discussion article for the meetup : Melbourne, practical rationality
[Link] Offense 101
by Alejandro1 — 2012-10-24T21:28:47.231Z

From Julian Sanchez, a brilliant idea unlikely to be implemented:

American politics sometimes seems like a contest to see which group of partisans can take greater umbrage at the most recent outrageous remark from a member of the opposing tribe. As a mild countermeasure, I offer a modest proposal for American universities. All freshmen should be required to take a course called “Offense 101,” where the readings will consist of arguments from across the political and philosophical spectrum that some substantial proportion of the student body is likely to find offensive. Selections from The Bell Curve. Essays from one of the New Atheists and one of their opponents, and from hardcore pro-lifers and pro-choicers. Ward Churchill’s “little Eichmanns” monograph. Defenses of eugenics, torture, violent revolution, authoritarianism, aggressive censorship, and absolute free speech. Positive reviews of the Star Wars prequels. Assemble your own curriculum—there’s no shortage of material.

For each reading, students will have to make a good faith, unironic effort to reconstruct the offensive argument in its most persuasive form, marshaling additional supporting evidence and amending weak arguments to better support the author’s conclusion. Points deducted if an observer can tell the student doesn’t really agree with the position they’re defending.

Only after this phase is complete will students be allowed to begin rebutting the arguments. Anyone who thinks it’s relevant to point out that the argument is offensive (or bigoted, sexist, unpatriotic, fascistic, communistic, whatever) will receive a patronizing look from the professor that says: “Yes, obviously, did you not read the course title? Let’s move on.” Insofar as these labels are shorthand for an argument that certain categories of views are wrong and can be rejected as a class, the actual argument will have to be presented.