trentmkelly/LessWrong-43k huggingface dataset:
Donation tradeoffs in conscientious objection
by  — 2012-12-27T17:23:50.704Z

Suppose that you believe larger scale wars than current US military campaigns are looming in the next decade or two (this may be highly improbable, but let's condition on it for the moment). If you thought further that a military draft or other forms of conscription might be used, and you wanted to avoid military service if that situation arose, what steps should you take now to give yourself a high likelihood of being declared a conscientious objector?
I don't have numbers to back any of this up, but I am in the process of compiling them. My general thought is to break down the problem like so: Pr(serious injury or death | conscription) * Pr(conscription | my conscientious objector behavior & geopolitical conditions ripe for war) * Pr(geopolitical conditions ripe for war), assuming some conscientious objector behavior (or mixture distribution over several behaviors).

If I feel that Pr(serious injury or death | conscription) and Pr(geopolitical conditions ripe for war) are sufficiently high, then I might be motivated to pay some costs in order to drive Pr(conscription | my conscientious objector behavior) very low.

There's a funny bit in the American version of the show The Office where the manager, Michael, is concerned about his large credit card debt. The accountant, Oscar, mentions that declaring bankruptcy is an option, and so Michael walks out into the main office area and yells, "I DECLARE BANKRUPTCY!"
In a similar vein, I don't think that draft boards will accept the "excuse" that a given person has "merely" frequently expressed pacifist views. So if someone wants to robustly signal that she or he is a conscientious objector, what to do? In my ~30 minutes of searching, I've found a few organizations that, on first glance, look worthy of further investigation and perhaps regular donations.

Here are the few I've focused on most:

Center on Conscience and War

Coffee Strong

War-Resister's International

The problems I'm thinking about along these lines include:

Whether or not the donation cost is worth it. There's no Giving What We Can type measure for this as far as I can tell, and even though I know from family experience that veteran mental illness can be very bad, I'm not convinced that donations to the above organizations provide a lot of QALY bang for the buck.

Another component of bang for the buck is how much the donation will credibly signal tha

...
[SEQ RERUN] The Mechanics of Disagreement
by MinibearRex — 2012-12-27T05:46:16.365Z

Today's post, The Mechanics of Disagreement was originally published on 10 December 2008. A summary (taken from the LW wiki):

Reasons why aspiring rationalists might still disagree after trading arguments.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Two Visions of Heritage, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Intelligence explosion in organizations, or why I'm not worried about the singularity
by sbenthall — 2012-12-27T04:32:32.918Z

If I understand the Singularitarian argument espoused by many members of this community (eg. Muehlhauser and Salamon), it goes something like this:

Machine intelligence is getting smarter.

Once an intelligence becomes sufficiently supra-human, its instrumental rationality will drive it towards cognitive self-enhancement (Bostrom), so making it a super-powerful, resource hungry superintelligence.

If a superintelligence isn't sufficiently human-like or 'friendly', that could be disastrous for humanity.

Machine intelligence is unlikely to be human-like or friendly unless we take precautions.

I am not particularly worried about the scenario envisioned in this argument. I think that my lack of concern is rational, so I'd like to try to convince you of it as well.*

It's not that I think the logic of this argument is incorrect so much as I think there is another related problem that we should be worrying about more. I think the world is already full of probably unfriendly supra-human intelligences that are scrambling for computational resources in a way that threatens humanity.

I'm in danger of getting into politics. Since I understand that political arguments are not welcome here, I will refer to these potentially unfriendly human intelligences broadly as organizations.

Smart organizations

By "organization" I mean something commonplace, with a twist. It's commonplace because I'm talking about a bunch of people coordinated somehow. The twist is that I want to include the information technology infrastructure used by that bunch of people within the extension of "organization".

Do organizations have intelligence? I think so. Here's some of the reasons why:

We can model human organizations as having preference functions. (Economists do this all the time)

Human organizations have a lot of optimization power.

I talked with Mr. Muehlhauser about this specifically. I gather that at least at the time he thought human organizations should not be counted as intelligences (or at least as intelligences with the potential to become superintelligences) because they are not as versatile as human beings.

So when I am talking about super-human intelligence, I specifically mean an agent that is as good or better at humans at just about every skill set that humans possess for achieving their goals. So that would include thi

...
Morality Isn't Logical
by Wei Dai — 2012-12-26T23:08:09.419Z

What do I mean by "morality isn't logical"? I mean in the same sense that mathematics is logical but literary criticism isn't: the "reasoning" we use to think about morality doesn't resemble logical reasoning. All systems of logic, that I'm aware of, have a concept of proof and a method of verifying with high degree of certainty whether an argument constitutes a proof. As long as the logic is consistent (and we have good reason to think that many of them are), once we verify a proof we can accept its conclusion without worrying that there may be another proof that makes the opposite conclusion. With morality though, we have no such method, and people all the time make moral arguments that can be reversed or called into question by other moral arguments. (Edit: For an example of this, see these posts.)

Without being a system of logic, moral philosophical reasoning likely (or at least plausibly) doesn't have any of the nice properties that a well-constructed system of logic would have, for example, consistency, validity, soundness, or even the more basic property that considering arguments in a different order, or in a different mood, won't cause a person to accept an entirely different set of conclusions. For all we know, somebody trying to reason about a moral concept like "fairness" may just be taking a random walk as they move from one conclusion to another based on moral arguments they encounter or think up.

In a recent post, Eliezer said "morality is logic", by which he seems to mean... well, I'm still not exactly sure what, but one interpretation is that a person's cognition about morality can be described as an algorithm, and that algorithm can be studied using logical reasoning. (Which of course is true, but in that sense both math and literary criticism as well as every other subject of human study would be logic.) In any case, I don't think Eliezer is explicitly claiming that an algorithm-for-thinking-about-morality constitutes an algorithm-for-doing-logic, but I worry that the characterization of "morality is logic" may cause some connotations of "logic" to be inappropriately sneaked into "morality". For example Eliezer seems to (at least at one point) assume that considering moral arguments in a different order won't cause a human to accept an entirely different set of conclusions, and maybe this is why. To fight this potential s...
META: Deletion policy
by Eliezer Yudkowsky — 2012-12-26T01:46:53.561Z

http://wiki.lesswrong.com/wiki/Deletion_policy

This is my attempt to codify the informal rules I've been working by.

I'll leave this post up for a bit, but strongly suspect that it will have to be deleted not too long thereafter. I haven't been particularly encouraged to try responding to comments, either. Nonetheless, if there's something I missed, let me know.
Group rationality diary, 12/25/12
by cata — 2012-12-25T21:51:47.250Z

This is the public group instrumental rationality diary for Christmas week. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to everyone who contributes, and I hope everyone is having a nice holiday!

Previous diary; archive of prior diaries.
Ideal Advisor Theories and Personal CEV
by lukeprog — 2012-12-25T13:04:46.889Z

Update 5-24-2013: A cleaned-up, citable version of this article is now available on MIRI's website.

Co-authored with crazy88

Summary: Yudkowsky's "coherent extrapolated volition" (CEV) concept shares much in common Ideal Advisor theories in moral philosophy. Does CEV fall prey to the same objections which are raised against Ideal Advisor theories? Because CEV is an epistemic rather than a metaphysical proposal, it seems that at least one family of CEV approaches (inspired by Bostrom's parliamentary model) may escape the objections raised against Ideal Advisor theories. This is not a particularly ambitious post; it mostly aims to place CEV in the context of mainstream moral philosophy.

What is of value to an agent? Maybe it's just whatever they desire. Unfortunately, our desires are often the product of ignorance or confusion. I may desire to drink from the glass on the table because I think it is water when really it is bleach. So perhaps something is of value to an agent if they would desire that thing if fully informed. But here we crash into a different problem. It might be of value for an agent who wants to go to a movie to look up the session times, but the fully informed version of the agent will not desire to do so — they are fully-informed and hence already know all the session times. The agent and its fully-informed counterparts have different needs. Thus, several philosophers have suggested that something is of value to an agent if an ideal version of that agent (fully informed, perfectly rational, etc.) would advise the non-ideal version of the agent to pursue that thing.

This idea of idealizing or extrapolating an agent's preferences1 goes back at least as far as Sidgwick (1874), who considered the idea that "a man's future good" consists in "what he would now desire... if all the consequences of all the different [actions] open to him were accurately forseen..." Similarly, Rawls (1971) suggested that a person's good is the plan "that would be decided upon as the outcome of careful reflection in which the agent reviewed, in the light of all the relevant facts, what it would be like to carry out these plans..." More recently, in an article about rational agents and moral theory, Harsanyi (1982) defined what an agent's rational wants as “the preferences he would have if he had all the relevant factual information, always reasoned with the g...
[SEQ RERUN] Two Visions of Heritage
by MinibearRex — 2012-12-25T02:20:58.314Z

Today's post, Two Visions Of Heritage was originally published on December 9, 2008. A summary:

Differences between Hanson and Yudkowsky on how they see heritage.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Are AIs Homo Economicus?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Godel's Completeness and Incompleteness Theorems
by Eliezer Yudkowsky — 2012-12-25T01:16:59.961Z

Followup to: Standard and Nonstandard Numbers

So... last time you claimed that using first-order axioms to rule out the existence of nonstandard numbers - other chains of numbers besides the 'standard' numbers starting at 0 - was forever and truly impossible, even unto a superintelligence, no matter how clever the first-order logic used, even if you came up with an entirely different way of axiomatizing the numbers.

"Right."

How could you, in your finiteness, possibly know that?

"Have you heard of Godel's Incompleteness Theorem?"

Of course! Godel's Theorem says that for every consistent mathematical system, there are statements which are true within that system, which can't be proven within the system itself. Godel came up with a way to encode theorems and proofs as numbers, and wrote a purely numerical formula to detect whether a proof obeyed proper logical syntax. The basic trick was to use prime factorization to encode lists; for example, the ordered list <3, 7, 1, 4> could be uniquely encoded as:

23 * 37 * 51 * 74

And since prime factorizations are unique, and prime powers don't mix, you could inspect this single number, 210,039,480, and get the unique ordered list <3, 7, 1, 4> back out. From there, going to an encoding for logical formulas was easy; for example, you could use the 2 prefix for NOT and the 3 prefix for AND and get, for any formulas Φ and Ψ encoded by the numbers #Φ and #Ψ:

¬Φ = 22 * 3#Φ

Φ ∧ Ψ = 23 * 3#Φ * 5#Ψ

It was then possible, by dint of crazy amounts of work, for Godel to come up with a gigantic formula of Peano Arithmetic [](p, c) meaning, 'P encodes a valid logical proof using first-order Peano axioms of C', from which directly followed the formula []c, meaning, 'There exists a number P such that P encodes a proof of C' or just 'C is provable in Peano arithmetic.'

Godel then put in some further clever work to invent statements which referred to themselves, by having them contain sub-recipes that would reproduce the entire statement when manipulated by another formula.

And then Godel's Statement encodes the statement, 'There does not exist any number P such that P encodes a proof of (this statement) in Peano arithmetic' or in simpler terms 'I am not provable in Peano arithmetic'. If we assume first-ord...
[LINK] Too Soon for Doom Gloom?
by Shmi — 2012-12-24T21:14:52.081Z

Came across this paper posted in gr-qc (General Relativity and Quantum Cosmology) of all places. Abstract:

The observation that we are among the first 10^11 or so humans reduces the prior probability that we find ourselves in a species whose total lifetime number of individuals is much higher, according to arguments of Carter, Leslie, Nielsen, and Gott. However, if we instead start with a prior probability that a history has a total lifetime number which is very large, without assuming that we are in such a history, this more basic probability is not reduced by the observation of how early in history we exist.

While I am skeptical of anything Don Page (Hawking's student and apparently an Evangelical Christian) writes on the topic of anthropics (he publishes stuff on Boltzmann Brains, for pity's sake), Stuart Armstrong and other resident experts in the area should be able to tell if this paper has any substance. If anything, it has a good list of references on the topic.
What if "status" IS a terminal value for most people?
by The Dao of Bayes — 2012-12-24T20:31:21.883Z

[Inspired by a few of the science bits in HP:MOR, and far more so by the discussions between Draco and Harry about "social skills". Shared because I suspect it's an insight some people would benefit from.]

One of the more prominent theories on the evolution of human intelligence suggests that humans involved intelligence, not to deal with their environment, but rather to deal with each other. A small intellectual edge would foster competition, and it would result in the sort of recursive, escalating loop that's required to explain why we're so SUBSTANTIALLY smarter than every other species on the planet.
If you accept that premise, it's obvious that intelligence should, naturally, come with a desire to compete against other humans. It should be equally obvious from looking at human history that, indeed, we seem to do exactly that.
Posit, then, that, linked to intelligence, there's a trait for politics - using intelligence to compete against other humans, to try and establish dominance via cunning instead of brawn.
And, like everything that the Blind Idiot God Evolution has created, imagine that there are humans who LACK this trait for politics, but still have intelligence.
Think about the humans who, instead of looking inwards at humanity for competition, instead turn outwards to the vast uncaring universe of physics and chemistry. Other humans are an obtainable target - a little evolutionary push, and your tribe can outsmart any other tribe. The universe is not nearly so easily cowed, though. The universe is, often, undefeatable, or at least, we have not come close to mastering it. Six thousand years and people still die to storms and drought and famine. Six thousand years, and we have just touched on the moon, just begun to even SEE other planets that might contain life like ours.
I never understood other people before, because I'm missing that trait.
And I finally, finally, understand that this trait even exists, and what it must BE like, to have the trait.
We are genetic, chemical beings. I believe this with every ounce of myself. There isn't a soul that defies physics, there is not a consciousness that defies neurology. The world, even ourselves, can be measured. Anger comes from a part of this mixture, as does happiness and love. They are not lesser for this. They are not!
This is not an interlude. It is woven in to the meaning of w...
Licensing discussion for LessWrong Posts
by Rick_from_Castify — 2012-12-24T10:00:44.076Z

If you've been following the announced partnership between LessWrong and Castify, you'll know that we would like to start offering the promoted posts as a podcast.
So far, everything offered by Castify is authored by Eliezer Yudkowsky who gave permission to have his content used. Because promoted posts can be written by those who haven't explicitly given us permission, we're reluctant to offer them without first working through the licensing issues with the community.
What we propose is that all content on the site be subject to the Creative Commons license which would allow content posted to LessWrong to be used for commercial use as long as the work is given proper attribution.
LessWrong management and Castify want feedback from the community before moving forward. Thoughts?

Edit: EricHerboso was kind enough to start a poll in the comments here.
[SEQ RERUN] Are AIs Homo Economicus?
by MinibearRex — 2012-12-24T05:36:38.091Z

Today's post, Are AIs Homo Economicus? was originally published on December 9, 2008. A summary:

It seems odd that predictions of AIs based on economics are the subject of complaints that the agents modeled are too human, since the usual criticism of economic models is the reverse.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Disjunctions, Antipredictions, Etc., and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
New censorship: against hypothetical violence against identifiable people
by Eliezer Yudkowsky — 2012-12-23T21:00:31.045Z

New proposed censorship policy:

Any post or comment which advocates or 'asks about' violence against sufficiently identifiable real people or groups (as opposed to aliens or hypothetical people on trolley tracks) may be deleted, along with replies that also contain the info necessary to visualize violence against real people.

Reason: Talking about such violence makes that violence more probable, and makes LW look bad; and numerous message boards across the Earth censor discussion of various subtypes of proposed criminal activity without anything bad happening to them.

More generally: Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored on the grounds that it makes LW look bad and that anyone talking about a proposed crime on the Internet fails forever as a criminal (i.e., even if a proposed conspiratorial crime were in fact good, there would still be net negative expected utility from talking about it on the Internet; if it's a bad idea, promoting it conceptually by discussing it is also a bad idea; therefore and in full generality this is a low-value form of discussion).

This is not a poll, but I am asking in advance if anyone has non-obvious consequences they want to point out or policy considerations they would like to raise. In other words, the form of this discussion is not 'Do you like this?' - you probably have a different cost function from people who are held responsible for how LW looks as a whole - but rather, 'Are there any predictable consequences we didn't think of that you would like to point out, and possibly bet on with us if there's a good way to settle the bet?'

Yes, a post of this type was just recently made. I will not link to it, since this censorship policy implies that it will shortly be deleted, and reproducing the info necessary to say who was hypothetically targeted and why would be against the policy.