trentmkelly/LessWrong-43k huggingface dataset:
A survey of anti-cryonics writing 
by Paul Crowley — 2010-02-07T23:26:52.715Z

(This was originally a link to a post on my blog, A survey of anti-cryonics writing. Eliezer asked me to include the entire text of the article here.)

For its advocates, cryonics offers almost eternal life. To its critics, cryonics is pseudoscience; the idea that we could freeze someone today in such a way that future technology might be able to re-animate them is nothing more than wishful thinking on the desire to avoid death. Many who battle nonsense dressed as science have spoken out against it: see for example Nano Nonsense and Cryonics, a 2001 article by celebrated skeptic Michael Shermer; or check the Skeptic's Dictionary or Quackwatch entries on the subject, or for more detail read the essay Cryonics–A futile desire for everlasting life by "Invisible Flan".

That it seems so makes me sad, because to my naive eyes it seems like it might work and I would quite like to live forever, but I know that I don't know enough to judge. The celebrated Nobel prize winning physicist Richard Feynman tells a story of a US general who spoke to him at a party and explained that one big challenge in desert warfare is keeping the tanks fuelled given the huge distances the fuel has to travel. What would really help, the general said, would be if boffins like Feynman could invent a sort of engine that was powered by sand. On this issue, I'm in the same position as the general; in the same way as a tank fuelled by sand seems plausible enough to him, it makes sense to me to imagine that however your brain stores information it probably has something to do with morphology and chemistry, so there's a good chance it might not evaporate right away at the instant of legal death, and that freezing might be a way to keep the information there long enough for future societies to extract it with their future-technology scanning equipment.

And of course the pro-cryonics people have written reams and reams of material such as Ben Best's Scientific Justification of Cryonics Practice on why they think this is exactly as plausible as I might think, and going into tremendous technical detail setting out arguments for its plausibility and addressing particular difficulties. It's almost enough to make you want to sign up on the spot.

Except, of course, that plenty of totally unscientific ideas are backed by reams of scientific-sounding documents good enough to fool non-experts like me. Back...
Limited kindness is unappreciated
by KatjaGrace — 2010-02-07T11:39:43.000Z

If you have not yet interacted with a person, you are judged neutrally by them. If you do something for them once, then you move up in their eyes. If you continue to benefit them you can move further up. If you stop you move to well below zero; you have actually slighted them. Even if you slow down a bit you can go into negative territory. This goes for many things humans offer each other from tea to sex. Why is limited attention worse than none?

One guess is that it’s an upshot of tit-for-tat. If I am nice to someone, they are nice to me in return, as obliged. Then I am obliged. Mentioning that the interaction has occurred an even number of times doesn’t get you off the hook; you always owe more friendly deeds.

Another potential reason is that when you haven’t interacted with someone they still have high hopes you will be a good person to know, whereas when you know them and cease to give them attention, you are demonstrably not. This doesn’t seem right, as strangers usually remain strangers, and people who have had an interest often return to it.

Perhaps un-friendliness is a punishment to encourage your future cooperation? People who have been useful in the past are a better target than others because they are presumably already close to being friendly again. If I’m wondering whether to phone you or not and I think you will be miffed if I haven’t it may push me over the line, whereas if we haven’t met and I think you might be miffed when we eventually do, I probably won’t bother because I probably will never meet you or want to anyway.

For whatever reason, this must reduce the occurrence of friendly behavior enourmously. Before you interact with someone you must ascertain that they are likely enough to be good enough for long enough that it’s worth the cost of their badmouthing or teary appeals to stay if you ever decide they’re not. This certainly limits my own friendliness – often I wouldn’t mind being helpful to strangers, but I’ve learned the annoying way how easy it is to become an obligated ‘friend’ just because you can’t bear to watch someone suffer on a single occasion. So other people prevent me from benefiting them with their implicit threat of obligation.

Interestingly, one situation where humans are nice to one another and not further obliged is when t...
BHTV: Eliezer Yudkowsky & Razib Khan
by MichaelGR — 2010-02-06T14:27:12.762Z

Link.

"Razib Khan has an academic background in the biological sciences, and has worked for many years in software. He is an Unz Foundation Junior Fellow. He lives in the western United States."

Razib's writings can be found on his blog, Gene Expression.
Ethics has Evidence Too
by Jack — 2010-02-06T06:28:24.157Z

A tenet of traditional rationality is that you can't learn much about the world from armchair theorizing. Theory must be epiphenomenal to observation-- our theories are functions that tell us what experiences we should anticipate, but we generate the theories from *past* experiences. And of course we update our theories on the basis of new experiences. Our theories respond to our evidence, usually not the other way around. We do it this way because it works better then trying to make predictions on the basis of concepts or abstract reasoning. Philosophy from Plato through Descartes and to Kant is replete with failed examples of theorizing about the natural world on the basis of something other than empirical observation. Socrates thinks he has deduced that souls are immortal, Descartes thinks he has deduced that he is an immaterial mind, that he is immortal, that God exists and that he can have secure knowledge of the external world, Kant thinks he has proven by pure reason the necessity of Newton's laws of motion.

These mistakes aren't just found in philosophy curricula. There is a long list of people who thought they could deduce Euclid's theorems as analytic or a priori knowledge. Epicycles were a response to new evidence but they weren't a response that truly privileged the evidence. Geocentric astronomers changed their theory *just enough* so that it would yield the right predictions instead of letting a new theory flow from the evidence. Same goes for pre-Einsteinian theories of light. Same goes for quantum mechanics. A kludge is a sign someone is privileging the hypothesis. It's the same way many of us think the Italian police changed their hypothesis explaining the murder of Meredith Kercher once it became clear Lumumba had an alibi and Rudy Guede's DNA and hand prints were found all over the crime scene. They just replaced Lumumba with Guede and left the rest of their theory unchanged even though there was no longer reason to include Knox and Sollecito in the explanation of the murder. These theories may make it over the bar of traditional rationality but they sail right under what Bayes theorem requires.

Most people here get this already and many probably understand it better than I do. But I think it needs to be brought up in the context of our ongoing discussion of normative ethics.

Unless we have reason to think about ethics differently, our normat...
Debate tools: an experience report
by Morendil — 2010-02-05T14:47:56.891Z

Follow-up to: Argument Maps Improve Critical Thinking, Software Tools for Community Truth-Seeking

We are here, among other things, in an attempt to collaboratively refine the art of human rationality.

Rationality is hard, because the wetware we run rationality on is scavenged parts originally intended for other purposes; and collaboration is hard, I believe because it involves huge numbers of tiny decisions about what information others need. Yet we get by, largely thanks to advances in technology.

One of the most important technologies for advancing both rationality and collaboration is the written word. It affords looking at large, complex issues with limited cognitive resources, by the wonderful trick of "external cached thoughts". Instead of trying to hold every piece of the argument at once, you can store parts of it in external form, refer back to them, and communicate them to other people.

For some reason, it seems very hard to improve on this six-thousand-year-old technology. Witness LessWrong itself, which in spite of using some of the latest and greatest communication technologies, still has people arguing by exchanging sentences back and forth.

Previous posts have suggested that recent software tools might hold promise for improving on "traditional" forms of argument. This kind of suggestion is often more valuable when applied to a real and relevant case study. I found the promise compelling enough to give a few tools a try, in the context of the recent (and recurrent) cryonics debate. I report back here with my findings.

I. Argunet

The first tool I tried was Argunet, an Open Source offering from the Institute of Philosophy in Berlin. I was seduced by the promise of reconstructing the logical structure of an argument, and by the possiblity of collaborating online with others on an argument.

Like other products in that category, the basic principle of operation of Argunet is that of a visual canvas, on which you can create and arrange boxes which represents statements, portions of an argument. Relationships between parts of an arguments are then materialized using links or arrows.

Argunet supports two types of basic relationship between statements, Supports and Attacks. It also supports several types of "inference patterns".

Unfortunately, when I tried using the Editor I soon found it difficult to the point of being unusable. The default expectation of being...
‘Cheap’ goals won’t explode intelligence
by KatjaGrace — 2010-02-05T14:00:46.000Z

An intelligence explosion is what hypothetically happens when a clever creature finds that the best way to achieve its goals is to make itself even cleverer first, and then to do so again and again as its heightened intelligence makes the the further investment cheaper and cheaper. Eventually the creature becomes uberclever and can magically (from humans’ perspective) do most things, such as end humanity in pursuit of stuff it likes more. This is predicted by some to be the likely outcome for artificial intelligence, probably as an accidental result of a smart enough AI going too far with any goal other than forwarding everything that humans care about.

In trying to get to most goals, people don’t invest and invest until they explode with investment. Why is this? Because it quickly becomes cheaper to actually fulfil a goal at than it is to invest more and then fulfil it. This happens earlier the cheaper the initial goal. Years of engineering education prior to building a rocket will speed up the project, but it would slow down the building of a sandwich.

A creature should only invest in many levels of intelligence improvement when it is pursuing goals significantly more resource intensive than creating many levels of intelligence improvement. It doesn’t matter that inventing new improvements to artificial intelligence gets easier as you are smarter, because everything else does too. If intelligence makes other goals easier a the same rate as it makes building more intelligence easier, no goal which is cheaper than building a given amount of intelligence improvement with your current intelligence could cause an intelligence explosion of that size.

Plenty of questions anyone is currently looking for answers to, such as ‘how do we make super duper nanotechnology?’, ‘how do we cure AIDS?’, ‘how do I get really really rich?’ and even a whole bunch of math questions are likely easier than inventing multiple big advances in AI. The main dangerous goals are infinitely expensive questions such as ‘how many digits of pi can we work out?’ and ‘please manifest our values maximally throughout as much of the universe as possible’. If someone were to build a smart AI and set it to solve any of those relatively cheap goals, it would not accidentally lead to an intelligence explosion. The risk is only wit...
A problem with Timeless Decision Theory (TDT)
by Gary_Drescher — 2010-02-04T18:47:07.959Z

According to Ingredients of Timeless Decision Theory, when you set up a factored causal graph for TDT, "You treat your choice as determining the result of the logical computation, and hence all instantiations of that computation, and all instantiations of other computations dependent on that logical computation", where "the logical computation" refers to the TDT-prescribed argmax computation (call it C) that takes all your observations of the world (from which you can construct the factored causal graph) as input, and outputs an action in the present situation.

I asked Eliezer to clarify what it means for another logical computation D to be either the same as C, or "dependent on" C, for purposes of the TDT algorithm. Eliezer answered:

For D to depend on C means that if C has various logical outputs, we can infer new logical facts about D's logical output in at least some cases, relative to our current state of non-omniscient logical knowledge. A nice form of this is when supposing that C has a given exact logical output (not yet known to be impossible) enables us to infer D's exact logical output, and this is true for every possible logical output of C. Non-nice forms would be harder to handle in the decision theory but we might perhaps fall back on probability distributions over D.

I replied as follows (which Eliezer suggested I post here).

If that's what TDT means by the logical dependency between Platonic computations, then TDT may have a serious flaw.

Consider the following version of the transparent-boxes scenario. The predictor has an infallible simulator D that predicts whether I one-box here [EDIT: if I see $1M]. The predictor also has a module E that computes whether the ith digit of pi is zero, for some ridiculously large value of i that the predictor randomly selects. I'll be told the value of i, but the best I can do is assign an a priori probability of .1 that the specified digit is zero.

The predictor puts $1M in the large box iff (D xor E) is true. (And that's explained to me, of course.)

So let's say I'm confronted with this scenario, and I see $1M in the large box.

The flaw then is that E (as well as D) meets your criterion for "depending on" my decision computation C. I'm initially unsure what C and E output. But if C in fact one-boxes here, then I can infer that E outputs False (or else the large box

...
"Put It To The Test"
by MBlume — 2010-02-03T23:09:37.378Z

Alt-rockers They Might Be Giants explain/advocate empiricism in a record aimed at young children.

Don't believe it 'cos they say it's so

If it's not true you have a right to know --

Put it to the test!

No, it's not quite Bayesian. The bridge ("A fact is just a fantasy, unless it can be checked") is more or less simply wrong. Still, I find the fact that the Ancient Art of Rationality is getting play at all pretty exciting. What do you all think? And what can we do to get more rationalist -- or even proto-rationalist -- ideas to youngsters?
A Much Better Life?
by Psychohistorian — 2010-02-03T20:01:57.431Z

(Response to: You cannot be mistaken about (not) wanting to wirehead, Welcome to Heaven)

The Omega Corporation
Internal Memorandum
To: Omega, CEO
From: Gamma, Vice President, Hedonic Maximization

Sir, this concerns the newest product of our Hedonic Maximization Department, the Much-Better-Life Simulator. This revolutionary device allows our customers to essentially plug into the Matrix, except that instead of providing robots with power in flagrant disregard for the basic laws of thermodynamics, they experience a life that has been determined by rigorously tested algorithms to be the most enjoyable life they could ever experience. The MBLS even eliminates all memories of being placed in a simulator, generating a seamless transition into a life of realistic perfection.

Our department is baffled. Orders for the MBLS are significantly lower than estimated. We cannot fathom why every customer who could afford one has not already bought it. It is simply impossible to have a better life otherwise. Literally. Our customers' best possible real life has already been modeled and improved upon many times over by our programming. Yet, many customers have failed to make the transition. Some are even expressing shock and outrage over this product, and condemning its purchasers.

Extensive market research has succeeded only at baffling our researchers. People have even refused free trials of the device. Our researchers explained to them in perfectly clear terms that their current position is misinformed, and that once they tried the MBLS, they would never want to return to their own lives again. Several survey takers went so far as to specify that statement as their reason for refusing the free trial! They know that the MBLS will make their life so much better that they won't want to live without it, and they refuse to try it for that reason! Some cited their "utility" and claimed that they valued "reality" and "actually accomplishing something" over "mere hedonic experience." Somehow these organisms are incapable of comprehending that, inside the MBLS simulator, they will be able to experience the feeling of actually accomplishing feats far greater than they could ever accomplish in real life. Frankly, it's remarkable such people amassed enough credits to be able to afford our products in the first place!

You may recall that a Beta version had an off switch, enabling users to deac...
Applying utility functions to humans considered harmful
by Kaj_Sotala — 2010-02-03T19:22:56.547Z

There's a lot of discussion on this site that seems to be assuming (implicitly or explicitly) that it's meaningful to talk about the utility functions of individual humans. I would like to question this assumption.

To clarify: I don't question that you couldn't, in principle, model a human's preferences by building this insanely complex utility function. But there's an infinite amount of methods by which you could model a human's preferences. The question is which model is the most useful, and which models have the least underlying assumptions that will lead your intuitions astray.

Utility functions are a good model to use if we're talking about designing an AI. We want an AI to be predictable, to have stable preferences, and do what we want. It is also a good tool for building agents that are immune to Dutch book tricks. Utility functions are a bad model for beings that do not resemble these criteria.

To quote Van Gelder (1995):

Much of the work within the classical framework is mathematically elegant and provides a useful description of optimal reasoning strategies. As an account of the actual decisions people reach, however, classical utility theory is seriously flawed; human subjects typically deviate from its recommendations in a variety of ways. As a result, many theories incorporating variations on the classical core have been developed, typically relaxing certain of its standard assumptions, with varying degrees of success in matching actual human choice behavior.
Nevertheless, virtually all such theories remain subject to some further drawbacks:
(1) They do not incorporate any account of the underlying motivations that give rise to the utility that an object or outcome holds at a given time.
(2) They conceive of the utilities themselves as static values, and can offer no good account of how and why they might change over time, and why preferences are often inconsistent and inconstant.
(3) They offer no serious account of the deliberation process, with its attendant vacillations, inconsistencies, and distress; and they have nothing to say about the relationships that have been uncovered between time spent deliberating and the choices eventually made.
Curiously, these drawbacks appear to have a common theme; they all concern, one way or another, temporal aspects of decision making. It is worth asking whether they arise because of some deep structural feature

...
False Majorities
by JamesAndrix — 2010-02-03T18:43:25.281Z

If a majority of experts agree on an issue, a rationalist should be prepared to defer to their judgment. It is reasonable to expect that the experts have superior knowledge and have considered many more arguments than a lay person would be able to. However, if experts are split into camps that reject each other's arguments, then it is rational to take their expert rejections into account. This is the case even among experts that support the same conclusion.

If 2/3's of experts support proposition G , 1/3 because of reason A while rejecting B, and 1/3 because of reason B while rejecting A, and the remaining 1/3 reject both A and B; then the majority Reject A, and the majority Reject B. G should not be treated as a reasonable majority view.

This should be clear if A is the koran and B is the bible.

Positions that fundamentally disagree don't combine in dependent aspects on which they agree. On the contrary, If people offer lots of different contradictory reasons for a conclusion (even if each individual has consistent beliefs) it is a sign that they are rationalizing their position.

An exception to this is if experts agree on something for the same proximal reasons. If pharmacists were split into camps that disagreed on what atoms fundamentally were, but agreed on how chemistry and biology worked, then we could add those camps together as authorities on what the effect of a drug would be.

If we're going to add up expert views, we need to add up what experts consider important about a question and agree on, not individual features of their conclusions.

Some differing reasons can be additive: Evolution has support from many fields. We can add the analysis of all these experts together because the paleontologists do not generally dispute the arguments of geneticists.

Different people might justify vegetarianism by citing the suffering of animals, health benefits, environmental impacts, or purely spiritual concerns. As long as there isn't a camp of vegetarians that claim it does not have e.g. redeeming health benefits, we can more or less add all those opinions together.

We shouldn't add up two experts if they would consider each other's arguments irrational. That's ignoring their expertise.

Original Thread