trentmkelly/LessWrong-43k huggingface dataset:
Irrational Upvotes
by mwaser — 2010-11-01T12:10:38.277Z

"This premise is VERY flawed" (found here) is the sole author-supplied content of a comment. There are no supporting links or additional content, only a one-sentence quote of the "offending" premise.

Yet, it has four upvotes.

This is a statement that can be made about any premise. It is backed by no supporting evidence.

Presumably, whoever upvoted it did so because they disagreed with the preceding comment (which, presumably, they downvoted -- unless they didn't have enough karma).

This *could* be viewed as rational behavior because it *does* support the goal of defeating the preceding comment but it does not support the LessWrong community. If premise is fatally flawed, then you should give at least some shred of a reason WHY or all you're doing is adding YOUR opinion.

This blog is "devoted to refining the art of human rationality". If the author is truly interested in refining his rationality, he has been given absolutely no help. He has no idea why his premise is flawed. He is now going to have to ask why or for some counter-examples. For his purposes (and the purposes of anyone else who doesn't understand or doesn't agree with your opinion), this post is useless noise clogging up the site.

Yet, it has four upvotes.

Is anyone else here bothered by this or am I way off base?
Berkeley LW Meet-up Saturday November 6
by LucasSloan — 2010-11-01T02:35:44.121Z

Last month, about 20 people showed up to the Berkeley LW meet-up. To continue the tradition of Berkeley Meetups, we will be meeting on Saturday, November 6 at 7 PM at the Starbucks at 2224 Shattuck Avenue. Last time, we chatted at the Starbucks for about 45 minutes, then went to get dinner and ate and talked under a T-Rex skeleton - we'll probably do something similar, so don't feel like you have to eat before you come. Hope to see you there!

ETA: As per User:Kevin's suggestion, we will instead be meeting at the Starbucks at 2128 Oxford Street.
Is cryonics evil because it's cold?
by ata — 2010-10-31T23:59:57.596Z

There have been many previous discussions here on cryonics and why it is perceived as threatening or otherwise disagreeable. Even among LWers who are not signed up and don’t plan to, I’d say there’s a good degree of consensus that cryonics is reviled and ridiculed to a very unjustified degree. I had a thought about one possible factor contributing to its unsavory public image that I haven’t seen brought up in previous discussions:

COLD is EVIL.

Well, no, cold isn’t evil, but “COLD is EVIL/THREATENING/DANGEROUS/HARSH/LONELY/UNLOVING/SAD/DEAD” seems to be a pretty common set of conceptual metaphors. You see it in figures of speech like “cold-hearted,” “in cold blood,” “cold expression,” “icy stare,” “chilling,” “went cold,” “cold calculation,” “the cold shoulder,” “cold feet,” “stone cold,” “out cold.” (Naturally, it’s also the case that WARM is GOOD/COMFORTING/SAFE/SOCIAL/LOVING/HAPPY/ALIVE, though COOL and HOT sort of go in their own directions.) Associating something with coldness just makes it seem more threatening and less benevolent. And besides, being that “COLD is DEAD,” it’s pretty hard to imagine someone as not really dead if they’re in a container of liquid nitrogen at -135ºC. (Even harder if it’s just their head in there… but that’s a separate issue.) There is already a little bit of research on the effects of some of the conceptual metaphors of coldness and the way its emotional content leaks onto metaphorically associated concepts (“Cold and lonely: does social exclusion literally feel cold?”; “Experiencing physical warmth promotes interpersonal warmth.”; any others?).

And indeed, it seems that repeatedly talking about it from the “it involves coldness [or ‘freezing’]” angle rather than the “it’s about preserving minds” angle pushes the right emotional buttons to make people feel negatively about it, given cryonics critics’ and ridiculers’ fondness for talking about people getting their heads “frozen” (“…the guys who had their heads sawed off and frozen…”) and referring to cryonics patients as “corpsicles....
Nils Nilsson's AI History: The Quest for Artificial Intelligence
by CarlShulman — 2010-10-31T19:33:39.378Z

I just noticed that AI pioneer and former Association for the Advancement of Artificial Intelligence (AAAI) head Nils Nilsson, has published his history of AI, The Quest for Artificial Intelligence: A History of Ideas and Achievements. The book is available as a free pdf from his website, with the pay version on Amazon, with reviews.
Imagine a world where minds run on physics
by cousin_it — 2010-10-31T19:09:04.250Z

This post describes a toy formal model that helps me think about self-modifying AIs, motivationally stable goal systems, paperclip maximizers and other such things. It's not a new result, just a way of imagining how a computer sitting within a world interacts with the world and with itself. I hope others will find it useful.

(EDIT: it turns out the post does imply a somewhat interesting result. See my exchange with Nesov in the comments.)

A cellular automaton like the Game of Life can contain configurations that work like computers. Such a computer may contain a complete or partial representation of the whole world, including itself via quining. Also it may have "actuators", e.g. a pair of guns that can build interesting things using colliding sequences of gliders, depending on what's in the return value register. The computer's program reasons about its model of the world axiomatically, using a proof checker like in my other posts, with the goal of returning a value whose representation in the return-value register would cause the actuators to affect the world-model in interesting ways (I call this the "coupling axiom"). Then that thing happens in the real world.

The first and most obvious example of what the computer could want is suicide. Assuming the "actuators" are flexible enough, the program could go searching for a proof that putting a certain return value in the register eventually causes the universe to become empty (assuming that at the beginning it was empty except for the computer). Then it returns that value and halts.

The second example is paperclipping. If the universe is finite, the program could search for a return value that results in a stable configuration for the entire universe with the most possible copies of some still-life, e.g. the "block". If the universe is infinite, it could search for patterns with high rates of paperclip production (limited by lightspeed in the automaton). In our world this would create something like the "energy virus" imagined by Sam Hughes - a rare example of a non-smart threat that sounds scarier than nanotech.

The third is sensing. Even though the computer lacks sensors, it will make them if the goal calls for it, so sensing is "emergent" in this formalization. (This part was a surprise for me.) For example, if the computer knows that the universe is empty except for a specified rectangle contai...
Ownership and Artificial Intelligence
by jferguson — 2010-10-31T15:44:38.802Z

(This is a subject that appears incredibly important to me, but it's received no discussion on LW from what I can see with a brief search. Please do link to articles about this if I missed them.)

Edit: This is all assuming that the first powerful AIs developed aren't exponentially self-improving; if there's no significant period of time where powerful AIs exist but they're not so powerful that the ownership relations between them and their creators don't matter, these questions are obviously not important.

What are some proposed ownership situations between artificial intelligence and its creators? Suppose a group of people creates some powerful artificial intelligence that appears to be conscious in most/every way--who owns it? Should the AI legally have self-ownership, and all the responsibility for its actions and ownership of the results of its labor that implies? Or, should strong AI be protected by IP, the way non-strong AI code already can be, treated as a tool rather than a conscious agent? It seems wise to implore people to not create AIs that want to have total free agency and generally act like humans, but that's hardly a guarantee that nobody will, and then you have the ethical issue of not being able to just kill them once they're created (if they "want" to exist and appear genuinely conscious). Are there any proposed tests to determine whether a synthetic agent should be able to own itself or become the property of its creators?

I imagine there aren't yet good answers to all these questions, but surely, there's some discussion of the issue somewhere, whether in rationalist/futurist circles or just sci-fi. Also, please correct me on any poor word choice you notice that needlessly limits the topic; it's broad, and I'm not yet completely familiar with the lingo of this subject.
Qualia Soup, a rationalist and a skilled You Tube jockey
by Raw_Power — 2010-10-31T14:18:40.790Z

Please have a look at this Youtube user account. I immediately thought of this site upon watching a couple of his clips. I am told we have tried here to publish some stuff in audiovisual fomat, but it didn't quite work out. Maybe we should contact this guy, perhaps we could profit from each other's work and experience? I am fairly hopeful that he could use some of the material here, and we could use some of his talent with the medium.

This video, for instance, looks like it was taken right out of this very blog
Are we trying to do things the hard way?
by NancyLebovitz — 2010-10-31T12:16:15.764Z

A TED talk about remarkable low-cost Indian products-- the Tata car which costs $2000 and is a real car, a $28 artificial lower leg which permits walking on rough ground, tree climbing, jumping, and running, and fast cheap drug development which starts with traditional Indian remedies. It's an example of something to defend because the effort is to develop products that very poor people can afford, so that incremental improvements and cost-cutting aren't good enough.

It leaves me wondering whether the process of creating FAI should be re-evaluated-- whether there's a built-in assumption of high personal costs which is unnecessary. That's wondering, not an absolute certainty, it's just that the $28 artificial lower leg shocked me into thinking about how much is being made harder than necessary.

Even if FAI is being worked on about as efficiently as possible, there may be a huge amount of possibility for making things easier in life generally.
Transhumanism and assisted suicide
by SeventhNadir — 2010-10-31T08:48:14.984Z

I would identify myself as holding many trans-humanist values and beliefs. That death and dying are not desirable and that it should, like smallpox, be eradicated. It also occurs to me however that I hold the belief that assisted suicide can sometimes be the right course of action. I can justify this sufficiently to myself, but I have a history of being a sophisticated arguer, so my ability to convince myself isn't great evidence.

Are my beliefs incoherent? I look at both issues and they both still feel right (I'm aware this does not make it so). Are these beliefs contradictory (and if so how)? Or are they justified by some hidden assumptions that I can't seem to acknowledge explicitly?

I get the idea that this is probably borderline appropriate for the discussion forum so I will delete this topic if asked to.
META: Tiered Discussions
by Pavitra — 2010-10-30T22:37:46.767Z

(Edit: It seems lots of people thought this was a terrible idea. I'm keeping the post as it was, though, mostly because I still think it's an interesting experiment and it ought to have been tried at least once somewhere on this site. Also, blah blah something about preserving the historical record so that earlier comments still make sense, whatever.)

You aren't allowed to know what this post says unless you can figure out what LW post this sentence is a clever reference to. The URL of that post is the CAST5 symmetric key for this one. Please help downvote spoilers into oblivion.
-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.10 (GNU/Linux)
jA0EAwMCtYf1bHFxvmRgyekK9+VOnIJEKESX8yr4CXk5IGX3rsoyS50Nsc+uCy85
413pFT1XlfX7UpRNjUPIlG5IjcFhMKhk4NUv32KEgBk7rbfCnPqIid6ry4Sb/QNC
RvgOhbTw1YY95+K9KMuZi67D0+Fak14jnL4ZrTTwgzl6dWaJmWnONpCK2hku8n9E
IZNFR5sGdxGdvmHRLvsqiVjZk0NP4ZyqN9bEAMIFOO4HcBISm24UyU46+leopqpE
K0dkirqKSL/7ZOXvk3s5cW9h7SOStUw9bo8mapHrkoPpPLmQmWB7FnJYY4omb5k+
5pGAS2qdXLQYvu1z7e8fyfMPiSqXFmGycM09tq5Un7y7ek63UHKkyIy29VuRa4uT
E88Yop/z0zodHoHruiDJLEN/JiWtitMouvpO/WzN4dJE1zOmQSTAiIWGUnvWUhc6
16m1dAPXR5+N5lkYRvPhi/tpTN96mZbesGBR0qOjheRssBMzRJhDAsZWt/Um/Qu3
au3Uxokq4UojOzJZSXLLYOhuBOa0nxNebp+Hcl/kbLkBe2WLgmVQY4EP8CVsMT0i
5PAYtwLpTmaakO/kwSe/ctd/Dr5KYOC+H68ciCXyRERQQrWczdI1gROPv+bmOp0R
TsQqGsWJhvuCMp4ZAsnj3HV/DhJKihb8F9TXwxjuC3tUghVrzzHUKk1ramGlWlK/
iMB9D2DWovBCbK00jfIQeZxu/kXBHns2DlcFwueGPShdarmtHCaWd/8wqChJ75sS
FupwvLKpVYcwa+hukNJi2BUgkfb/yrn4Y6vwhF+xF9D4MJrJG3mng4u7OnllifrZ
6OdUNYeQEG5P2Qkj1uu9hvJC8PP4vO648JjVEsaR7gtRouH3H1v5cKqElFpRlyED
qgkYdKzCfY96MOTj0b9BgEOw5a728F+rtSyDc+dcLWtFlSeuUc793YUvF6lxng2/
KlUyJ9dCDfSiTq+HsQH/kJHR8bmudomJC+ftnBoGxC5BLuQhC4gCPcaYM8evqWzP
kkR1OhjhWE9H+O2o53t75IIz/P2LUhwrfqGhBgD3PTAmxw94gbOz0Ckj3UjhKnTY
ID22NmrHRRZyJammi6TViHTWRNSLHKifMIGp0/hOC1o=
=sPUV
-----END PGP MESSAGE-----
Value Deathism
by Vladimir_Nesov — 2010-10-30T18:20:30.796Z

Ben Goertzel:

I doubt human value is particularly fragile. Human value has evolved and morphed over time and will continue to do so. It already takes multiple different forms. It will likely evolve in future in coordination with AGI and other technology. I think it's fairly robust.

Robin Hanson:

Like Ben, I think it is ok (if not ideal) if our descendants' values deviate from ours, as ours have from our ancestors. The risks of attempting a world government anytime soon to prevent this outcome seem worse overall.

We all know the problem with deathism: a strong belief that death is almost impossible to avoid, clashing with undesirability of the outcome, leads people to rationalize either the illusory nature of death (afterlife memes), or desirability of death (deathism proper). But of course the claims are separate, and shouldn't influence each other.

Change in values of the future agents, however sudden of gradual, means that the Future (the whole freackin' Future!) won't be optimized according to our values, won't be anywhere as good as it could've been otherwise. It's easier to see a sudden change as morally relevant, and easier to rationalize gradual development as morally "business as usual", but if we look at the end result, the risks of value drift are the same. And it is difficult to make it so that the future is optimized: to stop uncontrolled "evolution" of value (value drift) or recover more of astronomical waste.

Regardless of difficulty of the challenge, it's NOT OK to lose the Future. The loss might prove impossible to avert, but still it's not OK, the value judgment cares not for feasibility of its desire. Let's not succumb to the deathist pattern and lose the battle before it's done. Have the courage and rationality to admit that the loss is real, even if it's too great for mere human emotions to express.
DRAFT: Three Intellectual Temperaments: Birds, Frogs and Beavers
by multifoliaterose — 2010-10-30T17:49:15.846Z

Here is a draft of a potential top-level post which I'd welcome feedback on. I would appreciate any suggestions, corrections, additional examples, qualifications, or refinements.

Birds, Frogs and Beavers

The introduction of Birds and Frogs by Freeman Dyson reads

Some mathematicians are birds, others are frogs. Birds fly high in the air and survey broad vistas of mathematics out to the far horizon. They delight in concepts that unify our thinking and bring together diverse problems from different parts of the landscape. Frogs live in the mud below and see only the flowers that grow nearby. They delight in the details of particular objects, and they solve problems one at a time. I happen to be a frog, but many of my best friends are birds. The main theme of my talk tonight is this. Mathematics needs both birds and frogs. Mathematics is rich and beautiful because birds give it broad visions and frogs give it intricate details. Mathematics is both great art and important science, because it combines generality of concepts with depth of structures. It is stupid to claim that birds are better than frogs because they see farther, or that frogs are better than birds because they see deeper. The world of mathematics is both broad and deep, and we need birds and frogs working together to explore it.

Dyson is far from the first to have categorized mathematicians in such a fashion. For example, in The Two Cultures of Mathematics British mathematician Timothy Gowers wrote

The "two cultures" I wish to discuss will be familiar to all professional mathematicians. Loosely speaking, I mean the distinction between mathematicians who regard their central aim as being to solve problems, and those who are more concerned with building and understanding theories. This difference of attitude has been remarked on by many people, and I do not claim any credit for noticing it. As with most categorizations, it involves a certain oversimplication, but not so much as to make it useless. If you are unsure to which class you belong, then consider the following two statements.

(i) The point of solving problems is to understand mathematics better.
(ii) The point of understanding mathematics is to become better able to solve problems.

Most mathematicians would say that there is truth in both (i) and (ii). Not all problems are equally interesting, and one way of distinguishing the more interestin

...
QFT, Homotopy Theory and AI?
by ThomasR — 2010-10-30T10:48:13.992Z

What do you think about the new, exiting connections between QFT, Homotopy Theory and pattern recognition, proof verification and (maybe) AI systems? In view of the background of this forum's participants (selfreported in the survey mentioned a few days ago), I guess most of you follow those developments with some attention.

Concerning Homotopy Theory, there is a coming [special year](http://www.math.ias.edu/node/2610), you probably know Voevodsky's [recent intro lecture](http://www.channels.com/episodes/show/10793638/Vladimir-Voevodsky-Formal-Languages-partial-algebraic-theories-and-homotopy-category-), and [this](http://video.ias.edu/voevodsky-80th) even more popular one. Somewhat related are Y.I. Manin's remarks on the missing quotient structures (analogue to localized categories) in data structures and some of the ideas in Gromov's [essay](http://www.ihes.fr/~gromov/PDF/ergobrain.pdf).

Concerning ideas from QFT, [here](http://arxiv.org/abs/0904.4921) an example. I wonder what else concepts come from it?

BTW, whereas the public discussion focus on basic qm and on q-gravity questions, the really interesting and open issue is the special relativistic QFT: QM is just a canonical deformation of classical mechanics (and could have been found much earlier, most of the interpretation disputes just come from the confusion of mathematical properties with physics data), but Feynman integrals are despite half a century intense research mathematical unfounded. As Y.I. Manin called it in a recent interview, they are "an Eifel tower floating in the air". Only a strong platonian belief makes people tolerate that. I myself take them only serious because there is a clear platonic idea behind them and because number theoretic analoga work very well.
Ben Goertzel: The Singularity Institute's Scary Idea (and Why I Don't Buy It) 
by Paul Crowley — 2010-10-30T09:31:29.456Z

[...] SIAI's Scary Idea goes way beyond the mere statement that there are risks as well as benefits associated with advanced AGI, and that AGI is a potential existential risk.

[...] Although an intense interest in rationalism is one of the hallmarks of the SIAI community, still I have not yet seen a clear logical argument for the Scary Idea laid out anywhere. (If I'm wrong, please send me the link, and I'll revise this post accordingly. Be aware that I've already at least skimmed everything Eliezer Yudkowsky has written on related topics.)

So if one wants a clear argument for the Scary Idea, one basically has to construct it oneself.

[...] If you put the above points all together, you come up with a heuristic argument for the Scary Idea. Roughly, the argument goes something like: If someone builds an advanced AGI without a provably Friendly architecture, probably it will have a hard takeoff, and then probably this will lead to a superhuman AGI system with an architecture drawn from the vast majority of mind-architectures that are not sufficiently harmonious with the complex, fragile human value system to make humans happy and keep humans around.
The line of argument makes sense, if you accept the premises.
But, I don't.

Ben Goertzel: The Singularity Institute's Scary Idea (and Why I Don't Buy It), October 29 2010. Thanks to XiXiDu for the pointer.
Currently Buying AdWords for LessWrong
by Louie — 2010-10-30T05:31:43.667Z

So I'm trying to build more rationalists. To do this, I've invested a few hundred dollars of my own money to promote Less Wrong by buying low-cost AdWords on Google for different LW pages. I want to reach smart people with a really good article from Less Wrong that answers their question and draws them into our community so that the site's content can help improve their rationality. Based on buying AdWords before, I'd estimate that only 0.5-1% of people who click through to Less Wrong will actually get involved after reading an article, but since clicks only cost ~$0.04, that means it only costs me ~$6 to build a new rationalist and drastically improve someone's life. Seems like an excellent return on investment.
But to get a strong 1% conversion rate and really make an impact, I need to identify REALLY EXCELLENT Less Wrong content. Right now I'm experimenting by buying a lot of keywords related to quantum mechanics and sending people to http://lesswrong.com/lw/r8/and_the_winner_is_manyworlds/
My hope is that this page is useful and memorable enough that some small % of readers stick around and click through to other pages. My guess is that this isn't the ideal page to do this with but it's aiming in the right direction.
What page would you would want a new Less Wrong reader to find first? What answers a specific question they might have in such an impressive way that they would want to learn more about our community (perhaps many different pages for many different questions)?? Which articles are most memorable? Just looking at "Top" didn't yield any obvious choices... I felt like most of those articles were too META-META-META ... you'd need too much back knowledge for many of them. An ideally article would be more or less "stand-alone" so that any relatively intelligent person who doesn't have the whole LW corpus in their head already could just jump in and understand it immediately... and then branch out and explore LW from there.

So what do you think? Give me links to any landing pages you think would be worth promoting this way. You can write rough mini-ads as suggestions too if you'd like to be even more helpful. I'm looking forward to hearing your suggestions!