trentmkelly/LessWrong-43k huggingface dataset:
[video] Paul Christiano's impromptu tutorial on AIXI and TDT
by lukeprog — 2012-03-19T17:20:47.847Z

Paul Christiano was about to give a tutorial on AIXI and TDT, so I whipped out my iPhone and recorded it. His tutorial wasn't carefully planned or executed, but it may still be useful to some. Note that when Paul writes "UDT" on a piece of paper he really meant "TDT." :)

HD video download links: 1, 2.
[LINK] Nuclear winter: a reminder
by Stuart_Armstrong — 2012-03-19T11:48:44.038Z

Just a reminder that some of the old threats are still around (and hence that AI is not only something that can go hideously badly, but also some thing that could help us with the other existential risks as well):

http://blog.practicalethics.ox.ac.uk/2012/03/old-threats-never-die-they-fade-away-from-our-minds-nuclear-winter/

EDIT: as should have been made clear in that post (but wasn't!), the existential risks doesn't come from the full fledged nuclear winter directly, but from the collapse of human society and fragmentation of the species into small, vulnerable subgroups, with no guarantee that they'd survive or ever climb back to a technological society.
Suggestions on tech device/gear purchasing?
by jwhendy — 2012-03-19T03:39:42.768Z

I found out I won second place in an idea contest at work and am being granted ~$400 to spend at Best Buy (the web site for any unfamiliar). Originally, I believe the second place prize was going to be an iPad, but it looks like they've decided to just allow me to pick something in that ballpark price range.

I suspect there are a fair amount of tech saavy folks on LW and thought I'd inquire as to whether you've purchased a device or accessory (or anything from Best Buy-ish stores) that has brought you an increase in efficiency, usefulness, pleasure, etc. The idea of a tablet appeals to me, but I'm not entirely sure what I'd do with it. Also, a data plan is not in my budget, so many typical uses are not applicable in my case.

Anyway, just hoping to probe some collective knowledge about this decision. I'm not very knowledgeable on devices and/or how longer term usage/satisfaction matches expectations or even money spent.

Thanks for any assistance!
[SEQ RERUN] Hand vs. Fingers
by MinibearRex — 2012-03-19T03:13:45.486Z

Today's post, Hand vs. Fingers was originally published on 30 March 2008. A summary (taken from the LW wiki):

When you pick up a cup of water, is it your hand that picks it up, or is it your fingers, thumb, and palm working together? Just because something can be reduced to smaller parts doesn't mean that the original thing doesn't exist.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Initiation Ceremony, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Experience with Lumosity?
by AShepard — 2012-03-18T22:51:47.151Z

I just saw a commercial for Lumosity, which is a mental-skills training website. It seems like something that someone on LessWrong would have tried, but some googling of the site turns up only some passing mentions. Has anyone actually signed up and used it? Have you had results, and are they worth the subscription cost? (~$5-15/month, depending on subscription length).
6 Tips for Productive Arguments
by John_Maxwell — 2012-03-18T21:02:32.326Z

We've all had arguments that seemed like a complete waste of time in retrospect. But at the same time, arguments (between scientists, policy analysts, and others) play a critical part in moving society forward. You can imagine how lousy things would be if no one ever engaged those who disagreed with them.

This is a list of tips for having "productive" arguments. For the purposes of this list, "productive" means improving the accuracy of at least one person's views on some important topic. By this definition, arguments where no one changes their mind are unproductive. So are arguments about unimportant topics like which Pink Floyd album is the best.

Why do we want productive arguments? Same reason we want Wikipedia: so people are more knowledgeable. And just like the case of Wikipedia, there is a strong selfish imperative here: arguing can make you more knowledgeable, if you're willing to change your mind when another arguer has better points.

Arguments can also be negatively productive if everyone moves further from the truth on net. This could happen if, for example, the truth was somewhere in between two arguers, but they both left the argument even more sure of themselves.

These tips are derived from my personal experience arguing.

Keep it Friendly

Probably the biggest barrier to productive arguments is the desire of arguers to save face and avoid publicly admitting they were wrong. Obviously, it's hard for anyone's views to get more accurate if no one's views ever change.

This problem is exacerbated when arguers disparage one another. If you rebuke a fellow arguer, you're setting yourself up as their enemy. Admitting they were wrong would then mean giving in to an enemy. And no one likes to do that.

You may also find it difficult to carefully reconsider your own views after having ridiculed or berated someone who disagrees. I know I have in the past.

Both of these tendencies hurt argument productivity. To make arguments productive:

Keep things warm and collegial. Just because your ideas are in violent disagreement doesn't mean you have to disagree violently as people. Stay classy.

To the greatest extent possible, uphold the social norm that no one will lose face for publicly changing their mind.

If you're on a community-moderated forum like Less Wrong, don't downvote something unless you think the person who wrote it is

...
Meaning and having names for things vs knowing how they work
by Voltairina — 2012-03-18T19:25:05.666Z

I saw this post on a discussion thread earlier:

We routinely deny, or act in spite of, inconvenient truths. We can recognize that there is no meaning to love beyond evolutionary and chemical triggers, yet we fight for it just as fervently. Nihilists write books about nihilism despite it's admitted pointlessness. We are as blind as our very genes which multiply and propagate themselves despite our executioner sun which grows daily above our heads, eventually to the point of consuming everything we know. By the very act of living and pursuing human concocted dreams and desires, we are in a constant denial of our situation.

I wonder a few things. Is this sort of experience related to the spoiler effect people claim to experience when they know the ending of a good book in advance? If so, this (paper itself here, thanks gwern) might be relevant. Its a wired article, referring to a study where it appeared the opposite of the spoiler effect is actually true - knowing the ending in advance improves people's subjective experience of a good short story. Do people subjectively experience the spoiler effect anyways? I'm wondering if it isn't really something like this:

1.A person reads a short story they enjoy with a spoiler in mind.

2.A person anticipates the ending experience and ramp up for it

3.A person enjoys it more than if unspoiled.

4.person carries a "spoiler effect" meme, and engages in the mind projection fallacy to imagine what life would have been like had they been able to read the short story without a spoiler.

5.They decide to accept the validity of this projection, and this reinforces the meme, because they experience annoyance with their experience of the story compared to the imagined experience of the unspoiled story.

I also wonder whether there's a way to modify the mind projection fallacy to set up a feedback loop? Imagining how wonderful it will be to be in good shape while you're exercising, maybe, then fantasising about being in good shape, then thi...
The Best Comments Ever
by Grognor — 2012-03-18T13:02:09.744Z

Of the title of this discussion post, we already have an approximation in the most voted-for lists for Main and Discussion. There are many problems with this metric, however, a subset of which are:

Both sections are cluttered. The top comments for Main are full of rationality quotes (which are better accessed elsewhere), and the top comments for Discussion are full of polls. (Can we please have a non-stupid way of putting polls in comments?)

Both sections are biased by exposure. A comment that a lot of people see generally gets more karma than comments which not many people see. As Less Wrong's growth rate increases, and as time goes by, this will increasingly bias these sections toward newer comments. Also, comments made by well-known LWers will be seen more often and correspondingly upvoted more.

Joke comments oftentimes get a lot more comments than insightful comments.

Entire threads can have weird voting patterns that don't match quality.

These sections list number of upvotes, so extremely controversial comments appear in between unanimously good ones. (May not actually be a problem at all.)

People use karma as behavior reinforcers, so comments like "I'll transcribe this" get lots of votes.

Dozens more little things I won't try to list

So instead of lists of comments that made a lot of Less Wrong accounts go "eh, have a karma", let's have a list of Less Wrong comments that are so good you'll never forget them, and you wish you could do more than merely upvote.

This list will have its own problems, like being biased toward memorable comments, but I hope the two lists will complement each other and whatever's missing isn't terribly important.

I got the idea from a note about how "Discussion is better than Main, comments are better than Discussion" because social norms prevent certain types of Main post from being made.

Anyway, here are some proposed rules, of which all but the first two are mere suggestions:

Don't post your own comments.

Don't post more than one comment in this thread. If you find more other-person comments you want to add, edit yours to include them.

Say why you think the comment is good, even if it's only a line.

Bonus points for finding very old comments, or those from threads that got very little exposure.

No joke comments or comments that solely quote people.

Don't retrieve your comment from the list of top comments.

...
The AI design space near the FAI [draft]
by Dmytry — 2012-03-18T10:29:41.859Z

Abstract:

Nearly-FAIs can be more dangerous than AIs with no attempt at friendliness. The FAI effort needs better argument that the attempt at FAI decreases the risks. We are bad at processing threats rationally, and prone to very bad decisions when threatened, akin to running away from unknown into a minefield.

Nearly friendly AIs

Consider AI that truly loves mankind but decides that all of the mankind must be euthanized like an old, sick dog - due to chain of reasoning too long for us to generate when we test our logic of AI, or even comprehend - and proceeds to make a bliss virus - the virus makes you intensely happy, setting your internal utility to infinity; and keeping it so until you die. It wouldn't even take a very strongly superhuman intelligence to do that kind of thing. Treating life as if it was a disease. It can do so even if it destroys the AI itself. Or consider the FAI that cuts your brain apart to satisfy each hemisphere's slightly different desires. The AI that just wireheads everyone because it figured we all want it (and worst of all it may be correct).

It seems to me that one can find the true monsters in the design space near to the FAI, and even including the FAIs. And herein lies a great danger: bugged FAIs, the AIs that are close to friendly AI, but are not friendly. It is hard for me to think of a deficiency in friendliness which isn't horrifically unfriendly (restricting to deficiencies that don't break AI).

Should we be so afraid of the AIs made without attempts at friendliness?

We need to keep in mind that we have no solid argument that the AIs written without attempt at friendliness - the AIs that predominantly don't treat mankind in any special way - will necessarily make us extinct.

We have one example of 'bootstrap' optimization process - evolution - with not a slightest trace of friendliness in it. What did emerge in the end? We assign pretty low utility to nature, but non-zero, and we are willing to trade resources for preservation of nature - see the endangered species list and international treaties on whaling. It is not perfect, but I think it is fair to say that the single example of bootstrap intelligence we got values the complex dynamical processes for what they are, and prefers to obtain resources without disrupting those processes, even if it is slightly more expensive to do so, and is willing to divert small fracti...
Causation, Probability and Objectivity
by antigonus — 2012-03-18T06:54:33.513Z

Most people here seem to endorse the following two claims:

1. Probability is "in the mind," i.e., probability claims are true only in relation to some prior distribution and set of information to be conditionalized on;
2. Causality is to be cashed out in terms of probability distributions á la Judea Pearl or something.

However, these two claims feel in tension to me, since they appear to have the consequence that causality is also "in the mind" - whether something caused something else depends on various probability distributions, which in turn depends on how much we know about the situation. Worse, it has the consequence that ideal Bayesian reasoners can never be wrong about causal relations, since they always have perfect knowledge of their own probabilities.

Since I don't understand Pearl's model of causality very well, I may be missing something fundamental, so this is more of a question than an argument.
Three new papers on AI risk
by lukeprog — 2012-03-18T06:52:13.175Z

In case you aren't subscribed to FriendlyAI.tumblr.com for the latest updates on AI risk research, I'll mention here that three new papers on the subject were recently made available online...

Bostrom (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents.

This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent.

Yampolskiy & Fox (2012a). Safety engineering for artificial general intelligence.

Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence and robotics communities. We will argue that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, as are proposals to limit the negative effects of AIs by constraining their behavior. As an alternative, we propose a new science of safety engineering for intelligent artificial agents based on maximizing for what humans value. In particular, we challenge the scientific community to develop intelligent systems that have humanfriendly values that they provably retain, even under recursive self-improvement.

Yampolskiy & Fox (2012b). Artificial general intelligence and the human mental model.

When the first artificial general intelligences are built, they may improve themselves to far-above-human levels. Speculations about such future entities are already affected by anthropomorphic bias, which leads to erroneous analogies with human minds. In this chapter, we apply a goal-oriented understanding of intelligence to sh

...
[SEQ RERUN] Initiation Ceremony
by MinibearRex — 2012-03-18T04:49:18.772Z

Today's post, Initiation Ceremony was originally published on 28 March 2008. A summary (taken from the LW wiki):

Brennan is inducted into the Conspiracy

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was To Spread Science, Keep It Secret, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Fallacies as weak Bayesian evidence
by Kaj_Sotala — 2012-03-18T03:53:34.216Z

Abstract: Exactly what is fallacious about a claim like ”ghosts exist because no one has proved that they do not”? And why does a claim with the same logical structure, such as ”this drug is safe because we have no evidence that it is not”, seem more plausible? Looking at various fallacies – the argument from ignorance, circular arguments, and the slippery slope argument - we find that they can be analyzed in Bayesian terms, and that people are generally more convinced by arguments which provide greater Bayesian evidence. Arguments which provide only weak evidence, though often evidence nonetheless, are considered fallacies.

As a Nefarious Scientist, Dr. Zany is often teleconferencing with other Nefarious Scientists. Negotiations about things such as ”when we have taken over the world, who's the lucky bastard who gets to rule over Antarctica” will often turn tense and stressful. Dr. Zany knows that stress makes it harder to evaluate arguments logically. To make things easier, he would like to build a software tool that would monitor the conversations and automatically flag any fallacious claims as such. That way, if he's too stressed out to realize that an argument offered by one of his colleagues is actually wrong, the software will work as backup to warn him.

Unfortunately, it's not easy to define what counts as a fallacy. At first, Dr. Zany tried looking at the logical form of various claims. An early example that he considered was ”ghosts exist because no one has proved that they do not”, which felt clearly wrong, an instance of the argument from ignorance. But when he programmed his software to warn him about sentences like that, it ended up flagging the claim ”this drug is safe, because we have no evidence that it is not”. Hmm. That claim felt somewhat weak, but it didn't feel obviously wrong the way that the ghost argument did. Yet they shared the same structure. What was the difference?

The argument from ignorance

Related posts: Absence of Evidence is Evidence of Absence, But Somebody Would Have Noticed!

One kind of argument from ignorance is based on negative evidence. It assumes that if the hypothesis of interest were true, then experiments made to test it would show positive results. If a drug were toxic, tests of toxicity of reveal this. Whether or not this argument is valid depends on ...
LiveJournal Memes
by Douglas_Reay — 2012-03-18T02:56:46.571Z

On blogging websites just as LiveJournal, memes are often in the form of a question or set of questions which a blogger answers in their own blog, then challenges their readers to answer in the readers' blogs (thus spreading). It doesn't have to be the sort of question to which there is a 'correct' answer. More usually the meme spreads if the questions are interesting and the answers reveal something about the personality or interests of the blogger.

Here's a recent example:

A ticket for a flight into space (an orbit and a visit to the International Space Station) arrives through the post. Assume that you meet the relevant physical requirements. You can use it, transfer it, or sell it. What do you do and why?

It occurs to me that this sort of thing might be a very accessible way to introduce people to thinking about rationality. With the example I gave, you get people who would use the ticket because they know that if they didn't they'd always regret not having gone. However, if the question had been phrased as them receiving money, and a space flight ticket was only one of the things listed as purchasable for that sum, they wouldn't make the same decision.

Suppose LessWrong were to compose a LiveJournal meme. It would want to be made up of somewhere between 5 and 10 short (paragraph or less) and simple (easy reading comprehension, no obscure terms or prior reading required) questions designed to elicit answers interesting for blog readers to read, intended to introduce bloggers to the concept of a 'cognitive bias' and to thinking about what rationality actually is.

Do you have any suggestions for questions that would work well in such a meme?
I'm starting a game company and looking for a co-founder.
by Alexei — 2012-03-18T00:07:01.670Z

Summary: I am looking for co-founder(s) to start a game company with me. If you, or anyone you know, is interested, please contact me. (Alternatively, if you want to invest or provide funding, that would be very nice in its own right.)

It recently occurred to me that if reducing existential risk is indeed the most important goal, then I ought to actually do something about it. Turns out, for most mortals (including myself) the best option for reducing ex-risk is through donations. With that in mind, I'm going to start a game company. "Why a game company?" you might ask. Well:

* I've been making games since I was 13.

* I've hit my 10,000 hours of game programming a while ago. If I want to make a game, it will be made.

* I've studied a good amount of game design theory and have had some opportunities to put that knowledge to the test with success.

* I've worked for two different game companies. I've written games for computers, handhelds, and mobile devices.

* I've funded, designed, programmed, and published my own game.

* I am very familiar with the process of developing games. Everything from team structure to tools to game design.

I hope it's clear why starting a game company makes sense for me. Now, you might ask, "Why will you succeed?" Well, I have an answer for that too:

* Leverage all the rationality skills I've learned from LW.

* Leverage all other scientific knowledge: from psychology to statistics.

* I'm not attached to any particular game genre or game idea. Whatever gives the most ROI is good.

* There is a lot of low hanging fruit in terms of what games are easy to make and are almost guaranteed to be profitable. (Most people don't choose these ideas because they are easy, have been done before, or the people want to make other games.)

* Focus on making games as cheaply as possible. (Leverage 3rd party tools and other companies.)

* Have a structured approach to designing and developing a game, rather than an adhoc one like most companies have.

* The company will be built around scientific principle. (A/B testing is just one aspect of that.)

* The company will measure and analyze everything (or as much as possible), not only in the games it makes, but also in the company itself.

* The company will be completely transparent on the inside. If everyone knows everything about the company, it's more likely that they will find new

...