trentmkelly/LessWrong-43k huggingface dataset:
Metacontrarian Metaethics
by Will_Newsome — 2011-05-20T05:36:33.741Z

Designed to gauge responses to some parts of the planned “Noticing confusion about meta-ethics” sequence, which should intertwine with or be absorbed by Lukeprog’s meta-ethics sequence at some point.

Disclaimer: I am going to leave out many relevant details. If you want, you can bring them up in the comments, but in general meta-ethics is still very confusing and thus we could list relevant details all day and still be confused. There are a lot of subtle themes and distinctions that have thus far been completely ignored by everyone, as far as I can tell.

Problem 1: Torture versus specks

Imagine you’re at a Less Wrong meetup when out of nowhere Eliezer Yudkowsky proposes his torture versus dust specks problem. Years of bullet-biting make this a trivial dilemma for any good philosopher, but suddenly you have a seizure during which you vividly recall all of those history lessons where you learned about the horrible things people do when they feel justified in being blatantly evil because of some abstract moral theory that is at best an approximation of sane morality and at worst an obviously anti-epistemic spiral of moral rationalization. Temporarily humbled, you decide to think about the problem a little longer:

"Considering I am deciding the fate of 3^^^3+1 people, I should perhaps not immediately assert my speculative and controversial meta-ethics. Instead, perhaps I should use the averaged meta-ethics of the 3^^^3+1 people I am deciding for, since it is probable that they have preferences that implicitly cover edge cases such as this, and disregarding the meta-ethical preferences of 3^^^3+1 people is certainly one of the most blatantly immoral things one can do. After all, even if they never learn anything about this decision taking place, people are allowed to have preferences about it. But... that the majority of people believe something doesn’t make it right, and that the majority of people prefer something doesn’t make it right either. If I expect that these 3^^^3+1 people are mostly wrong about morality and would not reflectively endorse their implicit preferences being used in this decision instead of my explicitly reasoned and reflected upon preferences, then I should just go with mine, even if I am knowingly arrogantly blatantly disregarding the current preferences of 3^^^3 currently-alive-and-and-not-just-hypoth...
Lesswrongers from the German-speaking world, unite! 
by David Althaus — 2011-05-19T20:32:02.370Z

As far as I can tell, there never has been a LW-meetup in a german-speaking country. This is crazy!

There should be enough Lesswrongers from Germany, Austria or Switzerland to achieve a reasonable group size!

To be a bit more specific, how about meeting in Munich? It's relatively in the middle of the three countries, but it's fine if you propose another city!

I'm not sure about the date, but how about sometime in July or August? We would have plenty of time until then. But I'm happy if you propose another date!

I don't know if I'm merely shouting in the void, but I hope some aspiring rationalists are out there!

So If you're interested in such a meeting, please make a comment!

Added: Oh, and if you are not from a German-speaking country you are welcome, too!
Ottawa LW meetup, May 26, 7pm; Bayes study group, May 26, 9am
by Cyan — 2011-05-19T19:43:58.873Z

Less Wrong meeting:

Date: Thursday May 26, 7:00pm 'til whenever.

Venue: Bridgehead, 282 Elgin St. (corner at MacLaren).

Bayes study group: Anyone in the region interested in learning how to do Bayesian statistics is welcome to join us. We'll be using the statistical package R (http://cran.r-project.org/) as a platform, so bring your laptop if you have one.

Date: Thursday May 26, 9:00am to 10:30am. NB: 9 in the morning.

Venue: Jeanne Mance Building, Tunney's Pasture. Meet me in the lobby.
The Advantages of Being Technical
by XiXiDu — 2011-05-19T19:39:40.170Z

Someone told me that each equation I included in the book would halve the sales. I therefore resolved not to have any equations at all.

— Stephen Hawking, in his book A Brief History of Time

Whoever gave this advice to Mr. Hawking is probably partly responsible for the success, financial and otherwise, of his book. But as Paul K. Feyerabend said: "All methodologies have their limitations and the only ‘rule’ that survives is ‘anything goes’."

Open access online communities interested in high quality and productivity can benefit from a high degree of technical formality, including large amounts of mathematics.

Troll Filter

When confronted with an economic problem, first translate into mathematics, then solve the problem, then translate back into English and burn the mathematics.

— Alfred Marshall

Another valuable advice in and of itself. But expressing technical issues in a natural language can be a mixed blessing.

Everyone can understand natural language, or so they think. Often any understanding solely arrived at by natural language is vague and can lead people to mistakenly believe that they understand the issue in question sufficiently.

A little bit of knowledge is a dangerous thing. It can convince you that an argument this idiotic and this sloppy is actually profound. It can convince you to publicly make a raging jackass out of yourself, by rambling on and on, based on a stupid misunderstanding of a simplified, informal, intuitive description of something complex.

— Mark Chu-Carroll, The Danger When You Don’t Know What You Don’t Know

The introduction of formal language and a focus on hard problems filters out a large subgroup of people who might otherwise believe that they are eligible for membership, able contribute something useful or that the community is interested in their critique.

Less Vague

More generally, many of the objects demonstrated to be impossible in the previous posts in this series can appear possible as long as there is enough vagueness. For instance, one can certainly imagine an omnipotent being provided that there is enough vagueness in the concept of what “omnipotence” means; but if one tries to nail this concept down precisely, one gets hit by the omnipotence paradox. Similarly, one can imagine a foolproof strategy for beating the stock market (or some

...
[LINKs] Bitcoin hits mainstream; intelligent technical critique
by SilasBarta — 2011-05-19T18:30:35.402Z

Annie Lowrey discusses Bitcoin in Slate. No clear thesis, but important that it gets attention there. She gives a general overview, with emphasis on its benefits to fringe elements on society, and gives quick attack at the end. The attack seems misinformed, but it links to something more interesting, specifically...

A technical critique by Victor Grishchenko, PhD, who was mentioned here in the context of causal trees. He describes a few problems he sees with Bitcoin:

1) Asymmetry favors attackers, in that it takes a lot more effort to check for double spending than to attempt a double-spend, eventually requiring "supernodes" that have disproportionate influence over the network.

2) It needs to continuously spend spend cycles to stay free from attackers. He then describes an attack I don't quite understand that involves holding on to a discovered block and then broadcasting it at just the right time

3) It doesn't compare well against existing systems in terms of privacy, speed, or transaction cost. (I found this questionable because the system he's comparing it to is still subject to warrants, and Bitcoin takes significantly less time -- 1 hour or so -- to ensure a transaction than the wiring transfers Grishchenko discribes.)

Finally, he credits Bitcoin in being advantageous similarly to Bittorrent: the latter was clumsy and complicated compared to regular downloading, but could perform well enough in a niche niche to force change in the broader markets.
The Aliens have Landed!
by TimFreeman — 2011-05-19T17:09:16.761Z

"General Thud! General Thud! Wake up! The aliens have landed. We must surrender!" General Thud's assistant Fred turned on the lights and opened the curtains to help Thud wake up and confront the situation. Thud was groggy because he had stayed up late supervising an ultimately successful mission carried out by remotely piloted vehicles in some small country on the other side of the world. Thud mumbled, "Aliens? How many? Where are they? What are they doing?" General Thud looked out the window, expecting to see giant tripods walking around and destroying buildings with death rays. He saw his lawn, a bright blue sky, and hummingbirds hovering near his bird feeder.

Fred was trying to bring Thud up to speed as quickly as possible. "Thousands of them, General! 2376, to be precise. They gave us a map; we know where they all are. They aren't doing anything overt, but the problem is their computation! I have one here, if you'd like to look." Fred removed a black sphere two inches in diameter from his pocket and gave it to Thud.

Thud sat on his bed holding the small sphere and staring at it dumbfounded. "Okay, you think we should surrender to a few thousand small spheres. Why is that, exactly?" The sphere seemed a little flexible in Thud's hand. As he experimented a few seconds to see just how flexible, it collapsed in his hand, converting itself into a loose clump of alien sand that landed in his lap and started to dribble onto his bed and the floor. Thud stood up and brushed the rest of the sand off of his pyjamas and bed, and thought for a moment about where he left his vacuum cleaner bags. He was not impressed with these aliens.

Fred said "I don't think you wanted to do that, sir. Their ultimatum states that for every alien we destroy, they'll manufacture two in the outer reaches of the Solar System where we'll never find them!"

Thud said, "Okay, so now you think we should surrender to 2375 small spheres, and two or more small spheres that are out of the battlefield for the moment. Why is that?"

Fred said "Well, you remember a few years back when some people copied their brain state into a computer and posted it to the Internet? Apparently somebody copied the data across an unencrypted wireless link, the aliens picked it up with their radio telescopes, and now they are simulating those poor people in these black spheres and torturing the simulations! They sent us vide...
How to (un)become a crank
by cousin_it — 2011-05-19T17:03:29.787Z

Ahhh, a human interest post. Well, sort of. At least it has something besides math-talk.

In the extreme programming community they have a saying, "three strikes and you refactor". The rationalist counterpart would be this: once you've noticed the same trap twice, you'd be stupid to fall prey to it the third time.

Strike one is Eliezer's post The Crackpot Offer. Child-Eliezer thought he'd overthrown Cantor's theorem, then found an error in his reasoning, but felt a little tempted to keep on trying to overthrow the damned theorem anyway. The right and Bayesian thing to do, which he ended up doing, was to notice that once you've found your mistake there's no longer any reason to wage war on an established result.

Strike two is Emile's comment on one of my recent posts:

I find it annoying how my brain keeps saying "hah, I bet I could" even though I explained to it that it's mathematically provable that such an input always exists. It still keeps coming up with "how about this clever encoding?, blablabla" ... I guess that's how you get cranks.

Strike three is... I'm a bit ashamed to say that...

...strike three is about me. And maybe not only me.

There's a certain vibe in the air surrounding many discussions of decision theory. It sings: maybe the central insight of game theory (that multiplayer situations are not reducible to single-player ones) is wrong. Maybe the slightly-asymmetrized Prisoner's Dilemma has a single right answer. Maybe you can get a unique solution to dividing a cake by majority vote if each individual player's reasoning is "correct enough". But honestly, where exactly is the Bayesian evidence that merits anticipating success on that path? Am I waging war on clear and simple established results because of wishful thinking? Are my efforts the moral equivalent of counting the reals or proving the consistency of PA within PA?

An easy answer is that "we don't know" if our inquiries will be fruitful, so you can't prove I must stop. But that's not the Bayesian answer. The Bayesian answer is to honestly tally up the indications that future success is likely, and stop if they are lacking.

So I want to ask an object-level question and a meta-level question:

1) What evidence supports the intuition that, contra game theory, single-player decision theory has a "solution"?

2) If there's not much evidence supporting that intuition, how should I change my ac...
On the Anthropic Trilemma
by KatjaGrace — 2011-05-19T15:57:04.000Z

Eliezer’s Anthropic Trilemma:

So here’s a simple algorithm for winning the lottery: Buy a ticket. Suspend your computer program just before the lottery drawing – which should of course be a quantum lottery, so that every ticket wins somewhere. Program your computational environment to, if you win, make a trillion copies of yourself, and wake them up for ten seconds, long enough to experience winning the lottery. Then suspend the programs, merge them again, and start the result. If you don’t win the lottery, then just wake up automatically. The odds of winning the lottery are ordinarily a billion to one. But now the branch in which you win has your “measure”, your “amount of experience”, temporarily multiplied by a trillion. So with the brief expenditure of a little extra computing power, you can subjectively win the lottery – be reasonably sure that when next you open your eyes, you will see a computer screen flashing “You won!” As for what happens ten seconds after that, you have no way of knowing how many processors you run on, so you shouldn’t feel a thing.

See the original post for assumptions, what merging minds entails etc. He proposes three alternative bullets to bite: accepting that this would work, denying that there is “any meaningful sense in which I can anticipate waking up as myself tomorrow, rather than Britney Spears” so undermining any question about what you should anticipate, and Nick Bostrom’s response, paraphrased by Eliezer:

…you should anticipate winning the lottery after five seconds, but anticipate losing the lottery after fifteen seconds. To bite this bullet, you have to throw away the idea that your joint subjective probabilities are the product of your conditional subjective probabilities. If you win the lottery, the subjective probability of having still won the lottery, ten seconds later, is ~1. And if you lose the lottery, the subjective probability of having lost the lottery, ten seconds later, is ~1. But we don’t have p(“experience win after 15s”) = p(“experience win after 15s”|”experience win after 5s”)*p(“experience win after 5s”) + p(“experience win after 15s”|”experience not-win after 5s”)*p(“experience not-win after 5s”).

I think...
[SEQ RERUN] Tsuyoku vs. the Egalitarian Instinct
by badger — 2011-05-19T15:28:40.682Z

Today's post, Tsuyoku vs. the Egalitarian Instinct was originally published on March 28, 2007. A summary (from the LW wiki):

There may be evolutionary psychological factors that encourage modesty and mediocrity, at least in appearance; while some of that may still apply today, you should mentally plan and strive to pull ahead, if you are doing things right.

This post is part of a series rerunning Eliezer Yudkowsky's old posts so those interested can (re-)read and discuss them. The previous post was Tsuyoku Naritai! (I Want To Become Stronger), and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to discuss the Rerunning the Sequences series.
Example decision theory problem: "Agent simulates predictor"
by cousin_it — 2011-05-19T15:16:29.319Z

Some people on LW have expressed interest in what's happening on the decision-theory-workshop mailing list. Here's an example of the kind of work we're trying to do there.

In April 2010 Gary Drescher proposed the "Agent simulates predictor" problem, or ASP, that shows how agents with lots of computational power sometimes fare worse than agents with limited resources. I'm posting it here with his permission:

There's a version of Newcomb's Problem that poses the same sort of challenge to UDT that comes up in some multi-agent/game-theoretic scenarios.

Suppose:

The predictor does not run a detailed simulation of the agent, but relies instead on a high-level understanding of the agent's decision theory and computational power.

The agent runs UDT, and has the ability to fully simulate the predictor.

Since the agent can deduce (by low-level simulation) what the predictor will do, the agent does not regard the prediction outcome as contingent on the agent's computation. Instead, either predict-onebox or predict-twobox has a probability of 1 (since one or the other of those is deducible), and a probability of 1 remains the same regardless of what we condition on. The agent will then calculate greater utility for two-boxing than for one-boxing.

Meanwhile, the predictor, knowing that the the agent runs UDT and will fully simulate the predictor, can reason as in the preceding paragraph, and thus deduce that the agent will two-box. So the large box is left empty and the agent two-boxes (and the agent's detailed simulation of the predictor correctly shows the predictor correctly predicting two-boxing).

The agent would be better off, though, running a different decision theory that does not two-box here, and that the predictor can deduce does not two-box.

About a month ago I came up with a way to formalize the problem, along the lines of my other formalizations:

a) The agent generates all proofs of length up to M, then picks the action for which the greatest utility was proven.

b) The predictor generates all proofs of length up to N which is much less than M. If it finds a provable prediction about the agent's action, it fills the boxes accordingly. Also the predictor has an "epistemic advantage" over the agent: its proof system has an axiom saying the agent's proof system is consistent.

Now the predictor can reason as follows. It knows that the agent will find some proof t

...
Greg Egan and the Incomprehensible
by XiXiDu — 2011-05-19T10:38:32.600Z

In this post I question one disagreement between Eliezer Yudkowsky and science fiction author Greg Egan.

In his post Complex Novelty, Eliezer Yudkowsky wrote in 2008:

Note that Greg Egan seems to explicitly believe the reverse - that humans can understand anything understandable - which explains a lot.

An interview with Greg Egan in 2009 confirmed this to be true:

… I think there’s a limit to this process of Copernican dethronement: I believe that humans have already crossed a threshold that, in a certain sense, puts us on an equal footing with any other being who has mastered abstract reasoning. There’s a notion in computing science of “Turing completeness”, which says that once a computer can perform a set of quite basic operations, it can be programmed to do absolutely any calculation that any other computer can do. Other computers might be faster, or have more memory, or have multiple processors running at the same time, but my 1988 Amiga 500 really could be programmed to do anything my 2008 iMac can do — apart from responding to external events in real time — if only I had the patience to sit and swap floppy disks all day long. I suspect that something broadly similar applies to minds and the class of things they can understand: other beings might think faster than us, or have easy access to a greater store of facts, but underlying both mental processes will be the same basic set of general-purpose tools. So if we ever did encounter those billion-year-old aliens, I’m sure they’d have plenty to tell us that we didn’t yet know — but given enough patience, and a very large notebook, I believe we’d still be able to come to grips with whatever they had to say.

The theoretical computer scientist Scott Aaronson wrote in a post titled 'The Singularity Is Far':

The one notion I have real trouble with is that the AI-beings of the future would be no more comprehensible to us than we are to dogs (or mice, or fish, or snails). After all, we might similarly expect that there should be models of computation as far beyond Turing machines as Turing machines are beyond finite automata. But in the latter case, we know the intuition is mistaken. There is a ceiling to computational expressive power. Get up to a certain threshold, and every machine can simulate every other on

...
What bothers you about Less Wrong?
by Will_Newsome — 2011-05-19T10:23:01.201Z

Or, what do you want to see more or less of from Less Wrong?

I'm thinking about community norms, content and topics discussed, karma voting patterns, et cetera. There are already posts and comment sections filled with long lists of proposed technical software changes/additions, let's not make this post another one.

My impression is that people sometimes make discussion posts about things that bother them, and sometimes a bunch of people will agree and sometimes a bunch of people will disagree, but most people don't care that much (or they have a life or something) and thus don't want to dedicate a post just to complaining. This post is meant to make it socially and cognitively easy to offer critique.

I humbly request that you list downsides of existing policies even when you think the upsides outweigh them, for all the obvious reasons. I also humbly request that you list a critique/gripe even if you don't want to bother explaining why you have that critique/gripe, and even in cases where you think your gripe is, ahem, "irrational". In general, I think it'd be really cool if we erred on the side of listing things which might be problems even if there's no obvious solution or no real cause for complaint except for personal distaste for the color green (for example).

I arrogantly request that we try to avoid impulsive downvoting and non-niceness for the duration of this post (and others like it). If someone wants to complain that Less Wrong is a little cultish without explaining why then downvoting them to oblivion, while admittedly kind of funny, is probably a bad idea. :)
Should I be afraid of GMOs?
by Pavitra — 2011-05-19T01:29:09.128Z

I was raised to believe that genetically-modified foods are unhealthy to eat and bad for the environment, and given a variety of reasons for this, some of which I now recognize as blatantly false (e.g., human genetic code is isomorphic to fundamental physical law), and a few of which still seem sort of plausible.

Because of this history, I need to anchor my credence heavily downward from my sense of plausibility.

The major reasons I see to believe that GMOs are safe are:

I would probably think they were dangerous even if they were safe, due to my upbringing.

In general, whenever someone opposes a particular field of engineering on the grounds that it's unnatural and dangerous, they're usually wrong.

It's not quite obvious to me that introducing genetically-engineered organisms to a system is significantly more dangerous than introducing non-native naturally-evolved organisms.

The major reason I see to believe that GMOs are dangerous is:

I might believe they were safe even if they were dangerous, due to "yay science" (which was also part of my upbringing).

We are designing self-replicating things and using them without reliable containment, thereby effectively releasing them into the wild.

So: green goo, yes or no?