trentmkelly/LessWrong-43k huggingface dataset:
Mini advent calendar of Xrisks: Pandemics
by Stuart_Armstrong — 2012-12-06T13:44:22.443Z

The FHI's mini advent calendar: counting down through the big five existential risks. The fourth one is an ancient risk, still with us today: pandemics and plagues.

Pandemics

Current understanding: high
Most worrying aspect: the past evidence points to a risky future

The deathrates from infectious diseases follow a power law with a very low exponent. In layman’s terms: there is a reasonable possibility for a plague with an absolutely huge casualty rate. We’ve had close calls in the past: the black death killed around half the population of Europe, while Spanish Influenza infected 27% of all humans and killed one in ten of those, mostly healthy young adults. All the characteristics of an ultimately deadly infection already exist in the wild: anything that combined the deadliness and incubation period of AIDS with the transmissibility of the common cold.

Moreover, we know that we are going to be seeing new diseases and new infections in the future: the only question is how deadly they will be. With modern global travel and transport, these diseases will spread far and wide. Against this, we have better communication and better trans-national institutions and cooperation – but these institutions could easily be overwhelmed, and countries aren’t nearly as well prepared as they need to be.
A solvable Newcomb-like problem - part 3 of 3
by Douglas_Reay — 2012-12-06T13:06:24.638Z

This is the third part of a three post sequence on a problem that is similar to Newcomb's problem but is posed in terms of probabilities and limited knowledge.

Part 1 - stating the problem
Part 2 - some mathematics
Part 3 - towards a solution

In many situations we can say "For practical purposes a probability of 0.9999999999999999999 is close enough to 1 that for the sake of simplicity I shall treat it as being 1, without that simplification altering my choices."

However, there are some situations where the distinction does significantly alter that character of a situation so, when one is studying a new situation and one is not sure yet which of those two categories the situations falls into, the cautious approach is to re-frame the probability as being (1 - δ) where δ is small (eg 10 to the power of -12), and then examine the characteristics of the behaviour as δ tends towards 0.

LessWrong wiki describes Omega as a super-powerful AI analogous to Laplace's demon, who knows the precise location and momentum of every atom in the universe, limited only by the laws of physics (so, if time travel isn't possible and some of our current thoughts on Quantum Mechanics are correct, then Omega's knowledge of the future is probabilistic, being limited by uncertainty).

For the purposes of Newcomb's problem, and the rationality of Fred's decisions, it doesn't matter how close to that level of power Omega actually is. What matters, in terms of rationality, is the evidence available to Fred about how close Omega is to having to that level of power; or, more precisely, the evidence available to Fred relevant to Fred making predictions about Omega's performance in this particular game.

Since this is a key factor in Fred's decision, we ought to be cautious. Rather than specify when setting up the problem that Fred knows with a certainty of 1 that Omega does have that power, it is better to specify a concrete level of evidence that would lead Fred to assign a probability of (1 - δ) to Omega having that power, then examine the effect upon which option to the box problem it is rational for Fred to pick, as δ tends towards 0.

The Newcomb-like problem stated in part 1 of this sequence contains an Omega that it is rational for Fred to assign a less than unity probability of being able to perf...
[SEQ RERUN] Hard Takeoff
by MinibearRex — 2012-12-06T06:49:24.373Z

Today's post, Hard Takeoff was originally published on 02 December 2008. A summary (taken from the LW wiki):

It seems likely that there will be a discontinuity in the process of AI self-improvement around the time when AIs become capable of doing AI theory. A lot of things have to go exactly right in order to get a slow takeoff, and there is no particular reason to expect them all to happen that way.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Whither Manufacturing?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
A Probability Question
by JMiller — 2012-12-06T05:29:22.028Z

Hi, I am relatively new to this site, I am not sure if this is the right place to be posting.

I am sure many of you are familiar with the following probability riddle:

"Sarah is walking along the street when she encounters a man. With the man is his son. He tells Sarah that he has only one more child at home. She is asked, 'what is the probability that my child is a girl?'"

Since Sarah does not know whether the boy is the elder or younger sibling, she needs to take four possible states into account. The father either had:

1) a boy, then a girl

2) a girl, then a boy

3) two girls

4) two boys

Since 3 is impossible (Sarah knows there is at least one boy) that leaves three options. Two of those options imply a girl, the other implies a boy. Therefore, she can conclude that her probability estimate must be that it is 66.6% likely that there is a girl at home, and 33.3% likely that there is a boy.

Compare this to George's situation.

"George is walking along the street when he encounters a man. With the man is his son. He tells George that the boy with him is his oldest son, and that he has only one more child at home. He is asked, 'What is the probability that my child at home is a girl?'"

George's probability estimate is clear: either the man had a boy then a girl, or he had two boys. Therefore, it is 50% likely that the child at home is a girl.

My problem is this: I understand probability exists in the mind. The actual answer to the question is 100% one way or the other. Still, it seems like Sarah knows more about the situation, where George, by being given more information, knows less. His estimate is as good as knowing nothing other than the fact that the man has a child which could be equally likely to be a boy or a girl.

If the reply is something like "Well, Sarah actually knows less so her estimate is less likely to be right" then that is something she could have figured out on her own, and then realized that assigning probability .5 is best anyways. That seems wrong.

I know I must be making a mistake somewhere: why does it seem like George learns less by knowing more?

Thank you for your help.
Meetup : Less Wrong: Cleveland
by Raelifin — 2012-12-06T03:27:57.314Z

Discussion article for the meetup : Less Wrong: Cleveland

WHEN:
09 December 2012 03:00:23PM (-0500)

WHERE:
Cleveland

Four of us will be meeting on the 9th. If you're interested and in the area, send an email!

Discussion group: https://groups.google.com/forum/?fromgroups#!forum/less-wrong-cleveland

Location to be decided.

Discussion article for the meetup : Less Wrong: Cleveland
[Link] Eleven dogmas of analytic philosophy
by crazy88 — 2012-12-06T03:27:04.550Z

Closely related to some of Luke's recent discussions about philosophy, philosopher Paul Thagard has recently called for changes to the way we do philosophy:

I prefer an alternative approach to philosophy that is much more closely tied to scientific investigations. This approach is sometimes called “naturalistic philosophy” or “philosophy naturalized”, but I like the more concise term natural philosophy. Before the words “science” and “scientist” became common in the nineteenth century, researchers such as Newton described what they did as natural philosophy. I propose to revive this term to cover a method that ties epistemology and ethics closely to the cognitive sciences, and ties metaphysics closely to physics and other sciences.

In the same article, Thagard also lists eleven areas where modern philosophy goes awry. For example:

3. People’s intuitions are evidence for philosophical conclusions. Natural alternative: evaluate intuitions critically to determine their psychological causes, which are often more tied to prejudices and errors than truth. Don't trust your intuitions.

Source: Philosopher, Paul Thagard
Train Philosophers with Pearl and Kahneman, not Plato and Kant
by lukeprog — 2012-12-06T00:42:47.643Z

Part of the sequence: Rationality and Philosophy

Hitherto the people attracted to philosophy have been mostly those who loved the big generalizations, which were all wrong, so that few people with exact minds have taken up the subject.

Bertrand Russell

I've complained before that philosophy is a diseased discipline which spends far too much of its time debating definitions, ignoring relevant scientific results, and endlessly re-interpreting old dead guys who didn't know the slightest bit of 20th century science. Is that still the case?

You bet. There's some good philosophy out there, but much of it is bad enough to make CMU philosopher Clark Glymour suggest that on tight university budgets, philosophy departments could be defunded unless their work is useful to (cited by) scientists and engineers — just as his own work on causal Bayes nets is now widely used in artificial intelligence and other fields.

How did philosophy get this way? Russell's hypothesis is not too shabby. Check the syllabi of the undergraduate "intro to philosophy" classes at the world's top 5 U.S. philosophy departments — NYU, Rutgers, Princeton, Michigan Ann Arbor, and Harvard — and you'll find that they spend a lot of time with (1) old dead guys who were wrong about almost everything because they knew nothing of modern logic, probability theory, or science, and with (2) 20th century philosophers who were way too enamored with cogsci-ignorant armchair philosophy. (I say more about the reasons for philosophy's degenerate state here.)

As the CEO of a philosophy/math/compsci research institute, I think many philosophical problems are important. But the field of philosophy doesn't seem to be very good at answering them. What can we do?

Why, come up with better philosophical methods, of course!

Scientific methods have improved over time, and so can philosophical methods. Here is the first of my recommendations...

More Pearl and Kahneman, less Plato and Kant

Philosophical training should begin with the latest and greatest formal methods ("Pearl" for the probabilistic graphical models made famous in Pearl 1988), and the latest and greatest science ("Kahneman" for the science of human reasoning reviewed in Kahneman 2011). Beginning with Plato and Kant (and company), as most universities do today, both (1) filters for inexact thinkers, as Russell suggested, and...
Meetup : 16/12 London Meetup
by philh — 2012-12-05T23:51:51.774Z

Discussion article for the meetup : 16/12 London Meetup

WHEN:
16 December 2012 02:00:00PM (+0000)

WHERE:
Holborn, London

A meetup in the Shakespeare's Head pub by Holborn tube station. Everyone is welcome.

We also have a Google group.

Discussion article for the meetup : 16/12 London Meetup
AI "Boxing" and Utility Functions
by nigerweiss — 2012-12-05T23:44:21.496Z

So, I had this idea the other day when I was thinking about how to safely conduct research on potentially-FOOM-capable AI software. I'd like to sketch it out briefly and then get feedback on it.

So, this started out with the idea that an AI based on AIXI is, in some sense, safer than a fully functional AI, due to the existence of the anvil problem. Because AIXI can't conceive of its own nonexistence, it has no preference ordering over its own mortality, and won't (shouldn't) resist any attempt to shut it down. In other words, if AIXI starts to FOOM undesirably out of control, you actually can go pull the plug on it without fuss. Unfortunately, in terms of safety, the anvil problem gives AIXI a number of other undesirable properties: both third parties and the AI itself can modify its utility function at any time, for any reason, which is very unstable. However, I think a similar idea might be useful for reducing (though not eliminating) the existential risks posed by powerful optimization processes in the near term.

Say you have a piece of AI software ω, with an unFriendly instantaneous utility function, {maximize U}. You would like to use ω for some industrial application (say, manufacturing paperclips), but you're concerned about it FOOMing and resulting in human extinction. You decide to 'box' the AI, but, having read up on the subject, you are worried about it outsmarting you if you try to disable it.

So, you replace your original utility function U with a modified version in terms of U, U'.
U' = { max(U) | if ω exists

Ø | if ω !exist}

U' has several useful qualities. The agent will resist modifications to its utility function, while not resisting attempts to turn it off. It is entirely ambivalent towards its own existence. As a result, if it began to FOOM undesirably, stopping it would be fairly trivial. Furthermore, the AI would have no incentive to deceive us, so it'd be fairly easy to keep an eye on.

It should be noted that this is not in any way a final solution to the FAI problem. If the AI FOOMS quickly enough that we all die or get brain-hacked before we realize something's amiss, it could still all go very poorly. ...
Meetup : Berkeley: Boardgames
by Nisan — 2012-12-05T18:28:09.814Z

Discussion article for the meetup : Berkeley: Boardgames

WHEN:
05 December 2012 07:00:00PM (-0800)

WHERE:
Berkeley, CA

There will be a meetup tonight at Zendo as usual. I will not be around but Alex Mennen will be hosting. This will be a boardgames meetup. Come hang out and play a game from Zendo's game library, or bring a game of your own if you like.
For directions to Zendo, see the mailing list: http://groups.google.com/group/bayarealesswrong
or call me at:
http://i.imgur.com/Vcafy.png

Discussion article for the meetup : Berkeley: Boardgames
Absent Transhumanism and Transformative Technologies, Which Utopia is Left?
by diegocaleiro — 2012-12-05T12:54:30.200Z

Assume for the time being that it will forever remain beyond the scope of science to change Human Nature. AGI is also impossible, as is Nanotech, BioImmortality, and those things.

Douglas Adams mice finished their human experiment, giving to you, personally, the job of redesigning earth, and specially human society, according to your wildest utopian dreams, but you can't change the unchangeables above.

You can play with architecture, engineering, gender ratio, clothing, money, science grants, governments, feeding rituals, family constitution, the constitution itself, education, etc... Just don't forget if you slide something too far away from what our evolved brains were designed to accept, things may slide back, or instability and catastrophe may ensue.

Finally, if you are not the kind of utilitarian that assigns exactly the same amount of importance to your desires, and to that of others, I want you to create this Utopia for yourself, and your values, not everyone.

The point of this exercise is: The vast majority of folk not related to this community that I know, when asked about an ideal world, will not change human nature, or animal suffering, or things like that, they'll think about changing whatever the newspaper editors have been writing about last few weeks. I am wondering if there is symmetry here, and folks from this community here do not spend that much time thinking about those kinds of change which don't rely on transformative technologies. It is just an intuition pump, a gedankenexperiment if you will. Force your brain to face this counterfactual reality, and make the best world you can given those constraints. Maybe, if sufficiently many post here, the results might clarify something about CEV, or the sociology of LessWrongers...
Mini advent calendar of Xrisks: nanotechnology
by Stuart_Armstrong — 2012-12-05T11:02:18.885Z

The FHI's mini advent calendar: counting down through the big five existential risks. The third one is a also a novel risk: nanotechnology.

Nanotechnology

Current understanding: low
Most worrying aspect: the good stuff and the bad stuff are the same thing

The potential of nanotechnology is its ability to completely transform and revolutionise manufacturing and materials. The peril of nanotechnology is its ability to completely transform and revolutionise manufacturing and materials. And it’s hard to separate the two. Nanotech manufacturing promises to be extremely disruptive to existing trade arrangements and to the balance of economic power: small organisations could produce as many goods as much as whole countries today, collapsing standard trade relationships and causing sudden unemployment and poverty in places not expecting this.
And in this suddenly unstable world, nanotechnology will also permit the mass production of many new tools of war – from microscopic spy drones to large scale weapons with exotic properties. It will also weaken trust in disarmament agreements, as a completely disarmed country would have the potential to assemble an entire arsenal – say of cruise missiles – in the span of a day or less.
[SEQ RERUN] Whither Manufacturing?
by MinibearRex — 2012-12-05T05:10:23.157Z

Today's post, Whither Manufacturing? was originally published on December 2, 2008. A summary:

There's no general reason to suppose that nanotechnology will enable a boom in local production. The location of production is a trade off between economies and dis-economies of scale.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Recursive Self-Improvement, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Washington DC meetup- new sequence?
by rocurley — 2012-12-05T04:55:51.765Z

Discussion article for the meetup : Washington DC meetup- new sequence?

WHEN:
09 December 2012 11:55:42PM (-0500)

WHERE:
National Portrait Gallery Plaza, Washington, DC 20001, USA

We didn't pick out a meetup topic, so I'm going to propose we discuss the currently-in progress sequence.

Discussion article for the meetup : Washington DC meetup- new sequence?
How to incentivize LW wiki edits?
by lukeprog — 2012-12-05T04:39:36.394Z

How can we incentivize more productive activity on the LW wiki? There are many articles that could be created or expanded.

Are there previous discussions on this?

One suggestion: We could add a "good explanation!" button at the bottom of each article, and every time it is clicked, the user account responsible for the plurality of words in the current draft of the article gets 10 karma points. This requires that LW and LW-wiki accounts be synced, first.

How can this suggestion be improved?

What other suggestions do people have?
Mixed Reference: The Great Reductionist Project
by Eliezer Yudkowsky — 2012-12-05T00:26:24.471Z

Followup to: Logical Pinpointing, Causal Reference

Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.

- Death, in Hogfather by Terry Pratchett

Meditation: So far we've talked about two kinds of meaningfulness and two ways that sentences can refer; a way of comparing to physical things found by following pinned-down causal links, and logical validity by comparison to models pinned-down by axioms. Is there anything else that can be meaningfully talked about? Where would you find justice, or mercy?

...
...
...

Suppose that I pointed at a couple of piles of apples on a table, a pile of two apples and a pile of three apples.

And lo, I said: "If we took the number of apples in each pile, and multiplied those numbers together, we'd get six."

Nowhere in the physical universe is that 'six' written - there's nowhere in the laws of physics where you'll find a floating six. Even on the table itself there's only five apples, and apples aren't fundamental. Or to put it another way:

Take the apples and grind them down to the finest powder and sieve them through the finest sieve and then show me one atom of sixness, one molecule of multiplication.

Nor can the statement be true as a matter of pure math, comparing to some Platonic six within a mathematical model, because we could physically take one apple off the table and make the statement false, and you can't do that with math.

This question doesn't feel like it should be very hard. And indeed the answer is not very difficult, but it is worth spelling out; because cases like "justice" or "mercy" will turn out to proceed in a similar fashion.

Navigating to the six requires a mixture of physical and logical reference. This case begins with a physical reference, when we navigate to the physical apples on the table by talking about the cause of our apple-seeing experiences:

Next we have to call the stuff on the table 'apples'. But how, oh how can we do this, when grinding the universe and running it through a sieve will reveal not a single particle of appleness?

This part was covered at some length in the Reductionism sequence. Standard physics uses the same fundamental theory to describe the flight of a Bo...