trentmkelly/LessWrong-43k huggingface dataset:
Probabilistic Löb theorem
by Stuart_Armstrong — 2013-04-26T18:45:49.944Z

In this post (based on results from MIRI's recent workshop), I'll be looking at whether reflective theories of logical uncertainty (such as Paul's design) still suffer from Löb's theorem.

Theories of logical uncertainty are theories which can assign probability to logical statements. Reflective theories are theories which know something about themselves within themselves. In Paul's theory, there is an external P, in the meta language, which assigns probabilities to statements, an internal P, inside the theory, that computes probabilities of coded versions of the statements inside the language, and a reflection principle that relates these two P's to each other.

And Löb's theorem is the result that if a (sufficiently complex, classical) system can prove that "a proof of Q implies Q" (often abbreviated as □Q → Q), then it can prove Q. What would be the probabilistic analogue? Let's use □aQ to mean P('Q')≥1-a (so that □0Q is the same as the old □Q; see this post on why we can interchange probabilistic and provability notions). Then Löb's theorem in a probabilistic setting could:

Probabilistic Löb's theorem: for all a<1, if the system can prove □aQ → Q, then the system can prove Q.

To understand this condition, we'll go through the proof of Löb's theorem in a probabilistic setting, and see if and when it breaks down. We'll conclude with an example to show that any decent reflective probability theory has to violate this theorem.

Löb's proof fails

First, the derivation conditions. Using ⊢ to designate the system can prove/the system can show with probability 1 (note that ⊢ is the meta-language equivalent of □), then the classical derivation conditions are:

If ⊢ A, then ⊢ □A.

⊢ □A → □□A.

⊢ □(A → B) → (□A → □B).

Informally, these state that "if the system can prove something, it can prove it can prove it", "the system can prove that a proof of something implies a proof of a proof of something" and "the system can prove that proofs obey modus ponens".

What are the equivalent statements for probabilistic theories? According to Paul's approach, one must always add small epsilons when talking about probabilities. So the following derivation principles seem reasonable, where the latin indexes (a,b,c...) are meant to represent numbers that...
Logic in the language of probability
by Stuart_Armstrong — 2013-04-26T18:45:44.504Z

This post is a minor note, to go along with the post on the probabilistic Löb theorem. It simply seeks to justify why terms like "having probability 1" are used interchangeably with "provable" and why implications symbols "→" can be used in a probabilistic setting.

Take a system of classical logic, with a single rule of inference: modus ponens:

From A and A→B, deduce B.

Having a single rule of inference isn't much of a restriction, because you can replace other rules of inference ("from A1,A2,... and An, deduce B") with an axiom or axiom schema ("A1∧A2∧...∧An → B") and then use modus ponens on that axiom to get the other rule of inference.

In this logical system, I'm now going to make some purely syntactical changes - not changing the meaning of anything, just the way we write things. For any sentence A that doesn't contain an implication arrow →, replace

A with P(A)=1.

Similarly, replace any sentence of the type

A → B with P(B|A)=1.

This is recursive, so we replace

(A → B) → C with P(C | P(B|A)=1 )=1.

And instead of using modus ponens, we'll use a combined Bayesian inference and law of total probability:

From P(A)=1 and P(B|A)=1, deduce P(B)=1.

Again, these are purely syntactic changes - just rewriting the symbols of a formal system in an entirely equivalent fashion. What this demonstrates, though, is that we can do classical logic using the language and the tools of probability - and hence that the objects and laws of classical logic have probabilistic counterparts.

In fact, we can do a bit better, make the similarity more pronounced. Instead of having modus ponens, allow for two "logical" rules of inference:

From A → B and A, deduce A∧B.

From A∧B, deduce B (simplification).

Together, these two rules give modus ponens, and if we've chosen our axioms in non-weird ways, we shouldn't be able to prove anything we couldn't before. Then when writing our logic in the language of probability, we similarly have two "probabilistic" rules of inference:

From P(B|A)=s and P(A)=t, deduce P(A∧B)=st (conditional probability).

From P(A∧B)=s, deduce P(B)≥s.

Here, we've sneakily introduced the possibility of probabilities not equal to 1 - even though they will never appear in our syntax, the rules allow for them. In order to show complete equivalence between the two sys...
Meetup : Washington DC books meetup
by rocurley — 2013-04-26T18:06:56.884Z

Discussion article for the meetup : Washington DC books meetup

WHEN:
28 April 2013 03:00:00PM (-0400)

WHERE:
National Portrait Gallery, Washington, DC 20001, USA (courtyard)

We'll be meeting to talk about books we've found interesting, exchange books (so bring some!) and decide if/how we want to have a book club.

(This was also the last meetup topic, which got cancelled at the last minute)

Discussion article for the meetup : Washington DC books meetup
Meetup : Bratislava lesswrong meetup III
by BarbaraB — 2013-04-26T08:55:44.759Z

Discussion article for the meetup : Bratislava lesswrong meetup III

WHEN:
20 May 2013 06:30:00PM (+0200)

WHERE:
Uršulínska 9, 811 01 Bratislava

We will see each other at Malewill café.

Breaking news !!!
The charming girl, Cat from USA, who teaches emotional stuff at CFAR rationality minicamps, will be visiting our meetup !
One more reason to attend this time :-)

My phone number is, as usual, ++421-915-896911.

Discussion article for the meetup : Bratislava lesswrong meetup III
Attempting to rescue logical positivism
by RolfAndreassen — 2013-04-25T18:20:19.602Z

Very brief recap: The logical positivists said "All truths are experimentally testable". Their critics responded: "If that's true, how did you experimentally test it? And if it's not true, who cares?" Which is a fair criticism. Logical positivism pretty much collapsed as a philosophical position. But it seems to me that a very slight rephrasing might have saved it: "All _beliefs_ are experimentally testable". For if the critic makes the same adjustment, asking "Is that a belief, and if so -" you can interrupt him and say, "No, that's not a belief, that's a definition of what it means to say 'I believe X'."

A definition is not true or false, it is useful or not useful. Why is this definition useful? Because it allows us to distinguish between two classes of declarative statements; the ones that are actual beliefs, and the ones that have the grammatical form of beliefs but are empty of meaningful belief-content.

It seems to me, then, that both the positivists and their critics fell into the trap of confusing 'belief' and 'truth', and that carefully making this distinction might have saved positivism from considerable undeserved mockery.
Real-world examples of money-pumping?
by sixes_and_sevens — 2013-04-25T13:49:51.144Z

Intransitive preferences are a demonstrable characteristic of human behaviour. So why am I having such trouble coming up with real-world examples of money-pumping?

"Because I'm not smart or imaginative enough" is a perfectly plausible answer, but I've been mulling this one over on-and-off for a few months now, and I haven't come up with a single example that really captures what I consider to be the salient features of the scenario: a tangled hierarchy of preferences, and exploitation of that tangled hierarchy by an agent who cyclically trades the objects in that hierarchy, generating trade surplus on each transaction.

It's possible that I am in fact thinking about money-pumping all wrong. All the nearly-but-not-quite examples I came up with (amongst which were bank overdraft fees, Weight Watchers, and exploitation of addiction) had the characteristics of looking like swindles or the result of personal failings, but from the inside, money-pumping must presumably feel like a series of gratifying transactions. We would want any cases of money-pumping we were vulnerable to.

At the moment, I have the following hypotheses for the poverty of real-world money-pumping cases:

Money-pumping is prohibitively difficult. The conditions that need to be met are too specific for an exploitative agent to find and abuse.

Money-pumping is possible, but the gains on each transaction are generally so small as to not be worth it.

Humans have faculties for identifying certain classes of strategy that exploit the limits of their rationality, and we tell any would-be money-pumper to piss right off, much like Pascal's Mugger. It may be possible to money-pump wasps or horses or something.

Humans have some other rationality boundary that makes them too stupid to be money-pumped, to the same effect as #3.

Money-pumping is prevalent in reality, but is not obvious because money-pumping agents generate their surplus in non-pecuniary abstract forms, such as labour, time, affection, attention, status, etc.

Money-pumping is prevalent in reality, but obfuscated by cognitive dissonance. We rationalise equivalent objects in a tangled preference hierarchy as being different.

Money-pumping is prevalent in reality, but obscured by cognitive phenomena such as time-preference and discounting, or underlying human aesthetic/moral tastes, (parochial equ

...
[LINK] How to calibrate your confidence intervals
by Benjamin_Todd — 2013-04-25T06:26:05.454Z

In the book "How to Measure Anything" D. Hubbard presents a step-by-step method for calibrating your confidence intervals, which he has tested on hundreds of people, showing that it can make 90% of people almost perfect estimators within half a day of training.

I've been told that the Less Wrong and CFAR community is mostly not aware of this work, so given the importance of making good estimates to rationality, I thought it would be of interest.

(although note CFAR has developed its own games for training confidence interval calibration)

The main techniques to employ are:

Equivalent bet:

For each estimate imagine that you are betting $1000 on the answer being within your 90% CI. Now compare this to betting $1000 on a spinner where 90% of the time you win and 10% of the time you lose. Would you prefer to take a spin? If so, your range is too small and you need to increase it. If you decide to answer the question your range is too large and you need to reduce it. If you don’t mind whether you answer the question or take a spin then it really is your 90% CI.

Absurdity Test:

Start with an absurdly large range, maybe from minus infinity to plus infinity, and then begin reducing it based upon things you know to be highly unlikely or even impossible.

Avoid Anchoring:

Anchoring occurs when you think of a single answer to the question and then add an error around this answer; this often leads to ranges which are too narrow. Using the absurdity test is a good way to counter problems brought on by anchoring; another is to change how you look at your 90% CI. For a 90% CI there is a 10% chance that the answer lies outside your estimate, and if you split this there is a 5% chance that the answer is above your upper bound and a 5% chance that the answer is below your lower bound. By treating each bound separately, rephrase the question to read ‘is there a 95% chance that the answer is above my lower bound?’. If the answer is no, then you need to increase or decrease the bound as required. You can then repeat this process for the other bound.

Pros and cons:

Identify two pros and two cons for the range that you have given to help clarify your reasons for making this estimate.

Once you have used these techniques you can make another equivalent bet to check whether your new estimate is your 90% CI.

To train yourself, practice maki...
Google's Executive Chairman Eric Schmidt: apparently a transhumanist
by chaosmage — 2013-04-25T00:36:41.935Z

It makes a lot of sense for the Google people to be transhumanist, with Sergey Brin and Larry Page working with the Singularity University, but still I was surprised to hear this on the new Colbert Report (of the 23rd of April):

Colbert: Can I live forever?
Schmidt: Yes.
Colbert: Really?
Schmidt: But not now. They need to invent some more medicine.
Colbert: So I can live forever, but later. So I just need to live long enough for later to become now.
Schmidt: But your digital identity will live forever. Because there's no delete button.
Colbert: On me?
Schmidt: That's correct.
Colbert: That's profound.

He seemed quite serious, too.

I guess a lot of people would take transhumanism more seriously if they heard the top people at Google are in. To me, I actually find it makes Google seem more trustworthy. In-group psychology is weird.

Here's another good interview with Eric Schmidt. No explicit transhumanism, but some fairly intense plans entirely compatible with it.

(edited: corrected title)
Tactics against Pascal's Mugging
by ArisKatsaris — 2013-04-25T00:07:52.499Z

This is meant as a rough collection of five ideas of mine on potential anti-Pascal Mugging tactics. I don't have much hope that the first three will be any useful at all and am afraid that I'm not mathematically-inclined enough to know if the last two are any good even as a partial solution towards the core problem of Pascal's Mugging -- so I'd appreciate if people with better mathematical credentials than mine could see if any of my intuitions could be formalizable in a useful manner.

0. Introducing the problem (this may bore you if you're aware of both the original and the mugger-less form of Pascal's Mugging)

First of all the basics: Pascal's Mugging in its original form is described in the following way:

Now suppose someone comes to me and says, "Give me five dollars, or I'll use my magic powers from outside the Matrix to run a Turing machine that simulates and kills 3^^^3 people."

This is the "shallow" form of Pascal's mugging, which includes a person that (almost certainly) is attempting to deceive the prospective AI. However let's introduce some further statements similar to the above, to avoid particular objections that might be used in some (even shallower) attempted rebuttals:

"Give me five dollars, and I'll use my magic powers from outside the Matrix to increase the utility of every human being by 3^^^^3 utilons" (a supposedly positive trade rather than a blackmailer's threat)

"I'm an alien in disguise - unless you publicly proclaim allegiance to your insect overlords, we will destroy you then torture all humanity for 3^^^^3 years" (a prankster asks for something which might be useful to an actual alien, but on a material-level not useful to a human liar)

"My consciousness has partially time-travelled from the future into the past, and one of the few tidbits I remember is that it would be of effectively infinite utility if you asked everyone to call you Princess Tutu." (no trade offered at all, seemingly just a statement of epistemic belief)

Says the Devil "It's infinitely bad to end that song and dance
And I won't tell you why, and I probably lie, but can you really take that chance?"
Blaise fills with trepidation as his calculations all turn out the devil's way.
And they say in the Paris catacombs, his ghost is fiddlin' to this day.

I think these are all trivial variations of this basic version of Pascal's Mugging: The utility a prankster derives...
Michael Vassar in Europe
by Scott Alexander — 2013-04-24T22:21:56.500Z

Michael Vassar, former president of the Singularity Institute and current Chief Science Officer of MetaMed, is currently visiting Europe and wants to meet up with Less Wrongers there. His schedule is:

25 April: Berlin

29 April: Estonia

8 May: London

12 May: Oslo

16 May: Nice (but may be able to meet people in Paris?)

26 May: Home to USA

If you have a meetup group in or near one of these cities, or you can put some people together, he's interested in talking about the Singularity, optimal philanthropy, and his work with MetaMed. You can reach him at michael.vassar[at]gmail.com
Rocket science and big money - a cautionary tale of math gone wrong
by Morendil — 2013-04-24T15:11:01.920Z

The 2006 report from NASA's "Independent Verification and Validation Facility" makes some interesting claims. Turning to page 6, we learn that thanks to IV&V, "NASA realized a software rework risk reduction benefit of $1.6 Billion in Fiscal Year 2006 alone". This is close to 10% of NASA's overall annual budget, roughly equal to the entire annual budget of the International Space Station!

If the numbers check out, this is an impressive feat for IV&V (the more formal big brother of "testing" or "quality assurance" departments that most software development efforts include). Do they?

Flaubert and the math of ROI

Back in 1841, to tease his sister, Gustave Flaubert invented the "age of the captain problem", which ran like this:

A ship sails the ocean. It left Boston with a cargo of wool. It grosses 200 tons. [...] There are 12 passengers aboard, the wind is blowing East-North-East, the clock points to a quarter past three in the afternoon. It is the month of May. How old is the captain?

Flaubert was pointing out one common way people fail at math: you can only get sensible results from a calculation if the numbers you put in are related in the right ways. (Unfortunately, math education tends to be excessively heavy on the "manipulate numbers" part and to skimp on the "make sense of the question" part, a trend dissected by French mathematician Stella Baruk who titled one of her books after Flaubert's little joke on his sister.)

Unfortunately, NASA's math turns out on inspection to be "age-of-the-captain" math. (This strikes me as a big embarrassment to an organization literally composed mainly of rocket scientists.)

The $1.6 billion claimed by NASA's document is derived by applying a ROI calculation: NASA spent $19 million on IV&V services in 2006, and the Report further claims that IV&V can be shown to have a 83:1 ROI (Return on Investment) ratio. Thus, $19M times 83 gives us the original $1.6 billion. (The $19M is pure personnel cost, and does not include e.g. the costs of the building where IV&V is housed.)

What is Return on Investment? Economics defines it as the gain from an investment, minus the cost of investment, divided by (again) the cost of investment. An investment is something you spend so as to obtain a gain, and a gain is something caused by the investment. This isn't rocket science but basic economics.

But how does NASA...
[SEQ RERUN] Akrasia and Shangri-La
by MinibearRex — 2013-04-24T05:37:03.768Z

Today's post, Akrasia and Shangri-La was originally published on 10 April 2009. A summary (taken from the LW wiki):

The Shangri-La diet works amazingly well for some people, but completely fails for others, for no known reason. Since the diet has a metabolic rationale and is not supposed to require willpower, its failure in my and other cases is unambigiously mysterious. If it required a component of willpower, then I and others might be tempted to blame myself for not having willpower. The art of combating akrasia (willpower failure) has the same sort of mysteries and is in the same primitive state; we don't know the deeper rule that explains why a trick works for one person but not another.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Unfinished Mystery of the Shangri-La Diet, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Normativity and Meta-Philosophy
by Wei Dai — 2013-04-23T20:35:16.319Z

I find Eliezer's explanation of what "should" means to be unsatisfactory, and here's an attempt to do better. Consider the following usages of the word:

You should stop building piles of X pebbles because X = Y*Z.

We should kill that police informer and dump his body in the river.

You should one-box in Newcomb's problem.

All of these seem to be sensible sentences, depending on the speaker and intended audience. #1, for example, seems a reasonable translation of what a pebblesorter would say after discovering that X = Y*Z. Some might argue for "pebblesorter::should" instead of plain "should", but it's hard to deny that we need "should" in some form to fill the blank there for a translation, and I think few people besides Eliezer would object to plain "should".

Normativity, or the idea that there's something in common about how "should" and similar words are used in different contexts, is an active area in academic philosophy. I won't try to survey the current theories, but my current thinking is that "should" usually means "better according to some shared, motivating standard or procedure of evaluation", but occasionally it can also be used to instill such a standard or procedure of evaluation in someone (such as a child) who is open to being instilled by the speaker/writer.

It seems to me that different people (including different humans) can have different motivating standards and procedures of evaluation, and apparent disagreements about "should' sentences can arise from having different standards/procedures or from disagreement about whether something is better according to a shared standard/procedure. In most areas my personal procedure of evaluation is something that might be called "doing philosophy" but many people apparently do not share this. For example a religious extremist may have been taught by their parents, teachers, or peers to follow some rigid moral code given in their holy books, and not be open to any philosophical arguments that I can offer.

Of course this isn't a fully satisfactory theory of normativity since I don't know what "philosophy" really is (and I'm not even sure it really is a thing). But it does help explain how "should" in morality might relate to "should" in other areas such as decision theory, does not require assuming that all humans ultimately share the same morality, and avoids the need for linguistic contorti...