trentmkelly/LessWrong-43k huggingface dataset:
Probability Space & Aumann Agreement
by Wei Dai — 2009-12-10T21:57:02.812Z

The first part of this post describes a way of interpreting the basic mathematics of Bayesianism. Eliezer already presented one such view at http://lesswrong.com/lw/hk/priors_as_mathematical_objects/, but I want to present another one that has been useful to me, and also show how this view is related to the standard formalism of probability theory and Bayesian updating, namely the probability space.

The second part of this post will build upon the first, and try to explain the math behind Aumann's agreement theorem. Hal Finney had suggested this earlier, and I'm taking on the task now because I recently went through the exercise of learning it, and could use a check of my understanding. The last part will give some of my current thoughts on Aumann agreement.

Probability Space

In http://en.wikipedia.org/wiki/Probability_space, you can see that a probability space consists of a triple:

Ω – a non-empty set – usually called sample space, or set of states

F – a set of subsets of Ω – usually called sigma-algebra, or set of events

P – a function from F to [0,1] – usually called probability measure

F and P are required to have certain additional properties, but I'll ignore them for now. To start with, we’ll interpret Ω as a set of possible world-histories. (To eliminate anthropic reasoning issues, let’s assume that each possible world-history contains the same number of observers, who have perfect memory, and are labeled with unique serial numbers.) Each “event” A in F is formally a subset of Ω, and interpreted as either an actual event that occurs in every world-history in A, or a hypothesis which is true in the world-histories in A. (The details of the events or hypotheses themselves are abstracted away here.)

To understand the probability measure P, it’s easier to first introduce the probability mass function p, which assigns a probability to each element of Ω, with the probabilities summing to 1. Then P(A) is just the sum of the probabilities of the elements in A. (For simplicity, I’m assuming the discrete case, where Ω is at most countable.) In other words, the probability of an observation is the sum of the probabilities of the world-histories that it doesn't rule out.

A payoff of this view of the probability...
You Be the Jury: Survey on a Current Event
by komponisto — 2009-12-09T04:25:13.746Z

As many of you probably know, in an Italian court early last weekend, two young students, Amanda Knox and Raffaele Sollecito, were convicted of killing another young student, Meredith Kercher, in a horrific way in November of 2007. (A third person, Rudy Guede, was convicted earlier.)

If you aren't familiar with the case, don't go reading about it just yet. Hang on for just a moment.

If you are familiar, that's fine too. This post is addressed to readers of all levels of acquaintance with the story.

What everyone should know right away is that the verdict has been extremely controversial. Strong feelings have emerged, even involving national tensions (Knox is American, Sollecito Italian, and Kercher British, and the crime and trial took place in Italy). The circumstances of the crime involve sex. In short, the potential for serious rationality failures in coming to an opinion on a case like this is enormous.

Now, as it happens, I myself have an opinion. A rather strong one, in fact. Strong enough that I caught myself thinking that this case -- given all the controversy surrounding it -- might serve as a decent litmus test in judging the rationality skills of other people. Like religion, or evolution -- except less clichéd (and cached) and more down-and-dirty.

Of course, thoughts like that can be dangerous, as I quickly recognized. The danger of in-group affective spirals looms large. So before writing up that Less Wrong post adding my-opinion-on-the-guilt-or-innocence-of-Amanda-Knox-and-Raffaele-Sollecito to the List of Things Every Rational Person Must Believe, I decided it might be useful to find out what conclusion(s) other aspiring rationalists would (or have) come to (without knowing my opinion).

So that's what this post is: a survey/experiment, with fairly specific yet flexible instructions (which differ slightly depending on how much you know about the case already).

For those whose familiarity with the case is low:

I'm going to give you two websites advocating a position, one strongly in favor of the verdict, the other strongly opposed. Your job will be to browse around these sites to learn info about the case, as much as you need to in order to arrive at a judgment. The order, manner, and quantity of browsing will be left up to you -- though I would of course like to know how much you read in your response.

1. Site arguing defendants are gu...
Science - Idealistic Versus Signaling
by billswift — 2009-12-06T13:39:04.368Z

[This is a version of an first draft essay I wrote for my blog. I intend to write another version, but it is going to take some time to research, and I want to get this out where I can start getting some feedback and sources for further research.]

The responses to the recent leaking of the CRU's information and emails, has led me to a changed understanding of science and how it is viewed by various people, especially people who claim to be scientists. Among people who actually do or consume science there seem to be two broad views - what they "believe" about science, rather than what they normally "say" about science when asked.

The classical view, what I have begun thinking of as the idealistic view, is science as the search for reliable knowledge. This is the version most scientists (and many non-scientists) espouse when asked, but increasingly many scientists actually hold another view when their beliefs are evaluated by their actions.

This is the signaling and control view of science. This is the "social network" view that has been developed by many sociologists of science.

For an extended example of the two views in conflict, see this recent thread of 369 comments Facts to fit the theory? Actually, no facts at all! . PhysicistDave is the best exemplar of the idealistic view, with pete and several others having extreme signaling and control viewpoints.

I wonder how much of the fact that there hasn't been any fundamental breakthroughs in the last fifty years has to do with the effective takeover of science by academics and government - that is by the signaling and control view. Maybe we have too many "accredited" scientists and they are too beholden to government, and to a lesser extent other grant-making organizations - and they have crowded out or controlled real, idealistic science.

This can also explain the conflict between those who extol peer review, despite its many flaws, and downplay open source science. They are controlling view scientists protecting their turf and power and prerogatives. Anyone thinking about the ideals of science, the classical view of science, immediately realizes that open sourcing the arguments and data will meet the ends of extending knowledge much better than peer review, now that it is possible. Peer review was a stop gap means of getting a quick review of a paper that was necessary when the costs of distributing informa...
Parapsychology: the control group for science
by AllanCrossman — 2009-12-05T22:50:06.821Z

Parapsychologists are constantly protesting that they are playing by all the standard scientific rules, and yet their results are being ignored - that they are unfairly being held to higher standards than everyone else. I'm willing to believe that. It just means that the standard statistical methods of science are so weak and flawed as to permit a field of study to sustain itself in the complete absence of any subject matter.

— Eliezer Yudkowsky, Frequentist Statistics are Frequently Subjective

Imagine if, way back at the start of the scientific enterprise, someone had said, "What we really need is a control group for science - people who will behave exactly like scientists, doing experiments, publishing journals, and so on, but whose field of study is completely empty: one in which the null hypothesis is always true.

"That way, we'll be able to gauge the effect of publication bias, experimental error, misuse of statistics, data fraud, and so on, which will help us understand how serious such problems are in the real scientific literature."

Isn't that a great idea?

By an accident of historical chance, we actually have exactly such a control group, namely parapsychologists: people who study extra-sensory perception, telepathy, precognition, and so on.

There's no particular reason to think parapsychologists are doing anything other than what scientists would do; their experiments are similar to those of scientists, they use statistics in similar ways, and there's no reason to think they falsify data any more than any other group. Yet despite the fact that their null hypotheses are always true, parapsychologists get positive results.

This is disturbing, and must lead us to wonder how many positive results in real science are actually wrong.

The point of all this is not to mock parapsychology for the sake of it, but rather to emphasise that parapsychology is useful as a control group for science. Scientists should aim to improve their procedures to the point where, if the control group used these same procedures, they would get an acceptably low level of positive results. That this is not yet the case indicates the need for more stringent scientific procedures.

Acknowledgements

The idea for this mini-essay and many of its actual points were suggested by (or stolen from) Eliezer Yudkowsky's Frequentist Statistics are Frequently Subjective, though the idea might ha...
Nothing to see here
by KatjaGrace — 2009-12-05T09:00:28.000Z

New posts on Meteuphoric are unlikely for two weeks. I’m staying at my family’s new farm house in Tasmania, beyond Internet reach except for flickers of cell phone connectivity (via which I now write) and in the future. See you then.

Update 27 Dec: When internet companies guarantee two weeks they can mean some much longer amount of time. See you after that, or next time I move house.
Arbitrage of prediction markets
by taw — 2009-12-04T22:29:14.176Z

I've noticed something very curious on Intrade markets for 2012 Republican Presidential Nominee - Ron Paul gets 3.5% of getting a nomination - a value that's clearly (and spare me EMH here) wishful thinking of Ron Paul supporters more than any genuine estimate.

And this brings me to a question - if prediction markets overestimate chance of winning of some rare case, how can I profit from that? Naively if I know true chance is 1%, I win $3.5 99% of the time, and lose $97.5 1% of the time, for expected payoff of $2.5. But my maximum loss is 39x higher than my expected profit, and I won't be getting any money out of it for three more years.

I'd need to bet significant amount to earn any money out of it, and that would require accepting 39x as high maximum loss. No reasonably prediction market would accept this kind of leverage without some collateral, nor could I get any reasonable loan for it at rates that would make this arbitrage profitable.

The only way I can think of would be convincing someone with plenty of money that I'm right, and have him provide me with collateral for some (probably very high) portion of the payoff. But if results depend on my ability to convince rich people, that's not prediction market! None of this is a problem for people trying to artificially pump estimates for Ron Paul - they'll just take the loss, and write it off as marketing expense.

None of these problems occur if some position is vastly overestimated, like 60% estimate if I know true value to be 40% - this would be a cheap bet - maximum loss of $40 for expected profit of $20, and people who want to pump it need to take about as much risk as people who want to bring it back to the true value, not a lot more.

I'm confused. Is there some nice way to arbitrage this, or is this an inherent weakness of prediction markets and we should only trust positions they pick as leaders, not chances of their long tail?
Frequentist Statistics are Frequently Subjective
by Eliezer Yudkowsky — 2009-12-04T20:22:21.245Z

Andrew Gelman recently responded to a commenter on the Yudkowsky/Gelman diavlog; the commenter complained that Bayesian statistics were too subjective and lacked rigor. I shall explain why this is unbelievably ironic, but first, the comment itself:

However, the fundamental belief of the Bayesian interpretation, that all probabilities are subjective, is problematic -- for its lack of rigor... One of the features of frequentist statistics is the ease of testability. Consider a binomial variable, like the flip of a fair coin. I can calculate that the probability of getting seven heads in ten flips is 11.71875%... At some point a departure from the predicted value may appear, and frequentist statistics give objective confidence intervals that can precisely quantify the degree to which the coin departs from fairness...

Gelman's first response is "Bayesian probabilities don't have to be subjective." Not sure I can back him on that; probability is ignorance and ignorance is a state of mind (although indeed, some Bayesian probabilities can correspond very directly to observable frequencies in repeatable experiments).

My own response is that frequentist statistics are far more subjective than Bayesian likelihood ratios. Exhibit One is the notion of "statistical significance" (which is what the above comment is actually talking about, although "confidence intervals" have almost the same problem). Steven Goodman offers a nicely illustrated example: Suppose we have at hand a coin, which may be fair (the "null hypothesis") or perhaps biased in some direction. So lo and behold, I flip the coin six times, and I get the result TTTTTH. Is this result statistically significant, and if so, what is the p-value - that is, the probability of obtaining a result at least this extreme?

Well, that depends. Was I planning to flip the coin six times, and count the number of tails? Or was I planning to flip the coin until it came up heads, and count the number of trials? In the first case, the probability of getting "five tails or more" from a fair coin is 11%, while in the second case, the probability of a fair coin requiring "at least five tails before seeing one heads" is 3%.

Whereas a Bayesian looks at the experimental result and says, "I can now calculate the likelihood ratio (evidential flow) betwee...
Intuitive supergoal uncertainty
by JustinShovelain — 2009-12-04T05:21:03.942Z

There is a common intuition and feeling that our most fundamental goals may be uncertain in some sense. What causes this intuition? For this topic I need to be able to pick out one’s top level goals, roughly one’s context insensitive utility function, and not some task specific utility function, and I do not want to imply that the top level goals can be interpreted in the form of a utility function. Following from Eliezer’s CFAI paper I thus choose the word “supergoal” (sorry Eliezer, but I am fond of that old document and its tendency to coin new vocabulary). In what follows, I will naturalistically explore the intuition of supergoal uncertainty.
To posit a model, what goal uncertainty (including supergoal uncertainty as an instance) means is that you have a weighted distribution over a set of possible goals and a mechanism by which that weight may be redistributed. If we take away the distribution of weights how can we choose actions coherently, how can we compare? If we take away the weight redistribution mechanism we end up with a single goal whose state utilities may be defined as the weighted sum of the constituent goals’ utilities, and thus the weight redistribution mechanism is necessary for goal uncertainty to be a distinct concept.

Part of the intuition of supergoal uncertainty naturally follows from goal and planning uncertainty. What plan of action is best? How should I construct a local utility function for this context? The weight redistribution mechanism is then a result of gathering more evidence, calculating further, and seeing how this goal links up to one’s supergoals in the context of other plans.

It could be we are mistaken. The rules society sets up to coordinate behavior give the default assumption that there is an absolute standard by which to judge a behavior good or bad. Also, religions many times dictate that there is a moral absolute, and even if we aren’t religious, the cultural milieu make us consider the possibility that the concept “should” (and so the existence of a weight redistribution mechanism) can be validly applied to supergoals.

It could be we are confused. Our professed supergoal does not necessarily equal our actual supergoal but neither are they completely separate. So when we review our past behavior and introspect to determine what our supergoal is, we ge

...
Help Roko become a better rationalist! 
by Roko — 2009-12-02T08:23:37.643Z

Last time, I wrote about 11 core rationalist skills. Now I would like some help from the LW community: which of these skills am I good at, which ones am I bad at? Just to recap, the skills are:

Actually want an accurate map

Keep your eyes on the prize

Entangle yourself with the evidence

Be Curious

Aumann-update

Know standard Biases

Know Probability theory

Know your own mind

Be well calibrated

Use analytic philosophy

Resist Thoughtcrime

I'll post a description of each one of these skills as a comment, and if you think I am good at that skill, vote it up. If you think I am bad at it, vote it down. Don't be too shy - even if you are biased or uncertain - because over the course of many votes, these biases and errors will cancel out to some extent. (This is the "guess the number of beans in a jar by asking 50 people to guess and taking the average" method)

EDIT: We can also comment on each rationalist skill to say how well I am doing at that skill. Later today, I will do this myself.

Thanks in advance!
11 core rationalist skills
by Roko — 2009-12-02T08:09:05.922Z

An excellent way to improve one's skill as a rationalist is to identify one's strengths and weaknesses, and then expend effort on the things that one can most effectively improve (which are often the areas where one is weakest). This seems especially useful if one is very specific about the parts of rationality, if one describes them in detail.

In order to facilitate improving my own and others' rationality, I am posting this list of 11 core rationalist skills, thanks almost entirely to Anna Salamon.

Actually want an accurate map, because you have Something to protect.

Keep your eyes on the prize. Focus your modeling efforts on the issues most relevant to your goals. Be able to quickly refocus a train of thought or discussion on the most important issues, and be able and willing to quickly kill tempting tangents. Periodically stop and ask yourself "Is what I am thinking about at the moment really an effective way to achieve my stated goals?".

Entangle yourself with the evidence. Realize that true opinions don't come from nowhere and can't just be painted in by choice or intuition or consensus. Realize that it is information-theoretically impossible to reliably get true beliefs unless you actually get reliably pushed around by the evidence. Distinguish between knowledge and feelings.

Be Curious: Look for interesting details; resist cached thoughts; respond to unexpected observations and thoughts. Learn to acquire interesting angles, and to make connections to the task at hand.

Aumann-update: Update to the right extent from others' opinions. Borrow reasonable practices for grocery shopping, social interaction, etc from those who have already worked out what the best way to do these things is. Take relevant experts seriously. Use outside views to estimate the outcome of one's own projects and the merit of one's own clever ideas. Be willing to depart from consensus in cases where there is sufficient evidence that the consensus is mistaken or that the common practice doesn't serve its ostensible purposes. Have correct models of the causes of others’ beliefs and psychological states, so that you can tell the difference between cases where the vast majority of people believe falsehoods for some specific reason, and cases where the vast majority actually knows best.

Know standard Biases: Have conscious

...
The Difference Between Utility and Utility
by Matt_Simpson — 2009-12-02T06:16:18.125Z

Recently I argued that the economist's utility function and the ethicist's utility function are not the same. The nutshell argument is that they are created for different purposes - one is an attempt to describe the actions we actually take and the other is an attempt to summarize our true values (i.e., what we should do). I just ran across a somewhat older post over at Black Belt Bayesian arguing this very point. Excerpt:

Economics (of the neoclassical kind) models consumers and other economic actors as such utility maximizers... Utility is not something you can experience. It’s just a mathematical construct used to describe the optimization structure in your behavior...

Consequentialist ethics says an act is right if its consequences are good. Moral behavior here amounts to being a utility maximizer. What’s “utility”? It’s whatever a moral agent is supposed to strive toward. Bentham’s original utilitarianism said utility was pleasure minus pain; nowadays any consequentalist theory tends to be called “utilitarian” if it says you should maximize some measure of welfare, summed over all individuals... Take note: not all utility maximizers are utilitarians.

There’s no necessary connection between these two kinds of utility other than that they use the same math. It’s possible to make up a utilitarian theory where ethical utility is the sum of everyone’s economic utility (calibrated somehow), but this is just one of many possibilities. Anyone trying to reason about one kind of utility through the other is on shaky ground.
Why do aphorisms and cynicism go together?
by KatjaGrace — 2009-12-01T07:45:09.000Z

Why are aphorisms cynical more often than books are for instance?

A good single sentence saying can’t require background evidencing or further explanation. It must be instantly recognizable as true. It also needs to be news to the listener. Most single sentences that people can immediately verify as true they already believe. What’s left? One big answer is things that people don’t believe or think about much for lack of wanting to, despite evidence. Drawing attention to these is called cynicism.

HT to Robin Hanson for the question and to Francois de La Rochefoucauld for some examples:

We often forgive those who bore us, but we cannot forgive those whom we bore.

We promise according to our hopes; we fulfill according to our fears.

What often prevents us from abandoning ourselves to one vice is that we have several.

We confess to little faults only to persuade ourselves we have no great ones.

There are few people who are more often wrong than those who cannot suffer being wrong.

Nothing prevents us being natural so much as the desire to appear so.