trentmkelly/LessWrong-43k huggingface dataset:
Understanding and justifying Solomonoff induction
by gedymin — 2014-01-15T01:16:57.160Z

I've been trying to understand the uses and limitations of Solomonoff induction. Following the principle that in order to fully understand something you should explain it others, here's a try. I prefer to write such things in a form for dialogue, as that better reflects thought processes.
This is not a very-in-depth technical article - for example, cousin_it and Wen Dai has obviously spent more time pondering about SI. (I'm not a long-time LW reader, but have skimmed through existing LW and wiki articles on related topics before posting this.)

Alice. Hi, I'm interested in the question of why and when should I prefer simpler hypotheses. I've heard about Occam's razor and I've read about Solomonoff induction and the Universal Prior. Now I'm looking for a philosophical justification of the math. I'd like to have something akin to de Finetti's justification for probability theory - "if you don't believe in the axioms, you're going to be Dutch booked!".
Bob. You're welcome. Do you have any problems with the formulations?
A. I'm trying to understand how to connect the informal concept of Occam's razor with the mathematical formula of the Universal Prior in a meaningful way. Informally, a hypothesis is something that explains the data. Occam's razor tells us to prefer simpler hypotheses.
B. Well, yes.
A. But the Universal Prior seems to tell something that seems to be even stronger: that all shorter hypothesis are more likely! Clearly, that's not the case: if F and G are hypotheses, then "F OR G" is a hypothesis as well, and not less likely than either F or G individually! What's more, there exists a whole set of language and domain pairs where longer hypotheses have the same average probability. For one particular example consider propositional logic with AND, OR, and NOT operators. Because of symmetry between conjunction and disjunction, a well-formed 10-element formula is as likely to be a tautology (i.e. always correct) as a 100-element formula!

B. You're confused. In this formalism, you should interpret all hypotheses as programs for Universal Turing Machine. The machine reads a hypothesis from one of its tapes. It then executes the hypothesis, and outputs the result of the computation to another tape. The result is the data - the actual string, whose prefix we're observing, and whose remaining part we're trying to predict. Hypothesis is the input string - the string...
Functional Side Effects
by Scott Garrabrant — 2014-01-14T20:22:04.704Z

Cross Posted on By Way of Contradiction

You have probably heard the argument in favor of functional programming languages that functions act like functions in mathematics, and therefore have no side effects. When you call a function, you get an output, and with the exception of possibly the running time nothing matters except for the output that you get. This is in contrast with other programming languages where a function might change the value of some other global variable and have a lasting effect.

Unfortunately the truth is not that simple. All functions can have side effects. Let me illustrate this with Newcomb’s problem. In front of you are two boxes. The first box contains 1000 dollars, while the second box contains either 1,000,000 or nothing. You may choose to take either both boxes or just the second box. An Artificial Intelligence, Omega, can predict your actions with high accuracy, and has put 1,000,000 in the second box if and only if he predicts that you will take only the second box.

You, being a good reflexive decision agent, take only the second box, and it contains 1,000,000.

Omega can be viewed as a single function in a functional programming language, which takes in all sorts of information about you and the universe, and outputs a single number, 1,000,000 or 0. This function has a side effect. The side effect is that you take only the second box. If Omega did not simulate you and just output 1,000,000, and you knew this, then you would take two boxes.

Perhaps you are thinking “No, I took one box because I BELIEVED I was being simulated. This was not a side effect of the function, but instead a side effect of my beliefs about the function. That doesn’t count.”

Or, perhaps you are thinking “No, I took one box because of the function from my actions to states of the box. The side effect is no way dependent on the interior workings of Omega, but only on the output of Omega’s function in counterfactual universes. Omega’s code does not matter. All that matters is the mathematical function from the input to the output.”

These are reasonable rebuttals, but they do not carry over to other situations.

Imagine two programs, Omega 1 and Omega 2. They both simulate you for an hour, then output 0. The only difference is that Omega 1 tortures the simulation of you for an hour, while Omega 2 tries its best to simu...
What rationality material should I teach in my game theory course
by James_Miller — 2014-01-14T02:15:53.470Z

I will soon be teaching a 50 person game theory class at Smith College and I want to include material on how to be rational. What do you think I should teach/assign? The course has a one semester calculus requirement. Here is material I'm considering:

Litany of Gendlin, Humans are not automatically strategic, The map is not the territory, Ugh fields, Litany of Tarski, Branches of rationality, Twelve Virtues of Rationality, How to Be Happy, How to Beat Procrastination, Make Beliefs Pay Rent, Navigating disagreement, Superstimuli and the Collapse of Western Civilization, Efficient Charity, Bayesians vs. Barbarians, ARGUMENTS FROM MY OPPONENT BELIEVES, Are Your Enemies Innately Evil?, Reversed Stupidity Is Not Intelligence, How habits work and how you may control them, The Halo Effect, NEWTONIAN ETHICS, Planning Fallacy, The Good News of Situationist Psychology, Use the Try Harder, Luke, Lotteries: A Waste of Hope.
Philosophical Realism and Fairy Dust
by HalMorris — 2014-01-14T01:21:51.143Z

"Realism" in the philosophical sense has to be relative to something - Plato's essences, "collective imagination", society, truth are among the subjects that evoke comments that this person is a realist (considers the "something" to be real), and that one isn't.

A philosophical realist w.r.t. fairies is one who believes Fairies are real, while the non-realist says talk of fairies is due to overactive agency detectors or some such thing. It will tend to seem like the opposite of everyday use of the word "realism" -- at least if the subject is one non-philosophers would ever talk about.

I mention this only because I found it a bit difficult to get, and I think I've now "got" it. Correct me if you think I'm wrong.
Meetup : Boston - Aversion factoring and calibration
by Vika — 2014-01-13T23:24:15.085Z

Discussion article for the meetup : Boston - Aversion factoring and calibration

WHEN:
15 January 2014 07:00:00PM (-0500)

WHERE:
Citadel, 98 Elm St Apt 1, Somerville, MA, 02144

Victoria Krakovna will be running a session on Aversion Factoring and Calibration (based on a CFAR unit). While Goal Factoring is useful for understanding your motivations for things you do, Aversion Factoring helps you analyze the things you are averse to doing.

Cambridge/Boston-area Less Wrong Wednesday meetups are once a month at 7pm at Citadel (98 Elm St Apt 1 Somerville, near Porter Square). All other meetups are on Sundays.

Our default schedule is as follows:

—Phase 1: Arrival, greetings, unstructured conversation.

—Phase 2: The headline event. This starts promptly at 7:30pm, and lasts 30-60 minutes.

—Phase 3: Further discussion. We'll explore the ideas raised in phase 2, often in smaller groups.

Discussion article for the meetup : Boston - Aversion factoring and calibration
Meetup : Bratislava Meetup IX.
by Viliam_Bur — 2014-01-13T14:10:45.094Z

Discussion article for the meetup : Bratislava Meetup IX.

WHEN:
20 January 2014 06:00:00PM (+0100)

WHERE:
Bistro The Peach, Heydukova 21, Bratislava

The same place; lecture and discussion in Slovak language.

Stretávame sa na rovnakom mieste v rovnakom čase: pondelok o šiestej. Dnešným hosťom je Natália Blahová, sociálna poradkyňa a občianska aktivistka zaoberajúca sa právami detí. Téma je sociálny systém, altruizmus, občianska angažovanosť... a čokoľvek, čo vás bude zaujímať.

Technická poznámka: Pani Blahová príde až o siedmej, ale my sa stretneme klasicky o šiestej, čiže prvú hodinu bude voľná debata.

Discussion article for the meetup : Bratislava Meetup IX.
Anthropic Atheism
by  — 2014-01-13T05:00:23.495Z

(Crossposted from my blog)

I've been developing an approach to anthropic questions that I find less confusing than others, which I call Anthropic Atheism (AA). The name is a snarky reference to the ontologically basic status of observers (souls) in other anthropic theories. I'll have to explain myself.

We'll start with what I call the “Sherlock Holmes Axiom” (SHA), which will form the epistemic background for my approach:

How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?

Which I reinterpret as “Reason by eliminating those possibilities inconsistent with your observations. Period.” I use this as a basis of epistemology. Basically, think of all possible world-histories, assign probability to each of them according to whatever principles (eg occams razor), eliminate inconsistencies, and renormalize your probabilities. I won’t go into the details, but it turns out that probability theory (eg Bayes theorem) falls out of this just fine when you translate P(E|H) as “portion of possible worlds consistent with H that predict E”. So it’s not really any different, but using SHA as our basis, I find certain confusing questions less confusing, and certain unholy temptations less tempting.

With that out of the way, let’s have a look at some confusing questions. First up is the Doomsday Argument. From La Wik:

Simply put, it says that supposing the humans alive today are in a random place in the whole human history timeline, chances are we are about halfway through it.

The article goes on to claim that “There is a 95% chance of extinction within 9120 years.” Hard to refute, but nevertheless it makes one rather uncomfortable that the mere fact of one’s existence should have predictive consequences.

In response, Nick Bostrom formulated the “Self Indication Assumption”, which states that “All other things equal, an observer should reason as if they are randomly selected from the set of all possible observers.” Applied to the doomsday argument, it says that you are just as likely to exist in 2014 in a world where humanity grows up to create a glorious everlasting civilization, as one where we wipe ourselves out in the next hundred years, so you can’t update on that mere fact of your existence. Thi...
On Voting for Third Parties
by Scott Garrabrant — 2014-01-13T03:16:28.394Z

Cross Posted on my blog, By Way of Contradiction.

Anti-Trigger Warning: There is not really any politics in this post. I doubt it will kill your mind.

If your favorite candidate in an election is a third party candidate, should you vote for him?

This question has confused me. I have changed my mind many times, and I have recently changed my mind again. I would like to talk about some of the arguments in both directions and explain the reason for my most recent change.

Con 1) Voting for a third party is throwing your vote away.

We have all heard this argument before, and it is true. It is an unfortunate consequence of the plurality voting system. Plurality is horrible and there are all better alternatives, but it is what we are stuck with for now. If you vote for a third party, the same candidate would be elected as if you did not vote at all.

Pro 1) The probability that you vote changes the election is negligible. All your vote does is add one to the number of people who voted for a given candidate. Your vote for the third party candidate therefore matters more because it is changing a small number by relatively more.

This argument is actually an empirical claim, and I am not sure how well it holds up. It is easy to study the likelihood that you vote changes the election. One study finds that it roughly varies from 10^-7 to 10^-11 in America for presidential elections. However, it is not clear to me just how much your vote affects the strategies of political candidates and voters in the future.

Pro 2) The probability that your vote changes the election or future elections is negligible. The primary personal benefit for voting is the personal satisfaction of voting. This personal satisfaction is maximized by voting for the candidate you agree with the most.

I think that many people if given the choice between changing the next president between the two primary parties or being paid an amount of money equal to the product of the amount of gas they spent to drive to vote and 10^7 would take the money. I am not one of them but any of those people must agree that voting is a bad investment if you do not consider the personal satisfaction. However, I think I might get more satisfaction out of doing my best to change the election, rather than placing a vote that does not matter.

Con 2) Actually if you use a reflexive decision theory, you are much more likely to change ...
Meetup : West LA Meetup: Democracy
by Scott Garrabrant — 2014-01-13T01:07:04.880Z

Discussion article for the meetup : West LA Meetup: Democracy

WHEN:
15 January 2014 07:00:00PM (-0800)

WHERE:
10850 West Pico Blvd, Los Angeles, CA 90064, USA

How to get in: Go to the Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters. The entrance sign says "Lounge".

Parking is free for 3 hours.

Discussion: This week, we will be talking about democracy. What properties would a good voting system have? Why is there no perfect voting system? What is the best voting system? Is there any reasonable voting system that we have any chance of ever being implemented? Should I vote for a third party or a primary party candidate in the next election? Is democracy even worth it in the first place? In spite of the topic, this discussion will probably be abstract enough to not kill your mind.

No prior knowledge of or exposure to Less Wrong is necessary; this will be generally accessible.

Discussion article for the meetup : West LA Meetup: Democracy
Meetup : Lesswrong Boulder CO
by yakurbe0112 — 2014-01-12T19:37:13.792Z

Discussion article for the meetup : Lesswrong Boulder CO

WHEN:
16 January 2014 07:00:00PM (-0700)

WHERE:
2690 Baseline Rd, Boulder, CO 80305

Meet us here for awesome pizza and rationality. This week I'm going to see what I can learn about goal factoring and then tell you all about it.

Discussion article for the meetup : Lesswrong Boulder CO
Double-thick transistors and other subjective phenomena
by Manfred — 2014-01-12T19:32:41.358Z

If I'm running on a silicon computer, do I have twice as much subjective experience if my computer is twice as thick?

Why is this even a good question?

Consider a computer that was printed on a flat sheet. If we stick two of these computers (one a mirror image) together face to face, we get a thicker computer. And then if we peel them apart again, we get two thin computers! Suppose that we simulate a person using these computers. It makes sense that a person running on two thin computers has twice as much "experience" as a person running on just one (for example, in the Sleeping Beauty problem, the correct betting strategy is to bet as if the probability of making the bet in a given world is proportional to the number of thin computers). So if we take two people-computers and stick them together into one thicker person-computer, the thicker person contains twice as much "experience" as a thinner one - each of their halves has as much experience as a thin person, so they have twice as much experience.

Do I disagree? Well, I think it depends somewhat on how you cash out "experience." Consider the Sleeping Beauty problem with these computers - in the classic version, our person is asked to give their probability that they're in the possibility where there's one thin computer, or the world where there are two thin computers. The correct betting strategy is to bet as if you think the probability that there are two computers is 2/3 - weighting each computer equally.

Now, consider altering the experiment so that either there's one thin computer, or one double computer. We have two possibilities - either the correct betting probability is 1/2 and the computers seem to have equal "experience", or we bite the bullet and say that the correct betting probability is 2/3 for a double computer, 10/11 for a 10x thicker computer, 1000/1001 for a 1000x thicker computer, etc.

The bullet-biting scenario is equivalent to saying that the selfish desires of the twice-thick computer are twice as important. If one computer is one person, a double computer is then two people in a box.

But of course, if you have a box with two people in it, you can knock on the side and go "hey, how many of you people are in there? I'm putting in an order for chinese food, how many entrees should I get?" Instead, the double-thick computer is running exactly the same program as the thin computer, and w...
New (proposal for) monthly thread: Meetup Reports
by bartimaeus — 2014-01-12T15:50:07.423Z

If you had an interesting Less Wrong meetup recently, but don't have the time to write up a big report to post to Discussion, feel free to write a comment here. Even if it's just a couple lines about what you did and how people felt about it, it might encourage some people to attend meetups or start meetups in their area.

If you have the time, you can also describe what types of exercises you did, what worked and what didn't. This could help inspire meetups to try new things and improve themselves in various ways.

If you're inspired by what's posted below and want to organize a meetup, check out this page for some resources to get started! You can also check FrankAdamek's weekly post on meetups for the week.

Tell us about your meetup!
[LINK] Popular press account of social benefits of motivated reasoning
by Peter Wildeford — 2014-01-12T13:55:24.241Z

The Monkey Cage: "The Not Quite As Depressing Psychological Theory That Explains Washington":

In a landmark article in Behavioral and Brain Sciences, Hugo Mercier and Dan Sperber propose an “argumentative” theory of human reason which both directly acknowledges the problems of motivated reasoning and suggests that motivated reasoning in the right social contexts can be incredibly valuable and powerful.

Mercier and Sperber have a straightforward account of why human beings reason. They reason to win arguments by convincing others that they are right. This means that human beings suffer from “confirmation bias.” They are far, far better at finding justifications for why they are right than they are at thinking carefully about reasons why they might be wrong. [...]

However, where Mercier and Sperber depart from the skeptics is in pointing to the social value of reasoning. People are terrible judges of the flaws and weaknesses of their own arguments. However, they are much, much better at identifying weaknesses in the arguments of others. Furthermore, confirmation bias gives them good reason not only to try to confirm their own arguments, but also to try to demolish the arguments of people who disagree with them. This in turn means that groups — under the right conditions — are likely to be able to reach better judgments than any individual within the group. Real, substantial argument allows a kind of cognitive division of labor, in which different arguments get tested against each other.

[...]

When a group has to solve a problem, it is much more efficient if each individual looks mostly for arguments supporting a given solution. They can then present these arguments to the group, to be tested by the other members. This method will work as long as people can be swayed by good arguments, and the results reviewed . . . show that this is generally the case.
Even Odds
by Scott Garrabrant — 2014-01-12T07:24:25.537Z

(Cross-posted on my personal blog, which has LaTeX, and is easier to read.)

Let's say that you are are at your local less wrong meet up and someone makes some strong claim and seems very sure of himself, "blah blah blah resurrected blah blah alicorn princess blah blah 99 percent sure." You think he is probably correct, you estimate a 67 percent chance, but you think he is way over confident. "Wanna bet?" You ask.

"Sure," he responds, and you both check your wallets and have 25 dollars each. "Okay," he says, "now you pick some betting odds, and I'll choose which side I want to pick."

"That's crazy," you say, "I am going to pick the odds so that I cannot be taken advantage of, which means that I will be indifferent between which of the two options you pick, which means that I will expect to gain 0 dollars from this transaction. I wont take it. It is not fair!"

"Okay," he says, annoyed with you. "We will both write down the probability we think that I am correct, average them together, and that will determine the betting odds. We'll bet as much as we can of our 25 dollars with those odds."

"What do you mean by 'average' I can think of at least four possibilities. Also, since I know your probability is high, I will just choose a high probability that is still less than it to maximize the odds in my favor regardless of my actual belief. Your proposition is not strategy proof."

"Fine, what do you suggest?"

You take out some paper, solve some differential equations, and explain how the bet should go.

Satisfied with your math, you share your probability, he puts 13.28 on the table, and you put 2.72 on the table.

"Now what?" He asks.

A third meet up member takes quickly takes the 16 dollars from the table and answers, "You wait."

I will now derive a general algorithm for determining a bet from two probabilities and a maximum amount of money that people are willing to bet. This algorithm is both strategy proof and fair. The solution turns out to be simple, so if you like, you can skip to the last paragraph, and use it next time you want to make a friendly bet. If you want to try to derive the solution on your own, you might want to stop reading now.

First, we have to be clear about what we mean by strategy proof and fair. "Strategy proof" is clear. Our algorithm should ensure that neither person believes that they can increase their expected profit by lying about their probabilit...