trentmkelly/LessWrong-43k huggingface dataset:
Rationality Compendium: Principle 2 - You are implemented on a human brain
by ScottL — 2015-08-29T16:24:37.866Z

Irrationality is ingrained in our humanity. It is fundamental to who we are. This is because being human means that you are implemented on kludgy and limited wetware (a human brain). A consequence of this is that biases ↓ and irrational thinking are not mistakes, persay, they are not misfirings or accidental activations of neurons. They are the default mode of operation for wetware that has been optimized for purposes other than truth maximization.

If you want something to blame for the fact that you are innately irrational, then you can blame evolution ↓. Evolution tends to not to produce optimal organisms, but instead produces ones that are kludgy ↓, limited and optimized for criteria relating to ancestral environments rather than for criteria relating to optimal thought.

A kludge is a clumsy or inelegant, yet surprisingly effective, solution to a problem. The human brain is an example of a kludge. It contains many distinct substructures dating from widely separated periods of evolutionary development ↓. An example of this is the two kinds of processes in human cognition where one is fast (type 1) and the other is slow (type2) ↓.

There are many other characteristics of the brain that induce irrationality. The main ones are that:

The brain is innately limited in its computational abilities and so it must use heuristics ↓, which are mental shortcuts that ease the cognitive load of making a decision.

The brain has a tendency to blindly use salient or pre-existing responses to answers rather than developing new answers or thoroughly checking pre-existing solutions ↓.

The brain does not inherently value truth. One of the main reasons for this is that many of the biases can actually be adaptive. An example of an adaptive bias is the sexual over perception bias ↓ in men. From a truth-maximization perspective young men who assume that all women want them are showing severe social-cognitive inaccuracies, judgment biases, and probably narcissistic personality disorder. However, from an evolutionary perspective, the same young men are behaving in a more optimal manner. One which has consistently maximized the reproductive success of their male ancestors. Another similar example is the bias for positive

...
Asymptotic Logical Uncertainty: Irreducible Patterns
by Scott Garrabrant — 2015-08-28T22:00:27.000Z

This post is part of the Asymptotic Logical Uncertainty series. In this post, I will talk more about the exact assumptions we need to make about the sequence of sentences in the Benford Test.

I will start by making a few changes to the previous notation. First, we no longer need the output of L to be computable in bounded time. From now on, we can just say that L is a Turing machine that on input N eventually accepts if ϕN is provable, eventually rejects if ϕN is disprovable, and runs forever otherwise.

We will also change the sequence of sentences in the Benford test. Instead of "The first digit of A(n) in base 10 is 1," we will use "The first digit of 3↑n3 in base 10 is 1." This way, the reasonable answer to every question is 1log10 as opposed to before the powers of 10 were an exception. This change does not make the Benford test weaker, and the program we present will also pass the original Benford test, but it will make many definitions have fewer messy cases.

Finally, we will switch to deterministic machines. Instead of making a machine which outputs 1 with the correct probability, we will have a machine that outputs a probability. This clearly makes the problem no easier.

Here is the new version of the Benford test:

Let M be a Turing machine which on input N runs quickly and outputs a probability M(N), which represents the probability assigned to ϕN. We say that M satisfies the Benford test if
limn→∞M(sn)=1log(10),
where ϕsn= ``The first digit of 3↑n3 is a 1.''

In this post, we will not present a solution to the Benford test, but will be very explicit about the assumptions we need to make.

The fact that the best probability to assign to ϕsn is 1log(10) is dependent not only on the fact that frequency with which ϕsn is true tends to 1log(10), but also on the fact that the sequence of truth values of ϕsn contains no patterns that can be used to quickly compute a better probability on some subsequence. We therefore assume that this sequence of truth values is indistinguishable from a a sequence produced by a coin that outputs "true" with probability 1log(10). Formally, we are assuming that S={sn|n∈N} is an irreducible pattern with probability 1log(10) as defined below.

Fix a universal Turing machine UTM and an encoding scheme for machines, and let UTM(M,x) denote running the machine UTM to simulate M with input x.

Let S⊆N be an infinite subset of natural numbers

...
Calling references: Rational or irrational?
by PhilGoetz — 2015-08-28T21:06:46.872Z

Over the past couple of decades, I've sent out a few hundred resumes (maybe, I don't know, 300 or 400--my spreadsheet for 2013-2015 lists 145 applications). Out of those I've gotten at most two dozen interviews and a dozen job offers.

Throughout that time I've maintained a list of references on my resume. The rest of the resume is, to my mind, not very informative. The list of job titles and degrees says little about how competent I was.

Now and then, I check with one of my references to see if anyone called them. I checked again yesterday with the second reference on my list. The answer was the same: Nope. No one has ever, as far as I can recall, called any of my references. Not the people who interviewed me; not the people who offered me jobs.

When the US government did a background check on me, they asked me for a list of references to contact. My uncertain recollection is that they ignored it and interviewed my neighbors and other contacts instead, as if what I had given them was a list of people not to bother contacting because they'd only say good things about me.

Is this rational or irrational? Why does every employer ask for a list of references, then not call them?
Worst Case Bayes: Open Problems
by Scott Garrabrant — 2015-08-28T20:28:49.000Z

Here, I give a list of open problems related to the Worst Case Bayes prior, P, and provide some analysis.

1. Is P computably approximable?

I would not be surpirsed if there is an easy proof that I missed because I was focused on showing that the following specific procedure computably approximates P.

Start with assigning probability 1/2 to each sentence. Take a theorem prover which repeatedly outputs theorems of the form "Exactly one sentence in [finite list of sentences] is true." For each output of the theorem prover, observe the probabilities assigned to those sentences. If they do not sum to one, shift them so that they sum to one in the unique way that preserves the fact that Bayes score is independent of the model.

2. Does the above procedure converge to P?

3. Does the above procedure even converge?

4. Does the above procedure even converge to something whose Bayes score is independent of the model?

We probably need to make some assumptions about the theorem prover to get any of these.

5. Does P have the property that Bayes score is independent of the model?

I believe the answer to all of these questions is yes, assuming the theorem prover has good properties.

Clearly, 2 implies 1 and 3, 4 implies 3, and 4 together with 2 imply 5. What may be less obvious is that 4 actually implies 2, which means that a positive answer to question 4 would answer all of the other questions on the list.

To see that 4 implies 2, observe that if the procedure converges, the limit must be coherent, because coherence is equivalent to the property that whenever you have a sentence of the form “Exactly one sentence in [finite list of sentences] is true,” the probabilities in that list sum to 1. This is assuming that the theorem prover outputs every provable theorem of the correct form infinitely many times.

Since the procedure converges to a coherent prior, we showed in the previous post, there is no way to increase the Bayes score by any ε in all models simultaneously. Therefore there is no probability assignement which has worst case Bayes score ε greater than the Bayes score that the limit of the procedure gets in all models. Therefore the limit of the procedure maximizes worst case Bayes score, and by uniqueness equals P.

At first, 4 may seem like it should be obvious. Every step along the way, the Bayes score is independent of the model, and this Bayes score is increasing and bounded.

...
The Two-Update Problem: Monotonicity
by Scott Garrabrant — 2015-08-28T20:28:29.000Z

In his posts (1, 2) on the two-update problem, Abram Demski discussed a problem we see in existing proposals for logical priors. The existing proposals for logical priors work from a base theory T, and construct probability distributions PT, which represent a probability distributions on completions of that theory. The two-update problem is that it is not necessarily the case that PT(ϕ)=P∅(ϕ|T). This gives us two updates: One from putting sentences in the base theory, and one by performing a Bayesian update. Here, I want to talk about a weaker requirement for families of logical priors, where we only require that adding consequences of ϕ to the base theory does not decrease the probability of ϕ.

We say that a family of logical priors PT is monotonic if whenever T⊢ϕ→s for all s∈S, then PT(ϕ)≤PT∪S(ϕ). That is to say, if we only update on assumptions which are logical consequences of ϕ, then the probability of ϕ should only go up.

(I mentioned this before as one of the desirable properties in this post)

Theorem: The Demski prior is monotonic.

Proof: When sampling for the Demski prior with base theory T, for each infinite sequence of sentences sampled, either all the sentences in S are accepted as not contradicting previously sampled sentences or not.

In the case where all of the sentences in S are accepted, If we were to consider sampling the same infinite sequence of sentences for the Demski prior with base theory T∪S, then we would get the same complete theory in the end. Therefore, if we condition on the assumption that the infinite sequence causes all sentences in S to be accepted when the base theory is T, the probability that ϕ is accepted when the base theory is T is the same as the probability that ϕ is accepted when the base theory is T∪S.

On the other hand, if we condition on the assumption that the infinite sequence does not causes all sentences in S to be accepted when the base theory is T, then the probability that ϕ is accepted when the base theory is T is 0, while the probability that ϕ is accepted when the base theory is T∪S is non-negative.

Therefore, the probability that ϕ is accepted when the base theory is T is less than or equal to the probability that ϕ is accepted when the base theory is T∪S. □

Theorem: The Worst Case Bayes prior is not monotonic

Proof Sketch: Let A and B be two simple and independent sentences. We will consider a weighting on s

...
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-08-28T20:12:01.596Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
30 August 2015 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

Crossposted from mailing list. Meeting in courtyard, gathering from 3:00pm, hard start 3:30pm - until closing.

We'll be meeting to play games and/or just hang out and talk.

If you want to get people together for a big game, or a specific game, post [to the lesswrong-dc Google Group] looking for players and get an early start because they'll be kicking us out a little before 7pm.

Remember to bring games!

Upcoming Meetups:

Sep 6: What If

Sep 13: Sorting Hat

Sep 20: Fun and Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Maximize Worst Case Bayes Score
by Scott Garrabrant — 2015-08-28T18:56:42.000Z

This post has mostly been copied from something I posted on Less Wrong over a year ago. I am reposting it here, because I want to reference it in a future post related to the two-update problem. Here, I propose an logical prior. I do not think this prior is nearly as good as the modified Demski prior, or even the original Demski prior, but I am posting it has some interesting properties.

Given a consistent but incomplete theory, how should one choose a random model of that theory?

My proposal is rather simple. Just assign probabilities to sentences in such that if an adversary were to choose a model, your Worst Case Bayes Score is maximized. This assignment of probabilities represents a probability distribution on models, and choose randomly from this distribution. However, it will take some work to show that what I just described even makes sense. We need to show that Worst Case Bayes Score can be maximized, that such a maximum is unique, and that this assignment of probabilities to sentences represents an actual probability distribution. This post gives the necessary definitions, and proves these three facts.

Finally, I will show that any given probability assignment is coherent if and only if it is impossible to change the probability assignment in a way that simultaneously improves the Bayes Score by an amount bounded away from 0 in all models. This is nice because it gives us a measure of how far a probability assignment is from being coherent. Namely, we can define the "incoherence" of a probability assignment to be the supremum amount by which you can simultaneously improve the Bayes Score in all models. This could be a useful notion since we usually cannot compute a coherent probability assignment so in practice we need to work with incoherent probability assignments which approach a coherent one.

Now, let's move on to the formal definitions and proofs.

Fix some language L, for example the language of first order set theory. Fix a consistent theory T of L, for example ZFC. Fix a nowhere zero probability measure μ on L, for example μ(ϕ)=2−ℓ(ϕ), where ℓ(ϕ) is the number of bits necessary to encode ϕ.

A probability assignment of L is any function from L to the interval [0,1]. Note that this can be any function, and does not have to represent a probability distribution. Given a probability assignment P of L, and a model M of T, we can define the Bayes Score of

...
New LW Meetup: Rochester NY
by FrankAdamek — 2015-08-28T16:49:35.579Z

This summary was posted to LW Main on August 21st. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Rochester LessWrong and Transhumanists: 12 September 2015 01:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

Ann Arbor meetup: 22 August 2015 02:00AM
NH Meet-up: 25 August 2015 07:00PM
Oslo Lesswrong meetup, September: 27 September 2015 04:00PM
Warsaw August Meetup: 30 August 2015 03:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Sydney Meetup - August: 26 August 2015 06:30PM
Sydney Rationality Dojo - September: 06 September 2015 04:00PM
Sydney Meetup - September: 23 September 2015 06:30PM
Vienna: 26 September 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Cincinnati, Cleveland, Frankfurt, Helsinki, Marin CA, Ottawa, Pittsburgh, Portland, Salt Lake City, Southern ...
Is my brain a utility minimizer? Or, the mechanics of labeling things as "work" vs. "fun"
by contravariant — 2015-08-28T01:12:41.091Z

I recently encountered something that is, in my opinion, one of the most absurd failure modes of the human brain. I first encountered this after introspection on useful things that I enjoy doing, such as programming and writing. I noticed that my enjoyment of the activity doesn't seem to help much when it comes to motivation for earning income. This was not boredom from too much programming, as it did not affect my interest in personal projects. What it seemed to be, was the brain categorizing activities into "work" and "fun" boxes. On one memorable occasion, after taking a break due to being exhausted with work, I entertained myself, by programming some more, this time on a hobby personal project (as a freelancer, I pick the projects I work on so this is not from being told what to do). Relaxing by doing the exact same thing that made me exhausted in the first place.

The absurdity of this becomes evident when you think about what distinguishes "work" and "fun" in this case, which is added value. Nothing changes about the activity except the addition of more utility, making a "work" strategy always dominate a "fun" strategy, assuming the activity is the same. If you are having fun doing something, handing you some money can't make you worse off. Making an outcome better makes you avoid it. Meaning that the brain is adopting a strategy that has a (side?) effect of minimizing future utility, and it seems like it is utility and not just money here - as anyone who took a class in an area that personally interested them knows, other benefits like grades recreate this effect just as well. This is the reason I think this is among the most absurd biases - I can understand akrasia, wanting the happiness now and hyperbolically discounting what happens later, or biases that make something seem like the best option when it really isn't. But knowingly punishing what brings happiness just because it also benefits you in the future? It's like the discounting curve dips into the negative region. I would really like to learn where is the dividing line between which kinds of added value create this effect and which ones don't (like money obviously does, and immediate enjoyment obviously doesn't). Currently I'm led to believe that the difference is present utility vs. future utility, (as I mentioned above) or final vs. instrumental goals, and pleas...
Meetup : San Francisco Meetup: Projects
by maia — 2015-08-27T15:05:36.951Z

Discussion article for the meetup : San Francisco Meetup: Projects

WHEN:
31 August 2015 06:15:00PM (-0700)

WHERE:
1061 Market St #4, San Francisco, CA 94103

The last one went well, so we're doing this again. Bring something you're working on, especially something you might like help from other people on, and work on it at the meetup.

Roger can be reached at 301-458-0764 if you need help getting in. As always, feel free to show up late.

Discussion article for the meetup : San Francisco Meetup: Projects
Meetup : Discussions on Determining Between Right and Wrong
by APXAS — 2015-08-27T08:12:11.697Z

Discussion article for the meetup : Discussions on Determining Between Right and Wrong

WHEN:
02 September 2015 01:10:39AM (-0700)

WHERE:
11066 Santa Monica Blvd, Los Angeles, CA 90025

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: Ever wonder if what you're doing is right or wrong? Me too! After all, what's the point of all this "rationality" business if you it doesn't help you make better decisions? Seriously, this website is named "Less Wrong" surely at some point we should talk about how we figure out what wrong is. So if you feel like having a discussion about this, come on over. (Also, this sounds like one of those things that can spiral somewhere bad really easily, so we'll be focusing less on specific beliefs that people feel are right and wrong, and more on the different ways people interpret right and wrong, and how to recognize which is which.)

Recommended Reading: None! Just come on over!

No prior exposure to Less Wrong is required; this will be generally accessible

Discussion article for the meetup : Discussions on Determining Between Right and Wrong
Words per person year and intellectual rigor
by PhilGoetz — 2015-08-27T03:31:49.373Z

Continuing my cursory exploration of semiotics and post-modern thought, I'm struck by the similarity between writing in those traditions, and picking up women. The most-important traits for practitioners of both are energy, enthusiasm, and confidence. In support of this proposition, here is a photo of Slavoj Zizek at his 2006 wedding:

Having philosophical or logical rigor, or demonstrating the usefulness of your ideas using empirical data, does not seem to provide a similar advantage, despite taking a lot of time.

I speculate that semiotics and post-modernism (which often go hand-in-hand) became popular by natural selection. They provide specialized terminologies which give the impression of rigorous thought without requiring actual rigor. People who use them can thus out-publish their more-careful competitors. So post-modernism tends to drive rigorous thought out of any field it enters.

(It's possible to combine post-modern ideas and a time-consuming empirical approach, as Thomas Kuhn did in The Structure of Scientific Revolutions. But it's uncommon.)

If rigorous thought significantly reduces publication rate, we should find that the rigor of a field or a person correlates inversely with words per person-year. Establishing that fact alone, combined with the emphasis on publication in academics, would lead us to expect that any approach that allowed one to fake or dispense with intellectual rigor in a field would rapidly take over that field.

We should also then suspect that any supposedly rigorous field, or school within a field, which has a markedly high WPY, is bullshit.

I listed some prominent names from various fields under two categories, "rigorous" and "not rigorous". Then I counted the number of books and articles they published per year. I stop the year count at min(last year of publication, year of their death) because some people slow down in their last years. I estimate average page count for articles when they're given, and count 300 pages as 1 book. Sometimes I threw out the first publication when it appeared to be a thesis or other very early work written many years before anything else. I tried not to count patents, talks, letters, notes, reports, unpublished papers, translations, or republications. I counted autobiographies if they wrote them during their productive years, but

...