trentmkelly/LessWrong-43k huggingface dataset:
A Problem with the Modified Demski Prior
by abramdemski — 2015-09-13T23:37:45.000Z

The basic idea of the modified Demski prior is to let Turing machines enumerate sentences, using a Solomonoff-style prior. As such, we might hope that this prior would have the good properties of Solomonoff induction when applied to an empirical problem. This is not the case. The argument here comes out of discussions with Scott. (What follows is not a proof, but only a proof sketch with a hole at the end.)

The modified Demski prior is defined as follows. Interpret a universal machine U as enumerating sentences (giving us sentence codes with a special separator code between them, possibly forever). Define a distribution over such enumerations by feeding the machine random bits. Starting with a base theory (our axioms), keep extending the set by drawing new theories T from the distribution, and keeping those which are consistent with the axioms and with what we've kept so far. The limit of this process is one random extension of the base theory.

Consider the case of first-order logic with the axioms of PA, and an infinite stock of predicate symbols. (We really need only one additional predicate symbol, however.)

Suppose we've represented the problem as sequence induction, as we would for the Solomonoff distribution: we have a sequence Sn of 0s and 1s, which we want to predict. Encode this with a distinguished predicate not used for anything else, S(n).

Suppose that Sn encodes a stochastic problem, so that no deterministic machine predicts the sequence with 100% accuracy. Let's say it encodes flips of a biased coin which lands on True 30% of the time.

Consider only the first draw from U, and furthermore, only consider the cases where U enumerates sentences of the form S(n) or ¬S(n) exclusively. Because U is enumerating sentences rather than deciding sentences in sequence, it can skip as many sentences as it likes. Suppose that we've updated on N−1 sentences so far. For the Nth sentence, we make a prediction using a mixture distribution over all the machines T which haven't been wrong yet, weighted by their probability of coming from U. The mixture distribution consisting of machines which have made predictions for all S(n) so far is likely to be quite good, predicting very close to .3 on True for sufficiently large N. Those which have predicted few or none, however, may be quite bad. The problem is that these will have more weight, asymptotically. The argument is

...
The Two-Update Problem, Part 1
by abramdemski — 2015-09-13T23:01:13.000Z

This is a problem which Scott Garrabrant and I were thinking about quite a bit last fall, but moved on from as other problems in logical uncertainty captured our interest. As far as I know, it remains unsolved!

Scott and I were working on the problem of probabilistic priors, or as we've started calling them now, random extensions. Suppose you have a set (possibly empty) of axioms in a logic, and you want a "reasonable" probability distribution over the rest of the sentences, which are neither proved nor disproved by your set of sentences. This probability assignment is required to be coherent in the sense given by Paul Christiano or me: they "respect the logic", namely (using Paul's version):

For any ϕ and ψ, B(ϕ)=B(ϕ∧ψ)+B(ϕ∧¬ψ).

For any tautology ϕ, B(ϕ)=1.

For any contradiction ϕ, B(ϕ)=0.

Paul proves that this is equivalent to requiring P to be derived from a probability distribution on models (so that the probability of a sentence is the probability that the sentence is true).

Denote the random extension starting with theory T as PT(ϕ). We additionally require that for ϕ∈T, B(ϕ)=1.

We also wanted these probability distributions to be approximable, among other desirable properties.

The two-update problem began as the observation that, for my prior, adding a sentence ϕ to the initial theory is quite different from performing a Bayesian update on ϕ. As our research progressed, we noticed that this seemed to be true of all the proposals we came up with. There were two distinct ways to "update" on a sentence: either do a Bayesian update to make its probability 1, or add it to the initial set of axioms which are true before the prior's construction.

Stated as a formal problem, this may look easy to "solve". For finite non-empty T, we can simply re-define PT(ϕ)=P∅(ϕ|T). (There is a complication if you try to do this for infinite T, which is explained here. ) However, this seems to miss the point. Despite re-defining things in this way, if you look at the actual specification of the priors we were considering, a kind of non-bayesian update was quite visible. The non-bayesian update doesn't go away if we take away the set of axioms (AKA the theory we're extending into a probability distribution). Rather, it is visible in how we update on proofs.

As we approximate P to increasing degree, we observe proofs of one thing or another and incorporate this into the distribu

...
Group Rationality Diary, September 13-26
by Dahlen — 2015-09-13T08:27:02.294Z

This is the public group rationality diary for September 13th - 26th, 2015. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Archive of previous rationality diaries
Meetup : Tel Aviv: Black Holes after Jacob Bekenstein
by Squark — 2015-09-13T06:29:32.613Z

Discussion article for the meetup : Tel Aviv: Black Holes after Jacob Bekenstein

WHEN:
24 November 2015 08:00:00AM (+0300)

WHERE:
Yigal Alon 98, Tel Aviv

We will meet in Google Israel (Electra Tower) on floor 29, as always.

If you have trouble finding the location, feel free to call me (Vadim) at 0542600919.

Zohar Komargodski, a theoretical physicist from the Weizmann Institute of Science, will give a comprehensive review of the physics of black holes:

Physics in the past has progressed by connecting hitherto different concepts, for example: Space & Time, Electricity & Magnetism, Particles & Waves, and many others. We are now in the midst of another such exciting revolution, where Space-Time and Information Theory are being related. Jacob Bekenstein has laid out some of the basic concepts that appear in this surprising new developments. Thought experiments involving Black-Holes were central to the initial leaps that Jacob made.

The goal of the presentation is to describe in an informal fashion (requiring no particular prior knowledge of information theory or physics) what are Black-Holes, what did Jacob realise, what has been understood since his seminal papers, and what are the central remaining (formidable) challenges.

(Facebook event)[https://www.facebook.com/events/1666895726878967/]

Discussion article for the meetup : Tel Aviv: Black Holes after Jacob Bekenstein
Useful Wiki and special pages
by ScottL — 2015-09-13T04:33:34.176Z

This is a list of what I think are the most useful wiki and special pages on less wrong. If you know of any other pages that you think are useful, let me know and I will add them in. After I think enough time has passed, I will add the links below into the FAQ "Stuff to know about" section, if they are not already there.

Sequences - provides information on the main sequences

Rationality Materials - like the sequences page, but also has some links to standalone posts that are worth reading.

Unified list of new posts from both main and discussion

Popular pages - most viewed pages (includes wiki pages). You can also click on the "Main" and "Discussion" buttons and there will be a button for top scoring main posts and top scoring discussion posts. You can also change the filter for the time period (three months, this year, all time etc.). It is also possible to get the top scoring posts from both discussion and main.

Article summaries - contains a list of summaries of the articles on Less Wrong, in chronological order

Less Wrong Canon on Rationality - short abstracts of most of the topics on less wrong

Special threads - This is a collection of interesting discussion threads that have appeared on Less Wrong.

Categories - This page contains a list of categories for the pages on the wiki

Jargon - This is a short list of common terms and phrases used on LessWrong

FAQ page - main FAQ on wiki

John Maxwell IV/FAQ - another FAQ

Comment Formatting - information on how to use comment formatting

External resources - links to non less wrong resources that may be useful

Open problems on less wrong

Less Wrong meetup groups - information on all of the less wrong meetup groups

Rationality power tools - list of computer programs, e.g. anki, that assist with rationality.
Being unable to despair
by So8res — 2015-09-13T03:00:00.000Z
Link: https://mindingourway.com/being-unable-to-despair/

Sometimes, when people see that their life is about to get a lot harder, they start buckling down. Other times, they start despairing, or complaining, or preparing excuses so that they can have one ready when the inevitable failure hits, or giving up entirely and then failing with abandon. These next few posts assume that you have the former demeanor, and they might not be helpful to people who are inclined to respond to new difficulties with despair. Remember the law of equal and opposite advice! (For every person who needs a certain piece of advice, there is someone else who needs the opposite advice.)

With that said, I'm going to spend a few words giving some tips about how to have the former demeanor, if you want to. The first piece of pertinent advice is that the way you respond to challenges is context dependent; even if you've already been known to respond to some problems by despairing, there are likely other problems that you respond to by buckling down.

There is a specific mindset that, in my experience, makes it much easier to adopt the "buckle down" demeanor. This is the mindset where "not doing anything" doesn't seem like an available option in the action-space. I've written a bit before about how I think many people think there is a default "rest state", and this is a related concept: many people seem to think that there is a privileged "don't do anything" action, that consists of something like curling up into a ball, staying in bed, and refusing to answer emails. It's much easier to adopt the "buckle down" demeanor when, instead, curling up in a ball and staying in bed feels like just another action. It's just another way to respond to the situation, which has some merits and some flaws.

So this is my second piece of advice, if you want to be the sort of person who buckles down in the face of hardship: see the world in terms of possible responses. See curling up in bed and ignoring the world as just one possible response, rather than an escape hatch. Dispel the illusion that some actions are labeled "do nothing," and notice that those, too, are responses. There is no privileged null choice.

(That's not to say that it's bad to curl up in a ball on your bed and ignore the world for a while. Sometimes this is exactly what you need to recover. Sometimes it's what the monkey is going to do regardless of what you decide. The point is that when nature o...
The Heuristic About Representativeness Heuristic 
by BT_Uytya — 2015-09-12T23:15:38.547Z

(x-posted from my blog)

Some people think that the problem with representativeness heuristic is a base rate neglect. I hold that this is incorrect, and the problem is deeper than that, and simple use of a base rate isn't going to fix it. This makes the idea about "look at the base rate!" a heuristic as well.

The thing is, there is a fundamental difference between "How strongly E resembles H" and "How strongly H implies E". The latter question is about P(E|H), and this number could be used in Bayesian reasoning, if you add P(E|!H) and P(H)[1]. The former question — the question humans actually answer when asked to judge about whether something is likely — sometimes just could not be saved at all.

Several examples to get point across:

1) Conspiracy theorists / ufologists: naively, their existence strongly points to a world where UFOs exist, but really, their existence is very weak evidence of UFOs (human psychology suggests that ufologists could exist in a perfectly alienless world), and even could be an evidence against them, because if Secret World Government was real, we expect it to be very good at hiding, and therefore any voices who got close to the truth will be quickly silenced.

So, the answer to "how strongly E resembles H?" is very different from "how much is P(E|H)?". No amount of accounting for base rate is going to fix this.

2) Suppose that some analysis comes too good in a favor of some hypothesis.

Maybe some paper argues that leaded gasoline accounts for 90% variation in violent crime (credit for this example goes to /u/yodatsracist on reddit). Or some ridiculously simple school intervention is claimed to have a gigantic effect size.

Let's take leaded gasoline, for example. On the surface, this data strongly "resembles" a world where leaded gasoline is indeed causing a violence, since 90% suggest that effect is very large and is very unlikely to be a fluke. On the other hand, this effect is too large, and 10% of "other factors" (including but not limited to: abortion rate, economic situation, police budget, alcohol consumption, imprisonment rate) is too small of percentage.

The decline we expect in a world of harmful leaded gasoline is more like 10% than 90%, so this model is too good to be true; instead of being very strong evidence in favor, this analysis could be either irrelevant (just a random botched analys

...
[Link] Using mindkillers to promote rationality 
by Gleb_Tsipursky — 2015-09-12T15:44:37.030Z

As part of my broader project of promoting rationality to a wide audience , I published an article in Salon entitled "Get Donald Trump out of my brain: The neuroscience that explains why he’s running away with the GOP." I'd welcome your thoughts on this article itself, and also meta-comments on the strategy of using mindkillers such as politics to raise the sanity waterline by smuggling in rationality memes into such popular and populist venues.
How To Win The AI Box Experiment (Sometimes)
by pinkgothic — 2015-09-12T12:34:53.200Z

Preamble

This post was originally written for Google+ and thus a different audience.

In the interest of transparency, I haven't altered it except for this preamble and formatting, though since then (at urging mostly of ChristianKl - thank you, Christian!) I've briefly spoken to Eliezer via e-mail and noticed that I'd drawn a very incorrect conclusion about his opinions when I thought he'd be opposed to publishing the account. Since there's far too many 'person X said...' rumours floating around in general, I'm very sorry for contributing to that noise. I've already edited the new insight into the G+ post and you can also find that exact same edit here.

Since this topic directly relates to LessWrong and most people likely interested in the post are part of this community, I feel it belongs here. It was originally written a little over a month ago and I've tried to find the sweet spot between the extremes of nagging people about it and letting the whole thing sit just shy of having been swept under a rug, but I suspect I've not been very good at that. I have thus far definitely erred on the side of the rug.

How To Win The AI Box Experiment (Sometimes)

A little over three months ago, something interesting happened to me: I took it upon myself to play the AI Box Experiment as an AI.

I won.

There are a few possible reactions to this revelation. Most likely, you have no idea what I'm talking about, so you're not particularly impressed. Mind you, that's not to say you should be impressed - that's to contrast it with a reaction some other people have to this information.

This post is going to be a bit on the long side, so I'm putting a table of contents here so you know roughly how far to scroll if you want to get to the meat of things:

1. The AI Box Experiment: What Is It?

2. Motivation

2.1. Why Publish?

2.2. Why Play?

3. Setup: Ambition And Invested Effort

4. Execution

4.1. Preliminaries / Scenario

4.2. Session

4.3. Aftermath

5. Issues / Caveats

5.1. Subjective Legitimacy

5.2. Objective Legitimacy

5.3. Applicability

6. Personal Feelings

7. Thank You

Without further ado:

1. The AI Box Experiment: What Is It?

The AI Box Experiment was devised as a way to put a common rebuttal to AGI (Artificial General Intelligence) risk concerns to the test: "We could just keep the AI in a box and purely let it answer any questions its posed." (As a footnote, note that an AI...
Meetup : West LA: Fact-Checking
by abramdemski — 2015-09-11T23:12:25.417Z

Discussion article for the meetup : West LA: Fact-Checking

WHEN:
16 September 2015 07:00:00PM (-0700)

WHERE:
11066 santa monica blvd, la, ca

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: Checking assumptions is important. Sometimes that can be a matter of thinking things through yourself, but often we are better off looking for what other people have found. The first hurdle is to notice that we can check an assumption in this way -- often we do not even get that far. Once we've thought of actually checking, a simple search or wikipedia check may be sufficient. Sometimes, however, we need to dive deeper.

Where do you look for information? What do you do if a search brings up contradictory information? If an issue is not "settled science", do you give up? How do you decide whether an article is BS? How do you account for the relative weight of evidence from anecdotal answers to a question vs scientific research? How do you weight arguments against empirical evidence?

Recommended Reading:

This will be a bring-your-own-recommended-reading meetup! This forces you to practice the skill of investigating, differentiating useful references, and so on. Please make the assumption that no one else will do this and you'll have to be the only person bringing any useful information.

No prior exposure to Less Wrong is required; this will be generally accessible.

Discussion article for the meetup : West LA: Fact-Checking
Meetup : LW Scotland September Meetup
by cjb230 — 2015-09-11T19:54:34.985Z

Discussion article for the meetup : LW Scotland September Meetup

WHEN:
13 September 2015 02:00:00PM (+0100)

WHERE:
1a Dundas Street, Glasgow G1 2AH

As usual, this note is a mirror of the canonical meetup note, which is here: https://www.facebook.com/events/1694401560778891/

But at the time of posting, these are the details:

The September meetup will be on Sunday the thirteenth, in Glasgow. We'll start at 2pm at the Nero opposite Queen Street Station, and then move around the corner to Waxy O'Connors at 3:30pm.

This month's rough agenda is:

1 - Intros. What's the most (or least!) rational thing you did in the last month?

2 - A short Fermi estimation intro and game

3 - Discussion of part G of the Rationality book, "Against Rationalization", which is chapters 67 to 80.

We'll try to cover that in the cafe before moving to Waxy's. In Waxy's, we talk about whatever takes our fancy!

Locations:
Cafe Nero - 1a Dundas Street, Glasgow G1 2AH
Waxy O'Connor's - 44 W George St, Glasgow G2 1DH

The two are 30s walk apart.

Discussion article for the meetup : LW Scotland September Meetup
Political Debiasing and the Political Bias Test 
by Stefan_Schubert — 2015-09-11T19:04:49.452Z

Cross-posted from the EA forum. I asked for questions for this test here on LW about a year ago. Thanks to those who contributed.
Rationally, your political values shouldn't affect your factual beliefs. Nevertheless, that often happens. Many factual issues are politically controversial - typically because the true answer makes a certain political course of action more plausible - and on those issues, many partisans tend to disregard politically uncomfortable evidence.

This sort of political bias has been demonstrated in a large number of psychological studies. For instance, Yale professor Dan Kahan and his collaborators showed in a fascinating experiment that on politically controversial questions, people are quite likely to commit mathematical mistakes that help them retain their beliefs, but much less likely to commit mistakes that would force them to give up those belies. Examples like this abound in the literature.

Political bias is likely to be a major cause of misguided policies in democracies (even the main one according to economist Bryan Caplan). If they don’t have any special reason not to, people without special knowledge defer to the scientific consensus on technical issues. Thus, they do not interfere with the experts, who normally get things right. On politically controversial issues, however, they often let their political bias win over science and evidence, which means they’ll end up with false beliefs. And, in a democracy voters having systematically false beliefs obviously more often than not translates into misguided policy.

Can we reduce this kind of political bias? I’m fairly hopeful. One reason for optimism is that debiasing generally seems to be possible to at least some extent. This optimism of mine was strengthened by participating in a CFAR workshop last year. Political bias seems not to be fundamentally different from other kinds of biases and should thus be reducible too. But obviously one could argue against this view of mine. I’m happy to discuss this issue further.

Another reason for optimism is that it seems that the level of political bias is actually lower today than it was historically. People are better at judging politically controversial issues in a detached, scientific way today than they were in, say, the 14th century. This shows that progre...