trentmkelly/LessWrong-43k huggingface dataset:
The Two-Update Problem, Part 2
by abramdemski — 2015-08-22T00:47:05.000Z

In the first part, I explained that the two-update problem can be understood in two different ways: (1) adding an axiom ϕ to a logic before defining a probability distribution P for it is generally different from a Bayesian update on ϕ, and (2) improving our approximation of P upon observing a proof of ϕ is generally different from a Bayesian update. (These two forms appear to really be the same problem.)

This may not be fundamentally concerning: perhaps neither of these should be a Bayesian update. However, Paul Christiano's paradox of ignorance suggests that we will have some problems if we depart from Bayesian updates. I closed with the question of whether we can circumvent this by making the update-on-proofs into a Bayesian update; I'll address that question here.

What we would want to do is start with a probability distribution which is ignorant of logic, and do a Bayesian update on facts of logic as we prove them. (I think this suggestion was made once or twice elsewhere; I'll link to original sources as I find them, but let me know if you know of them.) Can this be made to work?

The simplest case is that we start out by assigning every statement a probability of 1/2, independent of all other statements. We then run our theorem-prover, and update on any proofs we find. This doesn't just mean updating single sentences as they are proved; indeed, this would do nothing to alter the probability of other sentences. The important thing is to eliminate combinations of sentence truth-assignments which are proven to be inconsistent. For example, if we find that ϕ⊢ψ, we rule out the states where ϕ is true and ψ is false. If ψ is a new sentence not yet touched by anything else, this modifies P(ϕ) by a likelihood ratio of 1:2 (making it less probable since its truth depends on a more constrained circumstance). Similarly, if ϕ is new, P(ψ) is modified by the likelihood ratio 2:1.

It follows that no matter how far along in the process we are, the likelihood ratio of ϕ is always being doubled or halved: it's halved when we encounter a proof of a new instance of ϕ⊢ϕ∨ψ (choosing ψ to select an untouched ϕ∨ψ), and doubled when we encounter a new instance of ϕ∧ψ⊢ϕ (again, chosen so that ϕ∧ψ is fresh). This means that the distribution can only converge if it converges to probabilities that are just 0 or 1. Furthermore, if ϕ is never proved or disproved, whether it converg

...
Robert Aumann on Judaism
by iarwain1 — 2015-08-21T19:13:05.917Z

Just came across this interview with Robert Aumann. On pgs. 20-27 he describes why and how he believes in Orthodox Judaism. I don't really understand what he's saying. Key quote (I think):

H (interviewer): Take for example the six days of creation; whether or not this is how it happened is practically irrelevant to one is decisions and way of conduct. It is on a different level.

A (Aumann): It is a different view of the world, a different way of looking at the world. That is why I prefaced my answer to your question with the story about the roundness of the world being one way of viewing the world. An evolutionary geological perspective is one way of viewing the world. A different way is with the six days of creation. Truth is in our minds. If we are sufficiently broad-minded, then we can simultaneously entertain different ideas of truth, different models, different views of the world.

H: I think a scientist will have no problem with that. Would a religious person have problems with what you just said?

A: Different religious people have different viewpoints. Some of them might have problems with it. By the way, I'm not so sure that no scientist would have a problem with it. Some scientists are very doctrinaire.

Anybody have a clue what he means by all this? Do you think this is a valid way of looking at the world and/or religion? If not, how confident are you in your assertion? If you are very confident, on what basis do you think you have greatly out-thought Robert Aumann?

Please read the source (all 7 pages I referenced, rather than just the above quote), and think about it carefully before you answer. Robert Aumann is an absolutely brilliant man, a confirmed Bayesian, author of Aumann's Agreement Theorem, Nobel Prize winner, and founder / head of Hebrew University's Center for the Study of Rationality. Please don't strawman his arguments or simply dismiss them!
Weekly LW Meetups
by FrankAdamek — 2015-08-21T16:23:04.874Z

This summary was posted to LW Main on August 14th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Ann Arbor meetup: 22 August 2015 02:00AM
LW Cologne meetup: 15 August 2015 05:00PM
Munich LessWrong Meetup August: Correctness Heuristics: 15 August 2015 07:48PM
NH Meet-up: 25 August 2015 07:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

San Francisco Meetup: Fun and Games: 17 August 2015 06:15PM
Sydney Meetup - August: 26 August 2015 06:30PM
Sydney Rationality Dojo - September: 06 September 2015 04:00PM
Sydney Meetup - September: 23 September 2015 06:30PM
Vienna: 15 August 2015 03:00PM
Washington, D.C.: Mini Talks: 16 August 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Cincinnati, Cleveland, Frankfurt, Helsinki, Marin CA, Ottawa, Pittsburgh, Portland, Salt Lake City, Southern California (...
Meetup :  Meetup : San Antonio Meetup
by Bryan-san — 2015-08-21T16:21:09.830Z

Discussion article for the meetup : Meetup : San Antonio Meetup

WHEN:
30 August 2015 12:30:00PM (-0500)

WHERE:
Yumi Berry, 12651 Vance Jackson Rd Suite #118 San Antonio Tx, 78230

Bubble tea, frozen yogurt, and discussion at Yumi Berry! All are welcome!

New Meetup to discuss rationality and all things LessWrong and meet the local community. Look for the sign that says Less Wrong!

Discussion article for the meetup : Meetup : San Antonio Meetup
Pro-Con-lists of arguments and onesidedness points
by Stefan_Schubert — 2015-08-21T14:15:36.306Z

Follow-up to Reverse Engineering of Belief Structures

Pro-con-lists of arguments such as ProCon.org and BalancedPolitics.org fill a useful purpose. They give an overview over complex debates, and arguably foster nuance. My network for evidence-based policy is currently in the process of constructing a similar site in Swedish.

I'm thinking it might be interesting to add more features to such a site. You could let people create a profile on the site. Then you would let them fill in whether they agree or disagree with the theses under discussion (cannabis legalization, GM foods legalization, etc), and also whether they agree or disagree with the different argument for and against these theses (alternatively, you could let them rate the arguments from 1-5).

Once you have this data, you could use them to give people different kinds of statistics. The most straightforward statistic would be their degree of "onesidedness". If you think that all of the arguments for the theses you believe in are good, and all the arguments against them are bad, then you're defined as onesided. If you, on the other hand, believe that some of your own side's arguments are bad, whereas some of the opponents' arguments are good, you're defined as not being onesided. (The exact mathematical function you would choose could be discussed.)

Once you've told people how one-sided they are, according to the test, you would discuss what might explain onesidedness. My hunch is that the most plausible explanation normally is different kinds of bias. Instead of reviewing new arguments impartially, people treat arguments for their views more leniently than arguments against their views. Hence they end up being onesided, according to the test.

There are other possible explanations, though. One is that all of the arguments against the thesis in question actually are bad. That might happen occassionally, but I don't think that's very common. As Eliezer Yudkowsky says in "Policy Debates Should Not Appear One-sided":

On questions of simple fact (for example, whether Earthly life arose by natural selection) there's a legitimate expectation that the argument should be a one-sided battle; the facts themselves are either one way or another, and the so-called "balance of evidence" should reflect this. Indeed, under the Bayesian definition of evidence, "strong evidence" is just that sort of evidence wh

...
Glossary of Futurology
by mind_bomber — 2015-08-21T05:51:45.602Z

Hi guys,

So I've been curating this glossary over at https://www.reddit.com/r/Futurology/. I want it to be sort of an introduction to future focused topics. A list of words that the layman can read and be inspired by. I try to stay away from household words (i.e. cyberspace), science fiction topics (i.e. dyson sphere), words that describe themselves (i.e. self driving cars), obscure and rarely used words (i.e. betelgeuse-brain), and words that can't be found in most dictionaries (i.e. Rocko's Basilisk (i've been meaning to remove that one)). Most of the glossary is from words and phrases I find on the /r/Futurology forum. I have a whole other list with potential words for the glossary that i collect just waiting for the day to be added (i.e particle accelerator, Aerogel, proactionary principle). I find curating the glossary to be more of an art than a science. I try to balance the list between science, technology, philosophy, ideology, and sociology. I like to find related topics to expand the list (i.e. terraforming & geoengineering). Even though the glossary is in alphabetical order i want it to read somewhat like a story.

Anders Sandberg of The Future of Humanities Institute, Oxford told me "I like the usefulness of your list..."

I'm interested to know what you guys think.

Glossary located below (the See /r/*.* is native to the reddit website. See /r/*.* links the glossary to subreddits (other reddit pages) related to that word or phrase on the reddit website):

3D printing - a process for making a physical object from a three-dimensional digital model, typically by laying down many successive thin layers of a material. See /r/3Dprinting.

Abolitionism - an ethical ideology based upon a perceived obligation to use technology to eliminate involuntary suffering and maximize happiness in all sentient life.

Accelerating change - a perceived increase in the rate of technological (and sometimes social and cultural) progress throughout history, which may suggest faster and more profound change in the future.

Artificial general intelligence (AGI) - the intelligence of a (hypothetical) machine that could successfully perform any intellectual task that a human being can. It is a primary goal of artificial intelligence research and an i...
Unlearning shoddy thinking
by MalcolmOcean — 2015-08-21T03:07:03.722Z

School taught me to write banal garbage because people would thumbs-up it anyway. That approach has been interfering with me trying to actually express my plans in writing because my mind keeps simulating some imaginary prof who will look it over and go "ehh, good enough".

Looking good enough isn't actually good enough! I'm trying to build an actual model of the world and a plan that will actually work.

Granted, school isn't necessarily all like this. In mathematics, you need to actually solve the problem. In engineering, you need to actually build something that works. But even in engineering reports, you can get away with a surprising amount of shoddy reasoning. A real example:

Since NodeJS uses the V8 JavaScript engine, it has native support for the common JSON (JavaScript Object Notation) format for data transfer, which means that interoperability between SystemQ and other CompanyX systems can still be fairly straightforward (Jelvis, 2011).

This excerpt is technically totally true, but it's also garbage, especially as a reason to use NodeJS. Sure, JSON is native to JS, but every major web programming language supports JSON. The pressure to provide citable justifications for decisions which were made for reasons more like "I enjoy JavaScript and am skilled with it," produces some deliberately confirmation-biased writing. This is just one pattern—there are many others.

I feel like I need to add a disclaimer here or something: I'm a ringed engineer, and I care a lot about the ethics of design, and I don't think any of my shoddy thinking has put any lives (or well-being, etc) at risk. I also don't believe that any of my shoddy thinking in design reports has violated academic integrity guidelines at my university (e.g. I haven't made up facts or sources).

But a lot of it was still shoddy. Most students are familiar with the process of stating a position, googling for a citation, then citing some expert who happened to agree. And it was shoddy because nothing in the school system was incentivizing me to make it otherwise, and I reasoned it would have cost more to only write stuff that I actually deeply and confidently believed, or to accurately and specifically present my best model of the subject at hand. I was trying to spend as little time and attention as possible working on school things, to free up more time and attention for working on my business, ...
On Overconfidence
by Scott Alexander — 2015-08-21T02:21:00.000Z

[Epistemic status: This is basic stuff to anyone who has read the Sequences, but since many readers here haven’t I hope it is not too annoying to regurgitate it. Also, ironically, I’m not actually that sure of my thesis, which I guess means I’m extra-sure of my thesis]

I.

A couple of days ago, the Global Priorities Project came out with a calculator that allowed you to fill in your own numbers to estimate how concerned you should be with AI risk. One question asked how likely you thought it was that there would be dangerous superintelligences within a century, offering a drop down menu with probabilities ranging from 90% to 0.01%. And so people objected: there should be options to put in only one a million chance of AI risk! One in a billion! One in a…

For example, a commenter writes that: “the best (worst) part: the probability of AI risk is selected from a drop down list where the lowest probability available is 0.01%!! Are you kidding me??” and then goes on to say his estimate of the probability of human-level (not superintelligent!) AI this century is “very very low, maybe 1 in a million or less”. Several people on Facebook and Tumblr say the same thing – 1/10,000 chance just doesn’t represent how sure they are that there’s no risk from AI, they want one in a million or more.

Last week, I mentioned that Dylan Matthews’ suggestion that maybe there was only 10^-67 chance you could affect AI risk was stupendously overconfident. I mentioned that was thousands of lower than than the chance, per second, of getting simultaneously hit by a tornado, meteor, and al-Qaeda bomb, while also winning the lottery twice in a row. Unless you’re comfortable with that level of improbability, you should stop using numbers like 10^-67.

But maybe it sounds like “one in a million” is much safer. That’s only 10^-6, after all, way below the tornado-meteor-terrorist-double-lottery range…

So let’s talk about overconfidence.

Nearly everyone is very very very overconfident. We know this from experiments where people answer true/false trivia questions, then are asked to state how confident they are in their answer. If people’s confidence was well-calibrated, someone who said they were 99% confident (ie only 1% chance they’re wrong) would get the question wrong only 1% of th...
[Repost] My First Insight After CFAR, or the Art of Not Needing to Ask Permission
by Ben Pace — 2015-08-20T23:00:00.000Z

I wrote this in the summer of 2015, in the week after I attended a CFAR workshop and met the in-person community for the first time. I no longer have the blog I wrote this to at the time, so I've reposted it to my new blog here for posterity.

At the CFAR workshops, we had tutoring sessions, in which we practiced helping another person solve a bug using a particular technique. I had three short opportunities to tutor a person in a particular technique, which allowed me to get better acquainted with the structure of the technique without being worried about my own bugs. Here’s an interesting conversation that occurred in at least two of the sessions:

Them: “So I’ve got this problem.?

Me: “Have you thought of any possible solutions??

Them: “Well, maybe doing X??

Me: “Oh, doing X?

Them: “Thank you!?

In the above conversation, in an open and curious tone of voice, I repeated back to the person exactly what they had said to me. Yet for them, this had turned something from a bad idea to a good one – I mean, they were enthusiastic in their thank-you’s! This was fascinating to me; it seemed that the idea was getting validation from being said aloud by someone else, in a positive tone of voice (I imagine if, as I’d said “x??, I’d raised the pitch of my voice skeptically and looked over my glasses at them as though they were being silly, they might not have taken to their idea so well). So I marked this as important.

At a later point in the workshop, Val talked about something he called ‘The Art of Not Needing Permission’. He gave an example to illustrate what he was pointing to (my thanks to Eliezer for permission to post this story):

When Eliezer began publishing HPMoR, many people gave him arguments against spending his time on it. When Eliezer gave reasons to think it would work, his reasons his reasons called on a causal model of successful fiction: he was modelling the reasons why fiction spreads, why people enjoy it and what he’d have to do to write fiction that would motivate people to change their lifestyles. The people who told him he was wrong, tended to give what felt to them like an outside view, arguments such as “It’s more than a little improbable that you’re a writer of the calibre required to do this? and “I don’t think that fan-fiction will be able to take off in this way.? I may be making the arguments look a little more respectable here, perhaps imagine...
Meetup : MelbLW Social Meetup
by Maelin — 2015-08-20T11:45:34.284Z

Discussion article for the meetup : MelbLW Social Meetup

WHEN:
21 August 2015 06:30:00PM (+1000)

WHERE:
328 Little Lonsdale Street, Melbourne

The Melbourne LessWrong social meetup is on again as scheduled! Sorry for the late notice.

Our social meetups happen on the third Friday of every month. They are informal events where we get together for a chat and often play a few games. The start and finish times are very loose, so feel free to arrive late or leave early or both - drop by for whatever time you like.

Where? Alchemist's Refuge, 328 Little Lonsdale St, Melbourne

When? From 6:30pm until late, Friday August 21st

Contact? If you have any questions, just text or call Richard on 0421231789

Dinner? Alchemist's Refuge stocks drinks but does not not serve food beyond packets of crisps, however there are a number of take-away places nearby (including some that deliver - the bar staff can give recommendations). We have a tradition of going to Stalactites (24hr Greek restaurant) for a souvlaki after the meetup if you would like to wait!

Games? The Refuge has a selection of games available to borrow and you are also welcome to bring any board games, card games, etc that you'd like to play. You should be able to organise a group to play pretty easily. Otherwise you can sit around and chat!

This event on Facebook: https://www.facebook.com/events/535496406603060/

Discussion article for the meetup : MelbLW Social Meetup
Vegetarian/Omnivore Ideological Turing Test Judging Round!
by Raelifin — 2015-08-20T01:53:29.016Z

Come one, come all! Test your prediction skills in my Caplan Test (more commonly called an Ideological Turing Test). To read more about such tests, check out palladias' post here.

The Test: http://goo.gl/forms/7f4pQfxB8I

In the test, you will be asked to read responses written by rationalists from LessWrong (and the Columbus Ohio LW group). These responses are either from a vegetarian or omnivore (as decided by a coin flip) and are either their genuine response or a fake response where they pretend to be a member of the other group (also decided by coin flip). If you'd like to participate (and the more, the merrier) you'll be asked to distinguish fake from real by assigning a credence to the proposition that a given response is genuine.

I'll be posting general statistics on how people did at a later date (probably early September). Please use the comments on this thread to discuss or ask questions. Do not make predictions in the comments. I got more entries than would be reasonable to ask people to judge, so if your entry didn't make it into the test, I'm sorry. We might be able to run a second round of judging. If you're interested in judging more entries, send me a PM or leave a comment. I tended to favor the first entries I got, when selecting who got in.