trentmkelly/LessWrong-43k huggingface dataset:
Group Rationality Diary, March 22 to April 4
by gjm — 2015-03-23T12:17:27.193Z

This is the public group rationality diary for March 22 - April 4, 2015. Here's the usual summary of what it's about:

It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.

Previous diary: February 15-28

Rationality diaries archive
Open thread, Mar. 23 - Mar. 31, 2015
by MrMind — 2015-03-23T08:38:37.302Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : San Francisco Meetup: Media Reccomendations
by rocurley — 2015-03-23T05:51:51.579Z

Discussion article for the meetup : San Francisco Meetup: Media Reccomendations

WHEN:
23 March 2015 06:15:00PM (-0700)

WHERE:
1061 Market St #4, San Francisco, CA 94103

(Make sure you go to the right address: We've been switching between 2 places).

We'll be meeting to discuss what people are reading, watching, listening to, etc.

As usual, feel free to show up late, and I can be reached at 301-458-0764.

Discussion article for the meetup : San Francisco Meetup: Media Reccomendations
LINK: Cosmologist and an Evangelical Christian Don Page on God and Cosmology
by Shmi — 2015-03-23T01:09:33.311Z

Last week's guest post on cosmologist's and General Relativity Expert's Sean Carroll's blog is by another prominent cosmologist, Don Page. Don argues for God and Christianity on Bayesian reasons:

I tend to favor a Bayesian approach in which one assigns prior probabilities based on simplicity and then weights these by the likelihoods (the probabilities that different theories assign to our observations) to get, when the product is normalized by dividing by the sum of the products for all theories, the posterior probabilities for the theories. [...]

For me, when I consider evidence from cosmology and physics, I find it remarkable that it seems consistent with all we know that the ultimate theory might be extremely simple and yet lead to sentient experiences such as ours. [...]

One might think that adding the hypothesis that the world (all that exists) includes God would make the theory for the entire world more complex, but it is not obvious that is the case, since it might be that God is even simpler than the universe, so that one would get a simpler explanation starting with God than starting with just the universe. [...]

...I have postulated that God loves mathematical elegance, as well as loving to create sentient beings, so something like this might explain both why the laws of physics, and the quantum state of the universe, and the rules for getting from those to the probabilities of observations, seem much simpler than they might have been, and why there are sentient experiences with a rather high degree of order. However, I admit there is a lot of logically possible variation on what God’s nature could be, so that it seems to me that at least we humans have to take that nature as a brute fact, analogous to the way naturalists would have to take the laws of physics and other aspects of the natural universe as brute facts. I don’t think either theism or naturalism solves this problem, so it seems to me rather a matter of faith which makes more progress toward solving it.

To me this post reads like a fascinating case of cognitive dissonance, self-deception, and rationalization. But maybe this is because I failed to construct a coherent argument for the other side. Unfortunately, I have a similar feeling about fervent proponents of Many Worlds, including Sean Carroll himself and Eliezer Yudkowsky, despite having a lot of respect for both of...
Meetup : LessWrong-like Meetup Hamburg
by Gunnar_Zarncke — 2015-03-22T20:53:40.977Z

Discussion article for the meetup : LessWrong-like Meetup Hamburg

WHEN:
27 March 2015 07:00:00PM (+0100)

WHERE:
Hamburg, Fehnweg 4

After some longer intermission the LessWrong Meetup Hamburg is back. Because most participants are also friends of me this is more a LessWrong-like meetup with a - hopefully - comparable focus. New guests are very welcome. Note that we will already have one guest from Bremen.

The topic of the meetup is effective sustainability. Many of us want to protect the environment and act responsible - but are our actions efficient? Or are we just collecting warm fuzzies?

Location is at my home which is reachable from central station in about 40 minutes via public transportations. You can get my phone number via LW private mail or via mail via my LW user page.

Discussion article for the meetup : LessWrong-like Meetup Hamburg
Just a casual question regarding MIRI
by Faustus2 — 2015-03-22T20:16:09.742Z

Currently I am planning to start a mathematics degree when I enter university, however my interest has shifted largely to computational neuroscience and related fields, so I'm now planning to switch to an AI degree when I go to study. Having said that, MIRI has always posed interesting problems to me, and I have entertained the thought of trying to do some work for MIRI before. And so my question boils down to this: Would there be any problem with taking the AI degree if I ever wanted to try my hand at doing some math for MIRI? Is a maths degree essential or would an AI degree with a good grasp on mathematics related to MIRI work just as well? Any thoughts or musings would be appreciated :)
Meetup : LessWrong Scotland
by cjb230 — 2015-03-22T18:44:33.255Z

Discussion article for the meetup : LessWrong Scotland

WHEN:
12 April 2015 02:00:00PM (+0000)

WHERE:
Biblos, 1 Chambers Street, Edinburgh EH1 1HR

We're going to meet in Edinburgh this time! I don't know the layout of the venue, but we'll be recognisable by books, stuffed owls, signs with Bayes' theorem on them, or some other LW trope.

We're starting a read-through of EY's "Rationality: From AI to Zombies." So this time we'll be discussing Book 1 - that's sections A to D.

Then we'll go on to whatever is on people's mind. Sometimes that's world domination, sometimes it isn't...

Hope to see some new faces then!

Discussion article for the meetup : LessWrong Scotland
How do I make sample size matter?
by  — 2015-03-22T17:53:31.405Z

In the introductory example in the Wikipedia article on the Bayesian theorem, they start out with a prior distribution for P(machine_ID | faulty_product)* and then updates this using a likelihood distribution P(faulty_product | machine_ID) to acquire a posterior distribution for P(machine_ID | faulty_product).

How did they come up with the likelihood distribution? Maybe they sampled 100 products from each machine and for each sample counted the number of faulty products. Maybe they sampled 1.000.000 products from each machine...

We don't know which sample size is used: the likelihood distribution doesn't reveal this. Thus this matter doesn't influence the weight of the Bayesian update. But shouldn't it do so? Uncertain likelihood distributions should have a small influence and vice versa. How do I make the bayesian update reflect this?

I read the links provided by somervta in the 'Error margins' discussion from yesterday, but I'm not skillful enough to adapt them to this example.

* technically they just make the prior distribution a clone of the distribution P(machine_ID) but I like to keep the identity across the Bayesian update so I gave the prior and the posterior distribution the same form: P(machine_ID | faulty_product).
Forum Digest: Reflective Oracles
by jessicata — 2015-03-22T04:02:38.000Z

Summary: This is a quick expository recap, with links, of writing (in papers and on this forum) on reflective oracles, through 3/21/15. Read this if you want to learn more about reflective oracles, or if you’re curious about what we’ve been working on lately!

Introduction

Solomonoff induction can be used to predict a sequence of bits produced by a computable environment. However, if the environment that produces the bits contains the Solomonoff inductor (or other Solomonoff inductors), then the environment will be uncomputable, and Solomonoff induction will no longer have any guarantee of accuracy. This motivates the study of the naturalized induction problem. This
is still an open problem, but we have recently made progress
using the framework of reflective oracles.

In this framework, programs are allowed to call a reflective
oracle that (probabilistically) predicts
the output of other programs (which may also call reflective oracles).
Roughly speaking, the oracle answers questions of the form:
"Is the probability that machine M returns 1 greater than
p?"? If the probability that M returns 1 is less than p, then the oracle
must return 0; if it is greater than p, it must return 1; and if
it is exactly p, then it may randomize between 0 and 1 arbitrarily.

It is
due to the randomization that paradoxes do not arise.
For example, we could define a program that asks the oracle if it
returns 1 with "greater than" 50% probability, and then
outputs 0 if this is true or 1 if it is false.
In this case, the oracle returns 1 half the time and 0 the
other half of the time for the query the program makes. Then the program also
returns 1 half the time, so the oracle was indeed allowed to randomize its
answer to the query.

While reflective oracles aren't a complete solution to the naturalized
induction problem, they are a useful model that lets us define naturalized algorithms and prove interesting properties about them. In particular, it is possible to define reflective variants of causal decision theory, Solomonoff induction, and AIXI.

Papers

Reflective oracles as a Foundation for Classical Game Theory,
Benja Fallenstein, Jessica Taylor, and Paul Christiano.
This paper formalizes reflective oracles, shows that they always exist, and defines
causal decision theory for agents in a world containing reflective oracles. When
multiple agents use this reflective version o

...
Meetup : West LA - Part A: Predictably Wrong
by AvivaLasVegas — 2015-03-22T03:11:50.622Z

Discussion article for the meetup : West LA - Part A: Predictably Wrong

WHEN:
25 March 2015 07:00:00PM (-0700)

WHERE:
11066 Santa Monica Blvd, Los Angeles, CA 90025

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: We will be discussing Part A of Rationality: From AI to Zombies. You are welcome to join us even if you have not completed the reading. However, the reading is very good, and it is free, and you should read it. We will be discussing roughly one part per week for a total of 26 weeks.

Recommended Reading:

Preface

Biases: An Introduction

Part A: Predictably Wrong (pages 7-42)

No prior exposure to Less Wrong is required; this will be generally accessible.

Discussion article for the meetup : West LA - Part A: Predictably Wrong
Summary and Lessons from "On Combat" 
by Gunnar_Zarncke — 2015-03-22T01:48:56.630Z

On Combat - The Psychology and hysiology of Deadly Conflict in War and in Peace by Lt. Col. Dave Grossman and Loren W. Christensen (third edition from 2007) is a well-written, evidence-based book about the reality of human behaviour in life-threatening situations. It is comprehensive (400 pages), provides detailed descriptions, (some) statistics as well as first-person recounts, historical context and other relevant information. But my main focus in this post is in the advice it gives and what lessons the LessWrong community may take from it.

TL;DR

In deadly force encounters you will experience and remember the most unusual physiological and psychological things. Innoculate yourself against extreme stress with repeated authentic training; play win-only paintball, train 911-dialing and -reporting. Train combat breathing. Talk to people after traumatic events.

Disclaimer: While I have read most of the book I have skimmed or skipped some sections I didn't found sufficiently relevant, applicable or new. These are mostly chapters in sections III and IV on warrior mind-set and history. I couldn't make myself read the section about the will to kill. I have to note that Grossman writes very balanced overall. He does strongly motivate the need for warriors and peace-keepers and urges for a suitable mind-set. This is to be expected of a distinguished West Point teacher. Nonetheless does he do so without indoctrination or lies. He tells the scientific truth about the harsh realities of war and deadly encounters and where he ventures into speculative explanations he says so. Where he inspires (there is a bit of hero cult between the lines) he mostly quotes people from the field. He also holds laudable views about the effect of violence on children.

He starts quickly with the hard facts.

Arousal Conditions

Grossman structures physiological arousal into conditions based mainly on heart rate:

White (baseline bpm) calm, ineffective ("for sheep")

Yellow (increased heart rate) alert and prepared (needed for fine motor control e.g. for aiming, mechanics)

Red (~120-140 bpm) optimum survival rate, at the top complex motor skill deteriorates

Gray - increasing deterioration but experience and training can compensate the set-in of Nlack

Black (definitely if >175 bpm) vasoconstriction, only gross motor skills, auomatic behavior, loss of perception

Condition Black

Black is th...
Is arrogance a symptom of bad intellectual hygeine?
by enfascination — 2015-03-21T19:59:41.057Z

I have this belief that humility is a part of good critical thinking, and that egoism undermines it. I imagine arrogance as a kind of mind-death. But I have no evidence, and no good mechanism by which it might be true. In fact, I know the belief is suspect because I know that I want it to be true — I want to be able to assure myself that this or that intolerable academic will be magically punished with a decreased capacity to do good work. The truth could be the opposite: maybe hubris breeds confidence, and confidence results? After all, some of the most important thinkers in history were insufferable.

Is any link, positive or negative, between arrogance and reasoning too tenuous to be worth entertaining? Is humility a pretty word or a valuable habit? I don't know what I think yet. Do you?
Seeking Estimates for P(Hell)
by  — 2015-03-21T15:44:44.558Z

I am trying to decide how to allocate my charitable donations between GiveWell's top charities and MIRI, and I need a probability estimate to make an informed decision. Could you help me?

Background on my moral system: I place a greater value on reducing high doses of suffering of conscious entities than merely preventing death. An unexpected, instant, painless death is unfortunate, but I would prefer it to a painful and chronic condition.

Given my beliefs, it follows logically that I would pay a relatively large amount to save a conscious entity from prolonged torture.

The possibility of an AI torturing many conscious entities has been mentioned1 on this site, and I assume that funding MIRI will help reduce its probability. But what is its current probability?

Obviously a difficult question, but it seems to me that I need an estimate and there is no way around it. I don't even know where to start...suggestions?

1 http://lesswrong.com/lw/1pz/the_ai_in_a_box_boxes_you/
Error margins
by  — 2015-03-21T11:05:23.858Z

Why don't probabilities come with error margins, or other means of describing uncertainty in their assessments?

If I evaluate a prior probability P(new glacial period starting within the next 100 years) to, say, 0.1, shouldn't I then also communicate how certain I feel about that judgement?
A scientist might make the same estimate but be more sure about it's accuracy than I.

In our everyday judgements we often use such package deals:
A: where's Jamie?
B: I think he went to the club house, but you know Jamie - he could be anywhere.

High P, high uncertainty

A: Where's Susie? Do you think she ran astray after that hefty argument?
B: no I'm certain she would *never* do that. She must have gone to a friends place.

High P, low uncertainty.
Rationality: From AI to Zombies online reading group
by  — 2015-03-21T09:54:33.020Z

Update: When I posted this announcement I remarkably failed to make the connection that the April 15th is tax day here in the US, and as a prime example of the planning fallacy (a topic of the first sequence!), I failed to anticipate just how complicated my taxes would be this year. The first post of the reading group is basically done but a little rushed, and I want to take an extra day to get it right. Expect it to post on the next day, the 16th

On Thursday, 16 April 2015, just under a month out from this posting, I will hold the first session of an online reading group for the ebook Rationality: From AI to Zombies, a compilation of the LessWrong sequences by our own Eliezer Yudkowsky. I would like to model this on the very successful Superintelligence reading group led by KatjaGrace. This is an advanced warning, so that you can have a chance to get the ebook, make a donation to MIRI, and read the first sequence.

The point of this online reading group is to join with others to ask questions, discuss ideas, and probe the arguments more deeply. It is intended to add to the experience of reading the sequences in their new format or for the first time. It is intended to supplement discussion that has already occurred the original postings and the sequence reruns.

The reading group will 'meet' on a semi-monthly post on the LessWrong discussion forum. For each 'meeting' we will read one sequence from the the Rationality book, which contains a total of 26 lettered sequences. A few of the sequences are unusually long, and these might be split into two sessions. If so, advance warning will be given.

In each posting I will briefly summarize the salient points of the essays comprising the sequence, link to the original articles and discussion when possible, attempt to find, link to, and quote one or more related materials or opposing viewpoints from outside the text, and present a half-dozen or so question prompts to get the conversation rolling. Discussion will take place in the comments. Others are encouraged to provide their own question prompts or unprompted commentary as well.

We welcome both newcomers and veterans on the topic. If you've never read the sequences, this is a great opportunity to do so. If you are an old timer from the Overcoming Bias days then this is a chance to share your wisdom and perhaps revisit the material with fresh eyes. All levels...
Consistent Plato
by Stuart_Armstrong — 2015-03-20T18:06:11.831Z

A putative new idea for AI control; index here. The idea here is somewhat underdeveloped.

I've been suggesting AI designs that are not exactly expected-utility maximisers, while wondering if such designs could be stable under self improvement/subagent creation/general stuff. And also wondering what designs such agents would converge to if they did self-improve.

This post is just a short note warning people to make sure they are consistent when they use non-expected utility maximalisation, and to be clear which parts are Platonic and idealised and which parts are not.

Let's take the reduced impact AI design as an example. In this, the agent has a kind of utility:

v = u - R,

where u is a standard utility and R is a penalty term. The penalty term involved a probability estimate for what a putative "super AI" would estimate, based on some observations it could make. Thus it is better described as R(P), where P is the probability estimator of that "Super AI".

When calculating expected v, the agent will use its own probability estimator P'. Initially, P' is just a less powerful version of P, one that can, moreover, estimate properties about P.

How could this possibly go wrong? I'm glad you asked ^_^ The obvious risk is that the agent acts to replace P with Q, such that R(Q) is as low as possible.

This could come about if P referred to some actual physical algorithm, possibly one written down somewhere the AI could get at it. Then re-writing the algorithm P could increase v by reducing R(P).

It is somewhat more difficult if P and P' are defined in the same place (maybe P is just P' run for a lot longer). Then modifying P could run the risk of degrading the AI's overall accuracy in predictions. However, this may be a price the AI is willing to pay, if the R(P) penalty is severe enough. More likely, the AI could degrade the quality of P but not P' by introducing pathologies that would only affect the first.

Even if P itself is safe from modification, the AI can still hack the system. Recall that it does not calculate P, it estimates it, using P' - so effectively using P'(P). Thus modifications of P' can affect the AI's estimate of R(P). Even if P and P'(P) cannot be changed, the AI can still improve P' for predictions of u, while leaving P'(P) unchanged. Thus the AI may become super-super-intelligent at predicting facts about the world, while remaining (deliberately...