trentmkelly/LessWrong-43k huggingface dataset:
Fighting Biases and Bad Habits like Boggarts
by palladias — 2014-08-21T17:07:43.674Z

TL;DR: Building humor into your habits for spotting and correcting errors makes the fix more enjoyable, easier to talk about and receive social support, and limits the danger of a contempt spiral.

One of the most reliably bad decisions I've made on a regular basis is the choice to stay awake (well, "awake") and on the internet past the point where I can get work done, or even have much fun. I went through a spell where I even fell asleep on the couch more nights than not, unable to muster the will or judgement to get up and go downstairs to bed.

I could remember (even sometimes in the moment) that this was a bad pattern, but, the more tired I was, the more tempting it was to think that I should just buckle down and apply more willpower to be more awake and get more out of my computer time. Going to bed was a solution, but it was hard for it not to feel (to my sleepy brain and my normal one) like a bit of a cop out.

Only two things helped me really keep this failure mode in check. One was setting a hard bedtime (and beeminding it) as part of my sacrifice for Advent. But the other key tool (which has lasted me long past Advent) is the gif below.

The poor kid struggling to eat his ice cream cone, even in the face of his exhaustion, is hilarious. And not too far off the portrait of me around 2am scrolling through my Feedly.

Thinking about how stupid or ineffective or insufficiently strong-willed I'm being makes it hard for me to do anything that feels like a retreat from my current course of action. I want to master the situation and prove I'm stronger. But catching on to the fact that my current situation (of my own making or not) is ridiculous, makes it easier to laugh, shrug, and move on.

I think the difference is that it's easy for me to feel contemptuous of myself when frustrated, and easy to feel fond when amused.

I've tried to strike the new emotional tone when I'm working on catching and correcting other errors. (e.g "Stupid, you should have known to leave more time to make the appointment! Planning fallacy!" becomes "Heh, I guess you thought that adding two "trivially short" errands was a closed set, and must remain 'trivially short.' That's a pretty silly error.")

In the first case, noticing and correcting an error feels punitive, since it's quickl...
Another type of intelligence explosion
by Stuart_Armstrong — 2014-08-21T14:49:50.430Z

I've argued that we might have to worry about dangerous non-general intelligences. In a series of back and forth with Wei Dai, we agreed that some level of general intelligence (such as that humans seem to possess) seemed to be a great advantage, though possibly one with diminishing returns. Therefore a dangerous AI could be one with great narrow intelligence in one area, and a little bit of general intelligence in others.

The traditional view of an intelligence explosion is that of an AI that knows how to do X, suddenly getting (much) better at doing X, to a level beyond human capacity. Call this the gain of aptitude intelligence explosion. We can prepare for that, maybe, by tracking the AI's ability level and seeing if it shoots up.

But the example above hints at another kind of potentially dangerous intelligence explosion. That of a very intelligent but narrow AI that suddenly gains intelligence across other domains. Call this the gain of function intelligence explosion. If we're not looking specifically for it, it may not trigger any warnings - the AI might still be dumber than the average human in other domains. But this might be enough, when combined with its narrow superintelligence, to make it deadly. We can't ignore the toaster that starts babbling.
An example of deadly non-general AI
by Stuart_Armstrong — 2014-08-21T14:15:55.051Z

In a previous post, I mused that we might be focusing too much on general intelligences, and that the route to powerful and dangerous intelligences might go through much more specialised intelligences instead. Since it's easier to reason with an example, here is a potentially deadly narrow AI (partially due to Toby Ord). Feel free to comment and improve on it, or suggest you own example.

It's the standard "pathological goal AI" but only a narrow intelligence. Imagine a medicine designing super-AI with the goal of reducing human mortality in 50 years - i.e. massively reducing human population in the next 49 years. It's a narrow intelligence, so it has access only to a huge amount of human biological and epidemiological research. It must gets its drugs past FDA approval; this requirement is encoded as certain physical reactions (no death, some health improvements) to people taking the drugs over the course of a few years.

Then it seems trivial for it to design a drug that would have no negative impact for the first few years, and then causes sterility or death. Since it wants to spread this to as many humans as possible, it would probably design something that interacted with common human pathogens - colds, flues - in order to spread the impact, rather than affecting only those that took the disease.

Now, this narrow intelligence is less threatening than if it had general intelligence - where it could also plan for possible human countermeasures and such - but it seems sufficiently dangerous on its own that we can't afford to worry only about general intelligences. Some of the "AI superpowers" that Nick mentions in his book (intelligence amplification, strategizing, social manipulation, hacking, technology research, economic productivity) could be enough to cause devastation on their own, even if the AI never developed other abilities.

We still could be destroyed by a machine that we outmatch in almost every area.
Why we should err in both directions
by owencb — 2014-08-21T11:10:59.654Z

Crossposted from the Global Priorities Project

This is an introduction to the principle that when we are making decisions under uncertainty, we should choose so that we may err in either direction. We justify the principle, explore the relation with Umeshisms, and look at applications in priority-setting.

Some trade-offs

How much should you spend on your bike lock? A cheaper lock saves you money at the cost of security.

How long should you spend weighing up which charity to donate to before choosing one? Longer means less time for doing other useful things, but you’re more likely to make a good choice.

How early should you aim to arrive at the station for your train? Earlier means less chance of missing it, but more time hanging around at the station.

Should you be willing to undertake risky projects, or stick only to safe ones? The safer your threshold, the more confident you can be that you won’t waste resources, but some of the best opportunities may have a degree of risk, and you might be able to achieve a lot more with a weaker constraint.

The principle

We face trade-offs and make judgements all the time, and inevitably we sometimes make bad calls. In some cases we should have known better; sometimes we are just unlucky. As well as trying to make fewer mistakes, we should try to minimise the damage from the mistakes that we do make.

Here’s a rule which can be useful in helping you do this:

When making decisions that lie along a spectrum, you should choose so that you think you have some chance of being off from the best choice in each direction.

We could call this principle erring in both directions. It might seem counterintuitive -- isn’t it worse to not even know what direction you’re wrong in? -- but it’s based on some fairly straightforward economics. I give a non-technical sketch of a proof at the end, but the essence is: if you’re not going to be perfect, you want to be close to perfect, and this is best achieved by putting your actual choice near the middle of your error bar.

So the principle suggests that you should aim to arrive at the station with a bit of time wasted, but not so much that you won’t miss the train even if something goes wrong.

Refinements

Just saying that you should have some chance of erring in either direction isn’t enough to tell you what you should actually choose. It can be a...
Productivity thoughts from Matt Fallshaw
by John_Maxwell — 2014-08-21T05:05:11.156Z

At the 2014 Effective Altruism Summit in Berkeley a few weeks ago, I had the pleasure of talking to Matt Fallshaw about the things he does to be more effective. Matt is a founder of Trike Apps (the consultancy that built Less Wrong), a founder of Bellroy, and a polyphasic sleeper. Notes on our conversation follow.

Matt recommends having a system for acquiring habits. He recommends separating collection from processing; that is, if you have an idea for a new habit you want to acquire, you should record the idea at the time you have it and then think about actually implementing it at some future time. Matt recommends doing this through a weekly review. He recommends vetting your collection to see what habits seem actually worth acquiring, then for those habits you actually want to acquire, coming up with a compassionate, reasonable plan for how you're going to acquire the habit.

(Previously on LW: How habits work and how you may control them, Common failure modes in habit formation.)

The most difficult kind of habit for me to acquire is that of random-access situation-response habits, e.g. "if I'm having a hard time focusing, read my notebook entry that lists techniques for improving focus". So I asked Matt if he had any habit formation advice for this particular situation. Matt recommended trying to actually execute the habit I wanted as many times as possible, even in an artificial context. Steve Pavlina describes the technique here. Matt recommends making your habit execution as emotionally salient as possible. His example: Let's say you're trying to become less of a prick. Someone starts a conversation with you and you notice yourself experiencing the kind of emotions you experience before you start acting like a prick. So you spend several minutes explaining to them the episode of disagreeableness you felt coming on and how you're trying to become less of a prick before proceeding with the conversation. If all else fails, Matt recommends setting a recurring alarm on your phone that reminds you of the habit you're trying to acquire, although he acknowledges that this can be expensive.

Part of your plan should include a check to make sure you actually stick with your new habit. But you don't want a check that's overly intrusive. Matt recommends keeping an Anki de...
Imaginary queues
by KatjaGrace — 2014-08-20T19:54:28.000Z

I thought of an interesting idea, then searched to see if anyone had done it. It seemed like not, so I wrote the below post. Then I looked once more, and found a few instances (finding one makes it easier to find others). I still think the proposal is good, and I have not much idea whether anyone doing it competently. However it is not so novel as hoped, and one must wonder why such things have not been successful enough for me to have heard of them (even after trying unusually hard to hear about them).

Instead of modifying the post, Robin Hanson suggested I put it up more or less as it was before I knew such a thing existed, and then I can compare the details of my suggestion to the details of systems that are real (or real minus any widespread adoption). This can make for a rare test of how different things look if you have an actual business providing a service, versus a daydream about an actual business providing a service. This seems like a good idea to me, so here you have it. The changes I made were adding this part, and removing a half-written couple of sentences at the end about how there didn’t seem to be anyone doing this, though there are some related things. I mean to compare it with a real system another time.

***

Consider how much time people spend in queues. Here is a proposal for reducing that dramatically, with a smartphone app.

The app controls a virtual queue. You tell your phone you want to join the queue. You continue shopping or whatever. Your phone pings you when you are near the front of the queue. You wander over and get in the physical line just before it is your turn.

There are many details to iron out here, but there seem to be plausible solutions:

How can people smoothly use this when there will be a large fraction of people who don’t have it?

In the simplest case, places that have multiple queues could have one dedicated one. Where this is not practical, the owner of the queue might have a smartphone or tablet at the front running a different version of the software which would allow others to add their names to the queue – in which case they can stand around as usual until called – or their phone numbers and a time estimate, in which case they can also wander a bit.

How do you know how long the queue is?

The app could trivially tell you this. Ultimately, it could also probably give you better stats on how

...
Anthropics doesn't explain why the Cold War stayed Cold
by KnaveOfAllTrades — 2014-08-20T19:23:26.132Z

(Epistemic status: There are some lines of argument that I haven’t even started here, which potentially defeat the thesis advocated here. I don’t go into them because this is already too long or I can’t explain them adequately without derailing the main thesis. Similarly some continuations of chains of argument and counterargument begun here are terminated in the interest of focussing on the lower-order counterarguments. Overall this piece probably overstates my confidence in its thesis. It is quite possible this post will be torn to pieces in the comments—possibly by my own aforementioned elided considerations. That’s good too.)
I
George VI, King of the United Kingdom, had five siblings. That is, the father of current Queen Elizabeth II had as many siblings as on a typical human hand. (This paragraph is true, and is not a trick; in particular, the second sentence of this paragraph really is trying to disambiguate and help convey the fact in question and relate it to prior knowledge, rather than introduce an opening for some sleight of hand so I can laugh at you later, or whatever fear such a suspiciously simple proposition might engender.)
Let it be known.
II
Exactly one of the following stories is true:
Story One
Recently I hopped on Facebook and saw the following post:
“I notice that I am confused about why a nuclear war never occurred. Like, I think (knowing only the very little I know now) that if you had asked me, at the start of the Cold War or something, the probability that it would eventually lead to a nuclear war, I would've said it was moderately likely. So what's up with that?”
The post had 14 likes. In the comments, the most-Liked explanation was:
“anthropically you are considerably more likely to live in a world where there never was a fullscale nuclear war”
That comment had 17 Likes. The second-most-liked comment that offered an explanation had 4 Likes.
Story Two
Recently I hopped on Facebook and saw the following post:
“I notice that I am confused about why George VI only had five siblings. Like, I think (knowing only the very little I know now) that if you had asked me, at his birth or something, the probability that he would eventually have had only five siblings and no more, I would've said it was moderately unlikely. So what's up with that?”
The post had 14 likes. In the comments, the mo...
Thought experiments on simplicity in logical probability
by Manfred — 2014-08-20T17:25:35.026Z

A common feature of many proposed logical priors is a preference for simple sentences over complex ones. This is sort of like an extension of Occam's razor into math. Simple things are more likely to be true. So, as it is said, "why not?"

Well, the analogy has some wrinkles - unlike hypothetical rules for the world, logical sentences do not form a mutually exclusive set. Instead, for every sentence A there is a sentence not-A with pretty much the same complexity, and probability 1-P(A). So you can't make the probability smaller for all complex sentences, because their negations are also complex sentences! If you don't have any information that discriminates between them, A and not-A will both get probability 1/2 no matter how complex they get.

But if our agent knows something that breaks the symmetry between A and not-A, like that A belongs to a mutually exclusive and exhaustive set of sentences with differing complexities, then it can assign higher probabilities to simpler sentences in this set without breaking the rules of probability. Except, perhaps, the rule about not making up information.

The question: is the simpler answer really more likely to be true than the more complicated answer, or is this just a delusion? If so, is it for some ontologically basic reason, or for a contingent and explainable reason?

There are two complications to draw your attention to. The first is in what we mean by complexity. Although it would be nice to use the Kolmogorov complexity of any sentence, which is the length of the shortest program that prints the sentence, such a thing is uncomputable by the kind of agent we want to build in the real world. The only thing our real-world agent is assured of seeing is the length of the sentence as-is. We can also find something in between Kolmogorov complexity and length by doing a brief search for short programs that print the sentence - this meaning is what is usually meant in this article, and I'll call it "apparent complexity."

The second complication is in what exactly a simplicity prior is supposed to look like. In the case of Solomonoff induction the shape is exponential - more complicated hypotheses are exponentially less likely. But why not a power law? Why not even a Poisson distribution? Does the difficulty of answering this question mean that thinking that simpler sentences are more likely is a delusion after a...
Polling Thread
by Gunnar_Zarncke — 2014-08-20T14:36:17.955Z

The next installment of the Polling Thread.

This is your chance to ask your multiple choice question you always wanted to throw in. Get qualified numeric feedback to your comments. Post fun polls.

These are the rules:

Each poll goes into its own top level comment and may be commented there.

You must at least vote all polls that were posted earlier than you own. This ensures participation in all polls and also limits the total number of polls. You may of course vote without posting a poll.

Your poll should include a 'don't know' option (to avoid conflict with 2). I don't know whether we need to add a troll catch option here but we will see.

If you don't know how to make a poll in a comment look at the Poll Markup Help.

This is a somewhat regular thread. If it is successful I may post again. Or you may. In that case do the following :

Use "Polling Thread" in the title.

Copy the rules.

Add the tag "poll".

Link to this Thread or a previous Thread.

Create a top-level comment saying 'Discussion of this thread goes here; all other top-level comments should be polls or similar'

Add a second top-level comment with an initial poll to start participation.
"Follow your dreams" as a case study in incorrect thinking
by cousin_it — 2014-08-20T13:18:02.863Z

This post doesn't contain any new ideas that LWers don't already know. It's more of an attempt to organize my thoughts and have a writeup for future reference.

Here's a great quote from Sam Hughes, giving some examples of good and bad advice:

"You and your gaggle of girlfriends had a saying at university," he tells her. "'Drink through it'. Breakups, hangovers, finals. I have never encountered a shorter, worse, more densely bad piece of advice." Next he goes into their bedroom for a moment. He returns with four running shoes. "You did the right thing by waiting for me. Probably the first right thing you've done in the last twenty-four hours. I subscribe, as you know, to a different mantra. So we're going to run."

The typical advice given to young people who want to succeed in highly competitive areas, like sports, writing, music, or making video games, is to "follow your dreams". I think that advice is up there with "drink through it" in terms of sheer destructive potential. If it was replaced with "don't bother following your dreams" every time it was uttered, the world might become a happier place.

The amazing thing about "follow your dreams" is that thinking about it uncovers a sort of perfect storm of biases. It's fractally wrong, like PHP, where the big picture is wrong and every small piece is also wrong in its own unique way.

The big culprit is, of course, optimism bias due to perceived control. I will succeed because I'm me, the special person at the center of my experience. That's the same bias that leads us to overestimate our chances of finishing the thesis on time, or having a successful marriage, or any number of other things. Thankfully, we have a really good debiasing technique for this particular bias, known as reference class forecasting, or inside vs outside view. What if your friend Bob was a slightly better guitar player than you? Would you bet a lot of money on Bob making it big like Jimi Hendrix? The question is laughable, but then so is betting the years of your own life, with a smaller chance of success than Bob.

That still leaves many questions unanswered, though. Why do people offer such advice in the first place, why do other people follow it, and what can be done about it?

Survivorship bias is one big reason we constantly hear successful people telling us to "follow our dreams". Successful people doesn't really know wh...
Meetup : September Rationality Dojo - Fixed and Growth Mindset
by MelbourneLW — 2014-08-19T07:52:29.425Z

Discussion article for the meetup : September Rationality Dojo - Fixed and Growth Mindset

WHEN:
07 September 2014 03:30:00PM (+1000)

WHERE:
Ross House Association, 247-251 Flinders Lane, Melbourne

[ATTN: Please remember the new location for the dojos: the Jenny Florence Room, Level 3, Ross House at 247 Flinders Lane, Melbourne.

3:30pm start / arrival - formal dojo activities will commence at 4:00pm.]

The Less Wrong Sunday Rationality Dojos are crafted to be serious self-improvement sessions for those committed to the Art of Rationality and personal growth. Each month a community member will run a session involving a presentation of content, discussion, and exercises.

Continuing the succession of immensely successful dojos, Helen will run a session on fixed and growth mindset.

As always, we will review the personal goals we committed to at the previous Dojo (I will have done X by the next Dojo). Scott Fowler recorded the commitments, if you didn't make it but would like to add your own goal to the records, send him a message (shokwave.sf@gmail.com).

The Dojo is likely to run for 2-3 hours, after which some people will get dinner together.

If you have any trouble finding the venue or getting in, call Louise on 0419 192 367.

If you would like to present at a future Dojo or suggest a topic, please fill it in on the Rationality Dojo Roster: http://is.gd/dojoroster

Discussion article for the meetup : September Rationality Dojo - Fixed and Growth Mindset
Steelmanning MIRI critics
by fowlertm — 2014-08-19T03:14:15.072Z

I'm giving a talk to the Boulder Future Salon in Boulder, Colorado in a few weeks on the Intelligence Explosion hypothesis. I've given it once before in Korea but I think the crowd I'm addressing will be more savvy than the last one (many of them have met Eliezer personally). It could end up being important, so I was wondering if anyone considers themselves especially capable of playing Devil's Advocate so I could shape up a bit before my talk? I'd like there to be no real surprises.

I'd be up for just messaging back and forth or skyping, whatever is convenient.