trentmkelly/LessWrong-43k huggingface dataset:
Pain, fear, sex, and higher order preferences
by Stuart_Armstrong — 2018-02-22T11:30:41.123Z

Suppose a footballer with a broken leg is steeling themselves to kick a penalty despite the pain. Suppose a statistics lecturer is about to take a plane to a conference, and is trying to overcome their fear of flying. Or suppose a manager is meeting a client, and is trying to overcome their sexual attraction and stay professional.

All three situations can be seen as a conflict between higher order rationality and lower order urges, but they are actually different. In every case, there is an instrumental reason to overcome the "urge", but the general attitude towards that urge is different.

For the footballer, they want to overcome their current pain avoidance instincts, but they don't want to get rid of those instincts entirely. They generally want to avoid pain, but, for the moment, they have something more important to do - win a game - and would want to put their pain aversion preference aside for the moment.

In contrast the statistician with a fear of flying wouldn't mind having that fear expunged for ever. They know that normal flying involves no large risks, so a fear of flying is wholly irrational.

Thus we see pain aversion as an endorsed preference; we wouldn't want to get rid of it. Fear of flying is an unendorsed preference: we would toss it out if we could.

What of the third example? Well, that depends on what the manager feels about their own sexual urges. They may be more or less endorsed, depending on various factors of the manager's values and social circumstances.

Higher order preferences?

For fear of flying, we can fit that fear into a simple narrative of lower order preferences being overruled - or not - by higher order rationality.

But it's not clear that pain aversion (or sexual desire) can be fit into the same narrative. Since we don't want to override our pain aversion, it might be fitting to see pain aversion as higher order preference itself - or at least "not overriding pain aversion" might be a higher order preference, while pain aversion is a lower order one.

One can complicate the picture further by considering more edge cases, but the simplest interpretation is that the simple "higher order versus lower order" does not fit pain aversion or (often) sexual desire.
TAPs 2
by alkjash — 2018-02-22T05:10:00.490Z

This is part 13 of 30 of Hammertime. Click here for the intro.

“Omit needless words!” cries the author on page 23, and into that imperative Will Strunk really put his heart and soul. In the days when I was sitting in his class, he omitted so many needless words, and omitted them so forcibly and with such eagerness and obvious relish, that he often seemed in the position of having shortchanged himself — a man left with nothing more to say yet with time to fill, a radio prophet who had out-distanced the clock. Will Strunk got out of this predicament by a simple trick: he uttered every sentence three times. When he delivered his oration on brevity to the class, he leaned forward over his desk, grasped his coat lapels in his hands, and, in a husky, conspiratorial voice, said, “Rule Seventeen. Omit needless words! Omit needless words! Omit needless words!”

~ The Elements of Style

There is nothing more essential to the practice of Hammertime than repetition, and no rationality technique that requires more repetitive practice than TAPs. Although we pick only three days to focus on them, it’s best to draw out the repetitive drilling of TAPs over a lifetime.

Day 13: TAPs

Previously: Day 3.

Triggers that Notice Themselves

The real skill with trigger-action planning is picking the right trigger. The best triggers are not only easy to notice, but hard to miss. It should not require effort and conscious attention to notice the trigger – the only conscious action occurs after the trigger calls the action to mind.

Three ways to find great triggers:

Sentimental value: there’s a process by which we naturally become attached to the items that accompany us through thick and thin. I am attached, for example, to the freckle on my right thumb, to a long-sleeve shirt gifted me by a childhood friend, to my Logitech gaming mouse – relic of a past life. Pay attention to these objects. Notice how they gain three-dimensionality. Inject them with meaning. For example, there’s a well of metaphysical space under my thumb freckle where I can store a preternatural calm for a minute of need.
Novelty: surprise is the easiest way to notice. Last month I made a number of purchases for Design, and their presence registered as unusual for weeks. Take advantage of new possessions to build micro-habits. My new welcome mat is a reminder to check my keys and phone before leaving the apartment. My new bean-bag

...
Robustness to Scale
by Scott Garrabrant — 2018-02-21T22:55:19.155Z

I want to quickly draw attention to a concept in AI alignment: Robustness to Scale. Briefly, you want your proposal for an AI to be robust (or at least fail gracefully) to changes in its level of capabilities. I discuss three different types of robustness to scale: robustness to scaling up, robustness to scaling down, and robustness to relative scale.

The purpose of this post is to communicate, not to persuade. It may be that we want to bite the bullet of the strongest form of robustness to scale, and build an AGI that is simply not robust to scale, but if we do, we should at least realize that we are doing that.

Robustness to scaling up means that your AI system does not depend on not being too powerful. One way to check for this is to think about what would happen if the thing that the AI is optimizing for were actually maximized. One example of failure of robustness to scaling up is when you expect an AI to accomplish a task in a specific way, but it becomes smart enough to find new creative ways to accomplish the task that you did not think of, and these new creative ways are disastrous. Another example is when you make an AI that is incentivized to do one thing, but you add restrictions that make it so that the best way to accomplish that thing has a side effect that you like. When you scale the AI up, it finds a way around your restrictions.

Robustness to scaling down means that your AI system does not depend on being sufficiently powerful. You can't really make your system still work when it scales down, but you can maybe make sure it fails gracefully. For example, imagine you had a system that was trying to predict humans, and use these predictions to figure out what to do. When scaled up all the way, the predictions of humans are completely accurate, and it will only take actions that the predicted humans would approve of. If you scale down the capabilities, your system may predict the humans incorrectly. These errors may multiply as you stack many predicted humans together, and the system can end up optimizing for some seeming random goal.

Robustness to relative scale means that your AI system does not depend on any subsystems being similarly powerful to each other. This is most easy to see in systems that depend on adversarial subsystems. If part of you AI system is suggest plans, and another part is trying to find problems in those plans, if you...
Don't Condition on no Catastrophes
by Scott Garrabrant — 2018-02-21T21:50:31.077Z

I often hear people say things like "By what date do you assign 50% chance to reaching AGI, conditioned on no other form of civilizational collapse happening first?" The purpose of this post is to make this question make you cringe.

I think that most people mentally replace the conditional with something like "if other forms of civilizational collapse were magically not a thing, and did not have to enter into your model." Further, I think this is the more useful question to discuss, as it makes it easier to double crux, or download other people's models. However, not everyone does this, and it is not the question being asked.

To illustrate the difference, consider Alice, who, if they ignored other civilizational collapse, would think that AGI arrival date is uniform over the next 100 years. However, they also think that if not for AGI, extinction level nuclear war will happen in the next 100 years, uniformly at random over the next 100 years. Alice is not concerned about any other catastrophes.

Alice has these two independent distributions on when each event will happen if counterfactually, the other were to magically not happen. However, the world is such that as soon as one of events happens, it causes the other event to not happen, because the world is made very different.

When asking about Alice's median AGI date, ignoring civilizational collapse, we would like to encourage her to say 50 years. However her median AGI date, conditional on no nuclear war happening first is actually 33 years. This is because conditioning on no nuclear war happening first biases towards AGI dates that are early enough to stop a counterfactual future nuclear war.

The form of the question I would like to ask Alice is as follows:

Take your distribution over ways the way the future can go, and sample a random future, f. If that future ends with nuclear war at time t, sample another world with the property that neither AGI nor any other catastrophe happens before time t. If that world ends with a non AGI catastrophe, redefine t to be the time of the catastrophe in that world, and repeat the process, until you get a world that ends with AGI, with no other catastrophe happening first. Use this as your new distribution over futures, and tell me the median AGI date.

Note that conditioning on no other catastrophe happening first is the same procedure, except wh... ,,,,
The Logic of Science: 2.2
by mpr — 2018-02-21T17:28:52.537Z
Link: http://pulsarcoffee.com/posts/los2-2/

This is the first technical post on my new blog, where I plan to continue writing about what I learn from self-study of mathematics. I'm currently reading E.T. Jaynes' Probability Theory: The Logic of Science, and Christopher Bishop's Pattern Recognition and Machine Learning. While I understand many LessWrong2.0 readers are far above my level in maths, maybe there are some who would benefit from and enjoy conversation about the sort of things I'm learning.
Yoda Timers 2
by alkjash — 2018-02-21T07:40:00.792Z

This is part 12 of 30 of Hammertime. Click here for the intro.

Anyone who can muster their willpower for thirty seconds, can make a desperate effort to lift more weight than they usually could. But what if the weight that needs lifting is a truck? Then desperate efforts won’t suffice; you’ll have to do something out of the ordinary to succeed. You may have to do something that you weren’t taught to do in school. Something that others aren’t expecting you to do, and might not understand. You may have to go outside your comfortable routine, take on difficulties you don’t have an existing mental program for handling, and bypass the System.

~ Make an Extraordinary Effort

I don’t know if I’ve ever made an extraordinary effort (that’s probably evidence I haven’t), but I’ve certainly made desperate efforts. The philosophy of Yoda Timers is that it might be enough to make desperate efforts all the time: to do the known thing as well and quickly as can be done. Past that is the realm of rare genius.

CFAR calls Yoda Timers “Resolve Cycles,” a sub-skill of Resolve – the ability to make a desperate effort. Least glamorous of all rationality techniques, Resolve deserves its own book. How much could you accomplish just by more brute force all the time?

Day 12: Yoda Timers

Previously: Day 2.

Resolve is the main skill being trained by Yoda Timers, but there are a number of other useful reasons to build timers and deadlines into your life. Today I’ll share three ideas to make the most out of Yoda Timers.

Yoda Deadlines

Sometimes, you surprise yourself with what can be done in five minutes. But sometimes, there are things that can’t be done in five minutes. In this case, the generalization of Yoda Timers is to set absurdly short deadlines for these tasks.

How long does it take to write a novel? NanoWriMo is a Yoda Deadline for one month.

How long does it take to solve long-standing research problems? The IMO says: sometimes, only four and a half hours.

How long does it take to turn your life around? How many people waste away for years or decades before accelerating back through life in the span of weeks, tipped by a single conversation or book or trip?

The short answer to all these questions is: you have no idea how fast you can be without practicing for speed.

The Harvard-MIT Math Tournament (HMMT) has an easier version, the Harvard-MIT November Tournament (HMNT), which is run for l...
Are you the rider or the elephant?
by Qiaochu_Yuan — 2018-02-21T07:25:04.371Z

Some recent threads seem to me to be pointing at a really fundamental tension that I don't know how to articulate in full. But here's a chunk of it:

When you think of your mind as divided into your System 1 / fast / unconscious / nonverbal / intuitive / emotional processes, which we'll call the "elephant," and your System 2 / slow / conscious / verbal / deliberate / analytic processes, which we'll call the "rider," which of those do you identify with? That is, when you say "I," does "I" refer to the elephant or to the rider (or both, or neither, or other)?

We used to talk about straw Vulcan rationality, the kind of rationality that is only about the rider and completely neglects the elephant, and how the kind of rationality that, say, CFAR is interested in is much more about getting the rider and the elephant to communicate with each other and work well together as a team, complementing each other's strengths and weaknesses.

We say this, but it still seems to me that many people I run into (for example, at introductory CFAR workshops) implicitly identify as their riders and treat their elephants as annoying pets that have to be managed so that they, meaning their riders, can get on with their lives. I think this is... "wrong" would be a type error, and also unkind. But I'm sad about it.

I'll out myself: I identify mostly with my elephant, and think of my rider as at best a helpful advisor for my elephant.

And I get the sense that all of the disagreements I've been navigating recently have been with people who identify as their riders and are deeply suspicious of their elephants, and also (by association) of my elephant; and that this has been the main driving force behind the disagreement.

I don't really know what to do with this. It seems like the polite thing to do is to mostly only engage people like this rider-to-rider (which is mostly what I'm doing right now, in this post), but there are a lot of important things - even important things for the art of rationality, from my perspective - that I think (and feel!) can only be communicated elephant-to-elephant, and setting aside how feasible it is to do this online, it seems to be unsettled whether people even want elephant-to-elephant communication happening on LW at all.

I'm also just worried about peop...
How to not talk about probability estimates
by boggler — 2018-02-21T03:19:19.645Z

I have recently begun asking for people's credence in interesting beliefs.

I was speaking with two of my peers at dinner yesterday, and our discussion chanced upon the topic of the existence of extraterrestrial life. Curious, I asked them what probability they would assign to the existence of life beyond Earth, and they both replied along the lines of "Definitely 100%. The universe is massive."

Surprised, I brought up the problematic corollary of having a 100% credence. "A belief of 100% probability in extraterrestrial life is equivalent to being unable to design a universe with all of our current knowledge of the world and which lacks extraterrestrial life. Yet, it is clearly possible to design such a universe."

"Wait, why are you designing a universe?"

"I'm not, but the claims are equivalent. If such a universe exists, then you cannot reasonably have a 100% belief in the existence of extraterrestrial life because what if our universe is that very universe you designed to not have extraterrestrial life."

"I still don't understand why you're designing universes."

The conversation unfortunately ended there, and in post-conversation analysis, I've been working out how to better articulate this claim.

I believe part of the problem is the verb "design". They are correct in that one cannot literally "design" a universe, and furthermore "design" may be conflated with "intelligent design". A clearer phrase would be "think of". Can you "think of" a universe such that extraterrestrial life does not exist? This is a concrete question. One could think up of such a universe, unlike the ability to design one.

Yet, even with this word replacement, how does one clearly articulate that "thinking of" an alternate, plausible universe implies it is unreasonable to hold a 100% credence in the existence of extraterrestrial life?

My natural impulse is to justify it by saying, "Well, what about the tiny chance that your hypothetical conception is the correct universe? That means that you can't hold a 100% belief because there's that small probability that we are currently in your life-only-on-Earth universe."

Unfortunately, I believe that this may not be effective. I have a sense that our disagreement lies in our different m...
Sex, Lies, and Dexamethasone
by Jacob Falkovich — 2018-02-20T19:56:42.149Z

CW: Here are some terms that appear in this post and give you some flavor of what it's about: clitoroplasty, childhood sexual abuse, autogynephilia, transvestite, institutional review board.

This is a review of Alice Dreger's Galileo's Middle Finger, a memoir about science and activism: when they get along and when they don't. The review separated into three parts: a quick review of the book as a reading experience, a summary of the book's story, and the lessons, intended and unintended, that I learned from it.

The Book

Pliny the Elder wrote:

True glory consists in doing what deserves to be written, in writing what deserves to be read...

Naturally, when reviewing novels I have focused on the second part of that quote. But Galileo's Middle Finger is non-fiction, and the story it tells is more important than the writing.

The book's author, Alice Dreger, certainly deserves to have a book written about her. There are scientists and activists on the left who stand up for truth and human rights when those are abused by conservatives, and there are scientists and activists on the right who defend them from leftist attacks. There are very few people who are both scientist and activist, who care about truth and human rights and see clearly that one can't flourish without the other, and who defend them repeatedly against attacks from all sides. Alice Dreger is one of those few.

But is this the best book that could be written about Dreger? I had the sense that she wrote it for the same reason she has done most other things: someone had to and no one else was going to. The writing deserves to be read, but it also deserves a lot more editing.

Dreger's stories are fascinating, but they're relayed in a disjointed narrative that in the space of a single page mixes times and places, historical exposition and pointless personal asides, corny jokes and medical horror stories. It's not very long, but it's 30% too long.

I wanted to address this up top because I want to discuss what's in GMF without coming back to consider it as a book that people have to read. To wrap up the book review part of the review, I can't do much better than to quote this:

Phrases the author uses that accidentally describe the book itself
“[E]xplicitly supportive of gay and transgender rights [...] – but [...] it had some truly obnoxious parts...
Why we want unbiased learning processes
by Stuart_Armstrong — 2018-02-20T15:10:38.000Z

Crossposted at Lesserwrong.

tl;dr: if an agent has a biased learning process, it may choose actions that are worse (with certainty) for every possible reward function it could be learning.

An agent learns its own reward function if there is a set R of possible reward functions, and there is a learning process P that maps world-histories (and policies) to distributions over R. Thus by interacting with the environment and choosing its own policies, the agent can learn which is the correct reward function it should be maximising.

Given a policy π, a history h, an environment μ, and a reward R, we can compute the expected probability of

R:

EμπP(R|h).

Then a learning process is unbiased if that expression is independent of π, and biased otherwise. Biased processes are less desirable, as they allow the agent to manipulate the process through its choice of policy.

Simple biased learning process

The most trivial example of a biased learning process is an agent that completely determins its reward by its actions.
Let R={R0,R1}, let the agent only act once with two actions available, {a0,a1}, (hence a choice of "policy" is a choice of action), and set

P(R0|a0)=P(R1|a1)=1.

Thus the agent can simply choose its reward function through its actions.

Note that some designs are a bit more sophisticated, and don't allow the agent to choose its reward function directly through its actions. But this doesn't matter, if the reward function is a consequence of anything that is a predictable consequence of the agent's actions (eg if the agent can trick/coerce/manipulate a human into saying "yes" or "no", and if P is determined by the human's response, it doesn't matter that P is not defined directly through the agent's actions: it is defined indirectly through them).

[Note that all P that involve learning about external facts are unbiased learning processes, so it's not as if unbiased means trivial]

Strictly dominated behaviour

Then an agent with a biased learning process that wants to maximise the expectation of the true reward, can sometimes follow strictly dominated policies.
That means that there is are policies π0 and π1, such that for all histories hi possible given πi, and all reward function R∈R,

R(h1)>R(h0).

And yet the agent will still choose π0 to maximise reward.

For example, with P and R defined as above, define R0 and R1 to be:

a0a1R023R101

Thus a1 is

...