trentmkelly/LessWrong-43k huggingface dataset:
Popular religions suggest extrapolated volition is non-existence and wireheading
by denisbider — 2018-02-09T00:06:30.957Z

I'm not sure if this is insightful enough to share here, but I'll try anyway.

A fair amount of wondering has been done about how FAI could figure out what humans actually want. A school of thought says persuasively that what we say we want is not what we actually want, so what we really want has to be extrapolated.

If we take the end-game promises of popular religions at face value, it occurs to me that Buddhism promises something between non-existence and wireheading (nirvana - "to blow out"), while Christianity promises wireheading (eternal bliss - heaven). I am not familiar enough with other religions to make statements about them.

In my anecdotal experience, it seems to me rationalists are quick to dismiss wireheading and non-existence as desirable possibilities. We experience this grasping desire to live, create, discover, and experience. We're not sure to what end, but we feel this indescribable zest, and we're convinced it's going to be great.

Look at people getting mental orgasms from Elon Musk launching a car into space. Whatever problems you have, space exploration is not going to solve them, and if your life is in harmony, you don't need space exploration. And yet there's this palpable zest about an unspoken implication... That perhaps an age of discovery is dawning, an age of adventure, an age of transcending our present problems and tackling larger ones. An age of being awesome.

There's a type of person that feels this zest, and this type is not a majority. The median person on Earth is confused by the world. They believe in things like Jesus Christ, and they press on in hope that adhering to divine guidance while they attempt to survive the trials and tribulations of life will be rewarded with not having to do this again. To such a person, the sight of two metal meteors descending from the sky with loud sonic booms, igniting engines and landing in synchrony does not necessarily inspire awe or enthusiasm as much as confusion and terror.

We, the few, are the seekers of something we cannot describe, and most of us find it hard to identify with the mindset of the median individual. The median individual does not want the excitement we seek, they just want an end... a release.
Science like a chef
by Adam Zerner — 2018-02-08T21:23:45.425Z

Alice: Hey honey, I made pasta with tomato sauce!

Bob: Great, let's eat!

Bob: Mmmmm, that's fantastic! It's even better than last time. It's got a sweeter, deeper flavor, which I like.

Alice: Thanks. Last time I only sautéed onions and garlic before adding the tomato puree, but this time I added carrots to the mix for some extra sweetness.

Bob: You know I love your cooking, but I always feel a twinge of skepticism whenever you try to explain why things taste the way they do. Yes, you added carrots to the sauté. But is that the only thing you did differently?

Alice: No. I added some butter along with the olive oil to give it some smoothness. Ummm. I threw some red wine in to the sauce. We didn't have any basil leaves today so I just left that out of the recipe.

Bob: So how do you know that the carrots are what made it taste sweeter? How do you know that it wasn't the butter, the red wine, or the absence of basil leaves?

Alice: Well, the red wine is acidic, so that wouldn't make it taste sweet. I think the butter helped make it taste richer and smoother. The basil leaves add flavor, so leaving them out probably subtracted from the deepness, rather than adding to it.

Bob: Sure, that all sounds plausible. But how can you know any of this?

Alice: It's just common sense. I know that butter is rich. I know that red wine is acidic. I know that basil is flavorful. So I know that adding them will have the corresponding effects on the sauce.

Bob: But isn't it possible that things work differently when you combine them? Sure, red wine is acidic by itself, but isn't it possible that when combined with sautéed onions, garlic and carrots, and olive oil and butter, and fried tomato paste, and whatever spices you used, and cooked. Isn't it possible that after all of that, some chemical reactions occur that cause the red wine to be sweet instead of acidic?

Alice: I don't think so. I think the sweetness came from the carrots. But yeah, I suppose it is possible. So what do you propose I do?

Bob: Cook like a scientist. Only change one variable at a time. Take note of the effect. So in this case, if you hypothesize that adding carrots to the sauté will add sweetness to the sauce, just add carrots to the sauté. Don't add red wine, or anything else.

Alice: Ok, let's suppose I did do that, and I found that carrots did in fact...
Write a Thousand Roads to Rome
by Screwtape — 2018-02-08T18:09:21.192Z

Epistemological Status: Pretty sure I'm on to something here, also very sure I'm restating the obvious, utterly confident that restating the obvious is the point.

Sometimes a piece of writing gets two very different responses. Half the commenters say something like "this is really obvious and a waste of time to write" and half of them say something like "this is revolutionary and an amazing insight I never would have reached on my own, thank you!" I think these posts that get this reaction are vital, possibly even more useful in some ways than a post breaking new ground.

First, a preamble describing how my brain is finicky.

When I was first learning Calculus in college, I had a hard time conceptually understanding what a derivative was. I could do the operation most of the time, but didn't know what that operation represented or what real thing those symbols on the page described. Someone suggested it was the rate of change and the second derivative was the rate of change in the rate of change, but that didn't mean anything to me. Someone else suggested it was velocity and the second derivative was acceleration, but I was honestly not great at physics either. I think someone suggested something about stock prices or economics, but I didn't have enough econ knowledge at the time to remember what exactly they said.

Then my professor recognized one of the equations I'd been messing around with on some scrap paper as describing the attack bonus from a game, and pointed out that the derivative of that would be how much better the character would get each level and the second derivative would be useful if characters got stronger at different rates.

"Oh, like an onion knight from Final Fantasy?" I asked.

The professor just gave a confused expression, but the concept had already clicked for me and later that week I was using derivatives at my gaming sessions as well as suddenly doing much better in my physics classes. The moral of the story is that if you want me to understand something, the best way is usually to relate it to a roleplaying game. I'm not the only one who thinks like this- a close friend of mine had no idea why I was worried about intelligence explosions until I reminded him of the Intellect Potion exploit in Morrowind. Yes, this is a silly way for a brain to work, but I think we all agree br...
Mental TAPs
by Logan Riggs — 2018-02-08T17:26:36.774Z

(Prereq: Knowledge of Trigger Action Plans (TAPs). alkjash's here & LifelongLeaner's here)

I converged (sort'of) upon the idea of TAPs on my own two summer's ago, but the difference between my idea (which we'll refer to as mental TAPs or mTAPs) and TAPs is continuous mental simulations.

TAP Problems

Usually with TAPs, you commit to the trigger-action once and mentally rehearse 5 or so times and that's it. This fails in several ways:

When the trigger doesn't happen every day (social fear, being confused, junk food temptations)
The trigger is not easily recognized/understood. (not always catching when you're confused, disappointed, stressed, mentally fatigued)
Doesn't take full advantage of wins/times you actually did the thing!

mTAP Solutions

The only difference is daily mental simulations/rehearsals of your TAPs. I have a sticky note near my toothbrush, so when I brush my teeth, I mentally rehearse my 9 TAPs a couple times each. This helps with (1) above due to being tied to a daily routine (or at least I hope I brush daily!).

Regarding (2), you do your mTAP, face the trigger in real life, update on what the trigger actually feels like in real life, and do an mTAP later that day with your updated trigger. Ex. I mentally simulate (being disappointed -> being curious), I go out and am actually disappointed that I didn't get to spend time with a friend, update on what feeling disappointed actually feels like with this data point, and finally hone in on that feeling when I mTAP that night.

Regarding (3), If you have a BIG WIN, you can hone in on that experience as when you mTAP. Last night I was red in my face embarassed due to social fear, but I still made the right choice despite that. I AM SO FREAKING PROUD OF MYSELF, lol, so I'm honing in on that experience this morning to build off of that win.

WARNING: my last TAP is a meta-TAP (brushing teeth-> look at sticky note and do mTAP). From n=4 experience, if you don't do this, it will become an obligation or you'll forget it.

Request: Has this idea already been explicitely stated elsewhere? Anything else regular old TAPs are missing?
Two kinds of Agency
by Elo — 2018-02-08T06:28:18.505Z

Original post: http://bearlamp.com.au/two-kinds-of-agency/

Let's talk about agency. This week I read Transform Your Self, as reviewed by Kaj Sotala in How I found and fixed the root problems behind my depression and anxiety after 20 years.

Transform your self explains that we each carry identities, self beliefs in a kind of database. A self identity is a completion to the phrase, "I am-". Separate from things that you do, "I drive a bus", or very broad strokes, "I am a human". Rather things that you can answer, for example, "I am kind".

Paul Graham talks about keeping your identity small. As he said, "The more labels you have for yourself, the dumber they make you.", Identity is baggage that can burden us. But Nate Soares says, caring about something is Something To Fight For. So which is it? It's both.

We both need less of the bad types of identity and more of the good types of identity. This is a class of problem of Joint over and under-diagnosis by Scott Alexander. Which is also that we can't simplify this model to "identity is good" or "identity is bad". If we do, we can no longer explain what is going on. We've lost information.

Jointly we need less, "I feel bad because I didn't get the assignment in on time" and more, "I feel good because I did the tasks that I care about". As per usual, Advice should come on a spectrum. And sometimes you have to hear the opposite advice for it to be applicable to your situation.

I've been working on the self concept of "agency". Building a personal database of examples of me exercising my agency. I seem to come up with two types of agency:

Agency Over Your Own Brain

This is the agency where:

In your head you can be anxious about making a phone call but "do it anyway".
You can feel grumpy but not dump that onto other people.
You can feel angry but not punch someone in the face.
You know you don't like taking tablets. So you bargain with yourself and decide you are allowed chocolate with the tablets and only if you take the tablets. Not at other times.
You don't like exercise so you throw away your bus pass so that you have to walk to work.
You know you will usually chicken out of social obligations, so you create a sunk cost by paying for a night course for 10 weeks

...
What Are Meetups Actually Trying to Accomplish?
by mingyuan — 2018-02-08T05:30:48.642Z

Disclaimer 1: I take responsibility for all opinions expressed here and note that they do not necessarily reflect the views of the people I interviewed or of my various employers.

Disclaimer 2: In this post I sometimes make claims on behalf of LW2. I am under the impression that this is mostly okay, but I want to clarify that I don't work for LessWrong.

Introduction

Background

LessWrong meetups have been taking place for almost a decade, and CFAR has a long history of encouraging people to start their own groups. But despite their popularity and staying power in the rationalist community, no one has yet solved meetups beyond the object level - that is, I haven't seen anyone give explicit models of what meetup groups are supposed to accomplish, nor even systematically examine what they've accomplished in the past.

I became interested in this a year ago, when I started my own meetup group and found that I was basically completely directionless, having not had much of a goal beyond 'start a meetup group'. In the intervening months, I’ve become increasingly interested in the theoretical underpinnings of the whole endeavor we call 'meetups', so in an attempt to figure out what the heck is going on, I interviewed members and organizers of eight rationality meetup groups from around the world and collected anecdata from dozens more.

Goals

I embarked on this project with the goal of answering two overarching questions:

What do meetups look like when they go right? What does it take to build a ‘successful’ rationality community, and is there a generalizable way to ensure a community’s success?
Why do some existential risk-focused people care about meetups, and should they? Is there a way to measure or determine what is being created by meetups?

I feel that I've gone some of the way towards answering both of these questions, but that I'm definitely not all the way there yet. Though I think this write-up captures the most important parts of what I've learned, there is a lot of other potentially valuable information that I left out for the sake of brevity, so if you think any of my claims seem unsubstantiated, I encourage you to ask for further details.

I intend to turn this post into a mostly static reference document for organizers, but there are hundreds of perspectives that I was not able to capture here, particularly from people who have many years of experience with this i...
Experiment: Separate  High/Low Productive Hours
by Raemon — 2018-02-08T04:10:05.669Z

Right now, I'm fortunate to be contracting at a tiny organization with extremely high trust. This meant I was able to pitch an offbeat idea:

I could bill for two types of hours:

High productivity hours.
Low/medium productivity hours.

I’d get paid roughly twice as much for the former.

This is something that obviously wouldn't hold up under a scaling organization, and even at our tiny size, we started with an assumption that "in 3 months, Goodheart's Law will probably have distorted the thing beyond usefulness". I think it’d only ever make sense for an employee to opt into, rather than an employer to expect. (I’m actually a bit worried about posting publicly about it, because even the prospect of it becoming a common thing might have weird consequences. But I think on net it’s a neat thing worth talking about)

We're one month in, so... still a chance of this getting distorted. But I currently bet you could maintain it indefinitely in non-scaling organizations where everyone trusts each other (and themselves) at least reasonably. At the very least, I think it's worth doing for at least a month, to a get a sense of how you work.

In January, I worked 75 low/med hours, and 25 high productivity hours. (I took a week off, and I generally work 5ish hours a day)

The payment rates are such that the average of them is approximately what I’d normally get paid (I think the net result of this has been somewhat less payment for me, since I ended up doing a lot more low/med hours than high productivity hours. But I still think this is net positive for me from a “gain more information about myself and grow more.”)

High Productivity hours are fuzzy, and sort of left up to me, but some factors going into high productivity hours:

i. My supervisor and I have to agree that a thing is The Most Important Thing.

This is the straightforward, non-weasel-out-able part, and why I think is most likely to be useful in the longer-term. It's easy to get distracted by things like "make a UI element look prettier" or "fix some random small bug that's annoying me, but isn't the most important thing" or "talk at meetings", none of which are "actually get the highest priority thing done".

Incentivizing the latter seems worth doing.

ii. High product hours should aim for "deliberate practice"

The original pitc...
Monthly Meta: Referring is Underrated
by Chris_Leong — 2018-02-08T01:36:48.200Z

For the next n months I'll be picking an issue within the community that I think is especially important to highlight. LW2.0 is not just a new site, but also an opportunity to do things differently and to ask ourselves how they could be better.

We all know that rationality is supposed to be about winning, but what does this mean? Let's assume that you already have some idea of what your problems are, but if you don't, then we can put not knowing what your problems are down as the first item on the list.

For each problem, there are generally two seperate strategies that we can take: 1) answering the problem by telling you how you can solve it or 2) referring you to other resources that can answer your question far better than we can. Of course, sometimes a combination of the two will work best, but this works well enough as a simple model.

A lot of rationality has been working towards getting better at answering. All of the discussion about biases and decision theory and psychology allow us to often give some pretty good answers.

Unfortunately, improving our ability to refer has recieved a lot less focus. Part of this may be related to status - if you are good at answering all of the credit flows to you, but if you are good at referring most of the credit goes to the person whose work you have referred them to. I suspect that improving referals is seriously underrated as rationality is of limited use if you don't have any concrete skills to utilise. If you want to start a business or improve your social skills or learn maths, then the main way that we can help you win is by pointing you to a great resource. And there's a valid question about how much the community wants to get into this, but if we're trying to systemise winning, it's not really something we can avoid.

Perhaps the lack of focus could be defended by questioning how easily tractable it is, but I'm not convinced that it has recieved enough attention yet for us to draw that conclusion. At the moment, we tend to solve this problems by sorting comments by the number of votes. As I pointed out in a previous post, this is far from the only way of doing this and one of the other techniques could be better.

While I listed a few ideas in that post, I really don't have a solid idea of what can be done about this problem. Instead, I'm just going to list some of t...
Hammertime Intermission and Open Thread
by alkjash — 2018-02-07T22:10:00.382Z

This post marks the end of the first cycle of Hammertime. Click here for intro.

Hammertime will return on Monday 2/19.

I want to close off the first cycle with some thoughts, and designate a place for discussion about the future of this sequence.

Discussion Topics

1. Sequences: Yea or Nay?

I’ve always felt that sequences are a valuable way to organize deeper thoughts and drive home a few central messages from several perspectives. However, the current format and culture on LW seem to radically favor short, independent chunks. (There is also the obvious problem that Sequence construction is not working.)

I’ve been posting daily for a while now but when I shifted from individual posts to a sequence, average Karma immediately dropped by a factor of about 2. It’s possible that people don’t bother upvoting the same sequence, or that my writing quality dropped, but if this is real signal that many more people would read a sequence if they are marketed as individual thoughts (and WordPress stats suggest this as well), that might be reason for me to stop writing sequences in the future, or at least collect sequences together only after they’re complete.

Possible actionable for meta: make posts in a Sequence share karma and/or a single slot on frontpage.

2. Repeat or Explore?

My original intention was to review 10 topics over three cycles, building up in the difficulty of problems solved. I think I will definitely return to and expand on several of the techniques we’ve seen already, but also add more topics. If people have favorite techniques (and hopefully references) they’d like to see in Hammertime, post them here.

3. Monotonicity of Progress

A big goal of mine is to solve the “Rationalist Uncanny Valley,” where beginning rationalists get worse at life before they get better. I can’t believe that this has to be the case; it seems to be symptomatic of a larger failure to develop the proper curriculum. I would like progress on rationality to be monotone – is there a good reason this should be difficult? It’d be great if we could compile a central list of “uncanny valley” failure modes.
Hammertime Day 10: Murphyjitsu
by alkjash — 2018-02-07T18:40:00.679Z

This is part 10 of 30 in the Hammertime Sequence. Click here for the intro.

Like, so pessimistic that reality actually comes out better than you expected around as often and as much as it comes out worse. It’s actually really hard to be so pessimistic that you stand a decent chance of undershooting real life.

Later in the day I will put up an open thread about the first cycle of Hammertime.

We finish up the first cycle with another post on planning. Murphyjitsu is CFAR’s method for planning which asks us to try to be so pessimistic as to undershoot real life.

Day 10: Murphyjitsu

Murphy’s Law states that anything that can go wrong will go wrong.

For our Mandarin-speaking readers, here’s a useful mnemonic: Murphy transliterates as 墨菲 (mo fei), which is homophonous to 莫非, “what if?” That’s why I think of Murphy’s Law as the What If Law.

In the course of making plans, Murphyjitsu is the practice of strengthening plans by repeatedly envisioning and defending against failure modes until you would be shocked to see it fail. Here’s the basic setup of Murphyjitsu:

Make a plan.
Imagine that you’ve passed the deadline and find out that the plan failed.
If you’re shocked in this scenario, you’re done.
Otherwise, simulate the most likely failure mode, defend against it, and repeat.

The first important sub-skill of Murphyjitsu is Inner Sim – the ability for System 1 to simulate failure modes.

Inner Sim

I have the suspicion that everyone is secretly a master at Inner Sim, the ability to instantly simulate failure. Imagine a friend declares to you their New Year’s Resolution: to write a novel, to go on a keto diet, to write a month-long sequence on instrumental rationality.

Now, listen for that internal scoffing – your System 1 instantly proliferates the future with all manner of obstacles. That’s Inner Sim at work.

If you’re anything like me, Inner Sim is better at predicting other people’s failure modes than your own. The mental move that helps apply Inner Sim introspectively is essentially Outside View: take your plan and imagine another person made them. What will go wrong?

Welp Mentality

Inner Sim does surprisingly little on its own.

I had a conversation with a rationalist friend (let’s call him “Alex”) that went something like this:

Alex: What’s bothering you?

Me: I’ve been terribly unproductive. I’m procrastinating on fellowship essays … they’re due in two weeks, and every time I think a...