Harley-ml/lesswrong huggingface dataset:
# Roleplaying As Yourself

(_This is a basic intuition pump I've found helpful in making decisions, and maybe you'll like it too.)_

For all its shortcomings, I think there was something quite useful about the "What Would Jesus Do?" meme within the Christian framework. Of course it's not a very sophisticated ethical guide, and it comes with all kinds of biases; but asking it does put the believer into a frame of mind that emphasizes things like compassion and duty, and it sometimes helps the believer generate options that weren't in their default solution space.

Is there a version of this handy tool for the consequentialist, with our muddled mixture of selfish and altruistic goals and impulses, and the added difficulty that we're looking to actually optimize rather hard?

The one that works best for me is a double roleplay.

* * *

Jernau Gurgeh, champion of strategic games across the galaxy, sits down to a nice futuristic immersive roleplaying game: The Orthonormal Experience. Gurgeh will be controlling a denizen of the early 21st century on Earth, someone with the online name of Orthonormal. Getting Orthonormal to do well by Orthonormal's own standards is Gurgeh's objective.

Just as our roleplaying games have game masters who can call out uncharacteristic plans, so too does Gurgeh's game. He can't simply use his superior vantage point to calculate the right stocks for Orthonormal to buy today and sell tomorrow, because Orthonormal couldn't do that except by luck. He can't even have Orthonormal think at peak performance on some days- there are character attributes (penalties like Anxiety Disorder) he has to play around.

But Gurgeh is able to think patiently, and strategically, about the various obstacles blocking Orthonormal's progress, and to guide Orthonormal's thoughts in plausible ways to work on these. There's a lot of points out there to be scored: better states to reach in Orthonormal's relationships, career, inner life, and more.

What would Gurgeh do?

* * *

One last note: a couple of the bugs with this approach can be confronted within the approach itself. If I decide that Gurgeh might do X, and I try X and fail, it can be tempting to get frustrated with myself. But this isn't what Gurgeh would do next! He'd take my failure as more data about what this character's current attributes are, and look for ways to work around that failure mode or to train the relevant attribute. And he'd probably give the character a short rest to recover mana before trying again.
# Scientist Fiction

[Rabbit fiction](https://www.goodreads.com/list/show/6260.Rabbits) is always about rabbits, but science fiction isn’t always about science. The _best_ science fiction, however, usually is. Science fiction transports the reader to strange and novel worlds; figuring out how those worlds work is often as important to the story as resolving the plot. And the best approach we have to figuring out how a world works is, in fact, science.

_Harry Potter_ is a pure fantasy book, but _[Harry Potter and the Methods of Rationality](http://www.hpmor.com/)_ is very much science fiction. The protagonist of the latter desperately applies the scientific method to understanding Hogwarts, rather than calmly accepting that reality is broken and stuffing his face full of ginger newts.

It’s an interesting exercise to categorize science fiction based on the _science _at the heart of each story. This can be anything from [microeconomics](https://medium.com/conversations-with-tyler/tyler-cowen-andy-weir-artemis-the-martian-7087b6873260), to theoretical [computer science](https://www.goodreads.com/book/show/17084967-wang-s-carpets), to [political philosophy](https://putanumonit.com/2017/04/21/book-review-too-like-the-lightning/). The two books I want to discuss today: Liu Cixin’s _[The Three-Body Problem](https://www.goodreads.com/book/show/20518872-the-three-body-problem)_ goes with the classic combination of math and astrophysics, and Jeff Vandermeer’s _[Annihilation](https://www.goodreads.com/book/show/17934530-annihilation)_ is a rare example of _ecology fiction_.

Both books aren’t leaving the reader to do the science all by themselves, they each are centered around scientist main characters. Three-Body Problem focuses on two Chinese physicists: astrophysicist Ye Wenjie and Wang Miao, a nanotechnology researcher. The main character of _Annihilation _is “the biologist”, because, well:

> There were four of us: a biologist, an anthropologist, a surveyor and a psychologist. I was the biologist. All of us were women this time, chosen as part of the complex set of variables that governed sending the expeditions.  
> \[…\]  
> I would tell you the names of the other three, if it mattered, but only the surveyor would last more than the next day or two.   
>      – _Annihilation_, page 4.

However, the two books have radically different takes on a core question of _what makes a scientist?_ One of the books’ answers I found deeply unsatisfying, while the other created a memorable character that may have altered my own life more than any work of fiction ever has.

[Continue reading on Putanumonit.com](https://putanumonit.com/2018/01/07/scientist-fiction/)
# 1/9/2018 Update - Frontpage Views

A few people had complained that the frontpage views (i.e. "Curated Content", "Frontpage", etc) were still a bit counterintuitive.

I made a couple quick changes:

*   make the buttons green, so that it's more clear you might want to click there at all
*   add a hover-menu, which hopefully improves discoverability and provides some context.
*   in particular, I hope that adding a hover-menu to the "..." more-options button gives it just the right amount of discoverability (i.e. enough that you can find it if you're looking for it or if you've been using the site awhile, but not a choice that newcomers are confronted with immediately)

Responding to Gwillen
---------------------

Gwillen had asked a question in a non-meta thread, and this seems like a good place to respond:

> People have the option to post either to personal blog or frontpage, right? Even people like me with not that much karma? So under what circumstances would a mod move a post from personal blog to frontpage? Might an author not object to that? I'm trying to understand whether I have a good model of the purpose of personal blog versus frontpage, and of what moderators can do, and what they normally do.

The exact details are still being fleshed out here, and the implementation will probably change over time as the site grows and moves from open-beta to launch-ready.

In general, the intent with LW2.0 is to focus on:

*   giving people a place to develop ideas, and improve their ability to make intellectual progress over time
*   design a system so that that the most important ideas bubble up into public common knowledge.

Right now, we want to make sure the site is usable, so we're making it easy for everyone to post to the front page, and only moving their posts _back_ to personal-blog-space if they violate the norms.

As our userbase grows over time, this would eventually result in the frontpage being flooded with content of varying quality. So the rough plan is to gradually ramp up the requirements for posting to Frontpage (eventually limiting it to people with higher karma who have a clear [understanding of the guidelines](https://www.lesserwrong.com/posts/tKTcrnKn2YSdxkxKG/frontpage-posting-and-commenting-guidelines)), with options to get personal blogposts moved to Frontpage when appropriate.

Right now we're erring on the side of moving posts from personal-blogs to frontpage fairly quickly. This may also change. (We've considered creating settings where users can flag if they _want_ a post to be moved to frontpage, but this would add some UI complexity that isn't necessarily worth it)

The personal blogs serve a few different purposes. The one I'm most confident will remain is as a space to discuss ideas shouldn't be front-and-center for new users. (i.e. discussion of politics, discussion of community-qua-community that isn't relevant if you're just starting to explore the site, and calls to action that are trying to persuade rather than explain).
# Surely, rhetorical question?

I just finished Daniel Dennett’s [Intuition Pumps And Other Tools for Thinking](https://smile.amazon.com/Intuition-Pumps-Other-Tools-Thinking/dp/0393348784/). In the book, Dan exposes the reader to several concepts, thought experiments and red flags one may find relevant to the quest of philosophy. I thought I’d share a couple I found very practical, along with an exercise where we can apply them.
# No, Seriously. Just Try It: TAPs

This next semester (I'm in university, so that's how I measure time) I'm working on developing my ability to better integrate arbitrary habits into my behavior. [Trigger-action-planning](https://www.lesserwrong.com/posts/v4nNuJBZWPkMkgQRb/making-intentions-concrete-trigger-action-planning) ([a more detailed explanation as well](https://www.lesserwrong.com/posts/vE7Z2JTDo5BHsCp4T/instrumental-rationality-4-2-creating-habits)) is LW's most concrete strategy for doing such a thing, so I've decided to [just try it](http://lesswrong.com/lw/5a5/no_seriously_just_try_it/). Starting 3 weeks ago and proceeding for the next 6 months I'm working on my personal approach to Trigger-action-planning (TAP for short).

  
My basic structure for this:

*   Each week (Sunday morning) design a new TAP, or redesign an old one.
*   Aim to follow through on the TAP in all applicable situations throughout the week.
*   At the end of each day (as part of an already existing review process) note if I did or did not follow through.
*   At the end of each week, look back and think on if the TAP was useful, what worked well, what was hard, and all that jazz.

The biggest things that's jumped out at me so far has been that not all TAPs are created equal. I think [conceptual similarity does not imply actionable similarity](https://mindlevelup.wordpress.com/2017/12/29/conceptual-similarity-does-not-imply-actionable-similarity/) applies very heavily to TAPs. In light of this, I'm approaching each week's TAP from a very implementation specific perspective, and then afterwards I'm going to think about what connections and universal principles might apply.

  
Here are some things I've noticed so far:

*   It's seems like there's a "mindfulness bootstrapping" problem, in that it often feels like my TAPs are only activated because of preexisting mindfulness triggers.
*   I've been putting the handle for each week's TAP on my phone lock screen. This was a very effective reminder originally, though it's already lost most of its power three weeks in. Mayhaps having a randomly generated background image would help prevent my mind from filtering out expected reminders. One google search and perusing the first page of results did not produce an app to do this.
*   I notice that sometimes there's a glimmer of noticing I'm in a context to activate the TAP, yet I don't. Part of this might be friction related to stopping whatever train of thought/awareness I'm currently on. Giving myself explicit permission to derail thoughts to execute a TAP could help.

I'll be posting updates on this project every so often. If you have had trouble with attempting TAPs or similar situational habit practices in the past, I'd love to hear about the specifics how what you've tried and what hasn't worked.
# Plan to Be Lucky

  

Like other rationalists, I’ve [written about](https://putanumonit.com/2016/11/23/climbing-the-horseshoe/) the dangers inherent in our instincts for tribalism. [Unlike other rationalists](https://twitter.com/yashkaf/status/953012916906807296), I think that instead of being suppressed tribalism should be indulged in a productive manner. That’s why I take a couple of hours each week, alone or with friends, to scream at a TV showing some sports game that involves Carolina, Tottenham or the Philadelphia Eagles.

So there I was on Saturday as the Eagles went into a playoff game against the Falcons as 3.5 point underdogs. Philadelphia had the best team in the league for most of the season, but in week 14 their star player Carson Wentz tore his ACL on a meaningless play and was replaced by Nick Foles, a loveable goofball whose career was so up-and-down-and-down-and-down that [he considered retiring from football](http://www.nbcsports.com/philadelphia/eagles/eagles-thankful-nick-foles-reconsidered-retiring) at age 26.

Foles, who hasn’t practiced with the rest of the Eagles’ first-string offense, played poorly in the last three weeks of the regular season. Every preview of the upcoming playoff game talked about the Eagles having to run the ball, pass conservatively, and win with their defense, given how unlikely it was that Foles could play well.

The game started, and Nick Foles dropped back to attempt his first playoff pass in four years. But instead of dumping the ball for a short pass, Foles launched a 42-yard bomb as if he were a battle-tested superstar.

The pass missed the receiver by ten feet.

And the thing I screamed at the TV was: _“Yes! That’s how you f%#&ing do it!!!”_

_Continue reading on [Putanumonit.](https://putanumonit.com/2018/01/15/plan-to-be-lucky/)_
# Conversational Presentation of Why Automation is Different This Time

I have been frustrated recently with my inability to efficiently participate in discussions of automation which crop up online and in person. The purpose of the post is to refine a conversational presentation of what I believe to be the salient concerns; the chief goals are brevity and clarity, but obviously corrections of fact supersede this.

Epistemic status: plausible causal conjecture.

I think the current wave of automation will be different from previous ones, in ways which make it more disruptive. There are three reasons for this:

**No Fourth Sector**: The economy has three broad sectors: agriculture, manufacturing, and services. The first wave was in agriculture, and people could find adjacent work or switched to working in manufacturing. The second wave was in manufacturing, and people could find adjacent work or switched to work in services. The current wave is affecting services, but there is no fourth sector of the economy left for workers to switch to.

**Skills Over Jobs**: Agricultural automation was largely about tasks: a digging machine, a seeding machine, a pulling machine. Manufacturing automation took this to the next level, with robots performing defined sequences of tasks. But in both cases these were specific - any task or series of tasks which had not been specifically automated was still work to be had. The new wave of automation is entire skillsets, like _apply this pattern_ or the _ability to speak_. This means when a job is lost to automation, all similar jobs are going away at the same time. There will be no adjacent work for people to switch to.

**Speed**: When automation was physical machines, they had to design them, and build them, and ship them, and customers had to rebuild their own factories to use manufacturing robots. Modern automation is largely software driven, so design and build are the same process, which is then practically free to copy and distribute. As soon as the method is ready, it can be picked up by businesses as fast as they can rent server space to run it. This gives local economies and institutions like government very little time to respond.

Automation is different this time because the problems we experienced last time will be more severe, and more widespread, and happen faster.
# Making Exceptions to General Rules

Suppose you make a general rule, ie. "I won't eat any cookies". Then you encounter a situation that legitimately feels exceptional , "These are generally considered the best cookies in the entire state". This tends to make people torn between two threads of reasoning:

1) Clearly the optimal strategy is to make an exception this one time and then follow the rule the rest of the time.

2) If you break the rule this one time, then you are likely to dismantle the rule and ending up not following it at all, so you should whether the rule is good on the whole, instead of for this particular opportunity

How can we resolve this? For a very, very long time I didn't want to take option 2) because I felt that taking a sub-optimal strategy was irrational. It may seem silly, but I found this incredibly emotionally compelling and I had no idea of how to respond to this with logic. Surely rationality should never require you to be irrational? It took me literally years to work out, but 1) is an incredibly misleading way of framing the situation. Instead, we should be thinking:

3) If future you will _always_ make the most rational decision, the optimal strategy is going to clearly be to make the exception. If there is a chance that it will cause you to fall off the path, either temporarily or permanently, then we need to account for these probabilities in the expected value calculation.

As soon as I realised this was the more accurate way of framing 1), all of its rhetorical power disappeared. Who cares about what is best for a perfectly rational agent? That isn't you. The other key benefit of this framing is that it pushes you to think in terms of probability. Too often I've thought, "I can make an exception without it becoming a habit". This is bad practise. Instead of thinking in terms of a binary "Can I?" or "Can't I?", we should be thinking in terms of probabilities. Firstly, because 0% probability is unrealistic, secondly because making a probability estimate allows you to better calibrate over time.

We can also update 2) to make it a more sophisticated argument as well.

4) If you break the rule this one time, then you risk dismantling the rule and ending up not following it at all. Further, humans tend to heavily biased towards believing that their future selves will make the decisions that they want it to make. So much so, that attempting the calculate this probability is hopeless. Instead, you should only make an exception if the utility gain would be so much that you would be willing to lose the habit altogether.

We still have two different ways of thinking of the problem, but at least they are more sophisticated than when we started.

(This article was inspired by seeing: [The Solitaire Principle](https://www.lesserwrong.com/posts/n8k5qL7w6iCWHDZrd/the-solitaire-principle-game-theory-for-one). I wanted to explain how I've progressed on this issue and I also wanted to have a post to link people to which is much shorter)
# Beware of black boxes in AI alignment research

Over the course of the [AI Alignment Prize](https://www.lesserwrong.com/posts/YDLGLnzJTKMEtti7Z/announcing-the-ai-alignment-prize) I've sent out lots of feedback emails. Some of the threads were really exciting and taught me a lot. But mostly it was me saying pretty much the same thing over and over with small variations. I've gotten pretty good at saying that thing, so it makes sense to post it here.

Working on AI alignment is like building a bridge across a river. Our building blocks are ideas like mathematics, physics or computer science. We understand how they work and how they can be used to build things.

Meanwhile on the far side of the river, we can glimpse other building blocks that we _imagine_ we understand. Desire, empathy, comprehension, respect... Unfortunately we don't know how these work inside, so from the distance they look like _black boxes_. They would be very useful for building the bridge, but to reach them we must build the bridge first, starting from our side of the river.

What about machine learning? Perhaps the math of neural networks could free us from the need to understand the building blocks we use? If we could create a _behavioral imitation_ of that black box over there (like "human values"), then building the bridge would be easy! Unfortunately, ideas like [adversarial examples,](https://blog.openai.com/adversarial-example-research/) [treacherous turns](http://lesswrong.com/lw/l9u/superintelligence_11_the_treacherous_turn/) or [nonsentient uploads](http://lesswrong.com/lw/ig8/9nwk/#9nr5) show that we shouldn't bet our future on something that imitates a particular black box, even if the imitation passes many tests. We need to understand how the black box works inside, to make sure our version's behavior is not only similar but based on the right reasons.

(Eliezer made the same point more eloquently a decade ago in [Artificial Mysterious Intelligence](http://lesswrong.com/lw/wk/artificial_mysterious_intelligence/). Still, with the [second round](https://www.lesserwrong.com/posts/4WbNGQMvuFtY3So7s/announcement-ai-alignment-prize-winners-and-next-round) of our prize now open, I feel it's worth saying again.)
# The Desired Response

A friend once told me a story of how an interaction with her mother changed her perspective on communication. She said that she had been going through a break up at the time, and was venting to her mother, when her mother responded with "Do you want my advice, or my sympathy?"  

Often times, when we have something important to talk about, we consider how we expect the other person to respond, and talk to people who will respond the kind of way we want. We may choose to speak to someone who we know can keep a secret, or who gives good advice, or who will say nothing at all, only listen. Sometimes we may turn to someone who won't want to talk about it at all, and will instead distract us from our problems. This can be great if you have a number of close friends who reliably respond in different ways, and you are able to predict this and make use of it. However, not everyone is in this situation, and so instead, we end up with interactions where people are not getting their desired responses.  

Take an interaction between example-humans Alice and Bob. Alice's goal in the conversation may be to be heard, whereas Bob may want to feel validated for what he says. If Alice is talking, Bob may point out something or make a clever remark. Instead of validating Bob, Alice feels like she hasn't been heard, and is quiet in response. Now Bob has not been validated either, and they are both sad.  

Sometimes, a person's desired response can be inferred. This is like how generally, if someone tells a joke, their desired response is for you to laugh, or at least acknowledge that the joke was funny. However, this isn't always easy to tell. Some of us may know the best thing for our best friend in a time of stress, but we don't always know what's right for someone else, even if we are close. Often times we will assume that what works for us will work for them, but then there is a risk of emotional damage. Trying to talk about a situation that someone wants to avoid thinking about may exacerbate the problem or cause tension between two people. Usually, it is better to ask a person what they need -- like how my friend's mother asked her.  

Going more deeply, some people may tend to have common desired responses, in general interactions. Think of the person constantly telling bad jokes, excited to hear people groan and laugh, or someone who loves giving advice and recommendations. The first of these two may love the validation they get from humor, whereas the second may want to feel helpful, and be appreciated for it. This goes far deeper than a single situational interaction; these people want these kinds of responses in everyday conversations. This may even tie to how we want others to see us (clever or kind) or how we most enjoy interacting with others (playing with ideas or working with people, receiving attention or giving it). A person's desired response may not always be the response that feels right, or the response that we want to give -- and that's perfectly okay. But it does give us insight into what drives them, what is important to them, and who they are as a person.
# How the LW2.0 front page could be better at incentivizing good content

(This should probably be in meta, but I'm not sure how to post to meta. Feel free to move if needed.)

Imagine tomorrow someone writes the best ever LW2.0 post and everyone upvotes it. How can front page visitors read that best and newest post? They must scroll down (no posts are visible above the fold) and then click (no posts are expanded by default).

Attention is a key scarce resource for incentivizing good content creation. Our current front page is an attention firehose pointed away from the best newest stuff. Where does it point instead? 1) Eliezer's sequences, which are finished, so we incentivize nothing. 2) SSC, which already gets a ton of visitors and where only Scott can post anyway, so we incentivize nothing.

Compare with how other websites do it. Reddit and HN put the best newest stuff at the top, though you have to click. Old LW used to put the best newest stuff at the top and expanded by default, so you could check the site first thing in the morning and get the awesomeness. SSC does the same.

Even worse, we have /daily which is more readable than the front page (you don't have to scroll). And getting a post into /daily doesn't require upvotes. The incentive gradient for writers is nowhere to be seen! On old LW, getting my posts promoted to Main felt special. On LW2.0, several of my posts were front paged and curated and it didn't feel special at all.

The best use of the attention firehose is incentivizing good content creation. We should make the front page work like a blog, with the best newest entries shown at the top and expanded. Move the sequences and SSC into a sidebar, like every other website does for good reason. Then we might finally enjoy some actual competition for the front page, which would be a good thing.
# The Tallest Pygmy Effect

Status: I thought this was a common economics term, but when I google it I get either unrelated or references [using it the way I expect](http://econlog.econlib.org/archives/2008/10/the_tallest_pyg.html) but not defining it. It’s a really useful term, so I’m going to attempt to make it a thing.

“Tallest Pygmy Effect” is when you benefit not from absolute skill or value at a thing, but by being better at it than anyone else.  For example, the US dollar is not that great a currency and the US economy is not that great an economy. However, the dollar is more stable than other currencies, so it becomes the currency of choice when you want stability. This high volume makes USD more stable and is in general good for the US economy (because e.g. US companies don’t have to take on currency risk when they borrow money).

Tallest pygmy effects are fragile, especially when they are reliant on self-fulfilling prophecies or network effects. If everyone suddenly thought the Euro was the most stable currency, the resulting switch would destabilize the dollar and hurt both its value and the US economy as a whole.
# A simpler way to think about positive test bias

Eliezer described "positive bias" (which I'll rename "positive test bias" for reasons explained in Unnamed's comment below) in an [LW post](http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/) and an [HPMOR chapter](http://www.hpmor.com/chapter/8). According to him, dealing with that bias requires a kind of mental gymnastics that doesn't come naturally to most people: "twisty negative thinking" or "flinching toward zero". You're supposed to devise tests that come out _false_ if your hypothesis is _true_. It's a bit confusing.

I think there's a simpler way to think about it. Positive test bias is just our bias toward strong hypotheses. You can deal with it by asking yourself, how can I test if my hypothesis is too strong?

*   The LW post about the bias mentions the Wason 2-4-6 task. If you're told that the number sequence 2-4-6 has property X, it's tempting to guess that property X means "ascending arithmetic progression". To test if that hypothesis is too strong, you need to try some other sequences that are arithmetic progressions but not ascending, and some that are ascending but not arithmetic progressions. Easy!
*   The HPMOR chapter describes Hermione spilling some soda on her robe, and a moment later it mysteriously becomes clean again. Hermione comes up with a hypothesis that her robe is magically self-cleaning. To test if that hypothesis is too strong, she can try spilling something else on her robe. There's no need for any counterintuitive thinking.

That technique is useful in other areas as well. For example, I often get carried away when writing posts, and end up with a draft full of wide-reaching conclusions. But then I ask myself, doesn't that sound a bit too strong? So I look for counterexamples, and the point I'm trying to make either dissolves or becomes much more robust.

Our bias toward strong hypotheses is especially harmful in politics. We will defend a hypothesis like "group X is to blame for everything" for its explanatory power, never noticing the real problem—that it's too strong. We'd all benefit from recognizing when it happens, and weakening our hypotheses until they match reality.
# Understanding-1-2-3

Epistemic Status: Seems worth sharing

Assumes Knowledge Of: [System 1 and System 2](http://bigthink.com/errors-we-live-by/kahnemans-mind-clarifying-biases) from [Thinking, Fast and Slow](https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman-ebook/dp/B00555X8OA/ref=sr_1_1?ie=UTF8&qid=1516664405&sr=8-1&keywords=think+fast+and+slow)

While in San Francisco this past week, I found myself in many complex and rapid conversations, with new ideas and concepts flying by the second. I hadn’t experienced this since least February’s AI Safety Unconference. It is wonderful to be pushed to one’s limits, but it means one must often rely on System-1 rather than System-2, or one will be left behind and lost. I now realize this is what happened to me in foreign language classes – I was using System-2 to listen and talk, and that doesn’t work _at all _if others don’t wait for you.

This meant that often I’d grasp what was being said in an intuitive sense using System-1, or agree or disagree in that fashion, without the time to unpack it. This can lead to great understanding, but also lead to waking up the next day having no idea what happened, or thinking about it in detail and realizing you didn’t understand it after all. So when people asked if I understood things, I instinctively started saying things like “I understand-1 but not 2.” I also did this for beliefs and agreements.

I debated explaining it but decided not to, as a test to see if others would intuit the meaning, most seemed to (or ignored the numbers, or came up with something equivalent in context); one person explicitly asked, and said he’d use it. I think this may be a valuable clarification tool, as it’s very different to understand-1 versus understand-2, or agree-1 versus agree-2, and seems better than saying something like “I _kind of _agree” or “I _think _I agree but I’m not sure,” which are both longer and less precise ways people often say the same thing.

My system-1 also added understand-3. Understand-3 means “I understand this well enough to teach it.” By extension, believe-3 or agree-3 means “I believe this, know how to convince others, and think others should be convinced.” To truly understand something, one must be able to explain, teach or defend it, which also means you and others can build solidly upon it. Writing has helped me turn much understanding-2 (and understanding-1) into understanding-3.
# Interpersonal Approaches for X-Risk Education

Much of the AI research community remains unaware of the Alignment Problem (according to my personal experience), and I haven't seen much discussion about how to deliberately expand the community (all I've seen to this effect is [Scott's A/B/C/D/E testing on alignment articles](http://slatestarcodex.com/2016/10/24/ai-persuasion-experiment-results/)).

  

Expanding the number of people aware of (and ideally, working on) the alignment problem is a high-leverage activity: a constant amount of effort spent educating someone in exchange for a chance of recruiting an ally who will work hard at our sides. Another metric by which we should evaluate approaches is whether we have to convince or simply educate; professors and high-status researchers may be more dismissive (possibly due to the [inside view](https://wiki.lesswrong.com/wiki/Outside_view), their wariness of strange-sounding ideas, and [overconfidence in their long-term predictions](https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert)), but their influence would be greater. On the other hand, a good friend in a CS or Math under-/post-graduate program may be more receptive.

  

In my case, I stumbled upon HP:MoR one year ago, read the Sequences, and then read more about Alignment and [CEV](https://intelligence.org/files/CEV.pdf). I appreciated that Alignment was a serious problem, but it wasn't until I got through _Superintelligence_ that I realized it's basically The Problem. Being in the second year of my doctorate program, I didn't know whether I was "too late" to start learning the math, "too far behind" people like Eliezer to make a difference. What I did know is that everyone can't defect - we need people to put in the work, and we probably need substantially more people doing so.

  

What happened to me took a lot of time and may be unrealistic to recommend to others. The articles Scott tested seem equally effective; instead, I'd like to discuss what social approaches work best for taking people from friend to friend-who-takes-alignment-seriously (while optimizing against effort expended), and whether this is an efficient use of our time.
# What are the Best Hammers in the Rationalist Community?

I've toyed with the idea of giving an intro to rationality, hoping I can find the 20% of the material that provides 80% of the gains (or the 5% that provides 50% of the gains).

Here and there, I've also been asked what the rationality community is about, and I've struggled to give an effective summary. I usually say the 3 keys are seeing in terms of concepts rather than words (a la rationalist Taboo or hyperdimensional thingspace), fighting motivated reasoning (since most people are already strong enough to prove others wrong and just need to turn that force upon themselves), and correcting for the heuristics and biases. But are these really the top 3 most important pieces? Even if they are, what other Hammers would take 4th, 5th, and 6th place?

The Hammers and Nails post got me thinking again: What are the most powerful pieces of rationalist lore? If I had 1 day to tutor a brilliant, but untrained mind, what are the most powerful techniques that can most easily be taught?

The comments under Hammers and Nails are full of people saying what their favorite and most powerful techniques are, but I'd like to make an explicit invitation here to share. What are your hammers? (see https://www.lesserwrong.com/posts/QzBuuNEqJGQFeWM4f/hammers-and-nails)
# "Slow is smooth, and smooth is fast"

I think it's worthwhile to signal boost bits of wisdom that turn out to be surprisingly useful. Apparently "Slow is smooth, and smooth is fast" is a Navy SEAL saying. The meaning is pretty clear. Practice slowly so that the correct motor patterns are ingrained. And perhaps equally importantly, _execute_ "slowly", that is to say, _don't rush._ Pushing your nervous system to perform faster than it's trained to will simply cause you to fumble what you're doing, and the end result is that your rushed performance is slower than if you had attempted a measured cadence in the first place.

On the rare occaisions that I've worked closely with high performers, be they artists or athletes or academics, I've noticed that they are all _slow_. The skilled artists never rush through their warmup exercises. The athletes are patient with their bodies. The academics will spend ten minutes on a task that might take me two minutes, because they're doing things like checking for consistency, giving themselves time to think things through and notice irregularities, and perhaps even formatting a table or graph to look nicer even if it's not necessarily ever going to be seen by anyone but themselves. The end result of all this "slowness" at the micro level is acceleration and efficiency at the macro level, and improved overall performance.

Show me somebody who gets things done at an unbelievable pace and I'll show you somebody whose life probably looks unbearably slow when you actually look at what they're doing on a daily basis.
# European Community Weekend 2018 Announcement

We are excited to announce this year's European LessWrong Community Weekend. For the fifth time, rationalists from all over Europe (and some from outside Europe) are **gathering in Berlin** to socialize, have fun, exchange knowledge and skills, and have interesting discussions.

The event takes place **September 7th to September 9th** and, like last year, it will be held in the beautiful Jugendherberge Wannsee which contains a large room for central events, several seminar rooms, and lots of comfortable spaces inside and out to socialize or relax.

This is a community-driven event. That means that while there will be a keynote and pre-planned content, the bulk of the schedule will be filled by the participants. There will be space to give talks, short or long, provide workshops, or just gather some people to do an activity together. In previous years we had the talks, lightning talks and workshops you would expect, as well as lighter activities such as morning-workouts, meditation sessions, authentic relating games, swimming in the lake and many more. Of course, there will also be time to reconnect with friends and form new connections with other aspiring rationalists.

**Some valuable information**

Most of the talks and discussions will be held in English, so you do not need to be able to speak German to attend.

The **ticket price of €150** includes accommodation for two nights, on-site meals (breakfast, lunch, dinner) and snacks, and a welcome lunch on **Friday at 12:00**.

The event wraps up **Sunday afternoon around 15:00**. In the days after the weekend, participants are invited to stay in Berlin a little longer to explore the city, go bouldering, play frisbee, etc. While this is not part of the official event, we will coordinate couch-surfing opportunities to avoid the need for hotels.

**tl;dr**

*   When? 7-9 September 2018
*   Where? [http://jh-wannsee.de](http://jh-wannsee.de)
*   How much? €150
*   Apply here: [http://tiny.cc/lwcw2018_signup](http://tiny.cc/lwcw2018_signup) and
*   Submit a contribution to support your application: [http://tiny.cc/lwcw2018_contribution](http://tiny.cc/lwcw2018_contribution)

If you have any question, please email us at [lwcw.europe@gmail.com](mailto:lwcw.europe@gmail.com).

Looking forward to seeing you there,

The Community Weekend organizers and LessWrong Deutschland e.V.
# Upvote/downvote amounts

In light of the variable voting powers that the new karma system has, I keep finding myself wanting to be able to vote less than my "full power" on a comment.

But in particular, the thing I think I always want in practice is to be able to _downvote_ less than that. Downvoting a comment by 4 feels like a huge slap in the face, especially if the comment already has 4 or fewer points. I think the default downvote weight should probably be 1, or n/2, or something less than the default upvote weight. Upvoting is a positive community interaction, and downvoting, while necessary, is sort of a necessary evil at best.

I'm not sure whether I'd argue in favor of going further to actually enable choosing the amount for up/down votes, or just down-weighting downvoting.
# The different types (not sizes!) of infinity

In a recent conversation, a smart and mathy friend of mine revealed they didn't understand Cantor's diagonal argument. Further questionning revealed that she was using the wrong concept of infinity. I thought I'd pass on my explanations from there.

What is infinity? Does one plus infinity make sense? What about one over infinity? Well, it all depends what concept of infinity you're using. Roughly speaking, infinity happens when you take a finite concept and add a "and then that goes on for ever and ever" at the end. There are four major examples:

1.  If you take the concept of "size" and push that to infinity, you get the infinite cardinals.
2.  If you take the concept of "ordered set" and push that to infinity, you get the infinite ordinals.
3.  If you take the concept of "increasing function" and push that to infinity, you get the poles and limits of continuous functions - a major part of real analysis.
4.  And if you take normal algebra and push that to infinity, you get the [hyperreals](https://en.wikipedia.org/wiki/Hyperreal_number).

To add to the confusion, there are normally only two symbols for infinity to go around - ω and ∞ - even though there are at least four very different concepts (honourable mention also goes to the infinities of complex analysis and algebraic geometry, which are like more limited versions of the real analysis one).

So, does ω+1 make sense (as something different from ω)? It does, for the ordinals and hyperreals only. What about ω-1, similarly? That only makes sense for the hyperreals. And -ω? That works for the hyperreals and real analysis. 1/ω? This is well defined for the hyperreals (where it is not equal to 0), and is arguably well-defined for real analysis, where it would be equal to 0.

Is ω times 2 the same as ω? "Yes", say real analysis and cardinals; "No", say ordinals and hyperreals. How about there being many different infinities of different sizes? That makes sense for all of these, except real analysis.

So when talking about "infinity", be careful that you're using the right concept of infinity, and don't mix one with another, or import the intuitions of one area into an arena where it doesn't make sense.

\*\*\*

  

**EDIT**: See also Scott Garrabrant's [comment](https://www.lesserwrong.com/posts/GhCbpw6uTzsmtsWoG/the-different-types-not-sizes-of-infinity/Ajwtd8BtHKdMyGN4x):

Starting with the natural numbers:

*   Cardinals: How many are there?
*   Ordinals: What comes next?
*   Limits: Where are they going?
*   Hyperreals: What is bigger than all of them?
# Biological humans and the rising tide of AI

The [Hanson-Yudkowsky AI-Foom Debate](http://intelligence.org/files/AIFoomDebate.pdf) focused on whether AI progress is winner-take-all. But even if it isn't, humans might still fare badly.

Suppose Robin is right. Instead of one basement project going foom, AI progresses slowly as many organizations share ideas with each other, leading to peaceful economic growth worldwide - a rising tide of AI. (I'm including uploads in that.)

With time, keeping biological humans alive will become a less and less profitable use of resources compared to other uses. Robin says humans can still thrive by owning a lot of resources, as long as property rights prevent AIs from taking resources by force.

But how long is that? Recall the displacement of nomadic civilizations by farming ones (which happened by force, not farmers buying land from nomads) or [enclosure](https://en.wikipedia.org/wiki/Enclosure) in England (which also happened by force). When potential gains in efficiency become large enough, property rights get trampled.

Robin argues that it won't happen, because it would lead to a slippery slope of AIs fighting each other for resources. But the potential gains from that are smaller, like a landowner trying to use enclosure on another landowner. And most such gains can be achieved by AIs sharing improvements, which is impossible with humans. So AIs won't be worried about that slippery slope, and will happily take our resources by force.

Maybe humans owning resources could upload themselves and live off rent, instead of staying biological? But even uploaded humans might be very inefficient users of resources (e.g. due to having too many neurons) compared to optimized AIs, so the result is the same.

Instead of hoping that institutions like property rights will protect us, we should assume that everything about the future, including institutions, will be determined by the values of AIs. To achieve _our_ values, working on AI alignment is necessary, whether we face a "basement foom" or "rising tide" scenario.
# RSS Feeds are fixed and should be properly functional this time

Just pushed a fix that should make RSS feeds work properly (you will have to resubscribe to the RSS feeds, if you did so a while ago). Thanks to [squirrelInHell](https://www.lesserwrong.com/posts/EfMBh23Nb9pdJtmG5/rss-troubles) who made me aware of the deeper problems with our RSS feed (that my RSS reader seemed to mostly automatically patch, but which other people ran into).
# OpenPhil's "Update on Cause Prioritization / Worldview Diversification"

OpenPhil recently posted an update on their overall cause prioritization strategy. In particular, exploring the concept of worldview diversification:

> When choosing how to allocate capital, we need to decide between multiple worldviews. We use “worldview” to refer to a highly debatable (and perhaps impossible to evaluate) set of beliefs that favor a certain kind of giving. Worldviews can represent a mix of moral values, views on epistemology and methodology, and heuristics. Worldviews that are particularly salient to us include:

> **• a “long-termist” worldview** that ascribes very high [value to the long-term future](https://www.openphilanthropy.org/blog/moral-value-far-future), such that it assesses grants by how well they advance the odds of favorable long-term outcomes for civilization;

> **• a “near-termist, human-centric”** worldview that assesses grants by how well they improve the lives of humans (excluding all animal welfare considerations) on a relatively shorter time horizon;

> **•** **a “near-termist, animal-inclusive”** view that focuses on a similarly short time horizon but ascribes significant moral weight to animals.

> If we evaluated all grants’ cost-effectiveness in the same terms (e.g., “Persons helped per dollar, including animals” or “Persons helped per dollar, excluding animals”), this would likely result effectively putting all our resources behind a single worldview.

> For reasons given below (and [previously](https://www.openphilanthropy.org/blog/worldview-diversification)), we don’t want to do this. Instead, we’re likely to divide the available capital into buckets, with different buckets operating according to different worldviews and in some cases other criteria as well. E.g., we might allocate X% of capital to a bucket that aims to maximize impact from a “long-termist” perspective, and Y% of capital to a bucket that aims to maximize impact from a “near-termist” perspective, with further subdivisions to account for other worldview splits (e.g., around the moral weight of animals) and other criteria.

> These buckets would then, in turn, allocate “their” capital to causes in order to best accomplish their goals; for example, a long-termist bucket might allocate some of its X% to [biosecurity and pandemic preparedness](https://www.openphilanthropy.org/focus/global-catastrophic-risks/biosecurity), some to causes that seek to improve decision-making generally, etc.

The [full post is quite long](https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy), but seemed useful both to highlight a useful way of thinking, and to generally stay up-to-date on how OpenPhil is strateg
# An experiment

Alexei's [postmortem of Arbital](https://www.lesserwrong.com/posts/kAgJJa3HLSZxsuSrf/arbital-postmortem) got me thinking about incentivizing good online explanations, so I got an idea for a test that we can run here and now:

What's your simplest, most vivid, most memorable explanation of _correlation does not equal causation_ for someone who's unaware of the concept? In your own words, no linking.