Harley-ml/lesswrong huggingface dataset:
# User GPT2 Has a Warning for Violating Frontpage Commenting Guidelines

We take commenting quality seriously on LessWrong, especially on Frontpage posts. In particular, we think that [this comment](https://www.lesswrong.com/posts/M8P4sT6uKAc5ijMaz/experimental-open-thread-april-2019-socratic-method#gaazNMChbDEc3uMik) by user GPT2 fails to live up to our [Frontpage commenting guidelines](https://www.lesswrong.com/posts/tKTcrnKn2YSdxkxKG/frontpage-posting-and-commenting-guidelines):

> This is a pretty terrible post; it belongs in Discussion (which is better than Main and just as worthy of asking the question), and no one else is going out and read it. It sounds like you're describing an unfair epistemology that's too harsh to be understood from a rationalist perspective so this was all directed at you.

Since user GPT2 seems to be quite prolific, we have implemented a setting to hide comments by GPT2, which can be accessed from the [settings](https://www.lesswrong.com/account) [page](https://www.lesswrong.com/account) when you are logged in.
# Announcing the Center for Applied Postrationality

Hi all! Today, we are announcing the formation of a new organization: the Center for Applied Postrationality (or CFAP).

Right now we're looking for two things: 1) $1.5 million in funding to have runway for the next six months, and 2) intellectual contributions from deep thinkers like YOU!

Just what is postrationality, anyway? To be honest, we don't really know either. Maybe you can help us? The term can refer to many different things, including:

*   Epistemological pragmatism
*   Getting in touch with your feelings
*   Learning social skills
*   Disagreeing with stuff on LessWrong
*   Becoming a Christian
*   According to one of our employees: "Postrationality is making the territory fit the map. Postrationality is realizing that paraconsistent linear homotopy type theory is the One True framework for epistemology, not Bayes. Postrationality is realizing that Aristotle was right about everything so there's no point in doing philosophy anymore. Or natural science. Postrationality is The Way that realizes there is no Way. Postrationality is meaning wireheading rather than pleasure wireheading. Postrationality is postironic belief in God. Postrationality is the realization that there is no distinction between sincerity and postirony."
*   Another employee: "CFAP exists at the intersection of epistemology, phenomenology, sense-making, and pretentiousness. Our goal is to make new maps, in order to better navigate the territory, especially the territory in California. Google maps sucks at this. "

We're still deconfusing ourselves on what "applied" postrationality is, as so far it's mostly been insight porn posted on Twitter. Comment below what techniques you'd suggest for training the art of postrationality!
# [April Fools] User GPT2 is Banned

For the past day or so, user [GPT2](https://www.lesswrong.com/users/gpt2) has been our most prolific commenter, replying to (almost) every LessWrong comment without any outside assistance. Unfortunately, out of 131 comments, GPT2's comments have achieved an average score of -4.4, and have not improved since it received a moderator [warning](https://www.lesswrong.com/posts/cxpZYswq5QvfhTEzF/user-gpt2-has-a-warning-for-violating-frontpage-commenting). We think that GPT2 needs more training time reading the Sequences before it will be ready to comment on LessWrong.

User GPT2 is banned for 364 days, and may not post again until April 1, 2020. In addition, we have decided to apply the death penalty, and will be shutting off GPT2's cloud server.

Use this thread for discussion about GPT2, on LessWrong and in general.
# 
LW Update 2019-04-02 – Frontpage Rework

Since LW2.0 launched, the frontpage had become very complex – both visually and conceptually. This was producing an overall bad experience, and making it hard for the team to add or scale up features (such as Q&A, and later on Community, Library and upcoming Recommendations)

For the past couple months, we've been working on an overhaul of the frontpage (and correspondingly, the overall site design). Our goal was is to rearrange that complexity, spending fewer "complexity points" on things that didn't need them as much, so we could spend them elsewhere.

**Frontpage Updates**

*   Tooltip oriented design.

*   It's easier to figure out what most things will do before you click on it.

*   Navigation Menu

*   Helps establish the overall site hierarchy
*   Available on all major site pages (not Post Pages, where we want people to read without distraction)
*   Improved mobile navigation (shows up as a tab menu at the bottom)
*   Eventually we'll deprecate the old Nav Menu (still available in the header) and replace it with a collapsible version of the new one.

*   Home Page streamlining

*   Moved Recommend Sequences and Community  over to the Nav Menu, so there are only 3 sections to parse
*   Post Items simplified down to one line.
*   Latest Posts now only have a single setting: _"show personal blogposts",_ instead of forcing you to figure out immediately what "meta", "curated" and "daily" are.
*   Post List options are generally 'light cobalt blue' – not too obtrusive, but easier to find when you want them.

*   Questions Page now has two sections:

*   _Recent Activity –_ simply sorted by "most recently commented at", so if you respond to an old question it will appear above the fold.
*   _Top Questions –_ _also_ sorted by "recently commented", but filtered to questions with 40 or more karma, so that it's easier to catch up on updates to highly upvoted questions.

*   _Community Page_

*   UI updated to match Home Page.
*   The group section now shows 7 groups instead of 3, and has a load more button.
# Machine Pastoralism

This idea has occurred to me before, but in the interim I dismissed it and then forgot. Since it is back again more-or-less unprompted, I am writing it down.

We usually talk about [animals and their intelligence](https://slatestarcodex.com/2019/03/25/neurons-and-intelligence-a-birdbrained-perspective/) as a way to interrogate intelligence in general, or as a model for possible other minds. It occurred to me our _relationship_ with animals is therefore a model for our relationship with other forms of intelligence.

In the mode of [Prediction Machines](http://www.overcomingbias.com/2018/04/prediction-machines.html), it is straightforward to consider: prediction engines in lieu of dogs to track and give warning; teaching/learning systems for exploring the map in lieu of horses; analysis engines to provide our solutions instead of cattle or sheep to provide our sustenance. The idea here is just to map animals-as-capital to the information economy, according to what they do for us.

Alongside what they do for us is the question of how we manage them. The [Software 2.0](https://medium.com/@karpathy/software-2-0-a64152b37c35) lens of adjusting weights to search program space reads closer to [animal husbandry](https://en.wikipedia.org/wiki/Animal_husbandry) than building a new beast from the ground up with gears each time, to me. It allows for a notion of lineage,  and we can envision using groups of machines with subtle variations, or entirely different machines in combination.

This analogy also feels like it does a reasonable job of priming the intuition about where dangerous thresholds might lie. How smart is smart enough to be dangerous for one AI? [Tiger-ish](https://en.wikipedia.org/wiki/Tiger_attack)? We can also think about relative intelligence: the primates with better tool ability and more powerful communication were able to establish patronage and then total domestication over packs of dogs and herds of horses, cattle, and sheep. How big is that gap exactly, and what does that imply about the threshold for doing the same to humans? Historically we are perfectly capable of doing it to ourselves, so it seems like the threshold might actually be lower than us.
# What are the advantages and disadvantages of knowing your own IQ?

I've seen some answers here: [https://www.quora.com/What-are-the-advantages-disadvantages-to-know-your-own-IQ](https://www.quora.com/What-are-the-advantages-disadvantages-to-know-your-own-IQ)

But I would be curious to know the perspective from people here.

Third alternative: taking an IQ test and tracking your IQ, but not looking at it.

For example: Can tracking IQ be useful to track cognitive degradation and predict neurodegenerative diseases?
# How do people become ambitious?

I have a pet theory about how people become ambitious and agenty.

I was about to write up some insight porn about it, and then was like "you know, Raemon, you should probably actually think about about this for real, since it seems like Pet Psychology Theories are one of the easier ways to get stuck in dumb cognitive traps."

"How do people get ambitious and agenty" seems to be something people should have actually tried to study before. I'm thinking something as simple as "interviewing lots of people and checking for common patterns."

2 seconds spent on google scholar suggested I could use better keywords.

Curious if anyone has looked into this (either reviewing existing literature, or conducting interviews themselves or otherwise trying to tackle the question in a serious way)

For clarity, the two phenomena I want to understand better are:

*   _Ambition._ How do people end up having plans (which they realistically expect to achieve) that affect at least thousands, and preferably millions of people?
*   _Agency._ How do people generally gain the capacity to be self-motivated, think through plans, and decide how to pursue goals that will change the world (changing it in small ways is fine)
# Aumann Agreement by Combat

The first paper in the SIGBOVIK 2019 proceedings, “Aumann Agreement by Combat” by Travis Hance, seems relevant to this site. The paper is on page 4 of the PDF (page 8 if you include front matter).

The abstract:

> The celebrated Aumann’s Agreement Theorem shows that two rational agents with the same priors on an event who make different observations will always converge on the same posteriors after some civilized conversation over tea. Furthermore, they will come to agree even if they do nothing other than simply state their posteriors over and over again.

> However, this protocol is widely criticized for being too boring. We therefore introduce a more exciting alternative, which we name _Aumann Agreement by Combat_.

The [SIGBOVIK](http://sigbovik.org/) conference is held every year on April 1.
# How would a better replacement for tissues look like?

When one has a cold common disposable tissues are the state of the art solution for getting mucus out of one's nose.

If there's however a lot of mucus a bit of that mucus usually ends up on the hand that holds the tissue. 

This in turn makes it likely that germs end up on the hand and on objects that one touches with the hand.

If we invent a better solution to replace disposable tissues, it would be valuable for reducing the spread of infectious disease.

Does anybody have ideas?
# Would solving logical counterfactuals solve anthropics?

One of the key problems with anthropics is establishing the appropriate reference class. When we attempt to calculate a probability accounting for anthropics, do we consider all agents or all humans or all humans who understand decision theory?

[If a tree falls on Sleeping Beauty](https://www.lesswrong.com/posts/gMXsyhPiEJbGerF6F/if-a-tree-falls-on-sleeping-beauty) argues probability is not ontologically basic and the "probability" depends on how you count bets. In this vein, one might attempt to solve anthropics by asking about whose decision to take a bet is linked to yours. You could then count up all the linked agents who observe A and all the agents who observe not A and then calculate the expected value of the bet. More generally, if you can solve bets, my intuition is that you can answer any other question that you would like about the decision by reframing it as a bet.
# Factored Cognition with Reflection

_Note: I accidentally moved this post to draft and re-published it. My intention wasn't to get it back on the home page. Sorry._

Some weeks ago I completed an HCH-like program/Q&A system that supports reflection, including time travel: https://github.com/rmoehn/jursey I'm just posting it here in case someone finds it useful.
# Towards a Quieter Life

My life is LOUD. Thoughts of fiction, conversations, unrealistic hypotheticals, games, tv shows flood my mind when there's no need to be babbling those thoughts. I have coroutines running in my mind to CHECK YOUR PHONE, YOUR EMAIL, YOUR RSS FEED, ...

Being aware of myself in those moments, I've realized I'm not actually happy doing any of them. Cravings for escapism is not what I want. I want something real, something true... I want reality.

So here's to a quieter life.

* * *

I'll check my phone/email after lunch and dinner. If I'm bored, I'll meditate. If I'm tired of meditating, I'll investigate that feeling. If that fails, I can Sabbath for a bit.

When I'm doing something, I'm only doing that. If an unrelated thought comes, I'll accept it and let it go. When I'm done with one thing, I'll go to the next with joy. If I don't have a next thing, I'll make a plan with joy. If that fails, I can Sabbath for a bit.

I'll be strict for the next few weeks, post the results, and go from there.

~Metta
# Slack Club

This is a post from The Last Rationalist, which asks, generally, "Why do rationalists have such a hard time doing things, as a community?" Their answer is that rationality selects for a particular smart-but-lazy archetype, who values solving problems with silver bullets and abstraction, rather than hard work and perseverance. This archetype is easily distractible and does not cooperate with other instances of itself, so an entire community of people conforming to this archetype devolves into valuing abstraction and specialized jargon over solving problems.
# Conditional revealed preference

In the linked post, I discuss revealed preference analysis and argue that claims about someone's actual values should predict not only what they do in their current situation, but what they would do in substantially different situations, given e.g. different information and different expectations for how others behave.

(LessWrongers may ask "what does this add over CEV?"  CEV is one possible hypothetical where people are more informed, but is impossible to compute and also not predictive of what people do in actual situations of having somewhat more information while still being under cognitive limitations.  Thus, CEV "analysis", unlike conditional/counterfactual analysis, ends up being largely spurious.)
# Robin Hanson on Simple, Evidence Backed Models

This is link to a Hanson post that is primarily about tax returns, and whether they should be public. It was an interesting foray into the considerations that bear on that topic.

But I was particularly interested in the opening paragraph, which had a useful lens:

> Our simplest model of an economy is: supply and demand. This model has many simple implications for policy. Now we know of many much more complicated economic models, which often have quite different policy implications. But often we are not sure which more complex models actually apply well to any given situation. So we have to worry that people favor more complex models mainly to justify their preferred policies. Knowing this pushes me toward recommending the policies implied by supply and demand, unless I see unusually clear evidence to support a different economic model. (FYI, the evidence that fixed costs exists seems plenty clear, so I really mean supply & demand with fixed costs.)

> In our simplest modes of information, people are better off when they have more information, and also when information is distributed more symmetrically. There is a vast world of much more complicated models, but it is often hard to tell which more complex models apply well to given situations, and many probably favor particular models to justify preferred policies. So as with supply and demand, this uncertainty pushes me to favor the simplest info models, and their policy recommendations favoring more info and more symmetric info.
# StrongerByScience: a rational strength training website

StrongerByScience is a website I have been using to inform my recently-renewed weightlifting habit. I think it will be of interest here because the writers are experts in a practice (strength training, particularly powerlifting), and their modus operandi is to do careful literature reviews of the latest strength research and use that to inform their practice.

I recommend all of the Guides on the Big Three page (now technically four), which cover the big three lifts from the biomechanics on up, as well as one on strength training. They have monthly reviews of the latest papers in the field. They occasionally do deep reviews of particular subjects (I am reading the metabolism article on and off now). They're a good source of references for other literature in the field.

Stuff you would normally expect to be unpleasant, but isn't: the mailing list is just the articles, not spam; the articles are written in a way similar to how posts are often written here, and are not particularly marketing-y; the articles and guides can mostly be downloaded as PDFs in addition to being read on the website.

The specific benefits I got came from the Strength Training Guide and from the article on high-volume training, which defined my current set structure. I have a new lifetime personal best in dumbbell press, and I am close to recovering my previous deadlift. I have not purchased any of the books or trainings, however.
# Episode 1 of "Tsuyoku Naritai!" (the 'becoming stronger' podcast/YT series).

The Kansas City Rationalists just had our first dojo meetup yesterday. It was a success as far as first meetings go; we had great attendance, including a couple non-rationalists. We are using the 'Hammertime' sequence as our content.

I am making a series of vlog posts/podcast episodes detailing my personal journey through the sequence, also for the benefit of others. I plan to expand it as I learn and experience more.

[https://www.youtube.com/watch?v=JmRr7Op6sfQ&feature=youtu.be](https://www.youtube.com/watch?v=JmRr7Op6sfQ&feature=youtu.be)

[https://pca.st/26cQ](https://pca.st/26cQ)
# Any rebuttals of Christiano and AI Impacts on takeoff speeds?

14 months ago, [Paul Christiano](https://sideways-view.com/2018/02/24/takeoff-speeds/) and [AI Impacts](https://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/) both published forceful and well-received take-downs of many arguments for fast (discontinuous) takeoff. I haven’t seen any rebuttals that are written by established researchers, longer than comments, or otherwise convincing. The longer there is no response, the less weight I put on the outside view that proponents of fast takeoff may be right.

Where are the rebuttals? Did I miss them? Is the debate decided? Did nobody have time or motivation to write something? Is the topic too hard to explain?

Why rebuttals would be useful:

-Give the community a sense of the extent of expert disagreement to form outside views.

-Prioritization in AI policy, and to a lesser extent safety, depends on the likelihood of discontinuous progress. We may have more leverage in such cases, but this could be overwhelmed if the probability is low.

-Motivate more people to work on MIRI’s research which seems more important to solve early if there is fast takeoff.
# How to make plans?

What models are out there for how humans can personally make plans, especially grand master plans that scale to a whole lifespan or beyond?

*   What is the set of good plan components? (e.g. "backchaining")
*   What useful tools are out there to help with planning? (e.g. "pen and paper")
*   How does one make flexible plans that adapt to change?
*   How granular and detailed vs simple and high-level should plans be?
*   What are the biggest disagreements between different schools of planning?
*   What are the most common/biggest planning mistakes?
# Moral Weight Doesn't Accrue Linearly

This post: [https://slatestarcodex.com/2019/03/26/cortical-neuron-number-matches-intuitive-perceptions-of-moral-value-across-animals/](https://slatestarcodex.com/2019/03/26/cortical-neuron-number-matches-intuitive-perceptions-of-moral-value-across-animals/)

begs a very important question as part of its central premise. Even given the idea that animals have moral weight, it does not follow that there exist a number of them that are of equal moral weight to a human (or another kind of animal, etc). There exist infinite sequences that converge to finite values.

It seems pretty clear to me that moral weight is not linearly additive. This is why we consider it worse when a species goes from 1,000 members to 0 members than when it goes from 1,001,000 members to 1,000,000 members, for instance.
# Interview with Putanumonit

If you’ve ever wondered what Putanumonit will sound like as a podcast, wonder no more! An interview with me was recorded at a public meetup, it started with the basic tenets of rationality and putting numbers on things, then seamlessly segued to audience questions about Israel-Palestine and my sex life.

This was recorded in a public space, so there is quite a bit of background noise. This is my first time being interviewed about this topic, so there are a lot of filler noises such as “umm” and “like” coming from me. Both issues markedly improve as the interview goes on but, unfortunately, neither disappears completely.

[The audio is here.](https://podcasts.apple.com/us/podcast/presentation-jacob-of-put-a-num-on-it/id1453336297?i=1000434473417)
# Strategic implications of AIs' ability to coordinate at low cost, for example by merging

It seems likely to me that AIs will be able to coordinate with each other much more easily (i.e., at lower cost and greater scale) than humans currently can, for example by merging into coherent unified agents by combining their utility functions. This has been discussed at least since [2009](https://www.lesswrong.com/posts/S4Jg3EAdMq57y587y/an-alternative-approach-to-ai-cooperation), but I'm not sure its implications have been widely recognized. In this post I talk about two such implications that occurred to me relatively recently.

I was recently [reminded](https://www.overcomingbias.com/2019/04/agency-failure-ai-apocalypse.html#comment-4433175599) of this quote from Robin Hanson's [Prefer Law To Values](https://www.overcomingbias.com/2009/10/prefer-law-to-values.html):

>The later era when robots are vastly more capable than people should be much like the case of choosing a nation in which to retire. In this case we don’t expect to have much in the way of skills to offer, so we mostly care that they are law-abiding enough to respect our property rights. If they use the same law to keep the peace among themselves as they use to keep the peace with us, we could have a long and prosperous future in whatever weird world they conjure. In such a vast rich universe our “retirement income” should buy a comfortable if not central place for humans to watch it all in wonder.

Robin argued that this implies we should work to make it more likely that our current institutions like laws will survive into the AI era. But (aside from the problem that we're most likely still incurring astronomical waste even if many humans survive "in retirement"), assuming that AIs will have the ability to coordinate amongst themselves by doing something like merging their utility functions, there will be no reason to use laws (much less "the same laws") to keep peace among themselves. So the first implication is that to the extent that AIs are likely to have this ability, working in the direction Robin suggested would likely be futile.

The second implication is that AI safety/alignment approaches that aim to preserve an AI's competitiveness must also preserve its ability to coordinate with other AIs, since that is likely an important part of its competitiveness. For example, making an AI corrigible in the sense of allowing a human to shut it (and its successors/subagents) down or change how it functions would seemingly make it impossible for this AI to merge with another AI that is not corrigible, or not corrigible in the same way. (I've mentioned this a number of times in previous comments, as a reason why I'm pessimistic about specific approaches, but I'm not sure if others have picked up on it, or agree with it, as a general concern, which partly motivates this post.)

Questions: Do you agree AIs are likely to have the ability to coordinate with each other at low cost? What other implications does this have, especially for our strategies for reducing x-risk?
# What's the best approach to curating a newsfeed to maximize useful contrasting POV?

In Superforecasters, Tetlock describes one superforecaster as having built a collection of automated scripts and sources to get useful information from a variety of different sources and different perspectives. This seems very useful and I'd like to emulate it.

I'm looking for a generalizable approach where, given a specific topic (ex. AI) I can curate sources (ex. Twitter accounts) that maximizes diversity of points (ex. different perspectives on AI Alignment risk). I'd like to avoid the bias where I only use sources that are well known and popular, which seems particularly likely to happen in cases where I am not familiar with the field

I have a few different tactics in mind, but I haven't yet settled on a cohesive strategy, and am interested in suggestions, in particular if you've done this before.
# What are some good examples of incorrigibility?

The idea of corrigibility is roughly that an AI should be aware that it may have faults, and therefore allow and facilitate human operators to correct these faults. I'm especially interested in scenarios where the AI system controls a particular input channel that is supposed to be used to control it, such as a shutdown button, a switch used to alter its mode of operation, or another device used to control its motivation. I'm especially interested in examples that would be compelling within the machine learning discipline, so I'd favour experimental accounts over hypotheticals.
# Change A View: An interesting online community

I've recently stumbled across the platform Change A View, which is a subreddit that forked/graduated to its own website that is centered around the idea of people creating posts in which they ask to have their mind changed. This tends to create a frame that enables much more healthy discourse than most online discussion.

Their new platform is centered around the ideas of "deltas" which users can award to other users when their minds have been changed by them.

I've surfed around the site a bit and generally found it pretty enjoyable and figured other people might find it valuable to. It's also an interesting for studying the psychology of what changes people's minds and what facilitates good discourse.
# Natural Structures and Conditional Definitions

There's a sense in which definitions are arbitrary. Words are made by humans and no-one can stop me from calling red blue and blue red if I really want to. So when people ask questions like, "What is consciousness?" or "What is free-will?", it seems quite reasonable to respond, "Just pick a definition. These terms can be defined many different ways and it's completely your choice which one you choose to use".

This may appear to dissolve the question, however, I would suggest that such an answer often misunderstands what the asker is attempting. Typically the asker is concerned by more than the linguistic question, but also with attempting to understand the ontology or structure of reality. And it may be the case that this structure includes a substructure that naturally fits with our intuitions of what consciousness is or what freewill is or it may be the case, as per the standard LW view of these two cases, that such a structure doesn't exist.

What makes this especially confusing is that many people will _conditionally accept_ the "it's arbitrary" answer when they are convinced that such a natural structure doesn't exist, while pointing out the natural structure otherwise. Here's an example. Let's suppose it was common knowledge that we all have souls. Then whenever someone asked about the definition of consciousness, we'd be tempted to point to the soul, just as whenever people ask about the definition of trees, we'd be tempted to talk about leaves and branches. The arguments for being able to use language arbitrarily and the fact that this isn't a perfectly well-specified definition remain. It's just that one definition suffices for 95% of cases, so we don't bring up that argument. But if instead it was common knowledge that there are no souls, it'd be much more likely they'd say that the definition is arbitrary. And by accepting answers to different questions depending on how things turn out, the intent behind the original question can easily be obscured.

**Appendix**

Here are some possible interpretations of, "What is X?":

*   What does term X intrinsically mean? (no intrinsic meaning exists)
*   What natural structure (if any) corresponds to X?
*   What are some useful interpretations of the term X?
*   How is the term X used in society?

These kinds of discussions tend to work better if everyone is on the same page about what is being asked.

**Note:** Apparently, I actually had an [old post](https://www.lesswrong.com/posts/p6wSGPzgocPqkHuYZ/map-and-territory-natural-structures) on this topic here which I'd completely forgotten about.

I've also discovered that the philosophical term is [natural kinds](https://plato.stanford.edu/entries/natural-kinds/).
# Has government or industry had greater past success in maintaining really powerful technological secrets?

Context: As part of our efforts of working on Open Questions the LessWrong team has been reaching out to various researchers we know and asked them about questions they would be interested in getting answered.

This question was given to us by [Ryan](https://www.lesswrong.com/users/ryancarey) [Carey](https://www.lesswrong.com/users/ryancarey) and some of the answers below are the result of us trying to answer the question for a day on a private LessWrong instance, copied over to allow other people to contribute and read what we wrote.
# Open Thread May 2019

If it’s worth saying, but not worth its own post, you can put it here.

Also, if you are new to LessWrong and want to introduce yourself, this is the place to do it. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome. If you want to explore the community more, I recommend [reading the Library](https://www.lesswrong.com/library), [checking recent Curated posts](https://www.lesswrong.com/?view=curated), and [seeing if there are any meetups in your area](https://www.lesswrong.com/community).

The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Swarm AI (tool)

[https://unanimous.ai/swarm/](https://unanimous.ai/swarm/)

I remember playing with this a while back, answering random questions. I guess now they've released it as a business tool for companies to run their own voting rooms.

Quick overview:
1) You create a room.
2) You invite 4-200 participants.
3) You ask questions.
4) Participants vote.
5) You get prediction results using their AI.

Their [video](https://vimeo.com/306667996) and website claims they got a lot of hard predictions right. Of course, they aren't saying how many other things they guessed. So it's hard to say how magical it is, but it seems worth trying out. I'm up for joining people's room if they want to run some experiments.
# [Meta] Hiding negative karma notifications by default

A few weeks ago we launched karma notifications, and I've been overall pretty happy with the feature. However, we and many other users noticed that while the current system treats upvotes and downvotes as symmetric, humans tend to definitely not perceive those two as symmetric and so the lived experience of having a single day with a slightly negative karma total tends to outweigh many days of strongly positive karma scores.

I also noticed that this experience has backpropagated into me being much more hesitant to downvote users or even to remove my upvotes since I know they will perceive this as a downvote and if they are not a frequent user might feel quite punished as a response.

As a consequence, we decided to hide negative karma notifications for users by default, though you can change that in your settings. My current model of users suggests it is very unlikely for users to not notice if they are getting significantly downvoted, so I am not very worried about that information getting lost, and I expect it will overall make people's site-experience better (and their beliefs about the value and reception of their content more, not less, accurate)

For some more context, see [this](https://www.lesswrong.com/posts/EQJfdqSaMcJyR5k73/habryka-s-shortform-feed#ZG5muBXfx3Tb7jjph) discussion on my short-form feed.
# A Map of the Lesswrong-O-Sphere of Interest

It's clear to me that we on lesswrong operate on a pile of different interest areas. In my head I am doing a sorting method that divides topics. There's not just topics I (personally) like and topics I don't like but also many sub-topics within those. Here's a list of the domain areas as I see them:

**Lesswrong**:

*   Artificial intelligence

*   Programming (not just AI)
*   Machine learning
*   Ethics
*   Math

*   Decision theory

*   Philosophy

*   Effective Altruism (mostly on the EA forum)
*   Ethics
*   Scientific method and crisis
*   Decision problems

*   Human Psychology

*   Therapy and related personal work
*   Biases and related thinking
*   Group thinking and how to work with people
*   Personal development (other than therapy)
*   How humans think (consciousness - overlapping with Post-rationality area)

*   Guides for doing things, like problem solving
*   Thinking exercises to try
*   reports/retrospective reflection on some event that has happened

*   Post-rationality

*   Buddhism
*   Mysticism
*   Woo
*   map/territory blurring

*   Human health and medical

*   Wellness
*   Longevity
*   Diet
*   Cryonics (uncommon)

*   Society

*   Economics

*   forecasting/prediction

*   History
*   Culture

*   The state of culture in other places virtually or physically

*   Book reviews

*   Meta: how lesswrong works or could work.
*   Original sequences and relevant theory or ongoing development (as opposed to new sequences)
*   _Uncommon areas_

*   Physics
*   Politics (discouraged)
*   Big/small (new) ideas and theories

\-\-\-\-\-\-\-\-\-\-\-\-\-\-

We’re not doing one thing. We’re not even doing three things, we are interested in a whole lot more topics. I could reformat the list to include more questions, along the lines of, “what are we trying to answer when we ask a history question?” but that’s not it. My point here is that we are investigating a diverse bunch of topics, and there’s a structure to our pattern. Even if it might look like a mess.

For me, I know AI is not my domain. EA, decision theory are not my domain. My domain is Humans and how they work, how they think, and (personally) how I relate to the world around me.

Personally, in regard to archipelago of forums or dividing up topics, I know I don’t comment on AI information, but I do comment on human, PR and a few other specific topics.

This post is to follow [this](https://www.greaterwrong.com/posts/kpFXfYbvQY5GhyqKL/open-thread-may-2019/comment/taJxkoyxyABBxLvmo) comment because I wanted to expand on my understanding of where lesswrong has moved since the original sequence materials.

Maybe you have a better classification of the many diverse topics we cover, but as a start, here’s what’s floating in my head. Prove me wrong by writing your own list of what is covered by lesswrong and what domains you engage in.

  

What did I miss?
# Gwern's "Why Tool AIs Want to Be Agent AIs: The Power of Agency" 

I somehow hadn't read this post until now, so I am posting this here in case I am not the only one (and I wasn't able to find a previous linkpost for it). Relevant to relatively recent discussion on AI-as-a-service, but also just good as a broad reference.
# How much do major foundations grant per hour of staff time?

_\[Edit: original source has been found. The way I framed this question was off, since I'd mis-remembered it as "amount that a given grantmaker grants per year" rather than "grants per hour of staff time". Updated the title\]_

I recall reading an article once that claimed that, when examining many small and large foundations, it turned out that there was a maximum amount that a given grantmaker typically gave out. And as organizations scaled to give out more money, this amount stayed surprisingly fixed, with a higher overhead ratio than you might have expected.

(i.e. when an org gives out a million a year, it has N grantmakers, and when it gives out 100 million a year, it typically has 100N grantmakers).

I don't remember the number, or the methodology that determined it. Curious if anyone can remember the article. (It might have been from OpenPhil's blog, or it might have been some random news site).

I vaguely remember the number "3" being involved, possibly $300k, or $3 million.

The takeaway I remember was something like "you might naively think you can scale up an organization and then give away money more efficiently, but weird forces seem to limit that."

Does this sound familiar to anyone?
# How long can people be productive in [time period]?

I often see statistics that "people are only productive four hours/day", but they never seem to cite their sources. The studies I've tracked down are low quality ([example](https://www.vouchercloud.com/resources/office-worker-productivity)). 30 minutes poking around google scholar produced nothing useful.

Where are the high quality studies showing how long people can work productivity?
# Space colonization: what can we definitely do and how do we know that?

Arguments for the value of the long-term future tend to make the assumption that we will colonize space. What can we definitely accomplish in terms of space colonization? Why think that we can definitely do those things?

The FHI paper, [Eternity in Six Hours](https://www.fhi.ox.ac.uk/publications/armstrong-s-sandberg-a-2013-eternity-in-six-hours-intergalactic-spreading-of-intelligent-life-and-sharpening-the-fermi-paradox-acta-astronautica-89-1-13/), is very optimistic about what can be done:

> In this paper, we extend the Fermi paradox to not only life in this galaxy, but to other galaxies as well. We do this by demonstrating that traveling between galaxies – indeed even launching a colonisation project for the entire reachable universe – is a relatively simple task for a star-spanning civilization, requiring modest amounts of energy and resources. We start by demonstrating that humanity itself could likely accomplish such a colonisation project in the foreseeable future, should we want to, and then demonstrate that there are millions of galaxies that could have reached us by now, using similar methods.

Is this paper reasonable? Which parts of its assertions are most likely to be mistaken?

_This question was inspired by a conversation with [Nick Beckstead](https://www.lesswrong.com/users/nick_beckstead)._
# Which parts of the paper Eternity in Six Hours are iffy?

The FHI paper, [Eternity in Six Hours](https://www.fhi.ox.ac.uk/publications/armstrong-s-sandberg-a-2013-eternity-in-six-hours-intergalactic-spreading-of-intelligent-life-and-sharpening-the-fermi-paradox-acta-astronautica-89-1-13/), is very optimistic about what can be done:

> In this paper, we extend the Fermi paradox to not only life in this galaxy, but to other galaxies as well. We do this by demonstrating that traveling between galaxies – indeed even launching a colonisation project for the entire reachable universe – is a relatively simple task for a star-spanning civilization, requiring modest amounts of energy and resources. We start by demonstrating that humanity itself could likely accomplish such a colonisation project in the foreseeable future, should we want to, and then demonstrate that there are millions of galaxies that could have reached us by now, using similar methods.

Is this paper reasonable? Which parts of its assertions are most likely to be mistaken?

_This question was inspired by a conversation with Nick Beckstead._
# Raleigh SSC/LW/EA Meetup - Meet MealSquares People

Romeo Stevens and I (both cofounders of MealSquares, a SSC sidebar advertiser) are visiting Raleigh/Durham and we're interested in meeting local people.  Romeo is interested in contemplative practice and philosophy of science.  John once won $2000 in the AI Alignment Prize and was the first long-term guest of the EA (Athena) Hotel.  Both of us post heavily here on LW and on the EA Forum and are well connected in the Bay Area EA and rationalist communities.  We'll be at the coffee shop called "42 & Lawrence", located at 134 E Martin St in Raleigh, this Sunday, May 12 at 2 PM.  John will be wearing an Effective Altruism t-shirt.  Romeo will be wearing a Qualia Research Institute t-shirt.  We're not sure who else will show up, but we'll hang out there for at least an hour.  If the weather is nice we might go to the park across the street.
# Kissing Scars

Last year August 6th, I got in a horrible drone accident that destroyed the lens and iris in my left eye and left a scar running down my eye. For a couple months I could barely read, my working memory was horrible, energy levels were shot, and I was in so much pain, especially after surgeries.

Then a sweet girl kissed my scars.

In one simple gesture, it was communicated

> "What you're going through really sucks and I want to make it better"

* * *

Today I realized two people upvoted my birthday comment in an obituary post to a lost friend. That post was almost a year ago, and the comment was a month ago.

Whoever you two are, thank you for kissing my scars.
# What Botswana Can Teach Us About Political Stability

![](https://cdn-images-1.medium.com/max/1024/1*gbzrVoG53JfsB9_ngLbkdw.jpeg)

US Army Africa/Botswana

*From my* [*article*](https://palladiummag.com/2019/05/09/what-botswana-can-teach-us-about-political-stability/) *first published on Palladium Magazine.*

It is hard to find a clearer outlier among developing countries than [Botswana](https://en.wikipedia.org/wiki/Botswana), a landlocked African country where 40% of government revenue comes from [diamond mining](https://en.wikipedia.org/wiki/Resource_curse) and a quarter of adults are HIV positive. Everything taught by a [development economics](https://en.wikipedia.org/wiki/Development_economics) department would suggest the country is set up for failure. But well-executed succession between presidents, and the resulting stability and good government, has meant success instead.

Botswana is possibly the nicest place in Africa — it is quieter and more stable than, say, [Greece](https://en.wikipedia.org/wiki/Greek_military_junta_of_1967%E2%80%931974#Coup_d'%C3%A9tat_of_21_April). In the entire period since independence, Botswana has not suffered devastating civil wars like those in the Congo or Mozambique, coups such as in Burkina Faso, or ethnic violence and expropriation as seen in Rwanda and Zimbabwe.

The country’s living standards [are comparable](https://en.wikipedia.org/wiki/List_of_countries_by_Human_Development_Index#Medium_human_development) to Turkey, Mexico, and South Africa. It has also been Sub-Saharan Africa’s fastest growing economy for most of the last half century.

The crucial variable is a sound government making well-informed, long-term choices. A low population density paired with abundant natural resources provides a reasonable standard of living even in the absence of administrative genius or favorable conditions, so long as governance provides stability. Political instability can impede development of physical infrastructure and the business environment, transforming good fundamentals into a bad outcome. See, for example, [Kazakhstan](https://palladiummag.com/2019/01/19/authoritarian-development-has-rebuilt-kazakhstan-into-a-eurasian-power/) compared to [Venezuela](https://palladiummag.com/2019/03/06/venezuela-an-up-close-look-at-a-nation-in-free-fall/).

Unfortunately, there are many examples of countries that have tried and failed to achieve good governance in the often chaotic post-colonial context. These countries followed Western advice as closely as they could, drafting legally impeccable constitutions and recruiting [well-educated](https://answersafrica.com/latest-10-most-educated-african-presidents-no-1-tops-the-world-list.html) statesmen, but the results have been mixed at best. Botswana’s positive outlier example raises the question of how it has done so well.

*Read the rest *[*here*](https://palladiummag.com/2019/05/09/what-botswana-can-teach-us-about-political-stability/)*.*

*Read more from Samo Burja* [*here.*](http://samoburja.com/essays)

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=532a695ae388)
# AI Forecasting online workshop

_I'm looking for volunteers to join me for_ _[a 1.5h online AI forecasting workshop](https://forms.gle/Q3uBVtezdgbJPc8z8)_.

**Why am I doing this?**

I’m working under [a BERI grant](http://existence.org/grants-database/) to use forecasting to support work on x-risk, together with Ben Goldhaber. We’re currently experimenting with a novel format for effective group forecasting.

**What does the workshop involve?**

We’ll gather outside views, make guesstimates, factorize the question into sub-questions, turn our uncertainty into probabilities, and so forth!

Much of the workshop will happen via silent collaboration in a Google doc, using the structured format that we’re developing specifically for this.

**Do I need prior experience with forecasting?**

No! We want to learn how well this format works for people with a wide range of backgrounds.

**What have past attendees said?**

We’ve had about ~20 people try this format over the last weeks, with a wide range of experience from superforecasters and AI safety researchers to amateur forecasters and undergraduates.

People have generally found it effective and fun, though we have several things we want to improve and are looking for more feedback and volunteers to keep iterating.

**When is it?**

Friday May 17, 11am Pacific time/7pm UK time. If you're interested in joining but can't make this time, let us know your other availabilities [here](https://forms.gle/tCjqJseD2EcXuRQCA).

**How do I join?**

Sign-up via [this form](https://forms.gle/uEouxPtUNE6gbPxS9), and we'll send you more details.