Harley-ml/lesswrong huggingface dataset:
# I Updated the List of Rationalist Blogs on the Wiki

I recently updated the list of rationalist community blogs. The new page is here: https://wiki.lesswrong.com/wiki/List\_of\_Blogs

Improvements:

-Tons of (active) blogs have been added

-All dead links have been removed

-Blogs which are currently inactive but somewhat likely to be revived have been moved to an inactive section. I included the date of their last post. 

-Blogs which are officially closed or have not been updated in many years are now all in the "Gone but not forgetten" section

Downsides:

-Categorizing the blogs I added was hard, its unclear how well I did. By some standard most rationalist blogs should be in "general rationality" 

-The blog descriptions could be improved (both for the blog-listings I added and the pre-existing listings)

-I don't know the names of the authors of Several blogs I added. 

I am posting this here because I think the article is of general interest to rationalists. In addition the page could use some more polish and attention. I also think it might be interesting to think about improving the lesswrong wiki. Several pages could use an update. However this update took a considerable amount of time. So I understand why many wiki pages are not up to date. How can we make it easier and more rewarding to work on the wiki?
# Nate Soares' Replacing Guilt Series compiled in epub Format

Hey everyone,

I really liked Nate Soares' [Replacing Guilt](http://mindingourway.com/guilt/) series, which has had a major positive impact on growing my intrinsic motivation.

Recently, I compiled all the posts into an ePUB for my own reading, and I thought it might be good to share it here if anyone would like to download it for their e-readers / on-the-go-reading. (I got Nate's permission first, so it's all good.)

[Google Drive link here.](https://drive.google.com/file/d/0B9G50JAvLnSlMnRkX1hMQl91Zkk/view?usp=sharing)
# Thoughts on civilization collapse

_Epistemic status: an idea I believe moderately strongly, based on extensive reading but not rigorous analysis._

We may have a dramatically wrong idea of civilization collapse, mainly inspired by movies that obsess over dramatic tales of individual heroism.

### Traditional view:

In a collapse, anarchy will break out, and it will be a war of all against all or small groups against small groups. Individual weaponry (including heavy weapons) and basic food production will become paramount; traditional political skills, not so much. Government collapse is long term. Towns and cities will suffer more than the countryside. The best course of action is to have a cache of weapons and food, and to run for the hills.

### Alternative view:

In a collapse, people will cling to their identified tribe for protection. Large groups will have no difficulty suppressing or taking over individuals and small groups within their areas of influence. Individual weaponry may be important (given less of a police force), but heavy weaponry will be almost irrelevant as no small group will survive alone. Food production will be controlled by the large groups. Though the formal "government" may fall, and countries may splinter into more local groups, government will continue under the control of warlords, tribal elders, or local variants. Cities, with their large and varied-skill workforce, will suffer less than the countryside. The best course of action is to have a stash of minor luxury goods (solar-powered calculators, comic books, pornography, batteries, antiseptics) and to make contacts with those likely to become powerful after a collapse (army officers, police chiefs, religious leaders, influential families).

Possible sources to back up this alternative view:

*   The book [Sapiens](https://www.amazon.com/Sapiens-Humankind-Yuval-Noah-Harari/dp/0062316095) argues that governments and markets are the ultimate enablers of individualism, with extended-family-based tribalism as the "natural" state of humanity.
*   The [history of Somalia](https://en.wikipedia.org/wiki/History_of_Somalia#Decentralization) demonstrates that laws and enforcement continue even after a government collapse, by going back to more traditional structures.
*   During China's [period of anarchy](https://en.wikipedia.org/wiki/History_of_the_Republic_of_China), large groups remained powerful: the nationalists, the communists, the Japanese invaders. The other sections of the country were generally under the control of local warlords.
*   [Rational Wiki argues](http://rationalwiki.org/wiki/Survivalist#Survivalism_in_practice) that examples of collapse go against the individualism narrative.
# That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi's paradox

**[That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi's paradox](https://arxiv.org/abs/1705.03394)**

Anders Sandberg, Stuart Armstrong, Milan M. Cirkovic

If a civilization wants to maximize computation it appears rational to aestivate until the far future in order to exploit the low temperature environment: this can produce a 10^30^ multiplier of achievable computation. We hence suggest the "aestivation hypothesis": the reason we are not observing manifestations of alien civilizations is that they are currently (mostly) inactive, patiently waiting for future cosmic eras. This paper analyzes the assumptions going into the hypothesis and how physical law and observational evidence constrain the motivations of aliens compatible with the hypothesis.

As far as I can tell, the paper's physics is correct (most of the energy comes not from burning stars but from the universe's mass).

However, the conclusions are likely wrong, because it's rational for "sleeping" civilizations to still want to round up stars that might be ejected from galaxies, collect cosmic dust, and so on.

The paper is still worth publishing, though, because there may other, more plausible ideas in the vicinity of this one. And it describes how future civilization may choose to use their energy.
# A new, better way to read the Sequences

A new way to read the Sequences:

[**https://www.readthesequences.com**](https://www.readthesequences.com)

It's also more mobile-friendly than a PDF/mobi/epub.

(The content is from the book — _Rationality: From AI to Zombies_. Books I through IV are up already; Books V and VI aren't up yet, but soon will be.)

_Edit:_ [Book V](https://www.readthesequences.com/Book-V-MereGoodness) is now up.

_Edit 2:_ [Book VI](https://www.readthesequences.com/Book-VI-BecomingStronger) is now up.

_Edit 3:_ A zipped archive of the site (for offline viewing) is now [available for download](https://www.readthesequences.com/About).
# Bet or update: fixing the will-to-wager assumption

(Warning: completely obvious reasoning that I'm only posting because I haven't seen it spelled out anywhere.)

Some people say, expanding on an idea of de Finetti, that Bayesian rational agents should offer two-sided bets based on their beliefs. For example, if you think a coin is fair, you should be willing to offer anyone a 50/50 bet on heads (or tails) for a penny. Jack called it the "will-to-wager assumption" [here](/lw/3cp/dutch_books_and_decision_theory_an_introduction/) and I don't know a better name.

In its simplest form the assumption is false, even for perfectly rational agents in a perfectly simple world. For example, I can give you my favorite fair coin so you can flip it and take a peek at the result. Then, even though I still believe the coin is fair, I'd be a fool to offer both sides of the wager to you, because you'd just take whichever side benefits you (since you've seen the result and I haven't). That objection is not just academic: using your sincere beliefs to bet money against better informed people is a bad idea in real world markets as well.

Then the question arises, how can we fix the assumption so it still says something sensible about rationality? I think the right fix should go something like this. If you flip a coin and peek at the result, then offer me a bet at 90:10 odds that the coin came up heads, I must either accept the bet or update toward believing that the coin indeed came up heads, with at least these odds. I don't get to keep my 50:50 beliefs about the coin and refuse the bet at the same time. More generally, a Bayesian rational agent offered a bet (by another agent who might have more information) must _either accept the bet or update their beliefs so the bet becomes unprofitable_. The old obligation about offering two-sided bets on all your beliefs is obsolete, use this one from now on. It should also come in handy in living room Bayesian scuffles, throwing some money on the table and saying "bet or update!" has a nice ring to it.

What do you think?
# Thought experiment: coarse-grained VR utopia

I think I've come up with a fun thought experiment about friendly AI. It's pretty obvious in retrospect, but I haven't seen it posted before. 

When thinking about what friendly AI should do, one big source of difficulty is that the inputs are supposed to be human intuitions, based on our coarse-grained and confused world models. While the AI's actions are supposed to be fine-grained actions based on the true nature of the universe, which can turn out very weird. That leads to a messy problem of translating preferences from one domain to another, which crops up everywhere in FAI thinking, [Wei's comment](/lw/6ha/the_blueminimizing_robot/4gsa) and [Eliezer's writeup](https://arbital.com/p/rescue_utility/) are good places to start.

What I just realized is that you can handwave the problem away, by imagining a universe whose true nature agrees with human intuitions by fiat. Think of it as a coarse-grained virtual reality where everything is built from polygons and textures instead of atoms, and all interactions between objects are explicitly coded. It would contain player avatars, controlled by ordinary human brains sitting outside the simulation (so the simulation doesn't even need to support thought).

The FAI-relevant question is: How hard is it to describe a coarse-grained VR utopia that you would agree to live in?

If describing such a utopia is feasible at all, it involves thinking about only human-scale experiences, not physics or tech. So in theory we could hand it off to human philosophers or some other human-based procedure, thus dealing with "complexity of value" without much risk. Then we could launch a powerful AI aimed at rebuilding reality to match it (more concretely, making the world's conscious experiences match a specific coarse-grained VR utopia, without any extra hidden suffering). That's still a very hard task, because it requires solving decision theory and the problem of consciousness, but it seems more manageable than solving friendliness completely. The resulting world would be suboptimal in many ways, e.g. it wouldn't have much room for science or self-modification, but it might be enough to avert AI disaster (!)

I'm not proposing this as a plan for FAI, because we can probably come up with something better. But what do you think of it as a thought experiment? Is it a useful way to split up the problem, separating the complexity of human values from the complexity of non-human nature?
# LessWrong 2.0 Feature Roadmap & Feature Suggestions

This post will serve as a place to discuss what features the new LessWrong 2.0 should have, and I will try to keep this post updated with our feature roadmap plans.

Here is roughly the set of features we are planning to develop over the next few weeks:

UPDATED: August 27th, 2017

Basic quality of life improvements:

1.  Improve rendering speed on posts with many comments
2.  _(A lot of improvements made, a lot more to come)_
3.  Improve usability on mobile
4.  _(After the major rework this is somewhat broken again, will fix it soon)_
5.  Add Katex support for comments and posts
6.  Allow merging with old LessWrong 1.0 accounts
7.  Fix old LessWrong 1.0 links DONE!
8.  Create unique links for each comment: **DONE!**
9.  Make comments collapsible
10.  Highlight new comments since last visit: **DONE!**
11.  Improve automatic spam-detection
12.  Add RSS feed links with adjustable karma thresholds
13.  Create better documentation for the page, with tooltips and onboarding processes
14.  Better search, including comment search and user search: **DONE!**

Improved Moderation Tools:

1.  New Karma system that weighs your votes based on your Karma
2.  Give moderators ability to suspend comment threads for a limited amount of time
3.  Give trusted post-authors moderation ability on their own posts (deleting comments, temporarily suspending users from posts, etc.)
4.  Add reporting feature to comments
5.  Give moderators and admins access to a database query interface to identify negative vote patterns

New Content Types:

1.  Add sequences as a top-level content-type with UI for navigating sequences in order, metadata on a sequence, and keeping track of which parts you've read **DONE!**
2.  Add Arbital-style predictions as a content block in posts (maybe also as a top-level content type)
3.  Add 'Wait-But-Why?' style footnotes to the editor
4.  Discussion page that structures discussions more than just a tree format ([here](https://www.figma.com/file/Jh9FqelcAtD6Pedg7J87Fsiv/Arbital-New-Claims) is a mockup I designed while working for Arbital, that I am style excited to implement)
5.  ...and we have many more crazy ideas we would like to experiment with

I will also create a comment for each of these under the post, so you can help us prioritize all of these. Also feel free to leave your own feature suggestions and site improvements in the comments.
# Distinctions of the Moment

I could use a better name for this (any suggestions?), but I'll use "distinctions of the moment" for the moment.

A distinction of the moment is a distinction made between two synonyms or near-synonyms (or sometimes antonyms), for the sake of pointing out a difference which is important to the discussion at hand. For example, I might contrast "knowledge" and "understanding" for the sake of a discussion about memorization-and-regurgitation style education vs focusing more on reasons. In a different conversation, I may well use "knowledge" vs "understanding" to talk about knowing-how vs knowing-what (and I might well call either of those two "understanding" and the other "knowledge" -- it doesn't matter, it's just a distinction of the moment).

The important thing about distinctions of the moment is that they just have to be made clear for the purpose of the conversation, and can be thrown away afterwards. This fails if someone thinks that they need to argue the definition of a word as if it were a sacred duty. Perhaps I make a distinction between "liberal" and "progressive" for the purpose of conversation, and someone takes issue with it because they've got a favorite definition of one or both of those terms which they think is importantly correct. This is not useful to the conversation.

A pet peeve of mine is that a lot of people seem to carry around distinctions-of-the-moment forever, as if they were true subtle differences in the meanings of words. I feel this way about distinguishing between "sympathy" and "empathy" -- I'm not sure quite why, but it seems to me that the distinction people tend to draw between the two is something which is just not generally useful enough to be more than a distinction-of-the-moment. The two words have different usage patterns, yes, but I don't think those usage patterns are really well-explained by "feeling emotion in response to others vs putting yourself in their shoes".
# Self-modification as a game theory problem

In this post I'll try to show a surprising link between two research topics on LW: game-theoretic cooperation between AIs (quining, Loebian cooperation, modal combat, etc) and stable self-modification of AIs (tiling agents, Loebian obstacle, etc).

When you're trying to cooperate with another AI, you need to ensure that its action will fulfill your utility function. And when doing self-modification, you also need to ensure that the successor AI will fulfill your utility function. In both cases, naive utility maximization doesn't work, because you can't fully understand another agent that's as powerful and complex as you. That's a familiar difficulty in game theory, and in self-modification it's known as the Loebian obstacle (fully understandable successors become weaker and weaker).

In general, any AI will be faced with two kinds of situations. In "single player" situations, you're faced with a choice like eating chocolate or not, where you can figure out the outcome of each action. (Most situations covered by UDT are also "single player", involving identical copies of yourself.) Whereas in "multiplayer" situations your action gets combined with the actions of other agents to determine the outcome. Both cooperation and self-modification are "multiplayer" situations, and are hard for the same reason. When someone proposes a self-modification to you, you might as well evaluate it with the same code that you use for game theory contests.

If I'm right, then any good theory for cooperation between AIs will also double as a theory of stable self-modification for a single AI. That means neither problem can be much easier than the other, and in particular self-modification won't be a special case of utility maximization, as some people seem to hope. But on the plus side, we need to solve one problem instead of two, so creating FAI becomes a little bit easier.

The idea came to me while working on [this mathy post on IAFF](https://agentfoundations.org/item?id=1532), which translates some game theory ideas into the self-modification world. For example, Loebian cooperation (from the game theory world) might lead to a solution for the Loebian obstacle (from the self-modification world) - two LW ideas with the same name that people didn't think to combine before!
# Rescuing the Extropy Magazine archives

Possibly of more interest to old school [Extropians](https://hpluspedia.org/wiki/Extropianism), you may be aware the defunct [Extropy Institute's website](http://www.extropy.org/) is very slow and broken, and certainly inaccessible to newcomers.

Anyhow, I have recently pieced together most of the early publications, 1988 - 1996 of '**Extropy: Vaccine For Future Shock'** later**,** **Extropy: Journal of Transhumanist Thought**, as a part of mapping the history of Extropianism.

You'll find some really interesting very early articles on neural augmentation, transhumanism, libertarianism, AI (featuring Eliezer), radical economics (featuring Robin Hanson of course) and even decentralised payment systems.

Along with the ExI mailing list which is not yet wikified, it provides a great insight into early radical technological thinking, an era mostly known for the early hacker movement.

Let me know your thoughts/feedback!

[https://hpluspedia.org/wiki/Extropy_Magazines](https://hpluspedia.org/wiki/Extropy_Magazines)
# Steelmanning the Chinese Room Argument

_(This post grew out of an [old conversation](/lw/ig8) with Wei Dai.)_

Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.

Can you, by observing the input-output behavior of the system, distinguish it from a person who _doesn't_ know the secret, or knows some other secret instead?

Clearly the only reasonable answer is "no, not in general".

Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?

Again, clearly, the only reasonable answer is "not in general".

Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?

A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
# Against lone wolf self-improvement

LW has a problem. Openly or covertly, many posts here promote the idea that a rational person ought to be able to self-improve on their own. Some of it comes from Eliezer's refusal to attend college (and Luke dropping out of his bachelors, etc). Some of it comes from our concept of rationality, that all agents can be approximated as perfect utility maximizers with a bunch of nonessential bugs. Some of it is due to our psychological makeup and introversion. Some of it comes from trying to tackle hard problems that aren't well understood anywhere else. And some of it is just the plain old meme of heroism and forging your own way.

I'm not saying all these things are 100% harmful. But the end result is a mindset of _lone wolf self-improvement_, which I believe has harmed LWers more than any other part of our belief system.

  

Any time you force yourself to do X alone in your room, or blame yourself for not doing X, or feel isolated while doing X, or surf the web to feel some human contact instead of doing X, or wonder if X might improve your life but can't bring yourself to start... your problem comes from believing that lone wolf self-improvement is fundamentally the right approach. That belief is comforting in many ways, but noticing it is enough to break the spell. The fault wasn't with the operator all along. _Lone wolf self-improvement doesn't work_.

  

Doesn't work compared to what? _Joining a class_. With a fixed schedule, a group of students, a teacher, and an exam at the end. Compared to any "anti-akrasia technique" ever proposed on LW or adjacent self-help blogs, joining a class works ridiculously well. You don't need constant willpower: just show up on time and you'll be carried along. You don't get lonely: other students are there and you can't help but interact. You don't wonder if you're doing it right: just ask the teacher.

  

Can't find a class? Find a club, a meetup, a group of people sharing your interest, any environment where social momentum will work in your favor. Even an online community for X that will reward your progress with upvotes is much better than going X completely alone. But any regular meeting you can attend in person, which doesn't depend on your enthusiasm to keep going, is exponentially more powerful.

  

Avoiding lone wolf self-improvement seems like embarrassingly obvious advice. But somehow I see people trying to learn X alone in their rooms all the time, swimming against the current for years, blaming themselves when their willpower isn't enough. My message to such people: give up. Your brain is right and what you're forcing it to do is wrong. Put down your X, open your laptop, find a class near you, send them a quick email, and spend the rest of the day surfing the web. It will be your most productive day in months.
# Online discussion is better than pre-publication peer review

Related: [Why Academic Papers Are A Terrible Discussion Forum](https://rationalconspiracy.com/2012/06/20/why-academic-papers-are-a-terrible-discussion-forum/), [Four Layers of Intellectual Conversation](https://www.facebook.com/yudkowsky/posts/10154888183439228)

During a [recent](/lw/pdd/the_doomsday_argument_in_anthropic_decision_theory/dwsa) [discussion](https://www.lesserwrong.com/posts/xQ9tMMk3RArodLtDq/intellectual-progress-inside-and-outside-academia) about (in part) academic peer review, some people defended peer review as necessary in academia, despite its flaws, for time management. Without it, they said, researchers would be overwhelmed by "cranks and incompetents and time-card-punchers" and "semi-serious people post ideas that have already been addressed or refuted in papers already". I replied that on online discussion forums, "it doesn't take a lot of effort to detect cranks and previously addressed ideas". I was prompted by Michael Arc and Stuart Armstrong to elaborate. Here's what I wrote in response:

My experience is with systems like LW. If an article is in my own specialty then I can judge it easily and make comments if it’s interesting, otherwise I look at its votes and other people’s comments to figure out whether it’s something I should pay more attention to. One advantage over peer review is that each specialist can see all the unfiltered work in their own field, and it only takes one person from all the specialists in a field to recognize that a work may be promising, then comment on it and draw others’ attentions. Another advantage is that nobody can make ill-considered comments without suffering personal consequences since everything is public. This seem like an obvious improvement over standard pre-publication peer review, for the purpose of filtering out bad work and focusing attention on promising work, and in practice works reasonably well on LW.

Apparently some people in academia have come to similar conclusions about how peer review is currently done and are trying to reform it in [various ways](http://pubs.acs.org/doi/full/10.1021/acs.chemmater.5b01917), including switching to [post-publication peer review](http://blog.scienceopen.com/2016/02/pre-or-post-publication-peer-review/) (which seems very similar to what we do on forums like LW). However it's troubling (in a "civilizational inadequacy" sense) that academia is moving so slowly in that direction, despite the necessary enabling technology having been invented a decade or more ago.
# 2017 LessWrong Survey

The [2017 LessWrong Survey](https://www.fortforecast.com/limesurvey/index.php/265172?lang=en) is here! This year we're interested in community response to the LessWrong 2.0 initiative. I've also gone through and fixed as many bugs as I could find reported on the last survey, and reintroduced items that were missing from the 2016 edition. Furthermore new items have been introduced in multiple sections and some cut in others to make room. You can now export your survey results after finishing by choosing the 'print my results' option on the page displayed after submission. The survey will run from today until the 15th of October.

You can take the survey below, thanks for your time. (It's back in single page format, please allow some seconds for it to load):

[Click here to take the survey](https://www.fortforecast.com/limesurvey/index.php/265172?lang=en)
# LW 2.0 Open Beta starts 9/20

Two years ago, I wrote [Lesswrong 2.0](/lw/n0l/lesswrong_20/). It’s been quite the adventure since then; I took up the mantle of organizing work to improve the site but was missing some of the core skills, and also never quite had the time to make it my top priority. Earlier this year, I talked with Oliver Habryka and he joined the project and has done the lion’s share of the work since then, with help along the way from Eric Rogstad, Harmanas Chopra, Ben Pace, Raymond Arnold, and myself. Dedicated staff has led to serious progress, and we can now see the light at the end of the tunnel.

So what’s next? We’ve been running the closed beta for some time at lesserwrong.com with an import of the old LW database, and are now happy enough with it to show it to you all. On 9/20, next Wednesday, we’ll turn on account creation, making it an open beta. (This will involve making a new password, as the passwords are stored hashed and we’ve changed the hashing function from the old site.) If you don't have an email address set for your account ([see here](/prefs/update/)), I recommend adding it by the end of the open beta so we can merge accounts. For the open beta, just use the Intercom button in the lower right corner if you have any trouble. 

Once the open beta concludes, we’ll have a vote of veteran users (over 1k karma as of yesterday) on whether to change the code at lesswrong.com over to the new design or not. It seems important to [look into the dark](/lw/iw/positive_bias_look_into_the_dark/) and have an escape valve in case this is the wrong direction for LW. If the vote goes through, we’ll import the new LW activity since the previous import to the new servers, merging the two, and point the url to the new servers. If it doesn’t, we’ll likely turn LW into an archive.

Oliver Habryka will be posting shortly with [his views on LW](/r/discussion/lw/pes/on_bottlenecks_to_intellectual_progress_in_the/) and more details on our plans for how LW 2.0 will further intellectual progress in the community.
# LW 2.0 Open Beta Live

The [LW 2.0 Open Beta is now live](https://www.lesserwrong.com/); this means you can create an account, start reading and posting, and tell us what you think.

Four points:

1) In case you're just tuning in, I took up the mantle of revitalizing LW through improving its codebase some time ago, and only made small amounts of progress until Oliver Habryka joined the project and put full-time engineering effort into it. He deserves the credit for the new design, and you can [read about his strategic approach here](/lw/pes/lw_20_strategic_overview/).

2) If you want to use your current LW account on LW2.0, we didn't import the old passwords, and so you'll have to use the reset password functionality. If your LW account isn't tied to a current email, [send a PM to habryka](/message/compose/?to=Habryka) on lesswrong and he'll update the user account details on lesserwrong. He's also working on improving the site and sleeping and things like that, so don't expect an immediate response.

3) During the open beta there will be a green message in the bottom right hand corner of the screen. This is called Intercom, and is how you can tell us about issues with the site and ask other questions.

4) The open beta will end with a vote of users with over a thousand karma on whether we should switch the lesswrong.com URL to point to the new code and database. If this succeeds, all the activity from the open beta and the live site will be merged together. If the vote fails, we expect to archive LW until another team comes along to revive it. We currently don't have a date set, but this will be announced a week in advance.
# LW 2.0 Site Update: 09/21/17

Hey! A few quick updates on edits done today (well, yesterday - we have late work hours).

Thanks Chris_Leong for making a [post](/posts/vtZsEerABCjhtgizX/beta-first-impressions) to document folks' bugs. In the comments ozymandias requested that the page containing every post published to the site organised by day, be more findable. That's now located in the menu (top left-hand corner).

Also in the comments philh wrote a comment of many requests; today we fixed a few of them.

*   You no longer need to include 'http://' at the start of your blog URL
    

*   You can now see a user's comments in their user page
    
*   You can now see your own comments in your user page
    

Other things we did:

*   There is a now page with all comments sorted by recency, findable on the frontpage just under the 'recent comments' section header
    
*   Fixed a bug where some users were unable to post to Meta
    
*   When you login, you no longer get redirected to the front page
    
*   Fixed formatting of RSS posts (more RSS changes coming soon)
    
*   If you click through to a recent comment, it also has a button to take you to its position in the comment thread.
    
*   Added various error messages for login and post editing and other things.
    

And answered an infinite number of support requests (where infinity = 56).
# Voting Weight Discussion

One of the distinguishing features of LW 2.0 is Voting Weight. People with more karma will be able to give higher-weighted upvotes and downvotes.

This is part of an overall plan to make it better at sorting signal-from noise. But it's a major change to the system and seemed worthy of a dedicated-discussion thread.

### Intended Outcomes

There are roughly three goals for the improved karma system:

**I. A reasonable pace of quality content on the front page, which is the** [**single conversation locus.**](https://www.lesserwrong.com/posts/8rYxw9xZfwy86jkpG/on-the-importance-of-less-wrong-or-another-single) (We're aiming for about 3-7 featured posts per week)

**II. A reasonable opportunity for people (newcomers and otherwise) to get their ideas seen, receive feedback.** (The front page involves an inherently limited supply of attention, so not everything can go there. But good posts should at least stick around on the Recent Posts page for awhile)

**III. Implement goals I and II in a way that is (and feels) fair.**

### Current Implementation of Voting Weight

The problem is that LessWrong is an _oddly specific_ community, and it won't be necessarily obvious to a newcomer what are the sorts of post we want to incentivize. So we want more experienced users who have a proven track epistemic record to be able to draw attention to posts more easily.

In light of Goal III, we don't want a high-karma user's vote to be _overwhelming_. So the current mathematical implementation is:

_floor(log_5(karma)+1)+1_

What this means in non-math-speak is that is that your voting weight looks something like:

0 karma: 1 voting weight  
5 karma: 2 voting weight  
25 karma: 3 voting weight  
125 karma: 4 voting weight  
625 karma: 5 voting weight  
...  
_... roughly capping out for practical purposes at approximately:  
_400,000 karma: 9 voting weight (currently no users are near this level)

You may have noticed that when you upvote or downvote a post, it changes by more than 1 point. This is why.

New posts also start with their user having upvoted them once, so if you have 25 karma, you'll start with 3 points. This is to reflect that people who've been around longer are more likely to be writing things that are worth paying attention to. (Although note that this initial upvote doesn't count towards their overall karma score)

Shifting to a voting-weight based system also makes it easier to notice people who are abusing the system and nullify their votes (preventing mass-downvote attacks)

### Thoughts?

The LW 2.0 team has some thoughts about how to refine the voting system into something more fine-grained. But before sharing those ideas it seemed good to solicit a more general discussion. (Some of us might share our thoughts in the comments, but those thoughts will be from our perspectives as users rather than site developers)
# LW 2.0 Site Update: 09/22/17

Another set of quick updates done today:

*   We had a bad bug where in about 50% of cases when a user created a post on the frontpage, it would neither show up on their userpage, nor on the frontpage, but only on the all-posts view. This is now fixed. Sorry for anyone who was confused by this!
    
*   Profiles should now accurately show both frontpage posts and personal blog posts
    
*   The karma/posts/comments icons now have tooltips that explain what they are
    
*   Imported all old blogposts for [Zvi](https://www.lesserwrong.com/users/zvi) and [Katja](https://www.lesserwrong.com/users/katjagrace) and set up automatic crossposting from their blogs
    
*   Added a new post link to a more accessible position (the user menu in the top right corner)
    
*   Fixed the sort order of comments on user's profile pages
    
*   This time actually fixed the LessWrong title in the top left corner and properly made it a link (i.e. you can now middle-click it and it should open the page in a new tab)
    
*   Fixed permission issues with a bunch of content in The Codex. The Codex should now be fully accessible
# Stubs of Posts I'd like to Write

Right now I feel like LW has too much "meandering around meta-community-stuff" and not enough "actually go out into the world, try to think about object level things, and then do stuff with that thinking stuff."

I'm not actually very good at the latter, but want to practice.

But, there's a bunch of ideas I have percolating that I _do_ think are pretty decent things worth pursuing on the meta-community-front, and the only thing wrong with them is that if I follow the incentive gradient that attracts them, I'll never get around the to object-thinking-stuff I want to do.

But, wanted to at least get them out of my head. Post stubs for the future:

*   **Benign and Malignant Demon Threads** _(i.e. how to tell when an internet discussion is about to go bad, and what to do)_
    
*   **Hard and Soft Mythology**: HPMOR, Steerswoman, and Hatchet as "rationality" stories, and the strengths of each.
    
*   **The Archipelago Model of Community Standards** _(many people agree the rationality community needs higher standards. They don't necessarily agree on what those standards should be. Different people want/need different things. One problem is confusion (or inevitable strife) over who owns which space. I think people should stake out spaces that they own and pro-actively foster whatever higher-standards they want there. Inspired by Scott Alexander's Archipelago post)_
    
*   Relatedly, **What Standards Do I Personally Want to Raise**, in myself and hopefully the people I interact with?
    
*   **Social Reality and Pressure Against Thinking Clearly** _(I mostly want other people who understand it better to write it, although by now I do have some ideas \[fake-edit:_ [_this isn't quite the thing_](https://www.lesserwrong.com/posts/5KdaNSsuqDJsda6eQ/rfc-on-an-open-problem-how-to-determine-probabilities-in-the) _I was pointing towards but is one of the things)_
    
*   **Lessons From the Steampunk World** _(What living on a boat for a week taught me about physicality and problem solving)_
    
*   **Building the Agency Pipeline** _(It seems like people need 1-on-1 conversations with people who are further along their path to help give them confidence/ambition to continue, but the people further along the path are often busy, and trying to make it_ too _legible who is more and less available to help people midway-through-the-agency-pipeline comes with weird knock-on effects)_
# NSFW Content on LW

There's some discussion of this on [my post](https://www.lesserwrong.com/posts/4DrybydEAh59vQiTs/the-typical-sex-life-fallacy) but I think it properly belongs on a meta post to which the mods can refer.

Points for consideration:

*   Do we want content about sex on LW at all? Yes, but only on people's own blogs? Yes, but only with a content warning? Yes, but only with some sort of NSFW tag? Yes, and with no limits?
    
*   It is probably a good idea to have at least a little idea what "about sex" means. Do [STD etymology fact posts](https://srconstantin.wordpress.com/2015/04/30/std-statistics/) count? Or posts about [rational romantic relationships](http://lesswrong.com/lw/63i/rational_romantic_relationships_part_1/)?
    
*   Cursing: is it off-limits entirely? Certain words only? (Which ones?) No excessive cursing?
    
*   What norms apply to titles?
    
*   Even if the content is content-warned or on people's own blogs, the recent comments show up in the recent comments feed, which could make some people feel uncomfortable browsing LW in public.
# Toy model of the AI control problem: animated version

A few years back, I came up with a [toy model](http://lesswrong.com/lw/mrp/a_toy_model_of_the_control_problem/) of the AI [control problem](https://en.wikipedia.org/wiki/AI_control_problem). It has a robot moving boxes into a hole, with a slightly different goal than it's human designers, and a security camera to check that it's behaving as it should. The robot learns to block the camera to get its highest reward.

I've been told that the model is useful for explaining the control problem quite a few people, and I've always wanted to program the "robot" and get an animated version of it. Gwern had a [live demo](http://www.gwern.net/docs/rl/armstrong-controlproblem/index.html), but it didn't illustrate all the things I wanted to.

So I programmed the toy problem in python, and generated a [**video with commentary**](https://www.youtube.com/watch?v=sx8JkdbNgdU).

![](https://i.imgur.com/nIdiBcK.png)

In this simplified version, the state space is sufficiently small that you can explicitly generate the whole table of Q-values (expected reward for taking an action in a certain state, assuming otherwise optimal policy). Since behaviour is deterministic, this can be updated in dynamical programming, using a full width backup. The number of such backups essentially measures the depth of the robot's predictive ability.

The most interesting depths of planning are:

*   At depth **0**, the robot behaves randomly.
    
*   At depth **3**, the robot crashes itself efficiently, to avoid loosing too much reward.
    
*   At depth **6**, the robot figures out how to get one box into the hole. The automated camera turns it off.
    
*   At depth **17**, the robot finally starts to deceive/conceal/manipulate, by blocking the camera and pushing two boxes into the hole. **Note that the robot's deception comes from the fact that its incentives are misaligned,** _**and**_ **that humans tried to control it.**
    
*   At depth **18**, the robot efficiently does the plan from depth 17.
    
*   At depth **20**, the robot does the maximally efficient plan: blocking the camera, and pushing all boxes into the hole.
    
*   At depth **32**, the robot has the correct Q-values for the maximally efficient plan.
    
*   At depth **45**, finally, the Q-value table is fully updated, and the robot will take maximally efficient, and, if need be, deceptive plans from any robot/box starting positions.
    

The code and images can be found [here](https://www.dropbox.com/sh/bflmein9d9oxjxe/AAA_A2AwQQJ00kveKDPEpsRea?dl=0).
# LesserWrong is now de facto the main site

My understanding was that the idea was for LesserWrong to run as a beta with a few users switching over and the main site still running until enough bugs had been fixed and enough missing features had been added so that we could have a vote and then decide whether to replace the old LessWrong.

What has actually happened is that all of the attention and worthwhile content has shifted over too LesserWrong, meaning that nothing is happening on LessWrong. This creates something of an issues:

*   If we encourage all the people remaining on LessWrong to shift to LesserWrong right now, then they will find a site that still has many bugs and which is missing many core features.
    
*   On the other hand, the status quo has costs as well. Anyone visiting LessWrong will find a forum that is not very lively and which doesn't contain particularly good content (because those in the know are posting to LesserWrong). In other words, all of the issues with lack of content and engagement on old LessWrong have now greatly increased.
    

I don't have any proposals. I simply wrote this to bring this issue to people's attention.

I have confidence in the new site. I think that enough people have switched over to LesserWrong and started posting quality content that we now have critical mass. So this isn't a disaster, but I thought it was important for someone to explicitly say what I'm sure a lot of other people are thinking.
# Rare Exception or Common Exception

**Proposition:** Pointing out that rare exceptions exist is usually a negative derailing tactic. However if you think your conversational partner is annoyingly pointing out a rare exception, double-check in case _they_ think they are pointing out a common exception.  
\-\-\-\-\-\-  
Human space is deep and wide, and there is a lot more variation in humanity than we really give credit to. This means that for almost any generalization you can make about people, there exists exceptions to the rule.

One rhetorical trick people sometimes use is to point out rare exceptions when the discussion is focusing on generalizations. For example, let's say Alice argues that five year olds shouldn't be allowed to vote because they aren't mature enough or knowledgeable enough. Bob replies "Some five year olds are more mature and knowledgeable than some thirty year olds."

Well... it's strictly _true_, in that there are some five year old prodigies that you could compare against some non-functional adults in a vegetative state. But it isn't really helpful to the conversation, and it tends to be derailing. This is a case where you are searching all of human-space to find your 0.001% exception and then going "AHA!"  
  
However, there are also times when there are _many_ people who fall outside the norm, and pointing out the exceptions is useful. If Clair says they don't use women to stock shelves because women aren't tall enough to reach the top shelf, and Derek points out "Some women are taller than the average man," this seems very different than the previous example. It would be rare to find a five year old that is more mature than a thirty-five year old, but not at all particularly rare to find a woman who is taller than a man (even though on average men are taller).  
  
I think sometimes misunderstandings occur when one person thinks they're pointing out a common exception, and their conversational partner thinks they are pointing out a rare exception. Maybe Bob really thinks that something like 10% of five-year-olds are more knowledgeable than about 10% of adults. If Alice assumes Bob is just pointing out the 0.001% exceptions that always exist in all human things ever, she is going to be more likely to assume Bob is arguing in bad faith, and miss out on finding their true disagreement.  
  
(I am bad at naming things, if anyone has an idea for what to call it when you point out a 0.001% exception.)