trentmkelly/LessWrong-43k huggingface dataset:
You’re Good Enough, You’re Smart Enough, and People Would Like You
by Zvi — 2017-04-09T12:49:27.000Z

Epistemic Status: Playing the odds

Originality Status: Very low. These are not unique views and this has been said many times before, so apologies if you’ve seen it many times before, but the message never gets through, so we keep trying.

The rationalist community can be intimidating. It is especially intimidating to those who are most suited to contributing. It is exactly those people who understand the need for high epistemic standards, who worry whether their writing skills are good enough to post, who worry if their thinking is good enough or their ideas original and interesting. Less Wrong, or even a person’s personal blog, can feel like a sacred space that one is loathe to profane with their unworthy presence.

That instinct is good! Those high standards, that sacredness, are what keep quality high, and make actions like ‘read the entire Less Wrong discussion section once a week and large percentages of the comments sections’ reasonable things to do. Our self-imposed standards allow us to almost skate by without any need for moderation. I am all about self-imposed high standards.

The problem is that our calibration is bad. The Fear has gone too far, and is keeping too many people quiet too often. Calibration is hard, and calibration of your own skill level is very hard. It speaks well of us that we would rather think of ourselves a level below where we are, than think of ourselves as a level above, but getting it right would be better still. We also are too afraid of trying to go one level too high, and not afraid enough of false humility. I am going to suggest that everyone adjust accordingly.

Consider levels on the following (0-5) online scale: Absent-Lurker-Commenter-Poster/Blogger-Organizer-Leader. You could have a similar offline scale: Absent-Silent-Talker-Presenter-Organizer-Leader.

My rule of thumb would be: Assume that you are ready to be one level higher than you think you are ready for.

If you are absent, you presumably are not reading this, but in case you are or someone can pass this advice along, I say that if you are interested in the ideas being discussed, go ahead and lurk. Show up to a meeting and listen; read some blog posts. Some of it won’t make sense. So what? Many of the times I have learned the most are in places where a lot of things did not make sense to me. Slowly, more and more of them did, and if some ...
Net Utility and Planetary Biocide
by madhatter — 2017-04-09T03:43:27.944Z

I've started listening to the audiobook of Peter Singer's Ethics in the Real World, which is both highly recommended and very unsettling. The essays on non-human animals, for example, made me realize for the first time that it may well be possible that the net utility on Earth over all conscious creatures is massively negative.

Naturally, this led me to wonder whether, after all, efforts to eradicate all consciousness on Earth - human and non-human - may be ethically endorsable.This, in turn, reminded me of a recent post on LW asking whether the possibility of parallelized torture of future uploads justifies killing as many people as possible today.

I had responded to that post by mentioning that parallelizing euphoria was also possible, so this should cancel things out. This seemed at the time like a refutation, but I realized later I had made the error of equating the two, utility and disutility, as part of the same smooth continuum, like [-100, 100] ∈ R. There is no reason to believe the maximum disutility I can experience is equal in magnitude to the maximum utility I can experience. It may be that max disutility is far greater. I really don't know, and I don't think introspection is as useful in answering this question as it seems intuitively to be, but it seems quite plausible for this to be the case.

As these thoughts were emerging, Singer, as if hearing my concerns, quoted someone or other who claimed that the human condition is one of perpetual suffering, constantly seeking desires which, once fulfilled, are ephemeral and dissatisfying, and therefore it is a morally tragic outcome for any of us to have emerged into existence.

Of course these are shoddy arguments in support of Mass Planetary Biocide, even supposing the hypothesis that the Earth (universe?) has net negative utility is true. For one, we can engineer minds somewhere in a better neighborhood of mindspace, where utility is everywhere positive. Or maybe it's impossible even in theory to treat utility and disutility like real-valued functions of physical systems over time (though I'm betting it is). Or maybe the universe is canonically infinite, so even if 99% of conscious experiences in the universe have disutility, there are infinite quantities of both utility and disutility and so nothing we do matters, as Bostrom wrote about. (Although this is actually not an arg...
Requesting Questions For A 2017 LessWrong Survey
by namespace — 2017-04-09T00:48:19.069Z

It's been twelve months since the last LessWrong Survey, which means we're due for a new one. But before I can put out a new survey in earnest, I feel obligated to solicit questions from community members and check in on any ideas that might be floating around for what we should ask.

The basic format of the thread isn't too complex, just pitch questions. For best chances of inclusion, however, it's best to include:

A short cost/benefit analysis of including the question. Keep in mind that some questions are too invasive or embarrassing to be reasonably included. Other questions might leak too many bits. There is limited survey space and some items might be too marginal to include at the cost of others.

An example of a useful analysis that could be done with this question(s), especially interesting analysis in concert with other questions. eg. It's best to start with a larger question like "how does parental religious denomination affect the cohorts current religion?" and then translate that into concrete questions about religion.

Some idea of how the question can be done without using write-ins. Unfortunately write-in questions add massive amounts of man-hours to the total analysis time for a survey and make it harder to get out a final product when all is said and done.

The last survey included 148 questions; some sections will not be repeated in the 2017 survey, which gives us an estimate about our question budget. I would prefer to not go over 150 questions, and if at all possible come in at many fewer than that. Removed sections are:

The Basilisk section on the last survey provided adequate information on the phenomena it was surveying, and I do not currently plan to include it again on the 2017 survey. This frees up six questions.

The LessWrong Feedback portion of the last survey also adequately provided information, and I would prefer to replace it on the 2017 survey with a section measuring the site's recovery, if any. This frees up 19 questions.

I also plan to do significant reform to multiple portions of the survey. I'm particularly interested in making changes to:

The politics section. In particular I would like to update the questions about feelings on political issues with new entries and overhaul some of the options on various questions.

I handled the calibration section poorly last year, and would like to replace it this year with an easi

...
What exactly is the "Rationality Community?"
by Raemon — 2017-04-09T00:11:44.364Z

[This is the second post in the Project Hufflepuff sequence, which I no longer quite endorse in its original form. But this post is particularly standalone]

I used to use the phrase "Rationality Community" to mean three different things. Now I only use it to mean two different things, which is... well, a mild improvement at least. In practice, I was lumping a lot of people together, many of whom neither wanted to get lumped together nor had much in common.

As Project Hufflepuff took shape, I thought a lot about who I was trying to help and why. And I decided the relevant part of the world looks something like this:

I. The Rationalsphere

The Rationalsphere is defined in the broadest possible sense - a loose cluster of overlapping interest groups, communities and individuals. It includes people who disagree wildly with each other - some who are radically opposed to one another. It includes people who don’t identify as “rationalist” or even as especially interested in “rationality” - but who interact with each other on a semi-regular basis. I think it's useful to be able to look at that ecosystem as a whole, and talk about it without bringing in implications of community.

There is no single feature defining people in the rationalsphere, but there are overlapping habits and patterns of thought. I'd guess that any two people in the cluster share at least one of the following features:

Being attentive to ways your mind is unreliable
Desire to understand objective reality.
Willingness to change one’s mind about ideas that are important to you.
Having goals, which you care about achieving badly enough to decide “if my current habits of thought are an obstacle to achieving those goals, I want to prioritize changing those habits.”
Ambitious goals, that require a higher quality of decision-making than most humans have access to.

People invested in the rationalsphere seem to have three major motivations:

Truthseeking - How do we improve our thinking? How do we use that improved-thinking to better understand the world?
Impact - A lot of things in the world could be a lot better. Some see this in moral terms - there are suffering people and unrealized potential and we have an obligation to help. Others see this purely in opportunity and excitement, and find the concept of “altruism” offputting or harmful. But they share this: an interest in having a big impact, while under

...
Why Rationality?
by Zvi — 2017-04-08T12:10:00.000Z

Previously: Responses to Tyler Cohen on Rationality

Recently: Yes, We Have Noticed the Skulls, When Rationalists Remade the World

Repeat as Necessary: Principle of Charity

A lot of people do not understand the appeal. I was surprised to find that Will Wilkinson was among them. Tyler shared a tweet-storm of his, which I will reproduce below:

1. Some thoughts on the controversy over the rationality community sparked by @tylercowen’s offhand comments to @ezraklein

2. Avoiding the most common errors in thought has surprisingly few personal benefits. Why try?

3. Vigilance against bias and fastidious Bayesianism has to appeal to something in you. But what? Why are you so worried about being wrong?

4. Some rationalists talk a lot about status signaling and tribe-based confirmation bias. What’s the theory of shutting this off?

5. Plato’s right. Will to truth is rooted in eros. Longing for recognition must be translated into a longing for a weird semantic relation.

6. Hume is right. Good things, like reason, come from co-opting and reshaping base motives. Reason’s enslaved to the impulse that drives it.

7. I see almost no interest among rationality folks in cultivating and shaping the arational motives behind rational cognition.

8. Bayes’ Law is much less important than understanding what would get somebody to ever care about applying Bayes’ Law.

9. If the reason people don’t is that it would destabilize their identity and relationships, maybe it’s not so great to be right.

10. Aristotle says a real philosopher is inhuman.

11. If the normal syndrome of epistemic irrationality is instrumentally rational, then you’ve got to choose your irrationality.

12. The inclination to choose epistemic rationality is evidence of being bad at life.

13. It’s understandable that these people should want a community in which they can feel better and maybe even superior about this.

14. We all need a home. If that desire for safety/acceptance/status produces more usefully true thoughts, that’s awesome.

(His later replies in the thread, which mostly happened after I read it the first time, are also worth looking at, and indicate the view that what has been done is good, but inadequate to the tasks at hand.)

At first this made me angry. I read this as a direct attack not only on my community, but even on the very idea of truth. How dare we ...
What conservatives and environmentalists agree on
by PhilGoetz — 2017-04-08T00:57:32.012Z

Today we had a sudden cold snap here in western Pennsylvania, with the temperature dropping 30 degrees F. I was walking through a white field that had been green yesterday, looking at daffodils poking up through the snow and feeling irritated that they'd probably die. It occurred to me that, if we could control the weather, people would probably vote for a smooth transition from winter to summer, and this would wreak some unforeseen environmental catastrophe, because it would suddenly make most survival strategies reliably sub-optimal.

This is typical environmentalist thinking: Whenever you see something in the environment that you don't like, stop and step back before trying to change it. Trust nature that there's some reason it is that way. Interfere as little as possible.

The classic example is forest fires. Our national park service used to try to stop all forest fires. This policy changed in the 1960s for several reasons, including the observation that no new Sequoia saplings had sprouted since the beginning of fire suppression in the 19th century. Fire is dangerous, destructive, and necessary.

It struck me that this cornerstone of environmentalism is also the cornerstone of social conservatism.

Leftist are social farmers; conservatives are social environmentalists

Most leftists view their own society [1] the same way farmers view the environment. They believe we can all agree on what crops to grow (what social values to have [2]), and we should clear out the stumps of old institutions, plant the seeds of the new, create the right rationally-informed infrastructure of irrigation, fertilization, crop rotation, etc., and pull out and burn the weeds of nostalgia, counter-revolution, and the bourgeoisie, and we will have a modern, rational society.

Conservatives view their own society the way environmentalists view the environment: as a complex organism best not lightly tampered with. They're skeptical of the ability of new policies to do what they're supposed to do, especially a whole bunch of new policies all enacted at once.

So, many liberals and many conservatives understand the law of unintended consequences (LUC), but the contexts they apply it in are disjoint.

Why?

If we broadly stereotype conservative Americans as Christian, and Christians as not believing in evolution, we can explai...
Meetup : Baltimore - SSC Edition
by l1n — 2017-04-07T13:14:52.802Z

Discussion article for the meetup : Baltimore - SSC Edition

WHEN:
09 April 2017 08:00:00PM (-0400)

WHERE:
Performing Arts and Humanities Building, 1000 Hilltop Cir, Baltimore, MD 21250, United States

We’ll be meeting to discuss ... anything. Suggested Reading: Guided By The Beauty Of Our Weapons from Slate Star Codex. Meetup is on the 4th floor by the English department. Follow the voices. For help getting into the building, please text 619-357-6633.

Format: We meet and start hanging out at 8:00, but don’t officially start doing the meetup topic until 8:15-8:30 to accommodate stragglers.

Discussion article for the meetup : Baltimore - SSC Edition
Agents that don't become maximisers
by Stuart_Armstrong — 2017-04-07T12:56:55.747Z

Cross-posted at the Intelligent Agent forum.

According to the basic AI drives thesis, (almost) any agent capable of self-modification will self-modify into an expected utility maximiser.

The typical examples are the inconsistent utility maximisers, the satisficers, unexploitable agents, and it's easy to think that all agents fall roughly into these broad categories. There's also the observation that when looking at full policies rather than individual actions, many biased agents become expected utility maximisers (unless they want to lose pointlessly).

Nevertheless... there is an entire category of agents that generically seem to not self-modify into maximisers. These are agents that attempt to maximise f(E(U)) where U is some utility function, E(U) is its expectation, and f is a function that is neither wholly increasing nor decreasing.

Intransitive example

Let there be a U with three action a0, a5, and a10 that set U to 0, 5, and 10, respectively.

The function f is 1 in the range (4,6) and is 0 elsewhere. Hence the agent needs to set the expectation of U to be in that range.

What will happen is that one action will be randomly removed from the set, and the agent will then have to choose among the remaining two actions. What possible policies can the agent take?

Well, there are three option sets the agent could face - (a0, a5), (a5, a10), and (a10, a0) - each with two options and hence 23=8 pure policies. Two of those policies - choosing always the first option in those ordered pairs, or choosing always the second option - are intransitive, as they rank no option above the other two.

But actually those intransitive options have an expected utility of (0+5+10)/3 = 5, which is just what the agent wants.

Even worse, none of the other (transitive) policies are acceptable. You can see this because each of the six transitive policies can be reached by taking one of the intransitive policies and flipping a choice, which must change the expected utility by ±5/3 or ±10/3, moving it out of the (4,6) range.

Thus there are no possible expected utility maximalisation that correspond to these options, as such maximalisations are always transitive.

Or another way of seeing this: the random policy of picking an action randomly has an expectation of (0+0+5+5+10+10)/6 = 5, so is also an acceptable policy. But for expected utility maximalisation, if the random policy...
Not all non-maximisers become maximisers
by Stuart_Armstrong — 2017-04-06T19:27:55.000Z

Because of the problems with optimisers, there's a search for other types of designs that can accomplish goals without the extreme optimisation pressure. Satisficers are one example of such designs, but satisficers aren't reflectively stable (a satisficer can design a successor agent that isn't a satisficer) and aren't even reflectively consistent (there are situations where a satisficer will prefer to become a maximiser rather than staying a satisficer).

The general failure mode of an other-ising design is that it turns into a maximiser.

Here's a design that doesn't turn into a maximiser, at least not directly. It's not useful for anything directly, but it does show some of the odd behaviours that certain agents can indulge in.

Doubly bounded satisficer

A Doubly-bounded satisficer (DBS) is one that has a utility function U and two bounds q<r. It's aim is to take actions that ensure the expected utility E(U) of U is bounded between q and r.

It's clear that, in general, the DBS won't make itself into a U-maximiser (since that might have an expectation above r) or into a U-minimiser (since that might have an expectation below q). But how exactly will it behave?

Assume for illustration purposes that 0≤U≤1, and that ϵ>0 is very small.

No stochasticity

If there is no stochasticity or uncertainty, the DBS knows that for every policy π there is a corresponding value Uπ of U. Therefore it will select a policy such that Uπ∈[q,r].

Alternately, it can simply turn itself into a U′q,r maximiser, where U′q,r=1 iff U∈[q,r], and U′q,r=0 otherwise.

Stochastic choice between outcomes

Let's add some uncertainty to the setup. The DBS knows that two numbers x and y will be drawn uniformly randomly from [0,1], and it will then have the choice between actions ax and actions ay. Then U∣ax=x and U∣ay=y.

Set [q,r] equal to [1/2−ϵ,1/2+ϵ]

In this case, maximising U′q,r (and breaking ties randomly) is still a good policy for the agent. But so is randomly choosing actions! Both these have an expected U of 1/2.

In contrast, maximising U′q,r and breaking ties by choosing the highest action, has an expected utility of around 2/3. So it's not the U′q,r-maximising that's doing the job here.

Set [q,r] equal to [2/3−ϵ,2/3+ϵ]

In this case, maximising U straight up is a good policy (and all good policies are close to that one). But that's an artefact of these particular values of q and r.

Se

...
Meetup : San Francisco Meetup: Deep Questions
by rocurley — 2017-04-06T05:37:48.215Z

Discussion article for the meetup : San Francisco Meetup: Deep Questions

WHEN:
10 April 2017 06:15:00PM (-0700)

WHERE:
1045 Mission St, San Francisco, CA 94103-2812, United States

Note: This is also an SSC meetup. If we get too many people, we'll move to the nearby park. If we do, I'll update this post, but you can also find out by calling when you get to the normal location.

We’re trying a new format this week!
How Deep Questions works: We’ll break into pairs. Each pair will take turns answering one of a selected list of open-ended meaningful questions (e.g. “What in life gets you really excited?”, “What’s one thing you’ve never told someone before about yourself that you feel comfortable sharing with a stranger?”, “What do you hope they say in your obituary?”, “Who do you admire most, and why do you admire that person so much?”). Each pair gets 12 minutes for both people to answer the question; after 12 minutes, the pairs rotate to take on the next question with a new person.
This meetup is more structured than usual; at the recent meta-meetup, we got feedback that more structured interaction might help shy people integrate better into the group. Please let us know what you think of this meetup format.
For help getting into the building, please call (or text, with a likely-somewhat-slower response rate): 301-458-0764.
Format:
We meet and start hanging out at 6:15, but don’t officially start doing the meetup topic until 6:45-7 to accommodate stragglers. Usually there is a food order that goes out before we start the meetup topic.
About these meetups:
The mission of the SF LessWrong meetup is to provide a fun, low-key social space with some structured interaction, where new and non-new community members can mingle and have interesting conversations. Everyone is welcome.
We explicitly encourage people to split off from the main conversation or diverge from the topic if that would be more fun for them (moving side conversations into a separate part of the space if appropriate). Meetup topics are here as a tool to facilitate fun interaction, and we certainly don’t want them to inhibit it.

Discussion article for the meetup : San Francisco Meetup: Deep Questions
Meetup : Chicago SSC Meetup / Rationality Reading Group
by mingyuan — 2017-04-05T18:10:34.000Z

Discussion article for the meetup : Chicago SSC Meetup / Rationality Reading Group

WHEN:
09 April 2017 01:00:00PM (-0500)

WHERE:
Harper Memorial Library Room 148, 1116 E 59th St, Chicago, IL 60637

This week, instead of our regular reading group meeting, we will be hosting the official Chicago SlateStarCodex meetup! On the latest SSC survey, 100 people expressed interest in a Chicago meetup, so we are hoping for a sizable turnout!

The reading for this week is optional, as this will be primarily a social event. But in case you're worried about not knowing what to talk about with fellow attendees, you can read Meditations On Moloch as a Schelling point.

Meetup details:

Date/time: Sunday, April 9th, 1-3 PM

Location: Harper Memorial Library Room 148, 1116 E 59th St, Chicago, IL 60637

Facebook event: https://www.facebook.com/events/1079805162123412/

Please RSVP on Facebook if you can, so I know how many people will be coming! If you have any questions, or if you would like to come but aren't able to make it at that time, please let me know.

See you on Sunday!

Claire

Discussion article for the meetup : Chicago SSC Meetup / Rationality Reading Group