trentmkelly/LessWrong-43k huggingface dataset:
"Flinching away from truth” is often about *protecting* the epistemology
by AnnaSalamon — 2016-12-20T18:39:18.737Z

Related to: Leave a line of retreat; Categorizing has consequences.

There’s a story I like, about this little kid who wants to be a writer. So she writes a story and shows it to her teacher.

“You misspelt the word ‘ocean’”, says the teacher.

“No I didn’t!”, says the kid.

The teacher looks a bit apologetic, but persists: “‘Ocean’ is spelt with a ‘c’ rather than an ‘sh’; this makes sense, because the ‘e’ after the ‘c’ changes its sound…”

“No I didn’t!” interrupts the kid.

“Look,” says the teacher, “I get it that it hurts to notice mistakes. But that which can be destroyed by the truth should be! You did, in fact, misspell the word ‘ocean’.”

“I did not!” says the kid, whereupon she bursts into tears, and runs away and hides in the closet, repeating again and again: “I did not misspell the word! I can too be a writer!”.

I like to imagine the inside of the kid’s head as containing a single bucket that houses three different variables that are initially all stuck together:

Original state of the kid's head:

The goal, if one is seeking actual true beliefs, is to separate out each of these variables into its own separate bucket, so that the “is ‘oshun’ spelt correctly?” variable can update to the accurate state of "no", without simultaneously forcing the "Am I allowed to pursue my writing ambition?" variable to update to the inaccurate state of "no".

Desirable state (requires somehow acquiring more buckets):

The trouble is, the kid won’t necessarily acquire enough buckets by trying to “grit her teeth and look at the painful thing”. A naive attempt to "just refrain from flinching away, and form true beliefs, however painful" risks introducing a more important error than her current spelling error: mistakenly believing she must stop working toward being a writer, since the bitter truth is that she spelled 'oshun' incorrectly.

State the kid might accidentally land in, if she naively tries to "face the truth":

(You might take a moment, right now, to name the cognitive ritual the kid in the story *should* do...
Improve comments by tagging claims
by Benquo — 2016-12-20T17:04:32.309Z

I used to think that comments didn’t matter. I was wrong. This is important because communities of discourse are an important source of knowledge. I’ll explain why I changed my mind, and then propose a simple mechanism for improving them, that can be implemented on any platform that allows threaded comments.

Should there be comments?

Comments are optional.

Organizing discourse as a forum with comments introduces substantial structural rigidity. If you don’t like one of the regular posters’ content, too bad, it’s still cluttering up your feed. If you want to start contributing to the discourse, this comes at the cost of pulling everyone else’s attention, willing or no. In forums with high barriers to entry, this means that the move equivalent to “start your own blog” is not available to everyone. In forums with low barriers to entry, this can mean dilution by low-value posts, leading to the departure of discriminating readers. If standards are ambiguous, this can lead to an adverse selection process in which the contributors who are most conscientious about respecting community standards err on the side of not posting, and the least conscientious contributors flood the forum with low-value content, accelerating the process of pushing away the most discriminating participants.

My prior model of how written discourse should work was that people should publish in whatever venue they thought appropriate - often their own blog. If they find someone else’s writing interesting and want to comment on it, they can write their own post, and link to the thing they want to talk about. This has a few benefits. A link-based attention economy means that in order to get the attention of the readers of some content, you generally have to get the attention of the person generating the content. This means that higher-quality writing gets attention, and limits the effectiveness of low-quality trolling. Another benefit is that it allows for partly overlapping clusters. If a blog you like keeps linking to another blog, you can decide whether you like that one, and if you do, you can start reading it directly. Thus, you’ll be directed to blogs that are “nearby” in your local network, but not everyone reading your blog has to read the same set of nearby blogs.

In the early 2000s, a natural experiment h...
Teaching an AI not to cheat?
by Florian_Dietz — 2016-12-20T14:37:53.512Z

I have been thinking about a technique in training AIs that I believe would be very useful. I would like to know if this is already known, or if it has been discussed at all.

I find that there are lots of different failure modes that people are worried about when it comes to AI. Maybe the AI misunderstands human intentions, maybe it deliberately misinterprets an order, maybe it associates the wrong sort of actions with the reward, etc.

If it was a game, many of these failure modes are what we would consider cheating. So why don't we just take this analogy and run with it:

Teach the AI to realize on its own what would be considered cheating by a human, and not to do anything that it identifies as cheating.

To do this, one could use the following technique:

Come up with games of increasing complexity, and let the AI play it in two stages:

In stage one, you introduce an artificial loophole into the game that makes winning it very easy. For instance, assuming the AI has already played chess before, so it can be assumed to understand the rules, give the AI the task to play a game of chess in which you simply do not check if the moves are legal. When the AI wins by cheating, i.e. via ordinarily illegal moves, reward it anyway.

In the second stage, the reward is far greater, but if the AI plays an illegal move, it now receives negative feedback.

Let the AI play many different games in these two stages. After a while, the AI will learn to identify what constitutes cheating, and to avoid doing so.

Start varying the amount of time during which cheating is allowed, to keep the AI on its toes. Sometimes, don't allow any cheating at all from the start.

If you train an AI in this manner:

-it would learn to understand how humans view the world (in some limited sense), as a human-centric viewpoint is necessary to understand what does and does not constitute cheating in human-designed games.

-it would be driven to adjust its own actions to match human preconceptions out of a fear of getting punished.

-if this AI were to "break out of the box" prematurely, there would be at least a chance that it would recognize that it was not supposed to get out of the box, that this constitutes cheating, and that it should get back in. This could even be tested by building a "box" of several layers and deliberately designing the inner layers to be hackable.
How does personality vary across US cities?
by JonahS — 2016-12-20T08:00:43.530Z

In 2007, psychology researchers Michal Kosinski and David Stillwell released a personality testing app on Facebook app called myPersonality. The app ended up being used by 4 million Facebook users, most of whom consented to their personality question answers and some information from their Facebook profiles to be used for research purposes.

The very large sample size and matching data from Facebook profiles make it possible to investigate many questions about personality differences that were previously inaccessible. Koskinski and Stillwell have used it in a number of interesting publications, which I highly recommend (e.g. [1], [2] [3]).

In this post, I focus on what the dataset tells us about how big five personality traits vary by geographic region in the United States.

The Five Factor Model of Personality

The Five Factor Model (FFM) or Big Five personality trait model is currently the dominant paradigm in personality research. The model is founded on the lexical hypothesis:

The lexical hypothesis is generally defined by two postulates. The first states that those personality characteristics that are most important in peoples' lives will eventually become a part of their language. The second follows from the first, stating that more important personality characteristics are more likely to be encoded into language as a single word.

When people are asked questions about whether various adjectives describe them (or describe someone who they know), their answers are pairwise correlated with one another. Applying factor analysis to the responses yields a small number of underlying factors that explain a large fraction of the variance common to the answers.

Empirically, it's been found that a model with 5 factors often fits the data well (though some researchers claim that one gets 6 or 7 factors if one uses a question battery that fully exhausts descriptive adjectives, see e.g. the HEXACO model of personality and The Big Seven Model of Personality and Its Relevance to Personality Pathology for more information).

The five factors referred to as the "Big Five" are labelled extraversion, neuroticism, agreeableness, conscientiousness and openness. I will describe these more below.

It's likely that the Big Five personality model falls far short of carving reality at its joints, and I'm in broad agreeme...
Guilt vs Shame, Pride vs Joy?
by ProofOfLogic — 2016-12-19T20:00:18.376Z

[Epistemic status: speculative. My goal here is only to articulate a plausible hypothesis given the data.]

The Moral Economy, which I mentioned in my post about my recent reading list, talks quite a bit about the crowding-out effect, in which placing external incentives on behavior decreases intrinsic drive for that behavior. This can cause incentives to have a smaller effect on behavior than classical economic theory would predict, or even have the complete opposite effect.

His main source of data is a large number of experiments in which participants across the world play simple games such as prisoner's dilemma, with modifications such as allowing players to punish each other or give bonuses for good behavior. The crowding-out effect is the most frequently observed phenomenon, but it isn't a predictable thing by a long shot: he observes a wide variety of behavior, even including what he calls "crowding in" (incentives increasing intrinsic pro-social motivation). So, there's one puzzle: what creates the different response patterns?

Another puzzle: the crowding-out effect suggests, heuristically, that capitalism would decrease pro-social motives. Incentivising behaviors with money made people more selfish in the experiments. Other priming experiments suggest that people get more selfish with the mere mention of money. Furthermore, experiments show that there is a "spillover" effect: playing games which crowd out non-selfish motives makes people play more selfishly in subsequent games. So, one might suspect that long-term exposure to capitalism would result in long-term crowding out, creating more selfish behavior in the games.

The opposite is true: exposure to capitalism highly correlates with altruistic behavior in these games. Hunter-gatherer societies which are based on mutual sharing of food are among the least cooperative, when it comes to the games. Why?

The book speculates that advanced economies engender a much higher degree of trust in strangers. If you regularly have economic interactions with strangers without being conned, you come to expect (and provide) a level of common decency.

Experiments also show that crowding-in, rather than crowding-out, can be engendered by allowing participants to discuss what actions should be punished before they start play, rather than making them anonymously dole out punishment without group consent. People from mar...
Stupid Questions December 2016
by pepe_prime — 2016-12-19T14:41:07.149Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
Open thread, Dec. 19 - Dec. 25, 2016
by MrMind — 2016-12-19T07:46:48.129Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
Multiplicitous
by Jacob Falkovich — 2016-12-18T16:39:52.000Z

P-Vices

Of all the terrible vices known to man, I guess NFL gambling and alternative medicine aren’t very terrible. Making medicine that doesn’t work (if it worked, it wouldn’t be alternative) is also a tough way to make money. But if you’re able to squeeze a p-value of 0.05 for acupuncture out of a trial that clearly shows that acupuncture has zero effect, you can make money and get a PhD in medicine!

It’s also hard to make money off of gambling on the NFL. However, you can make money by selling NFL gambling advice. For example, before the Eagles played as 6 point underdogs on the turf in Seattle after a 208 yard rushing game, gambling guru Vince Akins declared:

The Eagles are 10-0 against the spread since Dec 18, 2005 as an underdog and on turf after they’ve had more than 150 yards rushing last game.

10-0! Betting against the spread is a 50-50 proposition, so 10-0 has a p-value of 1/(2^10) = 1/1024 = 0.0009. That’s enough statistical significance not just to bet your house on the Eagles, but also to get a PhD in social psychology.

The easiest way to generate the p-value of your heart’s desire it to test multiple hypotheses, and only report the one with the best p-value. This is a serious enough problem when it happens accidentally to honest and well-meaning researchers to invalidate whole fields of research. But unscrupulous swindlers do it on purpose and get away it, because their audience suffers from two cognitive biases:

Conjunction fallacy.

Sucking at statistics.

No more! In this new installment of defense against the dark arts we will learn to quickly analyze multiplicity, notice conjunctions, and bet against the Eagles.

Hacking in Depth

[This part gets stats-heavy enough to earn this post the Math Class tag. If you want to skip the textbooky bits and get down to gambling tips, scroll down to “Reading the Fish”]

The easiest way to generate multiple hypotheses out of a single data set (that didn’t show what you wanted it to show) is to break the data into subgroups. You can break the population into many groups at once (Green Jelly Bean method), or in consecutive stages (Elderly Hispanic Woman method).

The latter method works like this: let’s say that you have a group of people (for example, Tajiks) who suffer from a medical condition (for example, descolada). Normally, exactly on...
Extortion and trade negotiations
by Stuart_Armstrong — 2016-12-17T21:39:30.166Z

To illustrate an old point - that it's hard to distinguish between extortion and trade negotiations - here's a schematic diagram of extortion, alternating actions by player B (blackmailer/extorter//blue) and V (victim//violet):

The extorter can let the default Def happen, or can instead do a threat, ending up in point T. Then the victim can resist (Res) or surrender (Sur). If the victim resists, the extorter has the option of carrying out their threat (C) or not doing so (¬C).

You need a few conditions to make this into a extortion situation:

Sur has to be the best outcome for B (or else the extortion has no point).

To make sure that T is a true threat, Sur has to be worse than Def for V, and C has to be worse that ¬C and Def for B.

And to make this into extortion, C has to be worse than Def for V.

A mere threat doesn't make this into extortion. Indeed, trade negotiations are a series of repeated threats from both sides, making offers with the implicit threat that they will walk away from the deal entirely if that offer is not accepted.

But if C is worse that Def for V, then this seems a true extortion: V will end up worse if they resist a extorter who carries out their threats, and they would have much preferred that the extorter not be able to make credible threats in the first place. And the only reason the extorter made the threats, was to force the victim to surrender.

Fairness and equity

Note that this is not about fairness or niceness. It's perfectly possible to extort someone into giving you fair treatment (depending on how you see the default point, many of the boycotts during the civil right movement would count as extortion).

The all important default

This model seems clear; so why is it so hard to identify extortion in real life? One key issue is disagreement over the default point. Consider the following situations:

During the Cuban missile crisis, it was clear to the Americans that the default was no nuclear missiles in Cuba, and the soviets were recklessly violating this default. To the USSR, the default was obviously that countries could station nuclear missiles on their allies' territories (like the Americans were doing in Turkey). Then, once the blockade was in place and the soviet ships were on their way, the default seems to be nuclear war (meaning that practically nothing could count as extortion in this situation).

T

...
Community needs, individual needs, and a model of adult development
by Vaniver — 2016-12-17T00:18:17.718Z

Sarah Constantin wrote:

Specifically, I think that LW declined from its peak by losing its top bloggers to new projects. Eliezer went to do AI research full-time at MIRI, Anna started running CFAR, various others started to work on those two organizations or others (I went to work at MetaMed). There was a sudden exodus of talent, which reduced posting frequency, and took the wind out of the sails.

One trend I dislike is that highly competent people invariably stop hanging out with the less-high-status, less-accomplished, often younger, members of their group. VIPs have a strong temptation to retreat to a "VIP island" -- which leaves everyone else short of role models and stars, and ultimately kills communities. (I'm genuinely not accusing anybody of nefarious behavior, I'm just noting a normal human pattern.) Like -- obviously it's not fair to reward competence with extra burdens, I'm not that much of a collectivist. But I think that potentially human group dynamics won't work without something like "community-spiritedness" -- there are benefits to having a community of hundreds or thousands, for instance, that you cannot accrue if you only give your time and attention to your ten best friends.

While I agree that the trend described in the second paragraph happens (and I also dislike the effects), I have another model that I think more tightly explains why the first paragraph happened. I also think that it's important to build systems with the constraint in mind that they work for the individuals inside those systems. A system that relies on trapped, guilted, or oppressed participants is a system at risk for collapse.

So in order to create great public spaces for rationalists, we don't just need to have good models of community development. Those can tell us what we need from people, but might not include how to make people fill those slots in a sustainable way.

To explain why lots of top bloggers left at once, let me present a model of adult development, drawn from George Vaillant’s modification of Erik Erikson’s model, as discussed in Triumphs of Experience. It focuses on 6 different ‘developmental tasks,’ rather than ‘stages’ or ‘levels.’ Each has success and failure conditions associated with it; a particular component of life goes either well or poorly. They’re also not explicitly hierarchical; one co...
Weekly LW Meetups
by FrankAdamek — 2016-12-16T16:51:40.277Z

This summary was posted to LW Main on December 16th. The following week's summary is here.

The following meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Baltimore Area / UMBC Weekly Meetup: 18 December 2016 08:00PM
Bay Area Winter Solstice 2016: 17 December 2016 07:00PM
[Berlin] AI safety Berlin: 07 January 2017 04:00PM
Canberra Secular Solstice: 18 December 2016 05:00PM
NY Solstice 2016 - The Story of Smallpox: 17 December 2016 06:00PM
[NYC] Solstice Day-After Megameetup: 18 December 2016 01:00AM
San Francisco Meetup: Short Talks: 19 December 2016 06:15PM
Vienna Meetup: 17 December 2016 03:00PM
Washington, D.C.: Game Theory: 18 December 2016 03:30PM

Locations with regularly scheduled meetups: Austin, Baltimore, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Prague, Research Triangle NC, San Francisco Bay Area, Seattle, St. Petersburg, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Cambridge UK, Chicago, Cincinnati, Cleveland, Frankfurt...