Selection bias

{{logicnav}}
{{cquote|I don't understand all the negativity around Russian roulette. Everyone I ever talked to who's played it is doing fine.|||[https://www.reddit.com/r/NoStupidQuestions/comments/y06ij9/comment/irrkc14/ u/robdingo36]}}
'''Selection bias''' is the bias that occurs in a survey or [[experiment]]al data when the selection of data points isn't sufficiently [[random]] to draw a general conclusion. If selection bias isn't accounted for or acknowledged, and the results are claimed to be widely applicable, it is known as the [[spotlight fallacy]].

==Examples==

Examples of selection bias are rife amongst online polls. These polls are completed only by visitors to the site, and so those with an interest in the website's mission or ones that agree with its POV are the only ones to be surveyed. While the informality of online polls is often clear and admitted to, it's important to not draw generalised conclusions from them.

A website devoted to preventing harassment of women unsurprisingly concluded that nearly all women were victims of harassment at some point;<ref>[http://www.stopstreetharassment.org/resources/statistics/ Stop Street Harassment - Statistics]</ref> meanwhile, a poll on [[alternative medicine]] managed to receive more votes than people who read the associated article, a sure sign that something very odd was happening.<ref>[http://www.smh.com.au/national/health/vote-on-alternative-medicine-falls-victim-to-dark-arts-of-the-internet-20120210-1skv6.html Vote on alternative medicine falls victim to dark arts of the internet]</ref> Polls by the [[right-wing]] news site [[WND]] are as famous for showing that their readership have right-wing views as for having ridiculous numbers of available options.<ref>[http://forums.wnd.com/index.php?pageId=3461&pollShowResults=1 WND - Why do you think CBS News ratings are so low?] As a random example.</ref> Similarly, [[Facebook]]'s recent "Questions" feature spreads via friends in similar circles who are likely to be in a similar area – so asking "what town is the best?" will be easily slanted. Reviews, of restaurants, films, or medical services for example, may also be highly biased towards the negative because people are more likely to rant on the internet after having a bad time, than they are to rave about something after a good or adequate time.

===In science===

Scientific selection bias is a massive problem and often great pains are taken to avoid it. In [[Andrew Wakefield]]'s heavily discredited paper on the [[MMR]] vaccine, he took 12 children who had behavioural disorders and attempted to [[vaccine hysteria|link them to vaccines]]. No attempt was made, in the paper, to look at children ''without'' behavioural disorders to see if their exposure to the MMR vaccine was greater or less. This would also be an example of [[confirmation bias]].

Selection bias doesn't just apply to people. With [[meta-analyses]] of existing research and scientific literature, the cited literature may be subjected to a selection bias based on several factors. Only certain types of studies might be included or only ones indexed in certain journals – certainly meta-analysis is restricted to data that is ''published'', and [[publication bias]] is a huge problem in itself. This is why good meta-analyses clearly state their search terms and criteria and include ''all'' papers that fit these terms, so selection bias is either minimised or at least acknowledged and controlled objectively. Often in [[pseudoscience]], the literature is chosen specifically to support a point of view. In this case, it is referred to as [[cherry picking]]. 

Time scale is also a possible biasing factor. Surveys done only at certain times or for too short a period of time will bias a result so that it is not applicable to other times. This is often seen in political approval ratings, where approval during a specific time frame (such as during a crisis or scandal) is not representative of public approval over the course of several years. A clear bias can be made if sample data is restricted to a certain time frame that emphasizes a change. [[RationalWiki]]'s pageviews between March 15<sup>th</sup> and March 21<sup>st</sup> 2011 went down significantly according to Alexa.com,<ref>[http://www.alexa.com/siteinfo/rationalwiki.org# Alexa.com - Rationalwiki.org, site info] - Observing the noise, it's possible to clip the data at several points to show many different trends.</ref> but this isn't representative of a longer trend that actually shows an increase. Again, intentional massaging of data by cutting off certain time periods is cherry picking, and greatly frowned up on [[science]] and [[statistics]], but is often the way [[global warming denialism]] presents data.

===In the anti-gender movement===
{{main|Gender ideology}}

Another example of selection bias is in several research articles conducted by anti-trans activists, notably [[TERF]]s. One notable example of this is the [[Littman 2018 ROGD study]], conducted using a sample of parents frequenting websites which are biased against gender affirmation of trans youth (without even surveying the trans youth themselves). Another example is an article published by the BBC in 2021 with a conclusion that FtM transition is a form of [[conversion therapy]] and MtF transition is a means to prey on cis lesbians; however, in this case, only lesbians which are in [[TERF_glossary#.23DropTheL|a transphobic organization]] were surveyed for the research, creating an innate bias.

==See also==
*[[Cherry picking]]
*[[Confirmation bias]]
*[[Dewey Defeats Truman]], an example of how bad polling prompted a newspaper to call an election prematurely for the losing candidate
*[[Spotlight fallacy]]
*[[Push poll]]
*[[Survivorship bias]] &ndash; a form of selection bias focusing on what has survived to the present and ignoring what must have been lost.
{{otherlang
|pt=Viés de seleção}}

==References==
{{reflist}}

{{logfal}}{{Cognitive biases}}

[[Category:Imprecision fallacies]]
[[Category:Research terms]]

Categories: Imprecision fallacies, Research terms