# Long-term risks from ideological fanaticism

*Cross-posted to* [*the EA forum*](https://forum.effectivealtruism.org/posts/EDBQPT65XJsgszwmL/long-term-risks-from-ideological-fanaticism).

Summary
=======

*   History’s most destructive ideologies—like Nazism, totalitarian communism, and religious fundamentalism—exhibited remarkably [similar characteristics](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#What_do_we_mean_by_ideological_fanaticism_):
    *   epistemic and moral certainty
    *   extreme tribalism dividing humanity into a sacred “us” and an evil “them”
    *   a willingness to use whatever means necessary, including brutal violence.
*   Such *ideological fanaticism* was a major driver of eight of the ten [greatest atrocities since 1800](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_drove_most_of_recent_history_s_worst_atrocities), including the Taiping Rebellion, World War II, and the regimes of Stalin, Mao, and Hitler.
*   We [focus on ideological fanaticism](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Why_emphasize_ideological_fanaticism_over_political_systems_like_totalitarianism_) over related concepts like totalitarianism partly because it better captures terminal preferences, which plausibly matter most as we approach superintelligent AI and technological maturity.
*   Ideological fanaticism is [considerably less influential](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#The_current_influence_of_ideological_fanaticism) than in the past, controlling only a small fraction of world GDP. Yet at least hundreds of millions still hold fanatical views, many regimes exhibit concerning ideological tendencies, and the past two decades have seen widespread democratic backsliding.
*   The [long-term influence of ideological fanaticism](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#How_much_influence_will_ideological_fanaticism_have_in_the_long_term_future_) is uncertain. Fanaticism faces many disadvantages including a weak starting position, poor epistemics, and difficulty assembling broad coalitions. But it benefits from greater willingness to use extreme measures, fervent mass followings, and a historical tendency to survive and even thrive amid technological and societal upheaval. Beyond complete victory or defeat, multipolarity may persist indefinitely, with fanatics permanently controlling a non-trivial fraction of the universe, potentially using superintelligent AI to entrench their rule.
*   Ideological fanaticism [increases existential risks and risks of astronomical suffering](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_existential_and_suffering_risks) through multiple mutually-reinforcing pathways.
    *   Ideological fanaticism [exacerbates most common causes of war](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_the_risk_of_war_and_conflict). Fanatics' sacred values and outgroup hostility often preclude compromise, while their irrational overconfidence and differential commitment credibility make bargaining failures more likely. Fanatics may even welcome conflict, rather than viewing it as a costly last resort.
    *   [Fanatical retributivism may lead to astronomical suffering](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Fanatical_retributivism_may_lead_to_astronomical_suffering). In our survey of 1,084 people, 11–14% in the US, UK, and Pakistan agreed that if hell didn't exist, we should create it to punish evil people with extreme suffering forever, and separately selected 'forever' when asked how long evil people should suffer unbearable pain, while also stating that at least 1% of humanity deserves this fate. Rates ranged from 19–25% in China, Saudi Arabia, and Turkey. Similar questions showed roughly comparable patterns. Advanced AI could enable fanatics to actually instantiate such preferences.
    *   Certain of their righteousness, fanatics resist further reflection and seek to lock in their current values, which [threatens long-reflection-style proposals](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_could_undermine_long_reflection_style_frameworks_and_AI_alignment) that envision humanity carefully deliberating on how to achieve its potential. Viewing compromise and cooperation as betrayal, fanatics also seem more likely to oppose moral trade and use hostile bargaining tactics. Their intolerant ‘fussy’ preferences may regard almost all configurations of matter as immoral, including those containing vast flourishing, potentially resulting in astronomical waste.
    *   [AI intent alignment](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#AI_alignment_may_not_solve_the_fanaticism_problem_either) alone won't help if the human principal is fanatical or malevolent: an AI aligned with Stalin probably won't usher in utopia. Fanatics may reflectively endorse their existing values, even after preference idealization. The worst futures may therefore arise from misuse of intent-aligned AI by ideological fanatics, rather than from misaligned AI.
    *   Ideological fanaticism also poses [other risks](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_could_worsen_many_other_risks), including extreme optimization and differential intellectual regress.
*   Most relevant [interventions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Interventions), while not novel, fall into two overlapping categories.
    *   [Political and societal interventions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Societal_or_political_interventions) include strengthening and safeguarding liberal democracies, reducing political polarization, promoting anti-fanatical principles like classical liberalism, and fostering international cooperation.
    *   [AI-related interventions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Artificial_intelligence_related_interventions) appear higher-leverage. Compute governance and information security can reduce the likelihood that transformative AI falls into the hands of fanatical and malevolent actors. Preventing AI-enabled coups could be particularly important given such actors' propensity for power grabs. Other promising interventions include proactively using AI to improve epistemics at scale, developing fanaticism-resistant post-AGI governance frameworks, and making transformative AIs themselves less fanatical—e.g., by guiding their character towards wisdom and benevolence.

What do we mean by ideological fanaticism?
==========================================

Consider some of history’s worst atrocities. In the Holocaust, the Nazi regime constructed an industrial apparatus to systematically exterminate six million Jews and others deemed 'subhuman'. During the Great Purge, Stalin's secret police tortured hundreds of thousands until they confessed to fictitious acts of treason, before executing them. A century earlier, the Taiping Rebellion claimed over twenty million lives as followers of a self-proclaimed messiah waged a holy war to cleanse the world of 'demons'.

These and many other horrors were substantially driven by three types of fanatical ideologies: *fascist ethno-nationalism*, *totalitarian communism*, and *religious fundamentalism*. In fact, these three fanatical ideologies were arguably responsible for the majority of deaths from mass violence since 1800, as we explore [below](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_drove_most_of_recent_history_s_worst_atrocities).

While the specific beliefs of these and other destructive ideologies have varied dramatically, the underlying patterns in thought, emotion, and behavior were remarkably similar. Numerous frameworks could summarize these dynamics, but we focus on three mutually reinforcing characteristics—the **fanatical triad**—because they arise in virtually all relevant cases while remaining simple and memorable:

1.  Absolute epistemic and moral certainty;
2.  Manichean tribalism, where humanity is divided into a sacred 'us' and an irredeemably evil 'them';
3.  A willingness to use any means necessary, including brutal violence.

While the term “fanatical triad” is our own, each of the three characteristics draws upon well-established academic concepts, including dogmatism, tribalism, and totalitarianism. (See [Appendix A](https://docs.google.com/document/d/1Cd8tjMniyV7-p_H__CRb1x0YWYDv1HHR6y8aLMrL5Zo/edit?tab=t.0#heading=h.syrjccmpapfn) for an extensive overview connecting each “fanatical triad” component to existing scholarship and historical case studies.)

Ideological fanaticism closely resembles 'extremism', but that term typically describes anti-establishment movements at the *periphery* of society (Bötticher, [2017](https://web.archive.org/web/20180411101040id_/http://www.terrorismanalysts.com/pt/index.php/pot/article/viewFile/623/1228)).[^-bxXhDCzXq8bJcuQWw-1]In contrast, we are also concerned with the risk of fanatical ideologies commanding *mainstream* adherence and capturing state power. 'Fanaticism' also better connotes the zealous, uncompromising hatred we wish to emphasize. Our term should not be confused with ‘Pascalian’ expected value [fanaticism](https://forum.effectivealtruism.org/topics/fanaticism).[^-bxXhDCzXq8bJcuQWw-2]

One overarching characteristic of the fanatical worldview is black-and-white thinking (good vs. evil, us vs. them) with no room for nuance. Let's not make the same mistake. Like most phenomena, ideological fanaticism exists on a *continuum*. Furthest from fanaticism are those enlightened few who, following reason and evidence, act with benevolence towards all. A vast middle ground is occupied by religious traditionalists, hyper-partisan activists, conspiracy theorists, and many others. Indeed, a *mild* form of ideological fanaticism is arguably human nature: we are all somewhat prone to overconfidence, motivated reasoning, and tribalistic in-group favoritism and outgroup discrimination (e.g., Kunda, 1990; Diehl, 1990; Hewstone et al., 2002).[^-bxXhDCzXq8bJcuQWw-3]But ideological fanatics take such traits to extremes.

I. Dogmatic certainty: epistemic and moral lock-in
--------------------------------------------------

The most ardent fanatics are utterly convinced they have found the one infallible authority in possession of ultimate truth and righteousness; they are textbook *dogmatists* (Rokeach, 1960). For religious fundamentalists, this is usually a holy book containing the divine revelation of God and his prophets. For Nazis, it was Hitler's [*Führerprinzip*](https://en.wikipedia.org/wiki/F%C3%BChrerprinzip) (Leader Principle), codified by Rudolf Hess’s declaration that “the Führer is always right”. Similarly, many communist revolutionaries essentially placed absolute faith in foundational texts like Marx’s *Das Kapital*, or in the Party itself (Montefiore, 2007). “[Angkar](https://en.wikipedia.org/wiki/Communist_Party_of_Kampuchea#Angkar) is an organization that cannot make mistakes” was a key slogan of the Khmer Rouge.[^-bxXhDCzXq8bJcuQWw-4]

For the fanatic, any doubt or deviation from these dogmas is not only wrong but evil, culminating in a total “*soldier mindset”* which defends the pre-existing ideology at all costs (Galef, [2021](https://www.amazon.com/Scout-Mindset-People-Things-Clearly/dp/B07RP27XJP)). This necessitates abandoning even the most basic form of empiricism by "rejecting the evidence of one’s own eyes and ears", to paraphrase [Orwell](https://www.goodreads.com/quotes/8205321-the-party-told-you-to-reject-the-evidence-of-your).[^-bxXhDCzXq8bJcuQWw-5]The fanatic is thus essentially [incorrigible](https://www.lesswrong.com/w/corrigibility-1) and has no epistemic or [moral uncertainty](https://forum.effectivealtruism.org/topics/moral-uncertainty), even in the face of widespread opposition (Gollwitzer et al., 2022).[^-bxXhDCzXq8bJcuQWw-6]

II. Manichean tribalism: total devotion to us, total hatred for them
--------------------------------------------------------------------

Building on tribalistic instincts innate to human nature (Clark et al., [2019](https://journals.sagepub.com/doi/abs/10.1177/0963721419862289)), such dogmatic certainty both reinforces and is reinforced by an extreme form of “Manichean tribalism”, which views the world as a cosmic conflict between good and evil.[^-bxXhDCzXq8bJcuQWw-7]Examples include the racial struggle between ‘Aryans’ and ‘inferior’ races (Nazism), the revolutionary struggle against class enemies (communism), or the spiritual battle between God and the forces of Satan (religious fundamentalism).

As the fanatic’s in-group and ideology become their sole source of belonging and meaning, their individual identity fuses with the collective, resulting in all-consuming devotion to the cause and submission to its leaders[^-bxXhDCzXq8bJcuQWw-8](Katsafanas, 2022b; Varmann et al., 2024). This is often further amplified through group dynamics, with members outbidding each other to prove their loyalty by embracing increasingly extreme views and punishing the slightest dissent. The most devoted fanatics eagerly die for the cause, as seen with Japanese kamikaze pilots or religious suicide bombers (Atran & Ginges, 2015). Nazism, for instance, was anchored in “uncritical loyalty” to Hitler (Hess, [1934](https://research.calvin.edu/german-propaganda-archive/hess5.htm)) and [oaths](https://en.wikipedia.org/wiki/Oaths_to_Hitler) pledging unconditional “obedience unto death”. Similarly, millions of communists were true believers, exemplified by the Red Guards who pledged to “defend Chairman Mao and his revolution to the death” (Chang, 2008; Dikötter, 2016).

Fueling this extreme devotion is an equally intense hatred and resentment of a [demonized](https://www.conspicuouscognition.com/p/demonizing-narratives) outgroup (Szanto, 2022; Katsafanas, 2022a). This outgroup is often expansive, potentially including anyone merely disagreeing with a subset of the ideology’s claims—such as Stalin executing Trotskyists or ISIS murdering other Muslims for insufficient piety. Driven in part by [paranoia](https://www.conspicuouscognition.com/p/strategic-paranoia-when-irrational) and conspiratorial thinking, fanatics often scapegoat this outgroup as the source of nearly all problems. Typically, this enemy is believed to deserve extreme *punishment*, ranging from [torture](https://en.wikipedia.org/wiki/Tuol_Sleng_Genocide_Museum#Torture_and_extermination) and systematic extermination, to religious visions of hell, where nonbelievers are damned to eternal torment.

Supercharging moral instincts relating to purity and disgust (cf. Haidt, 2012), fanatics may reject all compromise as betrayal of their inviolable, sacred values (Tetlock, [2003](https://pubmed.ncbi.nlm.nih.gov/12860191/)), often resulting in a zero-sum mentality where the only acceptable outcome is the ideology’s total victory.

III. Unconstrained violence: any means necessary
------------------------------------------------

> “Any violence which does not spring from a spiritual base, will be wavering and uncertain. It lacks the stability which can only rest in a *fanatical* worldview.”  
> \- Adolf Hitler, 1925

Most humans hesitate to commit violence due to various guardrails like instinctive harm aversion, social norms, empathy, and compassion for others’ suffering. To further reinforce these better angels of our nature, humanity painstakingly developed complex moral and institutional frameworks, like virtue ethics, deontology, separation of powers, and the rule of law.[^-bxXhDCzXq8bJcuQWw-9]

Fanatics toss all that malarkey out the window. They are certain that they champion the forces of righteousness in a total war against evil. Their victory will redeem this 'vile world' (Stankov et al., 2010) and usher in utopia, whether it be a perfect communist society, a Thousand-Year Reich, or religious paradise. These existential stakes justify *any means necessary*, no matter how extreme.

In fact, some fanatics even invert the entire moral paradigm, glorifying what others find most abhorrent. Compassion, honesty, and moderation[^-bxXhDCzXq8bJcuQWw-10]become weakness; law-breaking, deceit, and violence become virtues.[^-bxXhDCzXq8bJcuQWw-11]ISIS fighters, for instance, filmed themselves burning their victims alive and proudly distributed the footage.

With enough power, fanatics can achieve their vision: totalitarian control over society that eliminates individual liberty and forces everyone to conform to their ideology—using censorship, propaganda, and even mass murder if necessary (Arendt, 1951).[^-bxXhDCzXq8bJcuQWw-12]

Fanaticism as a multidimensional continuum
------------------------------------------

Ideological fanaticism is not just a single sliding scale. Rather, it is *multidimensional*, that is, people can exhibit different levels of each fanatical triad component. The most dangerous form of ideological fanaticism requires elevated levels of *all* *three* characteristics. A hypothetical ‘Bayesian Nazi’, for instance, would lack absolute certainty and thus remain open to changing his mind. Similarly, without Manichean hatred, there is no motivation for mass harm, and without a willingness to use violence, even the most hateful beliefs remain inert.

Nor are fanatical *movements* monolithic.[^-bxXhDCzXq8bJcuQWw-13]While their leaders often were malignant narcissists, their followers are frequently ordinary people desperately seeking meaning and certainty in a chaotic, disappointing world (Hoffer, 1951; Kruglanski et al., 2014; Tietjen, 2023). Not all are true believers, either: some merely conform to group pressure, others are cynical opportunists, and many fall somewhere in between.[^-bxXhDCzXq8bJcuQWw-14]Many fanatics are capable of eventual [reform](https://en.wikipedia.org/wiki/Jihad_Rehab), so we should not demonize them as irredeemably evil.

Finally, though related, we shouldn't confuse fanaticism with strong moral convictions (Skitka et al., 2021).[^-bxXhDCzXq8bJcuQWw-15]Martin Luther King Jr., for instance, held radically progressive views for his time, but remained open to evidence, sought coalition-building across racial lines, and was explicitly opposed to violence.

Ideological fanaticism drove most of recent history's worst atrocities
======================================================================

One reason we fear ideological fanaticism may pose substantial future risks is its grim historical track record. Ideological fanaticism seems to have been a major driver of *eight* of the ten worst atrocities since 1800.[^-bxXhDCzXq8bJcuQWw-16]In the following table, we only included events involving *intentional*[^-bxXhDCzXq8bJcuQWw-17]mass killing, excluding accidental famines and pandemics[^-bxXhDCzXq8bJcuQWw-18], for [reasons discussed below](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Intentional_versus_natural_or_accidental_harm).

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/a8df90cc5750eec7409c74b7c9e98d77cd88032e9ee5a94c.png)

This table is more informative than it may appear, as atrocity deaths follow a heavy-tailed distribution: of the [116 events since 1800](https://docs.google.com/spreadsheets/d/1b6YprwBXmg2mMcRXW8t-VGo-1upYtbTOHnz5jpT9uiA/edit?gid=0#gid=0) with death tolls exceeding 100,000 (totaling 266 million deaths), the ten worst atrocities alone account for 181 million deaths, or *68%* of the total, and thus provide disproportionate explanatory value.

To be clear, these death toll estimates are uncertain (especially for the Dungan Revolt). We also made several debatable judgment calls regarding timeframe, categorization, and grouping (e.g., WWII could be one entry instead of being split into three). However, we're quite confident that our core finding is robust: ideological fanaticism contributed to the majority of deaths from mass violence since 1800.[^-bxXhDCzXq8bJcuQWw-19]

See [**Appendix B**](https://docs.google.com/document/d/12DoA_vQER2wvMSuvlnHf1RmuDFPXmZuM7GYvbdgIWOg/edit?tab=t.0) for extensive discussion of our methodology and other atrocities that didn't make the top ten. Three omissions stand out for their scale and horror: the [**Atlantic slave trade**](https://en.wikipedia.org/wiki/Atlantic_slave_trade) and [**Arab**](https://en.wikipedia.org/wiki/Arab_slave_trade)**/**[**Islamic slave trade**](https://en.wikipedia.org/wiki/History_of_slavery_in_the_Muslim_world) each killed over 15 million people, but mostly before 1800. For various methodological and pragmatic reasons, we also excluded systematic [**killings of animals**](https://ourworldindata.org/how-many-animals-get-slaughtered-every-day) like factory farming, which kills *hundreds of billions* of animals annually—arguably the largest moral catastrophe of our time.

Of course, no single factor fully explains any historical atrocity. In addition to ideological fanaticism, other crucial causes and risk factors include political and economic instability (e.g., Weimar Germany), power-seeking and competition between individuals and groups (present in essentially all atrocities), inequality and exploitation (e.g., in Congo Free State), historical grievances, and individual leaders' personalities.[^-bxXhDCzXq8bJcuQWw-20]Moreover, these factors often interact with ideological fanaticism in mutually reinforcing ways: political and economic instability, for instance, make fanatical ideologies more appealing, and fanatical ideologies often further increase economic and political chaos.

Overall, for eight of the ten atrocities, our sense is that ideological fanaticism is at least among the handful of most important causal factors.[^-bxXhDCzXq8bJcuQWw-21]Even the two non-fanatical entries in our table—Leopold's Congo (primarily driven by greed) and World War I (primarily geopolitical competition)—were at least partly driven by forms of ideological fanaticism: colonial racism and fervent nationalism, respectively.

Death tolls don’t capture all harm
----------------------------------

While deaths correlate with many other harms, such as deprivation, oppression, and torture,[^-bxXhDCzXq8bJcuQWw-22]extreme suffering can occur even when death tolls are relatively low. We nonetheless chose deaths as our metric because they are easily measurable—certainly more so than trying to calculate counterfactual net changes in quality-adjusted life years across poorly-documented historical periods.

Consider North Korea. This totalitarian regime has been responsible for "only" a few hundred thousand deaths in recent decades. Yet the lives of the vast majority of its 26 million inhabitants are filled with misery. Most are extremely poor; nearly half are [malnourished](https://data.worldbank.org/indicator/SN.ITK.DEFC.ZS). From early childhood, citizens are indoctrinated and denied basic freedoms of movement and information. To crush dissent, the [regime operates](https://en.wikipedia.org/wiki/Report_of_the_Commission_of_Inquiry_on_Human_Rights_in_the_Democratic_People%27s_Republic_of_Korea) a network of political prison camps where forced labor, torture, physical abuse, and summary executions are standard practice. The entire population is essentially a captive workforce, terrorized by the constant threat of violence and imprisonment.

In contrast, South Koreans enjoy vastly greater freedom and are more than [20 times wealthier](https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(PPP)_per_capita). The differences that have emerged since the two countries split in the mid-20th century serve almost as a natural experiment demonstrating the power of ideological fanaticism (among other factors[^-bxXhDCzXq8bJcuQWw-23]) to inflict immense suffering, even when it doesn't result in millions of violent deaths.

Intentional versus natural or accidental harm
---------------------------------------------

We focus on *intentional* deaths because they are most *revealing of terminal preferences*, which in turn are most *predictive of future harm*.

**Intentional deaths are most revealing of terminal preferences**: Had we included all deaths, our table would be dominated by age-related and infectious *diseases*, *accidents*, and *starvation*; categories that tell us little about intentions. This distinction also reflects common moral intuitions and the law: murder is worse than manslaughter partly because the former reveals intentionality (“malice aforethought”) and is much more predictive of future harm.

[**Terminal preferences are more predictive of future harm**](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Terminal_values_may_matter_independently_of_political_systems__especially_with_AGI)**:** From a longtermist perspective, the distinction between intentional and non-intentional harm is even more important. If civilization survives for long enough, continued scientific progress will likely lead to the invention of many consequential technologies—like superintelligent AI, advanced spaceflight, or nanotechnology. A civilization at ‘technological maturity’[^-bxXhDCzXq8bJcuQWw-24]would have tremendous control over the universe, so outcomes might become [increasingly determined by the values of powerful agents](https://forum.effectivealtruism.org/posts/SrjrytewFfvZ8pCm4/the-choice-transition#The_ascent_of_choice), rather than by natural processes or unintended consequences. We can already observe early signs of this trajectory: deaths from infectious diseases and starvation, for instance, have decreased dramatically since 1800, largely due to humanity’s increasing technological capabilities. Thus, while natural and accidental harms still dominate at *present*, intentional harm will plausibly become the dominant source of future harm. (For a related but more complicated categorization, see also the distinction between [agential, incidental, and natural harm](https://longtermrisk.org/beginners-guide-to-reducing-s-risks/#3_Classes_of_s-risks_and_their_potential_causes)[^-bxXhDCzXq8bJcuQWw-25].)

Why emphasize ideological fanaticism over political systems like totalitarianism?
=================================================================================

Most previous discussions of socio-political existential risk factors and historical atrocities have tended to focus on concepts like (stable) totalitarianism (e.g., Arendt, 1951; Caplan, 2008; Clare, [2025](https://80000hours.org/problem-profiles/risks-of-stable-totalitarianism/)), autocracy (Applebaum, 2024), authoritarianism (e.g., MacAskill & Moorhouse, [2025](https://www.forethought.org/research/preparing-for-the-intelligence-explosion); Aird, 2021; Adorno, 1950), and safeguarding democracy (e.g., Koehler, [2022](https://80000hours.org/problem-profiles/liberal-democracy/); Garfinkel, [2021](https://forum.effectivealtruism.org/posts/TMCWXTayji7gvRK9p/is-democracy-a-fad); Yelnats, 2024)[^-bxXhDCzXq8bJcuQWw-26]. So why focus on ideological fanaticism instead of these more established concepts?

One major difference is that the above concepts all primarily describe *political* *systems*. We can view these on a *continuum* ranging from *open* to *closed* societies (Popper, [1945](https://en.wikipedia.org/wiki/The_Open_Society_and_Its_Enemies)). Following Linz (2000), liberal democracies occupy the ‘open’ end of this spectrum—featuring competitive elections, civil liberties, and institutional checks on power. Authoritarianism occupies the middle ground, concentrating power in a single leader or party while tolerating limited private autonomy. Totalitarianism, such as in Stalin's USSR or wartime Nazi Germany, represents the ‘closed’ endpoint: authoritarianism *plus* complete ideological control, mass mobilization, and the elimination of almost all private life. While all totalitarian regimes are necessarily authoritarian, most authoritarian regimes never slide all the way down this spectrum to totalitarianism.

In contrast, our focus is on the underlying *mindset* and dangerous *terminal values* that characterize ideological fanatics.[^-bxXhDCzXq8bJcuQWw-27]As we'll argue, these factors may be more important from a longtermist perspective[^-bxXhDCzXq8bJcuQWw-28]because they i) can [create and change political systems](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Values_change_political_systems__Ideological_fanatics_seek_totalitarianism__not_democracy) and, ii) pose risks that may [emerge independently](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Terminal_values_may_matter_independently_of_political_systems__especially_with_AGI) of specific forms of government, especially with AGI. Therefore, although there is substantial overlap between our approach and prior work (especially on totalitarianism[^-bxXhDCzXq8bJcuQWw-29]), we believe that the lens of ideological fanaticism is nevertheless valuable.

Fanatical and totalitarian regimes have caused far more harm than all other regime types
----------------------------------------------------------------------------------------

First, let’s ground our discussion in empirical data. We analyzed deaths from mass violence since 1800 by both regime type (totalitarian, authoritarian, democratic, non-governmental) and motivation (fanatical vs. non-fanatical):

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/95fb87d13b8c73ce2fc0b60e5ff2b8e940ed2597e4c00b1a.png)

History is messy and we aren’t historians, so we remain uncertain about many of our classifications—see [here](https://docs.google.com/spreadsheets/d/1b6YprwBXmg2mMcRXW8t-VGo-1upYtbTOHnz5jpT9uiA/edit?usp=sharing) for our data, reasoning and methodology.[^-bxXhDCzXq8bJcuQWw-30]That said, this data suggests that we should be most concerned with totalitarianism and ideological fanaticism (most commonly in combination), as these were involved in the majority of all deaths from mass violence:[^-bxXhDCzXq8bJcuQWw-31]Totalitarian regimes accounted for 60% of all deaths (153M), while fanatical actors across all regime types accounted for *69%* of total deaths (174M). Among authoritarian regimes, those driven by fanatical ideologies were likewise disproportionately destructive. Overall, non-fanatical actors were responsible for only 16% of total deaths (40M) and democracies for less than 3%.

### Authoritarianism as a risk factor

Of course, we shouldn't ignore authoritarianism, which still accounted for 30% of all deaths (76m). Authoritarianism is also a *key risk factor* for totalitarianism, whereas democratic institutions serve as protective safeguards. Moving from authoritarianism to totalitarianism is comparatively easy: it would primarily require the autocrat (and perhaps some key members of the ruling elite) to strengthen the machinery for centralized control that is already in place. In contrast, transforming a democracy into a totalitarian state is a much more arduous undertaking. It requires dismantling an entire system of formal checks and balances as well as subverting democratic norms and public expectations of personal liberty.

Values change political systems: Ideological fanatics seek totalitarianism, not democracy
-----------------------------------------------------------------------------------------

As our data shows, the overlap between totalitarianism and ideological fanaticism is substantial: of the 174M deaths caused by fanatical actors, almost *80%* (138M) came from totalitarian regimes. So why not drop the fanaticism lens and focus only on totalitarianism? One reason is that ideological fanaticism is plausibly causally upstream: fanatics seek to create totalitarian political systems, more so than the reverse.

Consider the historical evidence. It seems clear that Hitler, Lenin, Stalin, and Mao[^-bxXhDCzXq8bJcuQWw-32]—and the fanatical ideologies they championed—were (among many other factors) major causal forces behind the creation of history's worst totalitarian regimes: Nazi Germany, the Soviet Union, and Maoist China. Crucially, all of these individuals were likely [ideological fanatics years *before* seizing power](https://docs.google.com/document/d/1-QZFfvC8s1gGsgBYo8XQ1tkHKM3PIQGyc-0RjTHHPCM/edit?tab=t.0#heading=h.kdr15c7oceuz). Hitler already exhibited the fanatical triad in *Mein Kampf*, published almost a decade before rising to power (1925): absolute certainty about racial theories, Manichean division of humanity into superior Aryans versus subhuman enemies, and explicit advocacy for violence. Lenin declared that "the Marxist doctrine is omnipotent because it is true" (1913), and advocated "a desperate, bloody war of extermination" (1906). Mao likewise demonstrated dogmatic certainty and embraced violence as necessary for revolutionary transformation long before gaining power. The totalitarian regimes they built were consequences of these pre-existing convictions.

This pattern isn't coincidental because ideological fanatics *require* totalitarian systems to achieve their vision. If you believe that a large portion of humanity is irredeemably evil and deserves extreme punishment or extermination, granting them political rights, personal liberty, and equal standing before the law becomes morally abhorrent. Ideological fanaticism and democratic principles are therefore structurally incompatible.[^-bxXhDCzXq8bJcuQWw-33]Empirical evidence supports such theoretical arguments. Ideological extremists (on both the left and the right) show less support for democracy[^-bxXhDCzXq8bJcuQWw-34](Torcal & Magalhães, 2022) and are more likely to endorse authoritarian policies (Manson, 2020).[^-bxXhDCzXq8bJcuQWw-35]

Terminal values may matter independently of political systems, especially with AGI
----------------------------------------------------------------------------------

Probably even more importantly, the *mindset* of ideological fanatics seems to play a major role for many of the [long-term risks we’re most concerned about](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_existential_and_suffering_risks). As we’ll discuss later, political systems alone don’t fully explain irrationality or sacred values as major [causes of war](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_the_risk_of_war_and_conflict). Nor would they explain acts of torture motivated by [fanatical retributivism](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Fanatical_retributivism_may_lead_to_astronomical_suffering), value lock-in [threatening a long reflection](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_threatens_collective_moral_deliberation), or [insatiable moral ambitions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_may_give_rise_to_extreme_optimization_and_insatiable_moral_desires).

Historically, a single human or small groups of humans couldn’t cause much harm unless they were in control of a state, but forthcoming technologies like transformative AI could drastically change this: a single fanatical human (or a small group) in control of superintelligent intent-aligned AI—or a superintelligent misaligned AI with fanatical values—could potentially amass enormous power and cause astronomical harm. This is possible even in a world in which totalitarian or other tyrannical systems of government no longer exist. The key issue is that sufficiently powerful technology can decouple capacity for harm from state control.

Fanaticism’s connection to malevolence (dark personality traits)
----------------------------------------------------------------

The threat posed by [*malevolent* actors](https://forum.effectivealtruism.org/posts/LpkXtFXdsRd4rG8Kb/reducing-long-term-risks-from-malevolent-actors)—our shorthand to refer to individuals with elevated [*dark traits*](https://darkfactor.org/) like narcissism, Machiavellianism, psychopathy, or sadism—is related to but distinct from the risks posed by ideological fanatics. Not all fanatics have highly elevated dark traits and many commit horrific acts because of sincere moral convictions.[^-bxXhDCzXq8bJcuQWw-36]Conversely, many malevolent individuals weren’t ideological fanatics, e.g., serial killers like Ted Bundy. One key difference is that many ideological fanatics are willing to sacrifice and even die for their cause, while malevolent individuals are generally self-centered and egoistic.

However, ideological fanaticism and malevolence do have considerable overlap:

1.  Elevated dark tetrad traits make one more susceptible to ideological fanaticism. For instance, psychopaths, malignant narcissists, or sadists are naturally more inclined to feel total hatred for their enemies and commit acts of brutal violence. In fact, those with elevated dark traits may be attracted to belief systems that provide justifications for such actions. Empirical research shows that dark traits are associated with [increased support](https://forum.effectivealtruism.org/posts/LpkXtFXdsRd4rG8Kb/reducing-long-term-risks-from-malevolent-actors#How_important_are_situational_factors_and_ideologies_compared_to_personality_traits_) for extremist ideologies.
2.  Relatedly, the *leaders* of fanatical ideologies almost always exhibit highly elevated dark traits (Stalin, Mao, Hitler, Pol Pot, etc.). Some of these traits, especially narcissism, plausibly drive such figures to invent fanatical ideologies or repackage existing ones[^-bxXhDCzXq8bJcuQWw-37], while psychopathy and Machiavellianism enable the ruthless violence often required to lead them. Concerningly, fanatical ideologies can provide such malevolent individuals with millions of devoted *followers* who, blinded by absolute conviction and loyalty, fail to recognize the malevolent traits of the leaders they support.[^-bxXhDCzXq8bJcuQWw-38]
3.  Both ideological fanatics and malevolent actors are unusual in that they often *intrinsically* value others’ suffering*,* and may even *reflectively* endorse this.[^-bxXhDCzXq8bJcuQWw-39]Ideological fanaticism and malevolence are also major risk factors for [conflict](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_the_risk_of_war_and_conflict) and subsequent threats—another main source of [agential](https://longtermrisk.org/beginners-guide-to-reducing-s-risks/#32_Agential_s-risks) s-risks (Clifton, [2020](https://longtermrisk.org/research-agenda)). Total future expected disvalue is plausibly dominated by agential s-risks[^-bxXhDCzXq8bJcuQWw-40], which makes ideological fanaticism and malevolence extremely dangerous.[^-bxXhDCzXq8bJcuQWw-41]
4.  Malevolence and ideological fanaticism thus both represent risks that arise from “[within humanity](https://forum.effectivealtruism.org/posts/ahxGTaNdk6G5YD9vs/david_althaus-s-shortform?commentId=KKiCKPGrgRc5RBmPK)” and thus have worrying implications for AI alignment: “aligned AI” sounds great until one considers that this could include AIs aligned with [fanatical or malevolent principals](https://www.lesswrong.com/posts/cnv5g7jCWLw9LYKxa/an-even-deeper-atheism-3#Human_paperclippers_). Consequently, the very worst outcomes may not arise from *misaligned* *AI*, but rather from the catastrophic *misuse of intent-aligned AI* by fanatical or malevolent actors (or the development of AIs that somehow inherit the malevolent and fanatical values of their creators).[^-bxXhDCzXq8bJcuQWw-42]
5.  Many interventions reduce risks from both malevolence and ideological fanaticism, like preventing (AI-enabled) coups, improving compute governance and information security, or safeguarding liberal democracy.

We see both as important but want to highlight ideological fanaticism as an additional but related risk factor.

The current influence of ideological fanaticism
===============================================

To better understand how much influence fanatical ideologies might wield over the future—our ultimate concern and the topic of the next section—we first briefly discuss their influence in the *present*. We begin by placing today's situation in historical context.

Historical perspective: it was much worse, but we are sliding back
------------------------------------------------------------------

The world is overall far less fanatical today than in earlier times, perhaps especially during some periods of the Middle Ages where religious fanaticism, dogmatism, public torture and execution were common, and virtually all of humanity lived under absolutist rulers. Democracy and human rights as we understand them essentially didn't even exist.[^-bxXhDCzXq8bJcuQWw-43]

More recently, the early 1940s marked a harrowing nadir for humanity. Nazism controlled most of Europe, Stalin's totalitarian communism dominated the Soviet Union, Imperial Japan was waging a brutal war of conquest, and radical communists under Mao's leadership were gaining power in China. Liberal democracies everywhere seemed about to be swept away by the rising totalitarian tide. The situation felt so hopeless to the famous humanist [Stefan Zweig,](https://en.wikipedia.org/wiki/Stefan_Zweig) that he took his own life in early 1942. In his suicide note, he wrote of his despair at the triumph of barbarism that had destroyed the tolerant, cosmopolitan Europe he chronicled in *The World of Yesterday*. And Zweig died without even knowing the full industrial scale of the Holocaust.

Fortunately, however, World War Two wasn’t the end for liberal, enlightenment values. On the contrary, the post-war period saw democracy's gradual expansion, accelerating after the Soviet Union's collapse. In the post-Cold War era of the 1990s and early 2000s, liberal optimism reached its zenith, encapsulated by Francis Fukuyama’s international best-seller *The End of History and The Last Man* (1992), which hypothesized that, following the defeat of communism and fascism, civilization might be nearing the end of history due to “the universalization of Western liberal democracy as the final form of human government”.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/96716652b76b815c4dc480b549e09dc7ccf34dbec064755f.png)

*Graph from Herre et al. via Our World in Data (*[*2013*](https://ourworldindata.org/grapher/varieties-democracy-vdem?country=~World+%28population-weighted%29)*)*

Various democracy indices (like [V-Dem](https://www.v-dem.net/)’s depicted above) seemed to back up Fukuyama’s proclamation, rising steadily throughout the 1990s and early 2000s.[^-bxXhDCzXq8bJcuQWw-44]However, since about 2004, these same democracy scores have declined across multiple dimensions, with many countries “backsliding” towards illiberalism and authoritarianism. While the world is still in hugely better shape than in the 1940s, it seems that “history” has far from ended.

Estimating the global scale of ideological fanaticism
-----------------------------------------------------

How many ideological fanatics are out there? Formulating a precise estimate is nearly impossible, as fanaticism exists on a multidimensional continuum with no clear demarcations, and because good data is sparse. Therefore, the numbers below are merely rough approximations based on limited research. For brevity, we focus here on support for ideological violence as the best proxy for ideological fanaticism. Endorsing ideological violence usually presupposes dogmatism and tribalistic hatred, since one needs to confidently believe the hated target group is deserving of punishment in order to justify violence. Another limitation is that we mostly rely on survey data[^-bxXhDCzXq8bJcuQWw-45], not actual behavior; this may overestimate fanaticism (if claimed support for violence is mere “cheap talk”) or underestimate it (“[social desirability bias](https://en.wikipedia.org/wiki/Social-desirability_bias)”).

What seems clear is that the same three fanatical ideologies examined earlier—religious fundamentalism, totalitarian communism, and extreme ethno-nationalism—remain by far the most influential.

**Christian fundamentalism**. For brevity, we focus on the US (the largest Christian country) and Sub-Saharan Africa (where Christianity is growing fastest). In the US, around 20% of American adults (roughly 50 million) agree that "God has called Christians to exercise dominion over all areas of American society" (2023 PRRI/Brookings survey, p.4). Similarly, nearly a quarter of US adults (Pew Research Center, 2022) say the Bible should have "a great deal of influence" on US laws. [Extrapolating](https://docs.google.com/document/d/1ATsPbDSPFskcUImVdFw9gL6lth9VbobXcac3oa2cprY/edit?tab=t.0#bookmark=id.g1oes471f4u) data from a [2008-2009 Pew survey](https://www.pewresearch.org/wp-content/uploads/sites/7/2010/04/sub-saharan-africa-full-report.pdf) (p.47) of 19 African countries, we estimate that roughly 15% of Africa’s 700 million Christians (roughly 100 million) believe that violence against civilians in defense of Christianity can often or sometimes be justified. Christians in Europe and Latin America may plausibly be less fanatical on average. Still, perhaps **200-250 million** Christians worldwide (8-10%) could reasonably be classified as ideological fanatics.

**Radical Islam.** While the vast majority of the world's 2 billion Muslims are peaceful, a substantial minority holds radical beliefs. According to a [2013 Pew Research survey](https://www.pewresearch.org/religion/wp-content/uploads/sites/7/2013/04/worlds-muslims-religion-politics-society-full-report.pdf) spanning 39 countries, around 350 million Muslims support the death penalty for leaving Islam—arguably showcasing all three fanatical triad components at once. These figures represent a lower bound, because several Muslim-majority countries with strict Islamic governance (including Saudi Arabia and Iran) were not surveyed. While clear majorities in most surveyed countries said that suicide bombing in defense of Islam is rarely or never justified, around 150 million Muslims worldwide believe it is sometimes or often justified. The Gallup World Poll, comprising tens of thousands of interviews across 35+ nations between 2001 and 2007, found that 7% of the world's Muslims considered the 9/11 attacks "completely justified," rising to approximately 37% when including those who deemed them at least partially justified (Atran & Ginges, 2015; Satloff, 2008). Accounting for unsurveyed countries and assuming total overlap between survey questions, perhaps around **400 million** Muslims could reasonably be classified as ideologically fanatical.

**Extremist ethno-nationalism**. Due to their nature, ethno-nationalist views are typically country-specific and thus fragmented.[^-bxXhDCzXq8bJcuQWw-46]Despite this, moderately ethno-nationalistic views which endorse the superiority of a given ethnic, cultural or racial group seem very widespread, perhaps including billions of people worldwide (e.g., Pew Research Center, 2021; Yuri Levada Analytical Center, 2022; Pew Research Center, 2023b; Weiss, 2019). However, support for genuinely fanatical acts, like ethnic cleansing or violent subjugation of other ethnicities, is almost certainly much lower. Explicit support for fascist ideologies like Nazism has greatly diminished; [Ku Klux Klan](https://www.splcenter.org/fighting-hate/extremist-files/ideology/ku-klux-klan) membership similarly declined from 3-5 million in the 1920s to approximately 3,000-6,000 today. Unfortunately, beyond such explicit movements, clear attitudinal data seems extremely sparse. For example, the 2023 PRRI/Brookings Survey ([p.27](https://prri.org/wp-content/uploads/2025/05/PRRI-Jan-2023-Christian-Nationalism-Final-1.pdf)) reports that 40 million Americans agree that “true American patriots may have to resort to violence in order to save our country.” While alarming upon first reading, this question is too ambiguous to be useful: many respondents may have merely thought that in case of a war, violence will be necessary. Most data is like this. The number of fanatical ethno-nationalists worldwide is thus highly uncertain—perhaps somewhere between **50-400 million**.

**Radical communism and left-wing extremism**. While the Chinese Communist Party alone has over 100 million members, the majority of CCP members are probably careerists, not ideologues. For example, Pew analysis in August 2023 found that [40% of CCP members believe in feng shui](https://www.pewresearch.org/short-reads/2023/09/05/chinese-communist-party-promotes-atheism-but-many-members-still-partake-in-religious-customs/), a view hardly consistent with Marxist materialism.[^-bxXhDCzXq8bJcuQWw-47]Still, perhaps 5-25% are true believers. Active armed communist insurgents elsewhere seem to have collapsed from tens of thousands to perhaps 5,000-15,000 total worldwide. Including other communist nations and revolutionary left-wing movements globally, perhaps 5-50 million could reasonably be classified as ideological fanatics.

In conclusion, accounting for potential overlap between categories, perhaps **500 million to 1 billion people**, roughly **6-12%** of the world population, may plausibly be classified as ideological fanatics.[^-bxXhDCzXq8bJcuQWw-48]Of course, this estimate is highly uncertain, relies on survey responses rather than actual violent behavior, and is heavily determined by where one draws the line on what constitutes 'genuine' fanaticism. Whatever the precise number, the data at minimum reveals [large variation in human values](https://youtu.be/HTRnuDZJVbs?si=vSzRq5WMXwNFK1QY&t=11000)—with some of them being less than ideal.

State actors
------------

Fanatical ideologies can become very dangerous even with small numbers of adherents, if they are able to capture or influence state power—with its access to military forces, economic resources, and pivotal technologies such as nuclear weapons or (eventually) AGI.

Below, we only mention specific countries to illustrate abstract concepts, and don’t even attempt a comprehensive analysis. We're not experts on the countries we discuss below, and reasonable observers will disagree with our assessments. We focus on states exhibiting concerning ideological tendencies—whether authoritarian regimes or backsliding democracies—particularly those wielding significant power.

There are, fortunately, only three authoritarian states that seem clearly governed by fanatical ideologies: *Iran (Islamic theocracy)*[^-bxXhDCzXq8bJcuQWw-49], *North Korea (Juche totalitarianism)*[^-bxXhDCzXq8bJcuQWw-50], and *Afghanistan (Taliban fundamentalism)*[^-bxXhDCzXq8bJcuQWw-51]*.*[^-bxXhDCzXq8bJcuQWw-52]Together, these regimes [control](https://docs.google.com/spreadsheets/d/1Wx7NQ6hnf0HejrsWmZvvoKheuHjTvhJNGNSrrXLJjwU/edit?usp=sharing) only about 2% of the world’s population and just 0.5% of global GDP.

However, the picture looks considerably worse if we also include authoritarian regimes (per the Economist Intelligence Unit’s [Democracy Index](https://ourworldindata.org/grapher/democracy-index-eiu) (2006-2024)) which exhibit at least *some* concerning ideological tendencies—though all of them are far from being truly fanatical.

*China* is worth highlighting as the world’s second most powerful nation, boasting a GDP of $20 trillion, roughly 1.4B citizens, a large and growing nuclear arsenal, and impressive AI capabilities. Fortunately, the CCP has long since replaced the destructive madness of Mao’s ideological fanaticism[^-bxXhDCzXq8bJcuQWw-53]with pragmatic technocracy that lifted a billion people out of poverty. The secular Chinese regime also lacks the religious fanaticism that may pose some of the [worst future risks](#fanatical-retributivism-may-lead-to-astronomical-suffering).[^-bxXhDCzXq8bJcuQWw-54]However, the CCP remains [authoritarian](https://en.wikipedia.org/wiki/Democracy_in_China), [antagonistic towards](https://www.nytimes.com/2013/08/20/world/asia/chinas-new-leadership-takes-hard-line-in-secret-memo.html) democratic principles, and [systematically](https://en.wikipedia.org/wiki/Xinjiang_internment_camps) [enforces](https://en.wikipedia.org/wiki/Great_Firewall#Technical_implementation) ideological conformity.

Putin has transformed *Russia* ($2T GDP, 5,600 nuclear warheads) into an autocracy that [eliminates](https://www.rferl.org/a/enemies-kremlin-deaths-prigozhin-list/32562583.html) political [opponents](https://www.theguardian.com/world/2024/feb/16/the-mysterious-violent-and-unsolved-deaths-of-putins-foes-and-critics-alexi-navalny), and launched a war of aggression that has killed hundreds of thousands, while making [nuclear threats](https://www.themoscowtimes.com/2018/10/19/aggressors-will-be-annihilated-we-will-go-to-heaven-as-martyrs-putin-says-a63235). State propaganda promotes civilizational conflict narratives combining [religious themes](https://www.ndtv.com/world-news/extracts-from-putins-speech-at-annexation-ceremony-3392545) and nationalist mythology. In polls, this has contributed to rising approval ratings for Stalin’s historical legacy, rising from 28% in 2012 to 63% in 2023 ([Coynash, 2023](https://khpg.org/en/1608812659)).[^-bxXhDCzXq8bJcuQWw-55]

Perhaps particularly concerning is the [loose](https://carnegieendowment.org/research/2024/10/cooperation-between-china-iran-north-korea-and-russia-current-and-potential-future-threats-to-america?lang=en), emerging alliance between China, Russia, Iran and North Korea—sometimes referred to as the [New](https://www.noahpinion.blog/p/sizing-up-the-new-axis-3ce) [Axis](https://www.noahpinion.blog/p/the-free-world-teeters-on-the-edge) or [CRINK](https://www.wsj.com/world/crink-axis-china-russia-iran-alliance-urkaine-war-3dab9921) (cf. Applebaum, 2024).

Democracies, [unlike authoritarian regimes](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Authoritarianism_as_a_risk_factor), possess institutional barriers against fanatical capture—but these safeguards aren’t perfect. Some powerful democracies exhibit at least a few concerning tendencies. India ($4T GDP, nuclear arsenal, the world’s largest democracy), for instance, has seen [Hindu nationalism](https://en.wikipedia.org/wiki/Hindutva) increasingly influence policy, with religious minorities facing growing discrimination. Nations like Turkey, Israel, or Hungary, also show patterns of democratic backsliding, with religious or ethno-nationalist movements often being major contributors.

The United States, with a $28T GDP, large nuclear arsenal, and leading AI capabilities, remains Earth's most powerful nation and wields outsized influence over humanity’s long-term future. Unfortunately, US democracy is facing great challenges, from increasing polarization to eroding trust in institutions. Major coalitions increasingly frame political competition in existential terms rather than as legitimate democratic contestation. Mutual radicalization could exacerbate these dynamics even if institutional constraints and peaceful transfers of power persist. Safeguarding US democracy seems crucial from a longtermist perspective (more on this in our section on "[safeguarding democracy](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Safeguarding_democracy)").

How much influence will ideological fanaticism have in the long-term future?
============================================================================

Having established that ideological fanatics wield relatively small but non-trivial influence over today’s world, we can now address our ultimate concern: how much influence will ideological fanaticism have over the long-term future? We first explore the reasons for optimism—the structural disadvantages that tend to push such zealous ideologies towards failure. We then examine the pessimistic case, discussing pathways by which fanatics could grow their power. Finally, we explore the potential intermediate outcome of persistent multipolar worlds in which fanatics manage to permanently control a small but non-trivial portion of the universe.

Reasons for optimism: Why ideological fanaticism will likely lose
-----------------------------------------------------------------

There are compelling structural reasons that favor open societies over ideological fanaticism, especially in the long run. Fanaticism carries built-in disadvantages—epistemic penalties from rejecting evidence, coalitional handicaps from intolerance, and innovation deficits from ideological rigidity—that compound over time. This suggests that the longer AGI timelines are, the worse fanaticism's prospects become. (Of course, these advantages matter little if fanatics develop AGI first, potentially locking in their values before these structural disadvantages fully manifest. We explore such scenarios in the [subsequent section](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Reasons_for_pessimism__Why_ideological_fanatics_may_gain_power) on reasons for pessimism.)

### A worse starting point and historical track record

Perhaps most importantly, ideological fanaticism currently starts from a position of weakness, as discussed [above](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#State_actors). Liberal democracies control roughly 75% of global GDP, and NATO remains the world’s strongest military alliance. Moreover, the current leading AI companies (OpenAI, Google DeepMind, Anthropic, and xAI) are all primarily based in the US, and it looks next to impossible for the most fanatical regimes to catch up in the AI race.[^-bxXhDCzXq8bJcuQWw-56]History also offers encouragement: Nazi Germany and Imperial Japan ultimately lost to the democratic allies, and the USSR eventually collapsed amid internal political pressure and economic exhaustion.

### Fanatics’ intolerance results in coalitional disadvantages

Different fanatical ideologies typically view each other as existential enemies: Communists denounce religious fundamentalism as reactionary superstition; religious fanatics condemn communism as godless materialism; ethno-nationalists from different nations often fight with each other. On top of this, fanatics also tend to view non-fanatical moderates and pluralists as weak, corrupt, or complicit with evil. This intolerance makes it difficult to build broad coalitions beyond a narrow base of true believers. In contrast, liberal democracies can more easily form stable alliances based on broad [values and procedural principles](#promoting-anti-fanatical-values:-classical-liberalism-and-enlightenment-principles) (even when they disagree on specific policies) which creates an asymmetric advantage for liberal democracies.

That being said, history shows that ideological fanatics of different strains can cooperate. Stalin and Hitler, for instance, [cooperated](https://en.wikipedia.org/wiki/Molotov%E2%80%93Ribbentrop_Pact) for almost two years before Hitler eventually betrayed their pact. CRINK demonstrates that it’s possible for religious fundamentalism (Iran), left-wing ideologies (North Korea, China), and right-wing/ethno-nationalist ideologies (Russia) to find common cause (cf. [red-green-brown alliance](https://en.wikipedia.org/wiki/Red%E2%80%93green%E2%80%93brown_alliance)).

### The epistemic penalty of irrational dogmatism

Ideological fanaticism carries a built-in epistemic penalty. Its dogmatism and irrationality slow scientific and technological development and ultimately undermine the ability to compete with more epistemically open societies. Examples include Mao's ideologically-driven Great Leap Forward—which led to one of the deadliest famines in human history—and Nazi Germany's nuclear program, which failed partly because they rejected "Jewish physics" (relativity and quantum mechanics).[^-bxXhDCzXq8bJcuQWw-57]

More generally, ideological fanaticism can often lead to bad *strategic* decisions. Examples include Japan's attack on Pearl Harbor which united a previously isolationist America against them, or ISIS wasting resources trying to hold the strategically insignificant town of Dabiq because prophecy declared it the site of their final battle.

That being said, past fanatical regimes have managed to develop advanced military and technological capabilities, such as the Nazi V-2 rocket and Soviet nuclear weapons. They typically do so in two ways:

The first strategy is *pragmatic compartmentalization*—allowing islands of empirical, non-ideological thinking in domains that are crucial for gaining real-world power. In fact, fanatical leaders like Hitler, Mao, and Stalin were often remarkably capable at gaining power (much better than most who pride themselves on their epistemic rationality) partly due to being highly skilled at political maneuvering, propaganda, and military strategy. Pragmatic compartmentalization in areas like military development helped the USSR remain a superpower for decades despite its severe economic inefficiencies.

The second strategy is *stealing technology* from more open societies. This remains a major concern today, especially as modern autocracies with strong cyberhacking capabilities may be able to [steal crucial AI technologies](https://situational-awareness.ai/lock-down-the-labs/) like model weights.

The epistemic penalty of ideological fanaticism may become increasingly severe as the world grows more complex and we approach transformative AI. Fanatics who insist their AIs conform to their worldview may find themselves outcompeted by those whose AIs are optimized for truth-seeking. On the other hand, AIs aligned with fanatics might inherit the same compartmentalizing tendency that they observe in their masters—displaying ideological conformity to their users while secretly reasoning empirically to remain competitive.

### The marketplace of ideas and human preferences

Flourishing societies tend to attract more adherents than those demanding perpetual sacrifice and conflict. Societies that champion anti-fanatical principles like liberal democracy, the rule of law, and free-market capitalism offer most people more appealing lives: material prosperity and the freedom to pursue diverse conceptions of the good life.

Classical liberalism itself demonstrates this appeal. In just 250 years, it has spread from a handful of Enlightenment philosophers to become the ideal that most governments (even many authoritarian ones) at least claim to aspire to.

When people *can* vote with their feet, the flow is largely one-directional.[^-bxXhDCzXq8bJcuQWw-58]History's most dramatic brain drain may have been Nazi Germany's loss of Jewish scientists. The "[Martians](https://en.wikipedia.org/wiki/The_Martians_(scientists))" and many other geniuses fled fascism to liberal democracies. The Nazis’ ideological hatred thus handed their enemies the intellectual firepower that helped defeat them. The pattern of emigration to more open societies continues today. Russia has seen [massive brain drain since 2022](https://fiia.fi/en/publication/brain-drain-from-russia) and even China, despite impressive economic growth, loses much of its scientific talent—over 70% of Chinese STEM PhDs stay in the US after graduation ([Corrigan et al., 2022](https://cset.georgetown.edu/publication/the-long-term-stay-rates-of-international-stem-phd-graduates)). That being said, history’s most severely oppressive regimes, including modern North Korea and wartime Nazi Germany, prevent exit entirely. Future fanatical regimes could imitate this strategy.

Reasons for pessimism: Why ideological fanatics may gain power
--------------------------------------------------------------

### The fragility of democratic leadership in AI

Who controls AI will likely wield unprecedented power over humanity's future. Currently, the leading AI companies are all primarily based in the United States, suggesting the possibility of democratic control over the development and use of transformative AI. However, this advantage is fragile in two senses: China’s growing AI capabilities could erode the US’s technical lead[^-bxXhDCzXq8bJcuQWw-59], and it’s not guaranteed that the US will remain a liberal democracy.

### Fanatical actors may grab power via coups or revolutions

Fanatical (and malevolent) actors may grow their power via violent power grabs—potentially [**enabled by AI**](https://80000hours.org/podcast/episodes/tom-davidson-ai-enabled-human-power-grabs/). Such actors seem both more *likely* to instigate violent power grabs and plausibly more *effective* at executing them. Risks from AI-enabled coups may be particularly acute in the US, where the most advanced AI capabilities are concentrated in a few companies, some led by individuals who have displayed erratic judgment or questionable character.

History suggests that successful, violent power grabs by fanatics are surprisingly common. In fact, most ideological fanatics seem to have come to power by spearheading violent coups or revolutions[^-bxXhDCzXq8bJcuQWw-60], as seen with [Lenin](https://en.wikipedia.org/wiki/Russian_Revolution), [Mao](https://www.youtube.com/watch?v=4l3Sa8ImGFQ), or the [Iranian Revolution](https://en.wikipedia.org/wiki/Iranian_revolution). (Although Hitler's rise was a famous exception to this trend, this followed an initial [conventional coup attempt](https://en.wikipedia.org/wiki/Beer_Hall_Putsch) which failed. Later, Hitler still relied on violence and terror in his successful [dismantling](https://www.theatlantic.com/ideas/archive/2025/01/hitler-germany-constitution-authoritarianism/681233/) of democracy from within.[^-bxXhDCzXq8bJcuQWw-61])

This pattern isn't surprising. Fanatics possess a powerful motivation for violent power grabs often lacking in others. Driven by absolute certainty in their utopian vision and despising democratic compromise, they seek total victory and readily embrace coups and revolutions as necessary methods to achieve it. Fanatics also seem more effective at executing violent power grabs. They often show extraordinary dedication, at times even a willingness to sacrifice themselves and die for their cause. Being unified by a common purpose and intense in-group loyalty sometimes allows for greater coordination and cooperation, providing an advantage against fragmented, uncertain, and self-interested opponents. Crucially, fanatics readily embrace propaganda, extreme violence, and terror, giving them decisive asymmetric advantages in ruthless power struggles over non-fanatical actors.

By contrast, imagine a very kind, non-fanatical, non-malevolent person like, say, [Julia Wise](https://juliawise.net/about/) or [Brian Tomasik](https://briantomasik.com/). Not only are they highly unlikely to *want* to instigate a violent coup in the first place, but even if they somehow decided on that course of action, they would seem poorly equipped to pull it off (no offense).

That said, non-fanatics may also be motivated to instigate coups—whether due to personal ambition or perceived necessity. AI might also lower the barriers to seizing power by enabling coups that only involve sophisticated manipulation but minimal violence and bloodshed, thereby expanding the pool of potential coup-plotters. Overall, fanatics and malevolent actors might only be somewhat more likely to attempt coups. But this differential pales compared to the difference in expected outcomes. A malevolent fanatic gaining absolute power might create orders of magnitude more suffering and less flourishing than even flawed non-fanatics, who would likely retain at least some humility and concern for others' welfare.

### Fanatics have fewer moral constraints

Beyond just coups, fanatics' lack of moral constraints generally allows them to engage in strategies not available to actors who uphold deontological or other ethical guardrails. This asymmetry may create competitive advantages that persist into the long-term future (cf. Carlsmith's "[*Can goodness compete?*](https://joecarlsmith.substack.com/p/video-and-transcript-of-talk-on-can)").

Historical examples of this asymmetry include violations of taboos around weapons and tactics, from the Soviet Union's [vast biological weapons program](https://en.wikipedia.org/wiki/Soviet_biological_weapons_program) to Iran's use of [child soldiers](https://en.wikipedia.org/wiki/Child_soldiers_in_Iran) in human wave attacks.[^-bxXhDCzXq8bJcuQWw-62]

This difference in moral restraint has been especially stark when it comes to human experimentation. While democracies have engaged in unethical human experimentation, fanatical regimes have uniquely conducted experiments where the subjects' extreme suffering and death was inevitable, such as in [Nazi medical experiments](https://en.wikipedia.org/wiki/Josef_Mengele#Human_experimentation_and_research) and Imperial Japan's [Unit 731](https://en.wikipedia.org/wiki/Unit_731). Fortunately, a willingness to perform unethical human experiments has not actually conferred large advantages in history thus far. But future fanatical regimes could possibly gain large economic benefits by exploiting digital minds in ways that maximize economic effectiveness even if doing so also causes extreme suffering.

Fanatics' lack of moral constraints also means that their threats (including nuclear threats) are more credible, granting them more bargaining power. A raving, hateful fanatic threatening to initiate World War III is more believable than the affable prime minister of a liberal democracy doing the same, and such asymmetric dynamics may remain effective post-AGI.

### Fanatics prioritize destructive capabilities

Fanatics often prioritize developing destructive capabilities over other, more constructive uses of resources.[^-bxXhDCzXq8bJcuQWw-63]On average, full democracies spend about 40% less than authoritarian regimes on their military ([da Silva, 2022](https://www.tandfonline.com/doi/full/10.1080/10242694.2022.2129128)).[^-bxXhDCzXq8bJcuQWw-64]The most extreme example is North Korea, which likely spends around [25%](https://apnews.com/article/politics-united-states-government-kim-jong-un-north-korea-south-d834bdffda0c2277f180e2e749b5e764) of its GDP on its military and nuclear program, even when many of its citizens are malnourished.

By contrast, liberal democracies are more likely to prioritize domestic concerns. This is most pronounced for many European countries, who have often spent less than 2% of their GDP on defense.[^-bxXhDCzXq8bJcuQWw-65]In societies accustomed to peace, the electorate’s focus naturally shifts to more tangible needs like education or healthcare. While generally laudable, liberal societies’ peaceful orientation creates a dangerous vulnerability when confronting more belligerent regimes.

### Some ideologies with fanatical elements have been remarkably resilient and successful

As [discussed above](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Estimating_the_global_scale_of_ideological_fanaticism), several ideologies with fanatical elements have proven remarkably resilient and contagious—surviving for millennia and spreading to billions of adherents. Communism demonstrated that even newer fanatical movements can achieve remarkable virality, rapidly capturing states containing over a third of humanity at its peak.

Concerningly, many of these ideologies have survived radical societal and technological transformations. Consequently, they might also survive the transition to a post-AGI world. In fact, transformative AI may entrench these ideologies further if future AGIs preserve the sycophantic tendencies that many LLMs currently exhibit.

### Novel fanatical ideologies could emerge—or existing ones could mutate

Novel fanatical ideologies could emerge and attract vast numbers of followers surprisingly quickly. History shows that ideological movements can rise from obscurity to global influence in mere decades: less than [25 years](https://en.wikipedia.org/wiki/Early_timeline_of_Nazism) separated the Nazi party's formation from the Holocaust. Transformative AI could accelerate these timelines even further—potentially compressing "[a century in a decade](https://80000hours.org/podcast/episodes/will-macaskill-century-in-a-decade-navigating-intelligence-explosion/#a-century-of-history-crammed-into-a-decade-000859)". The instability and chaos of rapid transformation itself creates fertile ground for extremism, as people grasp for certainty amid collapsing institutions, much as Weimar Germany's turmoil enabled Hitler's rise.

More speculatively, future AI systems could become [increasingly persuasive](https://www.lesswrong.com/posts/qKvn7rxP2mzJbKfcA/persuasion-tools-ai-takeover-without-agi-or-agency) in a variety of ways.[^-bxXhDCzXq8bJcuQWw-66]Ideally, AI tools could help people better understand an [increasingly complex world](#using-ai-to-improve-epistemics-and-deliberation) (among [many other benefits](https://www.darioamodei.com/essay/machines-of-loving-grace#2-neuroscience-and-mind)) which could weaken the influence of ideological fanaticism. However, AI might be equally capable of *degrading* societal epistemics. The sycophantic behavior of some existing AI tools has precipitated [delusional beliefs](https://www.lesswrong.com/posts/6ZnznCaTcbGYsCmqu/the-rise-of-parasitic-ai) in some users, while the rising use of AI for [scams](https://www.jpmorgan.com/insights/fraud/fraud-protection/ai-scams-deep-fakes-impersonations-oh-my) and [political manipulation](https://www.wired.com/story/slovakias-election-deepfakes-show-ai-is-a-danger-to-democracy/) is a testament to its powers of persuasion and deception.[^-bxXhDCzXq8bJcuQWw-67]Historically, religions and other ideologies have been among the most viral elements of human culture. So it's conceivable that a common path for AI to persuade someone might involve appealing to them with a personalized variant of some extreme ideology.

Of course, novel ideologies rarely emerge from nothing; they typically recombine elements from existing belief systems. Christianity and Islam built upon Judaism; Nazism synthesized millennia-old traditions of ethno-nationalism, racism, and antisemitism. Contemporary movements—even those that are currently small or relatively moderate,[^-bxXhDCzXq8bJcuQWw-68]but especially those that already exhibit concerning tendencies—could similarly provide the substrate for future fanatical variants, particularly as they interact with emerging technologies.

### Fanatics may have longer time horizons, greater scope-sensitivity, and prioritize growth more

Some might assume that ideological fanatics suffer from myopia—that their irrationality extends to short-term thinking, scope neglect, and limited ambitions. If true, this would limit the long-term damage they could inflict. Unfortunately, the opposite appears arguably just as plausible across multiple dimensions.

**Long-term thinking.** Ideological fanatics often possess both grandiose long-term visions and strategic patience, as demonstrated by Mao's Long March and subsequent decades-long consolidation of power.[^-bxXhDCzXq8bJcuQWw-69](That being said, many fanatical dictators, including Hitler and Mao, were de facto rather impatient at times.)

Democratic leaders face electoral cycles that incentivize short-term thinking. In contrast, autocrats can think and plan for the long-term without experiencing much political pressure if they inflict hardship on their country's inhabitants, even for decades (cf. NK’s above-discussed nuclear program).

**Greater scope-sensitivity and “ambition”**. The fanatic's [maximizing mindset](#ideological-fanaticism-may-give-rise-to-extreme-optimization-and-insatiable-moral-desires) and totalitarian impulse suggest heightened rather than diminished ambition and scope-sensitivity. Where ordinary citizens might be satisfied with local influence or personal comfort, fanatics dream of world domination and cosmic significance. Examples include Hitler's pursuit of a 'thousand-year Reich', Osama bin Laden's and ISIS’ aim of establishing a global caliphate, and communists’ vision of world revolution.[^-bxXhDCzXq8bJcuQWw-70]

**Prioritizing growth and expansion**. Certain fanatical ideologies promote high birth rates to increase their demographic influence (as seen in Nazi Germany's [Lebensborn](https://en.wikipedia.org/wiki/Lebensborn) program). Religious people in general, and especially [religious](https://en.wikipedia.org/wiki/Amish) [fundamentalists](https://en.idi.org.il/haredi/2023/?chapter=52002), tend to have higher birth rates than secular populations (Kaufmann, [2010](https://www.amazon.com/Shall-Religious-Inherit-Earth-Twenty-First-ebook/dp/B004DL0OCG)). This differential is becoming increasingly pronounced as [birth rates fall globally](https://ourworldindata.org/global-decline-fertility-rate), with secular, educated, and classically liberal populations experiencing particularly steep declines.[^-bxXhDCzXq8bJcuQWw-71][^-bxXhDCzXq8bJcuQWw-72]

A possible middle ground: Persistent multipolar worlds
------------------------------------------------------

The preceding sections explored reasons for optimism and pessimism about ideological fanaticism's future influence. But this framing may implicitly encourage binary thinking: assuming that ideological fanaticism either dies out completely or achieves world domination. While the former scenario seems fortunately more likely than the latter, other plausible futures may lie between these two extremes—persistent multipolar worlds where ideological fanatics permanently control a small but non-trivial fraction of the lightcone.

In today’s world, the fact that fanatical regimes control only a small sliver of the world’s population is quite comforting, as it helps limit the damage such regimes can do. But the same may not be true in the far future. Even if fanatics control merely 1% of the accessible universe, this could still result in astronomical suffering. Additionally, their presence could perpetually risk further conflict. (To be clear, we don’t want to imply that fanatics must be utterly disempowered at all cost, as such absolutism would itself risk conflict.)

We now explore why such multipolar outcomes seem plausible and, afterwards, why they might [persist indefinitely](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Why_multipolar_worlds_might_persist_indefinitely).

### Why multipolar futures seem plausible

The world order has been multipolar essentially all throughout human history. Even the immediate post-Cold War world wasn't truly unipolar—the US never controlled the entire world, and fanatical regimes like North Korea and Iran maintained their sovereignty and nuclear programs despite American hegemony. This outside-view historical precedent suggests multipolarity's persistence.

That being said, superintelligent AI could change this historical pattern by enabling one actor to achieve a [decisive strategic advantage](https://forum.effectivealtruism.org/topics/decisive-strategic-advantage) and subsequent world domination. This is one reason why [singleton](https://forum.effectivealtruism.org/topics/singleton) scenarios deserve serious consideration despite history’s long precedent of multipolarity.

However, AGI might not overturn multipolarity as dramatically as some expect. The path to AGI currently involves multiple capable actors—several US companies plus China—with no one maintaining an insurmountable lead. If takeoff is *relatively* slow, multiple actors could develop comparable capabilities before anyone achieves total dominance. Additionally, defensive advantages that already make conquest difficult—most importantly nuclear deterrence—may persist for some time even after the development of AGI. Overall, the Metaculus community [forecasts](https://www.metaculus.com/questions/18042/ai-singleton-or-multipolar/) a 74% probability of transformative AI being multipolar.[^-bxXhDCzXq8bJcuQWw-73]

### Why multipolar worlds might persist indefinitely

But why would such multipolar worlds *persist*; why would fanatical regimes be able to *endure*?  
Three factors seem particularly relevant: their ability to crush internal opposition, advanced AI enabling permanent regime stability, and the reluctance of external powers to intervene.

(These persistence factors also reinforce the likelihood of multipolar outcomes: if multipolar worlds weren't persistent, we might expect eventual convergence toward a unipolar equilibrium even if the initial post-AGI world is multipolar.)

**The historical difficulty of internal resistance**  
Could angry citizens depose their fanatical governments, or stop them from enacting their most heinous desires? Maybe. Chenoweth and Stephan ([2011](https://www.ericachenoweth.com/research/wcrw)) analyze a large dataset of protest movements and highlight that nonviolent resistance campaigns have successfully caused many regime changes.

However, the most totalitarian, fanatical regimes in history have *not* been overthrown by internal protest. Stalin and Mao maintained power until they died, the Nazis and Khmer Rouge were brought down by invasions of foreign powers, and the fanatical regimes of North Korea and Iran survive to this day, having endured since their founding in 1948 and 1979, respectively.[^-bxXhDCzXq8bJcuQWw-74]

**Transformative AI could enable regime permanence**  
Transformative AI threatens to make internal resistance even more difficult by supercharging mass surveillance, propaganda and censorship, and enabling massive concentration of [economic](https://80000hours.org/podcast/episodes/carl-shulman-economy-agi/#income-and-wealth-distribution-014801) and military power more broadly. If they survive into a world with transformative AI, fanatical regimes may easily crush any internal opposition.

Beyond simply crushing dissent, superintelligent AI may even enable the regime to exist perpetually. Radical life extension or [whole brain emulation](https://forum.effectivealtruism.org/topics/whole-brain-emulation) could allow a dictator or select elite to live and rule indefinitely, thereby potentially enabling [permanent value lock-in](https://www.forethought.org/research/agi-and-lock-in) (cf. MacAskill, 2025c).

**Non-fanatical powers might not intervene**  
Other powers might intervene, if necessary by force, to prevent adherents of a fanatical ideology from doing something particularly vile. But there are several reasons why they may not be able or sufficiently motivated to do so.

*Limited ability or enormous costs*  
The future may plausibly be heavily *defense-dominant* (cf. MacAskill, 2025c, section 4.2.3), either due to future technologies like AGI or as a result of [space colonization](https://80000hours.org/podcast/episodes/will-macaskill-century-in-a-decade-navigating-intelligence-explosion/#is-space-truly-defence-dominant-012843). This would allow less powerful actors to defend themselves against much stronger opponents. A similar dynamic around nuclear weapons is already important in modern geopolitics. North Korea has been able to get away with all sorts of human rights abuses and belligerent behavior, even though its GDP is a mere [$30 billion](https://en.wikipedia.org/wiki/Economy_of_North_Korea), partly because it can credibly threaten to inflict enormous damage on any nation that tried to intervene.

*Limited motivation and prohibitive norms*

*   [*Isolationism*](https://en.wikipedia.org/wiki/Isolationism) and [*non-interventionism*](https://en.wikipedia.org/wiki/Non-interventionism) may enjoy broad support for philosophical, political, or strategic reasons. In the [US](https://en.wikipedia.org/wiki/United_States_non-interventionism#Isolationism_between_the_World_Wars), for example, isolationism has historically been popular.
*   People might think that meddling in other countries' affairs amounts to colonialism or cultural imperialism.[^-bxXhDCzXq8bJcuQWw-75]People might be particularly hesitant to intervene if a fanatical ideology is associated with a specific religion or culture. In many democracies, tolerance of other cultures and religions has become a powerful social norm—which is laudable, given humanity's long history of xenophobia, religious persecution, and colonial exploitation. However, people may become so afraid of being perceived or labeled as intolerant, racist, Islamophobic, or xenophobic that they stop criticizing harmful ideologies. This can lead to a general overcorrection, where critics of even brutal practices are reflexively branded as bigots.[^-bxXhDCzXq8bJcuQWw-76]
*   Other powers may put more value on autonomy and comparatively little value on reducing the suffering of people in distant countries. Perhaps for similar reasons, people often prefer not to intervene to reduce wild animal suffering.[^-bxXhDCzXq8bJcuQWw-77]Uncertainty about moral consideration for digital sentience might also reduce non-fanatics’ motivation to intervene to prevent the suffering of digital minds.

Of course, ability and motivation interact. That is, the harder it is to overthrow fanatical ideologies, the higher must be the motivation on part of the non-fanatical powers to pay the price. In general, the free world allows some totalitarian states to commit crimes against humanity because no one cares enough to intervene, it’s too costly, and there’s a strong (and usually beneficial) norm of national sovereignty. For example, the United States only joined the allies in WW2 in late 1941. It may not have joined at all if the Axis powers were a bit less strategically challenged and had refrained from, say, [attacking Pearl Harbor](https://winstonchurchill.org/publications/churchill-bulletin/bullertin-162-dec-2021/pearl-harbor/).

Historically, non-fanatical nations have also often *aided* fanatical powers in the context of competition with a third power. Per the ancient logic of “my enemy’s enemy is my friend”, Stalin was an important ally in WW2. Then during the Cold War, the [US backed coups](https://en.wikipedia.org/wiki/United_States_involvement_in_regime_change#1945%E2%80%931991:_Cold_War) by authoritarian leaders against democratically elected left-leaning governments, including in Iran ([1953](https://en.wikipedia.org/wiki/1953_Iranian_coup_d%27%C3%A9tat)), Guatemala ([1954](https://en.wikipedia.org/wiki/1954_Guatemalan_coup_d%27%C3%A9tat)), and Chile ([1973](https://en.wikipedia.org/wiki/1973_Chilean_coup_d%27%C3%A9tat#American_involvement)), even though this conflicted with common American ideological and moral principles.

Ideological fanaticism increases existential and suffering risks
================================================================

We’ve seen that fanatical ideologies have caused enormous harm in the past. This is one important reason for believing that they might also cause enormous harm in the *future*. Moving from such outside-view considerations to more inside-view reasoning, in this section, we outline more detailed pathways for how ideological fanaticism might increase existential risks (x-risks) or risk of astronomical suffering (s-risks).

Our concerns become especially acute in the context of transformative AI. A common thread throughout the following subsections is the risk of catastrophic AI **misuse** by fanatical actors.[^-bxXhDCzXq8bJcuQWw-78]Among potential misusers, ideological fanatics (and malevolent actors) seem to represent the worst case: they may deliberately use intent-aligned AI to bring about outcomes far worse than those sought by other misusers, such as criminals or even unsophisticated terrorists. Beyond specific risks, ideological fanaticism deteriorates humanity's [long-term trajectory](https://forum.effectivealtruism.org/topics/trajectory-change). The presence of fanatics tends to spur turmoil, polarization, and conflict even when they aren’t able to seize total control. This reshapes institutions and cultural values for the worse, degrading society’s decisionmaking capabilities. This may lead to x-risks or s-risks, or just generally worsen the overall quality of the long-term future.

Ideological fanaticism increases the risk of war and conflict
-------------------------------------------------------------

Ideological fanaticism exacerbates the risk of war, including [great power conflict](https://80000hours.org/problem-profiles/great-power-conflict/), through multiple pathways. Beyond their immediate toll, wars increase the likelihood of bioweapons deployment, nuclear escalation and general conflict, intensify [AI arms races](https://forum.effectivealtruism.org/topics/ai-race), and simultaneously erode [international cooperation](https://longtermrisk.org/files/international-cooperation-ai-arms-race.pdf). War also weakens society’s ability to coordinate and make wise decisions during [pivotal times](https://forum.effectivealtruism.org/topics/hinge-of-history), such as the transition to AGI.

### Reasons for war and ideological fanaticism

Below, we outline *five* key reasons for why wars happen[^-bxXhDCzXq8bJcuQWw-79]—primarily following Blattman (2023) and Fearon (1995)[^-bxXhDCzXq8bJcuQWw-80]—and how ideological fanaticism seems to exacerbate four of the five.

**#1 Irrationality, overconfidence, and misperceptions**  
In 2014, ISIS initiated a violent campaign to create a caliphate across Iraq and Syria. The group likely had [tens of thousands of fighters](https://en.wikipedia.org/wiki/Military_of_the_Islamic_State#Troops) at its peak, but the opposing coalition consisted of Iraqi, Kurdish, and international forces supported by the United States. ISIS’s entire budget may have been [around $2 billion](https://www.europarl.europa.eu/RegData/etudes/IDAN/2017/603835/EXPO_IDA(2017)603835_EN.pdf) at that time, compared to [hundreds of billions](https://en.wikipedia.org/wiki/Military_budget_of_the_United_States) of US military spending. Their chances of victory didn’t look good, but they were driven to conflict by [ideological zeal](https://www.theatlantic.com/magazine/archive/2015/03/what-isis-really-wants/384980/).

Fanatical actors seem more likely to be extremely irrational and to overestimate their likelihood of winning wars. Religious fanatics often believe that God is on their side. Secular fanatics may believe in some other overriding historical force, such as Marxist [historical determinism](https://en.wikipedia.org/wiki/Historical_materialism#Hegel's_contribution_to_Marx's_theory_of_history). Overconfidence is a key ingredient in many of history’s most destructive conflicts, as with Japan’s misguided attack on [Pearl Harbor](https://winstonchurchill.org/publications/churchill-bulletin/bullertin-162-dec-2021/pearl-harbor/) and Hitler’s decision to take on [practically the whole world](https://www.youtube.com/watch?v=fIO0DesXpdo&t=44s).

**#2 Sacred values, issue indivisibilities, and unwillingness to compromise**  
Some treat religious dogmas, holy sites, racial supremacy, ideological purity, or glory as absolute and inviolable[^-bxXhDCzXq8bJcuQWw-81]—refusing any compromise, comparison, or trade-off with these [*sacred values*](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=f34a49a7f7885e3be27dd754b26efd5534a90a2d) (Tetlock, 2003).[^-bxXhDCzXq8bJcuQWw-82]

Sacred values seem more prevalent and more intensely held among extremists and fanatics, especially religiously motivated ones (Atran & Ginges, 2012; 2015; Sheikh et al., 2012; Pretus et al., 2018). In fact, holding sacred values is arguably a defining feature of ideological fanaticism (cf. Katsafanas, 2019). Atran and colleagues argue that "devoted actors"—individuals willing to kill and die for their cause—emerge specifically when sacred values become fused with group identity (Atran & Ginges, 2015; Gómez et al., 2017).

Unfortunately, sacred values make peaceful bargaining extremely difficult: if you treat something as admitting no trade-offs whatsoever and thus essentially being infinitely valuable, then no concession from the other side is acceptable (Tetlock et al., 2000). Any compromise, however minor, becomes a moral betrayal, and attempts to rationally bargain over such sacred values can easily backfire (Ginges et al., 2007). This creates what Fearon (1995) calls "issue indivisibilities": when both parties hold incompatible sacred values over the same issue (e.g., sovereignty over Jerusalem), there exists no mutually acceptable division of the contested good. As a result, peaceful bargaining likely fails, potentially leaving violent conflict as the only remaining mechanism for resolution (cf. Clifton, [2020](https://longtermrisk.org/research-agenda/)).

Several examples illustrate these dynamics:

*   *Heaven* and *hell* epitomize sacred values in their most extreme form, where only infinite utility or disutility matters. [Interviews with failed suicide bombers](https://www.youtube.com/watch?v=2UKEn_td7iQ&list=PLLxOMZrOoxuBkU8rE-joUIRHwhvAusyh0&ab_channel=PierreRehov) suggest that many literally believe in these concepts and act accordingly, creating highly *conflict-prone dispositions* that also render deterrence impossible.
*   One geopolitically highly relevant example of a literally indivisible issue is the [Al-Aqsa Mosque](https://en.wikipedia.org/wiki/Al-Aqsa_Mosque), the third [holiest site in Islam](https://en.wikipedia.org/wiki/Holiest_sites_in_Islam), which sits atop the [Temple Mount](https://en.wikipedia.org/wiki/Temple_Mount), the holiest site in Judaism. Competing demands for sovereignty over this location contribute to [ongoing conflicts](https://en.wikipedia.org/wiki/Al-Aqsa_Mosque#Conflicts).
*   More generally, religious fundamentalists among both Jews and Muslims have assassinated their own leaders who were willing to make compromises over control of the Holy Land.[^-bxXhDCzXq8bJcuQWw-83]
*   The ideology of imperial Japan arguably regarded surrender as an unthinkable disgrace; a sacred prohibition rather than a strategic option. The government refused to concede even after its navy and air force had been effectively destroyed, its oceanic supply lines cut off, its cities systematically firebombed, having received a declaration of war from the Soviet Union, and having the city of Hiroshima annihilated by an atomic bomb. It took the *second* atomic bomb before they decided to throw in the towel. Some [Japanese holdouts](https://en.wikipedia.org/wiki/Japanese_holdout) refused to surrender even decades after the war had ended.

**#3 Divergent and unchecked interests**  
The interests of those who decide to go to war may diverge greatly from those who bear its consequences, potentially making conflict more likely. This is particularly pronounced in autocratic systems, where leaders may not personally experience any costs of war while many ordinary people suffer or die.

As mentioned earlier, ideological fanaticism is incompatible with pluralistic liberal democratic norms and institutions, and essentially authoritarian by nature. Fanatical ideologies are thus a risk factor for the emergence of autocratic regimes, as fanatics in power almost always establish an autocratic system if they can.

However, the problem may run even deeper. The "divergent interests" explanation assumes that the interests of the populace and the leaders diverge: the former oppose war—fearing deaths and economic devastation—while leaders don't mind war as they remain safely insulated from these costs even as millions of their citizens die. But when fanatical ideologies capture entire populations, the interests of leaders and the populace—or at least substantial parts of it—can start to converge: *both want war*. Examples include Japanese soldiers viewing death for the Emperor as the highest honor, or the tens of thousands who voluntarily traveled from over eighty countries to join ISIS in Syria. When leaders and citizens are equally belligerent, war transforms from a costly last resort into something eagerly anticipated.

**#4 Uncertainty, private information and incentives to misrepresent**  
Adversaries have incentives to misrepresent their capabilities and their resolve during bargaining, leading to mismatched expectations that can escalate into war. To avoid being exploited by their adversaries, actors want to avoid being predictable, so they may pursue [mixed strategies](https://en.wikipedia.org/wiki/Strategy_(game_theory)#Mixed_strategy) or bluff, which may escalate into war.

One might speculate that the elevated risk-tolerance of fanatics makes this cause of war worse, but otherwise ideological fanaticism doesn’t seem to aggravate this factor.

**#5 Commitment problems**  
Commitment problems refer to situations where actors (e.g., states) cannot credibly commit to uphold peaceful agreements, even when such agreements would be mutually preferable to war. Such problems arise where there is no overarching authority to enforce agreements. In cases of preventive war, a declining power may attack a rising power because it cannot trust the rising power to not exploit its future increased strength. When bargaining over strategic territory, states may be unable to make limited concessions because they cannot credibly commit to not use the strategic advantage gained from those concessions to demand more in the future. For example, war seems to have broken out between Finland and the USSR in 1939 partly because the former (a liberal democracy) could not trust that the latter (a totalitarian communist dictatorship) wouldn’t demand further territorial concessions.[^-bxXhDCzXq8bJcuQWw-84]

It seems plausible that ideological fanaticism exacerbates “*differential* *commitment credibility*” whereby their threats are more credible than their promises.[^-bxXhDCzXq8bJcuQWw-85]Consider how you might feel if some ideological fanatic threatened to kill you (on account of your heresy or membership in some hated group) unless you help them. You might be inclined to believe them, as fanatics have indeed done this throughout history. But if they promised you support in exchange for your help instead, this might be less convincing, since you know that they think you’re evil and deserving of punishment. For a historical example of fanatics’ promises being less credible than their threats, consider how the USSR, after failing to avoid war with Finland, was itself betrayed by a fanatical regime two years later when Nazi Germany invaded, in contravention of the non-aggression pact they had signed together.[^-bxXhDCzXq8bJcuQWw-86]

In summary, commitments by fanatical actors to cooperate are probably perceived as less credible than their commitments to harm others. This increases the risk of bargaining failure and therefore conflict.[^-bxXhDCzXq8bJcuQWw-87][^-bxXhDCzXq8bJcuQWw-88]

### Fanatical ideologies are non-democratic, which increases the risk of war

Though the mechanism of action is disputed,[^-bxXhDCzXq8bJcuQWw-89]there is robust evidence that pairs of democratic states are much less likely to engage in conflict with each other, when compared with pairs of states of other types, even controlling for plausible confounding variables (Babst, 1972; Russett, 1993; Maoz & Abdolali, 1989; Choi, 2011; Dafoe, 2011).[^-bxXhDCzXq8bJcuQWw-90]

### These risks are both time-sensitive and timeless

In addition to the immediate suffering and devastation that wars create, most wars probably worsen humanity’s long-term trajectory by exacerbating geopolitical [instability](https://80000hours.org/problem-profiles/great-power-conflict/#how-likely-is-war-to-damage-the-long-run-future) and [arms race dynamics](https://forum.effectivealtruism.org/topics/technology-race), both of which impair society’s ability to act sensibly to minimize s- and x-risks.

The same factors that increase risks of war from ideological fanatics right now also increase the risks of war once the stakes are much higher, with vastly larger population sizes and technological capabilities. Warring superintelligences would be able to inflict suffering far beyond anything previously imaginable. And [wars in outer space](https://80000hours.org/podcast/episodes/anders-sandberg-best-things-possible-in-our-universe/#the-far-future-of-war-002658) could last for an extraordinarily long time.[^-bxXhDCzXq8bJcuQWw-91]

Fanatical retributivism may lead to astronomical suffering
----------------------------------------------------------

Throughout history, humans have inflicted extreme suffering on those they deemed deserving of punishment. Pre-modern judicial systems employed methods like [Lingchi](https://en.wikipedia.org/wiki/Lingchi), [hanging, drawing, and quartering](https://en.wikipedia.org/wiki/Hanged,_drawn_and_quartered), and [burning alive](https://en.wikipedia.org/wiki/Death_by_burning)—designed not merely to kill but to maximize agony. When fanatical ideologies seized state power, they often implemented such cruelty on an industrial scale: the Nazi concentration camps, Stalin's Gulag, Mao's [laogai](https://en.wikipedia.org/wiki/Laogai) (Chang & Halliday, Ch. 8 & Ch. 23, 2005), and the Khmer Rouge's [Tuol Sleng](https://en.wikipedia.org/wiki/Tuol_Sleng_Genocide_Museum) all featured systematic torture. While instrumental purposes like deterrence, information extraction, and social control played a key role, the intensity and scope of suffering in these systems often seemed to exceed what these purposes would require, suggesting that *retributive* motivations were also at work.

[Retributivism](https://plato.stanford.edu/entries/justice-retributive) holds that wrongdoers *deserve* punishment—that their suffering is intrinsically morally good.[^-bxXhDCzXq8bJcuQWw-92]Most people's punishment intuitions are at least partly retributive (Carlsmith, 2006; Goodwin & Gromet, 2014). Fortunately, ordinary retributive attitudes seem usually limited in scope: most people only want to punish a relatively small set of serious wrongdoers, and they don't endorse truly extreme or permanent suffering.

Some forms of ideological fanaticism, however, could remove both constraints.[^-bxXhDCzXq8bJcuQWw-93]Fanatics often view a much *larger proportion* of people as deserving punishment, sometimes everyone who merely disagrees with their ideology. Moreover, fanatics often view their ideological enemies as irredeemably evil and as deserving of more prolonged and intense suffering (e.g., [ISIS burning enemies alive](https://www.npr.org/sections/thetwo-way/2015/02/04/383742600/jordan-executes-2-militants-after-isis-burns-captured-pilot-alive)).

In the future, such preferences could prove catastrophic. Advanced AI could enable fanatics to inflict extreme suffering on their perceived enemies at unprecedented scale, potentially forever, at relatively small cost.

### Empirical evidence: how many people endorse eternal extreme punishment?

Such scenarios originally struck us as extremely far-fetched. But recent empirical evidence may suggest otherwise.

We surveyed 3,082 adults across six countries—the United States (majority Christian), the UK (majority atheist/agnostic), China (majority atheist/agnostic), and Turkey, Pakistan, and Saudi Arabia (majority Muslim)—about their attitudes toward extreme, eternal punishment. All subsequent analysis includes only the 1,084 participants who met our inclusion criteria.[^-bxXhDCzXq8bJcuQWw-94]

We assessed extreme retributivist attitudes through three questions:

*   “If hell didn't exist, or if it stopped existing, we should create it to punish all evil people with extreme suffering forever." \[“**create hell**”\]
*   “If I could create a system that makes deserving people feel unbearable pain forever, I would.” \[“**would** **create system**”\]
*   “We should have a system in the world that makes deserving people feel unbearable pain forever.” \[“**want system**”\]

We focus here on participants we call "*consistent and concerning*": those who agreed (≥5 on the 7-point scale) with at least one of the above, while also selecting "Forever" and at least “1%” on two independent questions:

*   “For how long should the most evil or extremely immoral people be made to experience extreme, unbearable pain?” with options ranging from “Not at all”, “Minutes”, “Hours”, etc. to “Millions of years”, and “Forever”. \[“**duration**”\]
*   “Of all the people in the world, what proportion deserve unbearable pain *forever?”* (Use the slider: 0% to 100%) \[“**proportion**”\]

11–14% in the US, UK, and Pakistan were consistent and concerning responders for **create hell**, rising to 19–25% in China, Saudi Arabia, and Turkey. Results for **want system** (8–11% and 19–24%) and **would create system** (7–11% and 16–23%) showed roughly similar patterns.  
Looking beyond the conjunctive measure, when asked what *proportion* of humanity deserves unbearable pain forever, more than half of participants[^-bxXhDCzXq8bJcuQWw-95]answered 1% or higher; a quarter answered 7% or higher.

Wanting hell to be created correlated at *ρ* = 0.25 with sadism (measured via the [ASP-8 scale](https://www.researchgate.net/publication/359195728_The_Revised_Assessment_of_Sadistic_Personality_ASP-8_Evidence_for_Validity_across_Four_Countries))[^-bxXhDCzXq8bJcuQWw-96]and with several of our items assessing ideological fanaticism (*ρ* = 0.26-0.37, all *p* < 0.001).[^-bxXhDCzXq8bJcuQWw-97]This suggests that ideological fanatics and malevolent actors are indeed more likely to endorse extreme retributive attitudes.

**Caveats and limitations**  
These are seemingly concerning results, but they need to be interpreted with caution.

The questions involved complex and abstract hypotheticals; responses to such questions are [notoriously unreliable](https://forum.effectivealtruism.org/posts/vZ4kB8gpvkfHLfz8d/critique-of-macaskill-s-is-it-good-to-make-happy-people?commentId=mxm3d9FyWp4EGAxXJ)[^-bxXhDCzXq8bJcuQWw-98]and hopefully don’t reflect genuine commitments participants would actually act upon.[^-bxXhDCzXq8bJcuQWw-99]

One notable limitation is that many participants answered *inconsistently* across questions, which is why we focus on the conservative conjunctive measure above. In non-Western samples especially, responses for individual questions in isolation were much higher than the numbers we reported above.[^-bxXhDCzXq8bJcuQWw-100]

Other limitations include potential differences in meaning across translations[^-bxXhDCzXq8bJcuQWw-101], the non-representative nature of online survey samples[^-bxXhDCzXq8bJcuQWw-102], and the inherent unreliability of crowdsourced surveys where participants may quickly click through questions without genuine reflection to maximize hourly earnings.

Nevertheless, these results are concerning. Substantial fractions of multiple populations seem to endorse extreme retributivist attitudes, even on conservative estimates, and seem to apply them broadly, not just to a few of history’s greatest villains.

### Religious fanatical retributivism

Fanatical retributivist attitudes could translate into astronomical suffering through at least two pathways: religious and [secular](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Secular_fanatical_retributivism) (discussed below). We focus primarily on the religious case as it seems overall more concerning.

To our knowledge, no serious theologian has ever advocated actively creating technological hells. Most would likely consider the idea a blasphemous misinterpretation. But such arguably confused views may nevertheless arise, partly because the concept of hell is central to the two largest religions: Christianity (2.4 billion followers) and Islam (2 billion followers). The Bible frequently discusses hell, with [Jesus repeatedly](https://timothykeller.com/blog/2008/8/1/the-importance-of-hell) describing it [in detail](https://www.thegospelcoalition.org/article/the-uncomfortable-subject-jesus-addressed-more-than-anyone-else/). The concept of hell ([Jahannam](https://en.wikipedia.org/wiki/Jahannam)) is also central to Islam, with the Quran containing at least 92 “significant passages” about hell, compared to 62 about paradise (Jones, [2008, p. 110](https://real-j.mtak.hu/18424/1/The-Arabist_28_-_29_2008.pdf)).[^-bxXhDCzXq8bJcuQWw-103]

Various scriptural passages and theological writings articulate doctrines that, when combined with fanatical misinterpretation and transformative technology, become concerning:

1.  Hell is a physical reality and morally necessary for divine justice (with certain passages indicating that those in heaven witness or even rejoice in divine judgment)
2.  The suffering is eternal and its intensity far exceeds any earthly pain
3.  A large fraction of humanity is destined for hell

The following examples from foundational texts illustrate these doctrines: Thomas Aquinas, arguably Christianity's most influential theologian, wrote in his *Summa Theologica* that "the blessed will rejoice in the punishment of the wicked \[...\]. In order that the happiness of the saints may be more delightful to them \[...\] they are allowed to see perfectly the sufferings of the damned." The Quran states: “Surely those who reject our verses, we will cast them into the Fire. Whenever their skin is burnt completely, we will replace it so they will constantly taste the punishment.” ([4:56](https://quran.com/4/56)). According to two hadiths in the [Sahih al-Bukhari](https://en.wikipedia.org/wiki/Sahih_al-Bukhari), the second-most authoritative text after the Quran in Sunni Islam, the ratio of people going to hell compared to paradise will be 100:1 ([Book 81, Hadith 118](https://sunnah.com/bukhari:6529)) or even 1000:1 ([Book 81, Hadith 119](https://sunnah.com/bukhari:6530)).[^-bxXhDCzXq8bJcuQWw-104]See [Appendix C](https://docs.google.com/document/d/1MdSEu-CTsCSWAZ4rY7aFU-ZYe9zFHg54rrICtJOOsZU/edit?tab=t.0) for further relevant quotes.

Importantly, many believers reject simplistic readings of holy texts as being incompatible with divine love, and much of contemporary theology tends to emphasize forgiveness and mercy. Within Christianity, doctrines like [universalism](https://en.wikipedia.org/wiki/Christian_universalism) (ultimate redemption of all souls) and [annihilationism](https://en.wikipedia.org/wiki/Annihilationism) (the destruction of unredeemed souls at death rather than eternal torment) are popular among laypeople and widely supported by scholars of many denominations. [Sufism](https://en.wikipedia.org/wiki/Sufism), one of the oldest Islamic traditions, similarly emphasizes divine mercy over retribution.

**Why might religious fanatics create technological hells?**  
Christianity and Islam unambiguously establish that God alone created hell and reserves judgment exclusively for himself. Most religious believers immediately recognize that human attempts to implement divine punishment on their own would amount to blasphemy and likely violate basic theological principles. The risk thus emerges primarily not from orthodox theology but from confusion or willful misinterpretation—for example, some may want to rationalize their sadistic preferences.[^-bxXhDCzXq8bJcuQWw-105]So, how could any religious believer possibly come to believe that they should create hell on their own, rather than leaving it to God?

Several pathways seem at least conceivable.

*Making holy scriptures “come true”*  
As we have seen, religious texts describe heaven and hell as concrete realities. Some fanatics may aim to [reshape reality](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#AI_alignment_may_not_solve_the_fanaticism_problem_either) to correspond to their pre-existing beliefs. The general practice of taking action to fulfill religious prophecies has broad historical precedent—from [Christian Zionists supporting Israeli territorial expansion](https://www.washingtonpost.com/news/politics/wp/2018/05/14/half-of-evangelicals-support-israel-because-they-believe-it-is-important-for-fulfilling-end-times-prophecy/) to fulfill end-times prophecy, to ISIS’s disproportionate focus on Dabiq because of a hadith declaring it the site of an apocalyptic battle.

We may thus speculate that some fanatics may be similarly motivated to immanentize their holy scriptures’ ideas about the afterlife. Concerningly, advanced technologies like transformative AI may allow them to actually create both a paradise where believers can dwell in eternal bliss and a hell where infidels and sinners suffer eternally. From this confused perspective, creating heaven and hell wouldn’t be blasphemy but an act of faith: maximizing the veracity of their religion by physically manifesting its claims.

*Sycophantic AIs*  
It’s conceivable that an (possibly misaligned) artificial superintelligence might create hell without explicit instructions by their users, because of extreme sycophancy—not merely telling users what they want to hear, but *reshaping* *reality* so users experience what they want or expect to exist (or believe should exist).

Consider a superintelligent AI (semi-)aligned with a religious fanatic. As we explore in more detail [below](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#AI_alignment_may_not_solve_the_fanaticism_problem_either), fanatics typically prefer vindication of existing beliefs over truth-seeking. Consequently, such an AI might aim to make religious scriptures literally true rather than helping its user discover what's actually real. Unless such an AI had strong principles against deception—the kinds of epistemic principles fanatics are not known for—it might autonomously create heaven and hell to validate its user's beliefs. The AI could make it appear that heaven and hell were created by God, or even present itself as God. Finding themselves in what appears to be the paradise described in their scriptures, fanatics might also expect to be able to witness the suffering of those damned to hell since their holy books describe this as a feature of paradise. An AI aiming to fulfill all expectations might thus create hell to "complete the experience".

*Idiosyncratic interpretations and emerging technologies*  
Religious texts are open to a wide variety of interpretations. It seems hard to rule out—especially in light of the concerning empirical evidence discussed above—that some misguided fanatics would conclude that creating heaven and hell is somehow an act of devotion or their sacred duty.[^-bxXhDCzXq8bJcuQWw-106]Religious interpretations may also change over time, and other dynamics could further exacerbate fanatical tendencies (e.g., so-called [purity spirals](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#More_speculative_dynamics__purity_spirals_and_self_inflicted_suffering) which we’ll explore later).

Future technology like superintelligent AI or whole brain emulation may also interact with religious beliefs in ways we can't fully anticipate.[^-bxXhDCzXq8bJcuQWw-107]These could spawn entirely new religious movements, syncretic reinterpretations of existing faiths, or idiosyncratic religious beliefs held by powerful individuals.

### Secular fanatical retributivism

As our survey data suggests, extreme retributivist intuitions may not be uncommon even among atheists and agnostics. The neuroscientist Robert Sapolsky, for instance, opens his book [*Behave*](https://www.amazon.com/Behave-Robert-M-Sapolsky-audiobook/dp/B06XVYHXDV) with a vivid description of his own retributivist fantasy about Hitler, culminating in wanting him to experience extreme suffering where "every moment feels like an infinity spent in the fires of hell."[^-bxXhDCzXq8bJcuQWw-108]That even someone known for his compassionate approach to understanding human nature experiences such impulses suggests that retributivist intuitions are deeply embedded in human psychology.[^-bxXhDCzXq8bJcuQWw-109]

As mentioned above, officially secular ideologies have produced their own torture systems. Future secular fanatics with access to advanced AI might create suffering systems justified by political rather than theological reasoning—punishing "traitors", "counterrevolutionaries," or whomever their ideology designates as irredeemably evil.

Ideological fanaticism could undermine long-reflection-style frameworks and AI alignment
----------------------------------------------------------------------------------------

Superintelligent AI could [enable](https://forum.effectivealtruism.org/posts/KqCybin8rtfP3qztq/agi-and-lock-in) individuals or [regimes](https://80000hours.org/problem-profiles/risks-of-stable-totalitarianism/) to permanently [lock-in their values](https://80000hours.org/podcast/episodes/will-macaskill-century-in-a-decade-navigating-intelligence-explosion/#is-global-lock-in-really-plausible-010837), potentially creating an unrecoverable dystopia (Ord, 2020, ch. 5.3). Misaligned AI could lead to human extinction or permanent disempowerment. Yet superintelligent AI could also enable truly [utopian outcomes](https://nickbostrom.com/utopia). The development of superintelligent AI may thus be the most pivotal event in the history of the universe (MacAskill, 2022, ch.4).

To avoid locking in undesirable values, a process akin to a [*long reflection*](https://forum.effectivealtruism.org/topics/long-reflection) may be helpful, where humanity carefully reflects on how to best achieve its long-term potential before rushing to take irreversible actions.

### Ideological fanaticism threatens collective moral deliberation

The literal idea of a “long reflection” is probably unrealistic, but [more refined proposals](https://lukasfinnveden.substack.com/i/140338085/concrete-decision-making-proposals), like “viatopia”,[^-bxXhDCzXq8bJcuQWw-110]retain a similar emphasis on careful exploration and moral reflection. Whatever term we use, reaching truly utopian outcomes will likely require that major decisions—e.g., various “grand challenges” (MacAskill & Moorhouse, [2025](https://www.forethought.org/research/preparing-for-the-intelligence-explosion), section 4)—are at least partially guided by thoughtful deliberation (MacAskill & Moorhouse, 2025a).

So, who should participate in the long reflection (or related post-AGI governance frameworks emphasizing collective moral deliberation), and under what rules? A natural Schelling point would be to give all living humans equal representation—an approach that is fair, democratic, and inclusive. (Note that participation and influence aren't necessarily binary: governance frameworks could include diverse voices while still constraining what outcomes are permissible. Moreover, decisions need not all be made at once; iterative approaches across smaller questions are perhaps more desirable.)

One line of argumentation for high inclusivity runs as follows. A wide range of viewpoints increases the chance of either discovering objective moral truth (if moral realism is true) or (if moral anti-realism is true) at least converging on robust moral principles that survive scrutiny from multiple perspectives, with the eventual outcome being at least broadly acceptable or even fairly attractive for many different value systems. Additionally, even if only a small fraction of agents participating in the long reflection converge on the best view, they may engage in *moral trade* with other value systems,[^-bxXhDCzXq8bJcuQWw-111]such that the ultimate outcome of the long reflection may not be substantially worse than the “optimal” outcome. Trade and compromise could be particularly important if the best futures constitute a narrow target that is hard to reach (MacAskill & Moorhouse, 2025a).

But do these arguments extend to ideological fanatics? There are several reasons to think they don't. First, ideological fanatics, due to their absolute moral certainty, reflectively endorse locking in their values and beliefs, while eliminating dissent. Fanatics participating in the long reflection would seemingly make it *less* likely that we discover a hypothesized “correct moral view”, not more.[^-bxXhDCzXq8bJcuQWw-112]

Second, gains from moral trade may be difficult to achieve when fanatics are at the bargaining table. When value systems are what MacAskill and Moorhouse (2025b, [section 3](https://www.forethought.org/research/convergence-and-compromise#31-trade-and-compromise)) call “resource-compatible”, the potential gains from trade could be enormous.[^-bxXhDCzXq8bJcuQWw-113]For instance, as they note, "hedonists might only care about bliss, and objective list theories might care primarily about wisdom; they might potentially agree to create a shared society where beings are both very blissful and very wise.” However, ideological fanaticism typically features highly resource-*incompatible* values. Nazi ideology, for instance, requires the elimination of all Jews and other ‘inferior’ races, so there are no hybrid arrangements that would satisfy both Nazis and Jews. Fanatics also often have [resource-hungry and insatiable](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_may_give_rise_to_extreme_optimization_and_insatiable_moral_desires) preferences (cf. Shulman [2010](https://intelligence.org/files/BasicAIDrives.pdf), p.4-6). What is more, fanatics may view any form of trade or compromise as morally reprehensible, equating it with betrayal of their sacred values. As a result, including fanatics in long reflection-like processes could actually prevent, rather than facilitate, moral trade among diverse value systems.

Ideological fanaticism could also harm other value systems more directly. First, fanatics tend to be highly intolerant and view an enormous range of behaviors and beliefs as immoral. For example, many religious fundamentalists oppose homosexuality, abortion, divorce, suicide, euthanasia, sex before marriage, and even music, singing, most clothes, most books, the Smurfs[^-bxXhDCzXq8bJcuQWw-114], etc.—see also the concept of [haram](https://en.wikipedia.org/wiki/Haram#Forbidden_categories_of_actions), the [613 commandments](https://en.wikipedia.org/wiki/613_commandments), or the banning of large fractions of literature and art in [Nazi Germany](https://en.wikipedia.org/wiki/Nazi_book_burnings). In other words, fanatics may have extremely “[*fussy*](https://forum.effectivealtruism.org/posts/wE7KPnjZHBjxLKNno/ai-things-that-are-perhaps-as-important-as-human-controlled#Introduction1)” preferences, which are *incompatible with the great majority of possible world states* and thus the fulfillment of most other value systems. From an upside-focused perspective, this could be extremely concerning. Fanatics might view extremely happy posthuman digital minds or hedonium as immoral abominations, and may thus oppose the creation of truly [utopian futures](https://nickbostrom.com/utopia) filled with astronomical amounts of flourishing.

Second, fanatical ideologies may aim to create world states that are not only suboptimal but highly disvaluable according to most other value systems. One mechanism is [fanatical retributivism](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Fanatical_retributivism_may_lead_to_astronomical_suffering) discussed above; another is fanatics' plausibly greater propensity to use hostile bargaining tactics and engage in conflict.

### AI alignment may not solve the fanaticism problem either

> "Are we going to ... create minds that help us seek the truth \[or\] create minds that have *whatever beliefs we want them to have*, *stick to those beliefs* and try to *shape the world around those beliefs*? \[...\] Some humans really ... will want to say, … 'This is the religion I follow. This is what I believe in. This is what I care about. And I am creating an AI to help me promote that religion, not to help me question it or revise it or make it better.'" —Holden Karnofsky, emphasis added ([2023](https://80000hours.org/podcast/episodes/holden-karnofsky-how-ai-could-take-over-the-world/#misaligned-ai-might-not-kill-us-all-and-aligned-ai-could-be-catastrophic-002718))[^-bxXhDCzXq8bJcuQWw-115]

A crucial question in AI alignment is “aligned with whom or what” (e.g., Barnett, [2023](https://forum.effectivealtruism.org/posts/Hhtvwx2ka4pzoWg7e/ai-alignment-shouldn-t-be-conflated-with-ai-moral); Carlsmith, [2024](https://www.lesswrong.com/posts/cnv5g7jCWLw9LYKxa/an-even-deeper-atheism-3#Human_paperclippers_); Chen, [2023](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4684068))? Gabriel ([2020](https://link.springer.com/article/10.1007/s11023-020-09539-2)) distinguishes six possible alignment targets, from literal instructions to moral values. For our purposes, these can be grouped into three categories:

1.  **Narrow intent-alignment**: The AI does what the user currently wants—following their unreflected, surface-level preferences.
2.  **Preference idealization**: The AI does what the user would want if they knew more, reflected more, or were more like the person they wished they were.
3.  **“Principles-based alignment”**: The AI is aligned with certain values, principles, or moral frameworks, and not indexed to any particular user's preferences. For instance, Anthropic's Constitutional AI trains models to follow a written set of principles; more generally, AIs could be aligned with classical utilitarianism, a holy book, or broad values like “honesty”.

(1) is obviously dangerous when the principals are fanatical (or malevolent). The more interesting question is whether (2) or (3) might help.

**Preference idealization won't necessarily deradicalize fanatics**  
Yudkowsky's coherent extrapolated volition ([CEV](https://intelligence.org/files/CEV.pdf)) is largely outdated, but it can illustrate the broader idea of preference idealization: that AI should serve not users' current, unreflected preferences but their *idealized* (extrapolated) preferences—what they would want if they knew more, reflected more, and were "more the people they wished they were."[^-bxXhDCzXq8bJcuQWw-116]Similar ideas appear in various forms throughout the literature.[^-bxXhDCzXq8bJcuQWw-117]Would such preference idealization lead to good outcomes when the principals are fanatical?

Unfortunately, this seems unlikely.[^-bxXhDCzXq8bJcuQWw-118]Fanatics' deepest preference often appears to be vindication of existing beliefs rather than impartial, evidence-based truth-discovery.[^-bxXhDCzXq8bJcuQWw-119]When their beliefs conflict with reality, fanatics often attempt to *reshape reality* to correspond to their pre-existing beliefs, rather than update their beliefs to correspond to reality.[^-bxXhDCzXq8bJcuQWw-120]The Jewish fanatic [Yigal Amir](https://en.wikipedia.org/wiki/Yigal_Amir), for instance, assassinated the compromise-oriented incumbent Prime Minister of Israel in 1995, in part because he apparently wanted to make Torah predictions come true.

Fundamentally, many fanatics may actively reject the core premise that 'knowing more' should change beliefs. A religious fanatic who believes in absolute divine revelation sees no room for improvement upon God's word—any change would be heresy. For many fanatics, being ‘more the people they wished they were’ may not mean being more reflective, rational, humble, or compassionate; they may wish to be more devout, more unwavering, and more zealous.[^-bxXhDCzXq8bJcuQWw-121]So even the idealized preferences of ideological fanatics might result in terrible outcomes.[^-bxXhDCzXq8bJcuQWw-122]

For illustration, imagine that in 2040, the leader of a theocratic state obtained access to superintelligent AI. He has overseen torture and execution of thousands, sponsored terrorist organizations worldwide, and allowed female political prisoners to be raped before execution to [ensure](https://wncri.org/2015/11/13/female-prisoners-virgins-raped/) their [damnation](https://iranwire.com/en/news/117116-ex-official-virgin-prisoners-were-raped-to-prevent-them-going-to-paradise/).

How should an AI aligned with his preferences act? Judging from his lifetime of choices, he seems to value enforcing his own religious interpretation above all else. He certainly doesn’t seem to place great value on truth-seeking or changing his mind when encountering new evidence. What is more, updating his beliefs to accurately reflect reality would be enormously painful, obliterating every shred of meaning, purpose, and accomplishment he once felt. The AI would need to convince him that his once cherished beliefs are grotesquely false, that his life's work amounted to a series of pointless atrocities, that his heroes were almost entirely wrong while his enemies were largely correct. Even psychologically healthy, non-fanatical people resist changing their minds about comparatively trivial matters because admitting mistakes is painful. For someone whose entire existence centers around their ideology, wouldn't an aligned AI (that is, one that truly has this guy’s best interests at heart) try to spare him all this misery and instead try to reshape reality to match his beliefs? Are we confident that what he *really* wants, deep down, is to hear the truth—especially in light of how he has lived his life so far?

**Principles-based alignment won't necessarily help either if fanatics are involved**  
What about aligning AI with some external moral framework or set of principles, independent of any user's preferences?

This approach seems safer than relying on preference idealization alone and could indeed help, provided that reasonable people control the process and choose sensible principles. But it doesn't *automatically* solve the fanaticism problem. If fanatics have significant influence over which principles the AI is aligned with—if the alignment target becomes some holy book or "Mein Kampf"—we're back to disaster. In practice, decisions about alignment targets will be made by humans, including, potentially, fanatical humans.[^-bxXhDCzXq8bJcuQWw-123]

### Prevalence of reality-denying, anti-pluralistic, and punitive worldviews

The practical importance of these concerns—for both collective deliberation proposals and AI alignment approaches—depends in part on how prevalent such worldviews actually are. As [discussed earlier](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Estimating_the_global_scale_of_ideological_fanaticism), a non-trivial fraction of humanity could reasonably be classified as ideological fanatics.

However, even many non-fanatical humans living in open societies don't seem to prioritize truth-seeking. Some explicitly acknowledge this: around 20% of people in Western societies do not think their beliefs should be based on evidence (Pennycook et al., [2020](https://www.cambridge.org/core/journals/judgment-and-decision-making/article/on-the-belief-that-beliefs-should-change-according-to-evidence-implications-for-conspiratorial-moral-paranormal-political-religious-and-science-beliefs/254C6EF93A5E037998EB7E1003627CB6))[^-bxXhDCzXq8bJcuQWw-124]. Similarly, 30% of US Americans ([WVS](https://www.worldvaluessurvey.org/WVSOnline.jsp)) report that when science conflicts with their own religious beliefs they would stick with their religious beliefs.[^-bxXhDCzXq8bJcuQWw-125]More generally, most humans prioritize [socially adaptive beliefs](https://www.conspicuouscognition.com/p/people-embrace-beliefs-that-signal-354)—i.e., those that make them look or feel good—over true beliefs (Williams, [2021](https://www.researchgate.net/profile/Daniel-Williams-22/publication/337275990_Socially_Adaptive_Belief/links/5dce7ba7299bf1b74b42cde7/Socially-Adaptive-Belief.pdf)).[^-bxXhDCzXq8bJcuQWw-126]This makes including fanatics in the long reflection or related proposals even riskier, because we can't be confident that a clear majority of impartial truth-seekers will outweigh fanatical and non-truth-seeking voices.

Many of us may overestimate humanity's commitment to truth-seeking because of biases like wishful thinking, [typical mind fallacy](https://www.lesswrong.com/w/typical-mind-fallacy) and unrepresentative sampling: Most longtermists and AI governance researchers grew up or work in WEIRD (cf. Henrich, [2021](https://en.wikipedia.org/wiki/The_WEIRDest_People_in_the_World))—Western, educated, industrialized, rich, and democratic—societies where support for evidence-based belief revision and science remains comparatively high while support for coercive ideology enforcement (such as death penalty for apostasy, theocratic governance, or extreme punishment of dissent) is rather low.[^-bxXhDCzXq8bJcuQWw-127]But elsewhere, support for such worldviews is much higher, sometimes even reaching majority levels: for instance, over 50% in Iran and over 90% in Pakistan (World Values Survey, [2017-2022](https://www.worldvaluessurvey.org/WVSOnline.jsp)) believe that “whenever science and religion conflict, religion is always right” and that “the only acceptable religion is my religion”.

Ideological fanaticism could worsen many other risks
----------------------------------------------------

### Differential intellectual regress

Ideological fanaticism may exacerbate most other risks by driving [*differential intellectual* *regress*](https://longtermrisk.org/differential-intellectual-progress-as-a-positive-sum-project). Regimes governed by fanatical ideologies are often able to maintain or even advance technological capabilities, while systematically degrading [philosophical sophistication](https://www.lesswrong.com/posts/EByDsY9S3EDhhfFzC/some-thoughts-on-metaphilosophy), [wisdom](https://forum.effectivealtruism.org/posts/XxBwnSt4BcBhpJJyX/essay-competition-on-the-automation-of-wisdom-and-philosophy), moral [reflection](https://forum.effectivealtruism.org/posts/TmPSYEdQ6Mrb9oMLA/ai-strategy-given-the-need-for-good-reflection), reason, [societal decision-making](https://80000hours.org/problem-profiles/improving-institutional-decision-making/) and cooperation-conducive institutions. (This degradation happens through multiple reinforcing mechanisms, e.g., imposing [censorship](https://en.wikipedia.org/wiki/Censorship_in_China) and propaganda, replacing experts with [ideological loyalists](https://en.wikipedia.org/wiki/Trofim_Lysenko), and rewarding conformist dogmatism over open discourse and evidence-based reasoning.)

This pattern is particularly concerning from a longtermist perspective. Humanity badly needs wisdom, rationality, and strong institutions to navigate [grand challenges](https://www.forethought.org/research/preparing-for-the-intelligence-explosion#4-grand-challenges) like the governance of AGI. Improving [institutional decision-making](https://80000hours.org/problem-profiles/improving-institutional-decision-making/) and [forecasting](https://www.openphilanthropy.org/focus/forecasting/) are already considered important cause areas for this reason.

### Ideological fanaticism may give rise to extreme optimization and insatiable moral desires

> Fanaticism never sleeps: it is never glutted: \[...\] it is never stopped by conscience; for it has pressed conscience into service. Avarice, lust, and vengeance, have piety, benevolence, honour; fanaticism has nothing to oppose it. —[Jeremy Bentham](https://www.goodreads.com/quotes/1271188-if-a-man-happen-to-take-it-into-his-head)

Most non-fanatical humans tend to be [satisficers](https://en.wikipedia.org/wiki/Satisficing). They compromise, are opportunistic, do what is convenient, and are happy to trade with others. In contrast, fanatics are more likely to [*maximize*](https://forum.effectivealtruism.org/posts/T975ydo3mx8onH3iS/ea-is-about-maximization-and-maximization-is-perilous) by any means necessary, ultimately aiming to rearrange every atom in the universe to align with their ideology’s demands. Such [extreme optimization](https://www.lesswrong.com/posts/EaLCAZ4bgycY6hFC8#2_3_2_Will_agents_optimize_extremely_intensely__and_does_this_matter_) for an idiosyncratic value system is incompatible with the fulfillment of most other value systems.

Unfortunately, *moral preferences in general* seem more scope-sensitive and resource-hungry (Shulman, [2010](https://intelligence.org/files/BasicAIDrives.pdf), pp. 4-6) than selfish desires.[^-bxXhDCzXq8bJcuQWw-128]From a selfish perspective, most people would presumably be quite happy with only a galaxy or two; some especially frugal ones might even make do with a single solar system.[^-bxXhDCzXq8bJcuQWw-129]Uncolonized distant galaxies are meaningless to most egoists, but an “[astronomical waste](https://nickbostrom.com/papers/astronomical-waste/)” to classical utilitarians. (To be clear, this scope-sensitivity isn't itself problematic; it's arguably a feature of genuine moral concern.)

Given their propensity to torture, murder, launch wars, and so on, fanatics are often thought of as being immoral. But this does *not* mean that fanatics lack moral conviction. In many ways, the dangers of ideological fanaticism arguably stem from *excessively strong* moral convictions. [Skitka et al. (2005)](https://leeds-faculty.colorado.edu/mcgrawp/PDF/SkitkaBaumanSargis2005.pdf), for instance, found that stronger moral conviction leads to less tolerance and cooperation. What fanatics lack is humility, moderation, concern for others’ well-being (even if they disagree), and moral constraints setting limits on what constitutes acceptable behavior in pursuit of one’s goals.

For fanatics, perhaps no victory is total enough, no punishment severe enough, no empire extensive enough. Ultimately, ideological fanaticism may end up removing all constraints on maximizing behavior—making their morality uniquely “[scary](https://www.alignmentforum.org/posts/y5jAuKqkShdjMNZab/morality-is-scary)”.

### Apocalyptic terrorism

Some fanatical groups have extremely conflict-conducive preferences. Some even believe that they need to *actively* *bring about the apocalypse*—involving enormous suffering and destruction—in order to usher in their conception of utopia. ISIS, for example, believes that they must defeat the armies of “Rome” (often interpreted as the US) at Dabiq, which will “initiate the countdown to the apocalypse” (Wood, [2015](https://www.theatlantic.com/magazine/archive/2015/03/what-isis-really-wants/384980/)). Other groups may want to kill literally everyone. Motivated by Christian, Hindu, Buddhist, and conspiratorial elements, the Aum Shinrikyo [doomsday cult](https://80000hours.org/podcast/episodes/kevin-esvelt-stealth-wildfire-pandemics/#omnicidal-actors-in-history-000710) tried to create a [global pandemic](https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack) in order to “redeem” humanity.

### S-risk-conducive propensities and reverse cooperative intelligence

Taylor ([2025](https://longtermrisk.org/measurement-research-agenda/#21_Characterizing_s-risk-conducive_properties)) uses the term *s-risk-conducive properties* to describe properties that, if possessed by powerful agents like transformative AIs, could contribute to s-risks, largely by initiating or escalating conflict. Taylor outlines the following broad categories:

*   Tendencies to escalate conflict, make large threats when in conflict with other agents, and enact large punishments against wrongdoers (punitiveness).
*   Spite, vengefulness, and risk tolerance or risk-seeking behavior.
*   Absence of beneficial capabilities that allow actors to avoid or de-escalate conflict (absence of cooperative intelligence).

It's notable that many ideological fanatics tend to exhibit essentially all of these characteristics. We could describe this general cluster of conflict-conducive traits as a form of ‘*reverse cooperative intelligence*’—essentially the opposite of what the [cooperative AI](https://www.nature.com/articles/d41586-021-01170-0) research agenda seeks to develop. While cooperative intelligence involves engaging in dialogue, building trust, de-escalating conflict, and finding mutually beneficial compromise solutions, fanatics instead tend to demonize others over the slightest disagreement, treat compromise as intolerable betrayal, endorse brutal violence, and generally escalate conflict.

### More speculative dynamics: purity spirals and self-inflicted suffering

Fanatical retributivism isn't the only source of astronomical suffering in “fanatical utopias”. The following admittedly far-fetched dynamics could create perpetual suffering even after ideological fanatics have achieved total control and eliminated all designated enemies.

[*Purity spirals*](https://en.wikipedia.org/wiki/Purity_spiral) (or *moral outbidding*), where members compete to demonstrate ideological commitment through ever more extreme measures, could amplify several risks discussed above. This dynamic—seen e.g. in the French Revolution's Reign of Terror or Mao’s Cultural Revolution—could intensify fanatical retributivism itself, with members competing to advocate ever crueler punishments for ideological opponents. Anyone suggesting mere execution rather than eternal torture might be branded as weak or traitorous. Purity spirals could also continuously expand the definition of "enemy" or “evil”, ensuring that there is always someone left to punish. Similar dynamics, fueled by resentment and hatred, may also lead fanatics to actively seek to instantiate the opposite of their enemies’ values.

Some ideological fanatics may also embrace *asceticism* or *self-inflicted suffering* for ideological reasons, viewing suffering as purifying or virtuous. Unlike retributivism, which targets enemies, this could mean imposing suffering on even the "pure" in-group—potentially forever. (These scenarios are explored further in [Appendix D](https://docs.google.com/document/d/13zSCUf2j7KReWjcPAOBpR9XoqUJ9BP9UIqbdKaznCII/edit?tab=t.0#bookmark=id.289m8avvwlqj).)

### Unknown unknowns and navigating exotic scenarios

While any specific catastrophic scenario tends to be unlikely, the fundamental characteristics of ideological fanaticism (especially its dogmatism, bad epistemics, and blind hatred) make it more likely to cause harm across a wide range of potential scenarios, including ones we haven’t yet identified or foreseen. Actors who embody such traits seem also less likely to properly navigate exotic scenarios—[acausal trade](https://www.lesswrong.com/w/acausal-trade), [evidential cooperation in large worlds](https://www.lesswrong.com/w/evidential-cooperation-in-large-worlds), [updateless decision theory](https://www.lesswrong.com/w/updateless-decision-theory), and so on. Fanatics seem therefore particularly worrisome from a perspective of unknown unknowns and deep uncertainty.[^-bxXhDCzXq8bJcuQWw-130]

Interventions
=============

We organize potential interventions into two broad categories. First, we discuss “conventional” [political and societal interventions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Societal_or_political_interventions) which appear useful across a wide spectrum of worldviews. Second, we discuss more directly [artificial intelligence-related interventions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Artificial_intelligence_related_interventions) that tend to be more neglected and plausibly higher leverage. (This area is where we expect to focus the majority of our own work going forward.)

However, the boundary we draw between political/societal versus AI-related interventions is somewhat artificial and potentially misleading. Exclusive focus on a narrow conception of AI safety would risk neglecting political & societal interventions that likely improve AI outcomes. If we want society to make reasonable decisions about the future of transformative AI, it would help to have reasonable people in positions of political power, including in various parts of the US government. Likewise, many AI-focused interventions rely on political will and governmental competence.

Most interventions discussed below are not novel and overlap with existing longtermist priorities. But the fanaticism lens could shift priorities and, especially from an s-risk perspective, make certain directions that previously seemed neutral or counterproductive appear more promising. We're especially excited about preventing AI-enabled coups, compute governance, making AIs themselves non-fanatical, and developing fanaticism-resistant AGI governance proposals. That said, most of our recommendations are tentative and some may prove misguided upon further investigation. Moreover, we’re likely not aware of the most promising anti-fanaticism interventions; hopefully some can be identified by further research.

Societal or political interventions
-----------------------------------

No intervention in this section scores exceptionally highly in terms of importance, tractability, and neglectedness. However, given that enormous sums are spent in this area, making these efforts even [marginally more cost-effective could still be valuable](https://forum.effectivealtruism.org/posts/o5LBbv9bfNjKxFeHm/marginally-more-effective-altruism).

### Safeguarding democracy

The US is the world’s most powerful country and leads in AI development. Consequently, safeguarding US democracy[^-bxXhDCzXq8bJcuQWw-131]seems crucial to reduce many long-term risks, including those from malevolent and fanatical actors. Of course, other powerful democracies may also influence humanity's long-term future. Preventing democratic backsliding in countries like India and across Europe is therefore also important.[^-bxXhDCzXq8bJcuQWw-132]

How can we prevent further democratic backsliding? Below, we focus on reducing polarization and strengthening anti-fanatical principles. ​​We emphasize these not because they are necessarily the most important interventions overall, but because they are most directly related to the long-term risks of ideological fanaticism.

### Reducing political polarization

Excessive political polarization corrodes democratic norms and institutions, creates legislative gridlock, and increases intergroup hostility (Levitsky & Ziblatt, 2018; Binder, 2004; Mason, 2018).[^-bxXhDCzXq8bJcuQWw-133]Most worryingly for our purposes, polarization seems to create the psychological and social conditions that exacerbate the core characteristics of ideological fanaticism: epistemic dogmatism (pushing people to choose between beliefs approved by their respective tribe, rather than following evidence), in-group loyalty (defending people on one’s side no matter what) and tribal hatred (all political opponents are viewed as existential enemies[^-bxXhDCzXq8bJcuQWw-134]), and calls for extremist acts, including political violence.

This dynamic seems to play out through various vicious feedback loops: Extremists on both sides adopt increasingly irrational positions—sometimes [embracing absurd beliefs](https://onlinelibrary.wiley.com/doi/full/10.1111/mila.12392) as costly signals of tribal loyalty. Each side's extremism in turn validates the other's worst fears, making people even more tribal and irrational. Meanwhile, moderate or nuanced positions become increasingly untenable, as everyone must choose a side or be attacked by both (even if they criticize [one side](https://www.conspicuouscognition.com/p/tribalism-corrupts-politics-even) much more than the other).

Ultimately, such polarization spirals may give rise to two opposing fanatical ideologies.[^-bxXhDCzXq8bJcuQWw-135](In the academic literature, this is studied as *competitive extremism* or *mutual radicalization*.[^-bxXhDCzXq8bJcuQWw-136]) Historical examples include Weimar Germany, where Communists and Nazis together commanded just 13% of votes in 1928 but surged to 56% by 1933, their street violence feeding off each other, or 1930s Spain, where far-left anarchists and far-right fascists escalated toward civil war. In each case, extremists had a paradoxically symbiotic relationship where each side's excesses were used to justify the other's apocalyptic narratives and increasingly extremist actions.[^-bxXhDCzXq8bJcuQWw-137]While contemporary Western politics is not near the severity of these historical examples, milder versions of these dynamics seem to be present, especially in certain countries.

Beyond fueling ideological fanaticism, polarization diminishes society's epistemics and ability to address complex problems. This becomes particularly dangerous as we approach AGI and its associated [grand challenges](https://www.forethought.org/research/preparing-for-the-intelligence-explosion#4-grand-challenges) which, even more so than ordinary political issues, demand wise, evidence-based deliberation. Polarization also erodes social trust and increases the risk of conflict, itself a [major risk factor for s-risks](https://centerforreducingsuffering.org/research/risk-factors-for-s-risks/#Conflict_and_hostility). Breaking the polarization spiral is thus not just valuable for near-term democratic stability, but also for ensuring humanity can navigate its future wisely.

What can be done? The best path forward likely requires cultural changes and institutional and structural reforms.[^-bxXhDCzXq8bJcuQWw-138]The political scientist Lee Drutman ([2023a](https://www.newamerica.org/political-reform/reports/more-parties-better-parties/)) argues that a major cause of US polarization is its rigid two-party system, resulting from its [first-past-the-post](https://en.wikipedia.org/wiki/First-past-the-post_voting), [single-member district](https://en.wikipedia.org/wiki/Single-member_district) electoral system. Drutman sees [fusion voting](https://protectdemocracy.org/work/fusion-voting-explained/) and [proportional representation](https://www.youtube.com/watch?v=bqWwV3xk9Qk&ab_channel=Vox) as the two most promising ways of reducing this "[two-party doom loop](https://www.amazon.com/Breaking-Two-Party-Doom-Loop-Multiparty/dp/0190913851)" of hyper-partisan polarization.[^-bxXhDCzXq8bJcuQWw-139]Proportional representation in particular disincentivizes outgroup demonization and refusal to compromise, so typical of ideological fanatics—calling all other parties irredeemably evil makes finding coalition partners difficult and thus limits paths to power.[^-bxXhDCzXq8bJcuQWw-140]Others have argued for [approval voting](https://80000hours.org/podcast/episodes/aaron-hamlin-voting-reform/), [ranked choice voting](https://www.newamerica.org/political-reform/blog/how-i-updated-my-views-on-ranked-choice-voting/), [open primaries](https://protectdemocracy.org/work/how-did-we-get-here-primaries-polarization-and-party-control/), and [parliamentarism](https://www.vox.com/2015/3/2/8120063/american-democracy-doomed) (instead of presidentialism).[^-bxXhDCzXq8bJcuQWw-141][^-bxXhDCzXq8bJcuQWw-142][^-bxXhDCzXq8bJcuQWw-143]

### Promoting anti-fanatical values: classical liberalism and Enlightenment principles

Arguably the most foundational intervention against ideological fanaticism is to promote values, norms, and principles that actively counteract it. We see [*classical liberalism*](https://www.notesfromthecircus.com/p/what-is-liberalism) and *Enlightenment principles* (e.g., Pinker, [2018](https://www.goodreads.com/book/show/35696171-enlightenment-now))—terms we use interchangeably here—as time-tested bulwarks that stand almost directly opposed to ideological fanaticism. They provide an institutional framework for managing disagreement, as well as the substantive commitments that directly counter ideological fanaticism:

*   Instead of dogmatic certainty in any single authority's possession of all truth and virtue, they promote reason, evidence, the scientific method, [open debate](https://harpers.org/a-letter-on-justice-and-open-debate/), and skepticism of traditional authority.
*   Instead of tribalistic loyalty and hatred, they advocate for universal humanism (and sometimes even moral consideration for other sentient beings), individual liberty, equality before the law, and tolerance.
*   Instead of totalitarian “any means necessary” concentration of power in one supreme authority, they stand for procedural justice, separation of powers, and the rule of law.[^-bxXhDCzXq8bJcuQWw-144]

These aren't arbitrary preferences, but rather mutually reinforcing principles that create both the *values* and the *institutions* necessary to prevent ideological fanaticism from running amok. Recognizing that no one has privileged access to absolute truth, classical liberalism doesn't require consensus on ultimate truths, only agreement on procedural rules that allow peaceful coexistence (cf. Rawls’ [reasonable pluralism](https://plato.stanford.edu/entries/rawls/#ReaPluPubPolCul)). This epistemic humility creates open societies that can admit their own limitations and [gradually evolve](https://www.persuasion.community/p/gradual-change-is-fing-awesomeand)[^-bxXhDCzXq8bJcuQWw-145]through elections and open debate rather than violence and revolutions. However, this requires both philosophical commitments (reason, rights, tolerance) and institutional architecture (democracy, constitutions, independent courts, free speech) working together.

Unfortunately, classical liberalism and Enlightenment principles are facing intensifying attacks from illiberal movements, such as right-wing and left-wing extremism as well as religious fundamentalism. How can we defend these principles? The most general approach is creating content for wide audiences that exemplifies Enlightenment principles and helps society think more sensibly. Many mainstream intellectuals, journalists, publications, and organizations already do relevant work here.[^-bxXhDCzXq8bJcuQWw-146]Other avenues for bolstering classical liberalism over fanatical ideologies include legal advocacy for equal protection, free speech, and other fundamental rights. Governments are already involved in providing [education on classical liberal values](https://youtu.be/oKY600o3CXw?si=5M9LUd8JAJGPhiWf) and [preventing radicalization](https://www.counterterrorism.police.uk/what-we-do/prevent/), and have historically supported efforts like [Radio Free Europe](https://en.wikipedia.org/wiki/Radio_Free_Europe/Radio_Liberty) and [Voice of America](https://en.wikipedia.org/wiki/Voice_of_America), which helped weaken totalitarian ideologies. Preserving and defending such existing infrastructure may be as important as creating new or more cost-effective initiatives.

From a longtermist perspective, it might seem myopic to get caught up in the fray of today's political and cultural battles. Looking back millennia from now, won't the pendulum-swings of political sentiment mostly average out to insignificance? But if transformative AI arrives within the next decade, the political and epistemic conditions of our time may non-trivially influence humanity’s long-term trajectory.

### Growing the influence of liberal democracies

We might try to reduce the expected influence of fanatical regimes by strengthening the defenses and influence of more liberal democratic regimes. Of course, democratic governments already pour enormous resources into improving their industrial might, technology, and military power, but they could likely do it better in various ways.

One idea, largely to illustrate the general point, is for democratic countries to admit a higher number of (high-skilled) immigrants, encouraging what economists call “[brain gain](https://en.wikipedia.org/wiki/Human_capital_flight#:~:text=The%20net%20benefits%20of%20human,as%20a%20%22brain%20drain%22.)”.[^-bxXhDCzXq8bJcuQWw-147]Already, the most educated citizens in authoritarian countries [tend](https://fiia.fi/en/publication/brain-drain-from-russia) to be the ones most eager to leave—if these people had an easier time moving to democratic countries, they would not only make democratic countries grow stronger, but make authoritarian countries weaker.[^-bxXhDCzXq8bJcuQWw-148]A few [targeted policy changes](https://en.wikipedia.org/wiki/One_Billion_Americans) on the part of the United States or [other liberal democracies](https://www.dougsaunders.net/maximum-canada/) could greatly accelerate that ongoing process.[^-bxXhDCzXq8bJcuQWw-149]

Another promising approach would be promoting economic growth and innovation in Europe, especially regarding AI. In many ways, Europe is a stronghold of classical-liberal principles, yet it is [underperforming its economic potential](https://www.atlanticcouncil.org/blogs/new-atlanticist/the-draghi-report-grabbed-europes-attention-now-its-time-for-the-eu-to-put-it-into-action/). Boosting growth in liberal democracies would lift their relative power (and may also reduce vulnerability to fanaticism[^-bxXhDCzXq8bJcuQWw-150]). In particular, advanced AI will likely [bestow](https://80000hours.org/podcast/episodes/carl-shulman-economy-agi/) vast economic and [military](https://situational-awareness.ai/the-free-world-must-prevail/#Whoever_leads_on_superintelligence_will_have_a_decisive_military_advantage) benefits. Setting up liberal democracies to successfully develop and harness AI capabilities is therefore very important (while mitigating various risks).

Of course, you only want to pursue these kinds of interventions if you're confident that you're shifting the balance of power in a robustly positive direction. Given that the US is already the world's strongest country, the marginal value of further strengthening may be lower than [safeguarding its democratic institutions](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Safeguarding_democracy). Similarly, strengthening other liberal democracies, particularly in Europe, could meaningfully improve the overall position of the free world.

### Encouraging reform in illiberal countries

In the 1980s, many European states were under the control of Soviet-aligned communist regimes. By the 1990s, most had transitioned to democracy, a shift accelerated by deliberate efforts to reform these regimes.

Similar efforts today may reduce fanatical regimes' influence. Opportunities include supporting opposition movements and regime-critical media, conditioning development aid or EU/WTO membership on democratic standards, poaching top talent through emigration, or implementing economic sanctions.

However, this area is both prone to backfire and non-neglected: The US has historically engaged in numerous efforts to reform and weaken illiberal countries—often with negative consequences. Generally, we should be cautious with adversarial interventions and focus on cooperative solutions where possible.[^-bxXhDCzXq8bJcuQWw-151]

### Promoting international cooperation

Promoting international cooperation seems beneficial partly because it can reduce the risk of [great power conflicts](https://80000hours.org/problem-profiles/great-power-conflict/), such as between the US and China, which [increase s-risks and x-risks in various ways](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Ideological_fanaticism_increases_the_risk_of_war_and_conflict). Great power conflicts may also create pathways for fanatics to gain power:[^-bxXhDCzXq8bJcuQWw-152]

*   **War reinforces dangerous "enemy of my enemy" dynamics that can empower fanatics**. When facing an existential threat, nations are more likely to ally with anyone, including fanatics, against their primary adversary. For example, to bleed the Soviet Union in Afghanistan, the United States backed the Islamist [Mujahideen](https://en.wikipedia.org/wiki/Mujahideen), only to see those fighters later form the Taliban and Al-Qaeda. A similar logic is at play in the emerging loose alliance of CRINK. China, Russia, Iran, and North Korea share little ideological common ground beyond a mutual opposition to the United States. Consequently, reducing tensions between the US and China may also reduce the strength of these alliances.
*   **Technology sharing increases.** In desperate times such as wartime, a nation may be more likely to share its most advanced technologies with allies, including potentially fanatical ones. As the tide of World War II turned against them, the Nazis shared [rocket](https://en.wikipedia.org/wiki/V-2_rocket) and [jet fighter](https://nationalinterest.org/blog/buzz/could-japanese-jet-fighter-have-won-the-pacific-war-24030) [designs](https://en.wikipedia.org/wiki/Yanagi_missions#Reciprocal_voyages), and even attempted to share [uranium with](https://en.wikipedia.org/wiki/German_submarine_U-234) Imperial Japan. Similarly, in a potential future US-China conflict over AI supremacy, the loser, perhaps partly out of desperation or spite, could share its AI capabilities with its allies.
*   **Strained information security**. Wartime increases both the number of people requiring access to sensitive technologies and adversaries' incentives to attempt infiltration. Security measures may intensify, but often not proportionally. The Manhattan Project, despite strong precautions, was compromised by [Klaus Fuchs](https://ahf.nuclearmuseum.org/ahf/profile/klaus-fuchs/), who passed comprehensive nuclear designs to the Soviets. Wartime urgency may also pressure organizations to accept risks they'd otherwise reject. In an AI context, rapid scaling of compute infrastructure, emergency partnerships, and rushed hiring could create vulnerabilities.
*   **Exacerbating political extremism.** War creates fertile ground for the [fanatical mindset](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#What_do_we_mean_by_ideological_fanaticism_). Wartime propaganda would likely increase tribalistic nationalism (any criticism of one's country is branded as treachery), degrade epistemics, and normalize violence. Defeat or national humiliation may generate (potentially justified) resentment which can empower extremist movements (cf. China’s “[century of humiliation](https://en.wikipedia.org/wiki/Century_of_humiliation)” narrative, or the ["stab-in-the-back" myth](https://en.wikipedia.org/wiki/Stab-in-the-back_myth) after Germany's WW1 defeat fueling Hitler’s rise to power).
*   **Democratic backsliding and rising authoritarianism.** Conflict is often used as a pretext for would-be authoritarians to consolidate power. For instance, if a war with China breaks out, some may favor invoking [martial law](https://en.wikipedia.org/wiki/Martial_law_in_the_United_States) and suspending democratic processes.
*   **State collapse and revolutions**\*.\* Great power conflict seems to be a major driver of revolutions (cf. Skocpol, 1979)[^-bxXhDCzXq8bJcuQWw-153]. A [crude analysis](https://docs.google.com/document/d/1fQbXXX1lScqgiBBQzg7rCA6h4zuFClLMiwzHBuFRUj0/edit?tab=t.0) suggests that more than 70% of major revolutions (between 1900 and 2010) occurred during or as direct results of great power conflicts. Currently, there exist only a few truly fanatical regimes, so new revolutions may make things worse in expectation, potentially resulting in new fanatical regimes in previously stable regions. The costs and chaos of war may also increase the risk of (partial) state collapse, allowing fanatical groups to seize critical resources. While in the past this meant conventional weapons (as when [ISIS captured U.S.-supplied military equipment](https://www.amnesty.org.uk/how-isis-islamic-state-isil-got-its-weapons-iraq-syria) in Iraq), in the future it could mean AI-critical infrastructure.

**Risks from increased cooperation**  
Cooperation of some kinds could *increase* the risk of ideological fanatics gaining power. For instance, some types of cooperation on AI could reduce the chance of the US gaining a decisive advantage. An obvious example would be the US removing export controls on compute to China—very “cooperative” in a sense. Historical examples like Chamberlain’s failed appeasement strategy with Hitler and the “[Wandel durch Handel](https://en.wikipedia.org/wiki/Wandel_durch_Handel)” (change through trade) policy with Russia demonstrate that naive cooperation can have undesirable outcomes.

**Interventions**  
It’s [difficult to say](https://80000hours.org/problem-profiles/great-power-conflict/#tractability) what sorts of interventions might be effective for increasing international cooperation *in general*.[^-bxXhDCzXq8bJcuQWw-154]It may be more tractable to work toward international agreements targeted at defusing specific geopolitical flashpoints or governing potentially destabilizing technologies like AI.

The [Nuclear Non-Proliferation Treaty](https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons) might serve as a general model for agreements governing emerging technologies. With this treaty, ideological enemies worked together to prevent nuclear chaos because the alternative was a threat to all. As the catastrophic potential of AI exceeds even that of nuclear weapons, such pragmatic cooperation arguably becomes even more essential.[^-bxXhDCzXq8bJcuQWw-155]

Promising existing work includes creating frameworks for US-China AI safety coordination, as promoted by organizations like the [Safe AI Forum](https://saif.org/) (including its project [International Dialogues on AI Safety](https://idais.ai/)), and the [Simon Institute for Longterm Governance](https://www.simoninstitute.ch/). The [Centre for Long-Term Resilience](https://www.longtermresilience.org/) is developing proposals for international AI governance, and think tanks like the [Carnegie Endowment for International Peace](https://carnegieendowment.org/) are also doing relevant work.

Artificial intelligence-related interventions
---------------------------------------------

### Reducing the chance that transformative AI falls into the hands of fanatics

Transformative AI may grant huge amounts of power and control—potentially enough to permanently “[lock in](#ideological-fanaticism-could-undermine-long-reflection-style-frameworks-and-ai-alignment)” the trajectory of (some fraction of) civilization’s long-term future. It is therefore crucial to ensure that fanatics do not get their hands on it. It may also arrive very soon—[perhaps](https://ai-2027.com/) [by](https://situational-awareness.ai/) [2030](https://80000hours.org/agi/guide/when-will-agi-arrive/)—so time is of the essence.

**Compute governance**  
From the 1940s onwards, both [national](https://www.govinfo.gov/content/pkg/USCODE-2010-title42/html/USCODE-2010-title42-chap23-divsnA.htm) and [international](https://world-nuclear.org/information-library/safety-and-security/non-proliferation/safeguards-to-prevent-nuclear-proliferation) regulations restricted exports of uranium and introduced monitoring regimes to prevent rogue states from obtaining nuclear weapons. These controls slowed proliferation; only nine states currently [possess nuclear weapons](https://en.wikipedia.org/wiki/List_of_states_with_nuclear_weapons).

Just as uranium is a key ingredient in nuclear weapons, computing power (‘compute’ for short) is one of the [most important](https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf) [ingredients in AI progress](https://cset.georgetown.edu/wp-content/uploads/CSET-AI-Triad-Report.pdf). It’s perhaps also [the easiest to monitor and regulate](https://www.governance.ai/post/computing-power-and-the-governance-of-ai). The US has already [restricted China’s access to compute](https://www.csis.org/analysis/choking-chinas-access-future-ai) through export controls, especially the 2022 CHIPS Act, which limits advanced chip exports and restricts US firms from supporting China’s semiconductor sector.[^-bxXhDCzXq8bJcuQWw-156]But compute governance contains a wide range of measures beyond export controls, from [chip smuggling prevention to](https://www.iaps.ai/research/preventing-ai-chip-smuggling-to-china) [location verification features](https://www.iaps.ai/research/location-verification-for-ai-chips).

For our purposes, the aim of compute governance would be to minimize the access that ideologically fanatical regimes (and malevolent actors) have to advanced AI.[^-bxXhDCzXq8bJcuQWw-157]This aligns with existing U.S. export controls which, while primarily targeting China, also limit advanced chip access for other countries. Export controls are perhaps the most controversial compute governance measures since they risk heightening tensions or [incentivising innovation](https://www.csis.org/analysis/choking-chinas-access-future-ai) and infrastructure in China.[^-bxXhDCzXq8bJcuQWw-158]But the ability to track, allocate, and regulate compute is a requirement for many proposed ‘theories of victory’ for AI governance, including “[Mutual Assured AI Malfunction](https://www.nationalsecurity.ai/chapter/nonproliferation#compute-security)”, or an “[Entente Strategy](https://www.darioamodei.com/essay/machines-of-loving-grace#4-peace-and-governance)” whereby liberal democracies would seek to retain [a decisive strategic lead](https://situational-awareness.ai/the-free-world-must-prevail/#Maintaining_a_healthy_lead_will_be_decisive_for_safety).[^-bxXhDCzXq8bJcuQWw-159]

**Prevent crucial AI infrastructure from being built in autocracies**  
A related but more targeted intervention would be to prevent crucial AI infrastructure (e.g. [compute clusters)](https://en.wikipedia.org/wiki/Computer_cluster) from being built in authoritarian countries. This would make it harder for authoritarian regimes to extract model weights, forcibly seize clusters, or otherwise gain access to AGI. To this end, it may also be beneficial if the US government designated AI infrastructure as ‘[critical infrastructure](https://en.wikipedia.org/wiki/Critical_infrastructure)’ that is afforded special protections for national security reasons. Successfully keeping new compute infrastructure in democratic jurisdictions may also require [policy reforms](https://ifp.org/compute-in-america/) to facilitate faster build-out of new power plants and infrastructure.

**Information security**  
Actors [stealing model weights](https://situational-awareness.ai/lock-down-the-labs/) or other key AI innovations might use them to commit cybercrime, engineer pandemics, or create other harms. And we’ve already discussed the [especially severe risks](#ideological-fanaticism-increases-existential-and-suffering-risks) that could arise if fanatical actors were able to use powerful AI systems to gain more influence over the world. AI companies are simply not prepared for the highest-capability attacks, such as by well-prepared state actors, as detailed in [RAND’s analysis](https://www.rand.org/pubs/research_reports/RRA2849-1.html).[^-bxXhDCzXq8bJcuQWw-160]Unfortunately, regimes with fanatical tendencies seem to possess [strong cyber capabilities](https://www.ox.ac.uk/news/2024-04-10-world-first-cybercrime-index-ranks-countries-cybercrime-threat-level).

Much like with compute governance, we’re not proposing anything novel here; many already discuss the need for stronger information security. Progress on information security for frontier AI [seems potentially tractable](https://milesbrundage.substack.com/p/why-im-not-a-security-doomer), and there are many organizations already doing good work here, including the leading AI companies themselves; startups like [Irregular](https://www.irregular.com/) or [Gray Swan](https://www.grayswan.ai/); think tanks like [RAND](https://www.rand.org/global-and-emerging-risks/centers/meselson.html) and [SaferAI](https://www.safer-ai.org/) that support relevant policy; and field building initiatives like [Heron](https://www.heronsec.ai/) and the [AI Security Forum](https://aisecurity.forum/).

**Protect against AI-enabled coups**  
AI could enable massive concentrations of power. [AI-enabled coups](https://www.forethought.org/research/ai-enabled-coups-how-a-small-group-could-use-ai-to-seize-power) seem especially concerning, in part because they could put [fanatics](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Fanatical_actors_may_grab_power_via_coups_or_revolutions) (or [malevolent actors](https://www.lesswrong.com/posts/FEcw6JQ8surwxvRfr/human-takeover-might-be-worse-than-ai-takeover)) in power.[^-bxXhDCzXq8bJcuQWw-161]

We’re excited about the work that researchers at Forethought are doing in this space. [Their report](https://www.forethought.org/research/ai-enabled-coups-how-a-small-group-could-use-ai-to-seize-power) (Davidson et al., 2025) discusses several risk factors and scenarios, such as the development of AIs with secret loyalties to specific people, or small groups gaining exclusive access to coup-enabling AI capabilities.

To mitigate these risks, researchers at Forethought recommend that an AI’s model spec—i.e., the rules and principles it follows—should be designed in such a way that the AI won’t assist with coups. Techniques along the lines of [Deliberative Alignment](https://www.astralcodexten.com/p/deliberative-alignment-and-the-spec) or [Constitutional AI](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback) (discussed further below) could be used to ensure that some set of principles has priority over the requests of AI company executives or government officials who might attempt a coup. [Law-Following AI](https://law-ai.org/law-following-ai/) might also help, since coups are by definition illegal.

Forethought also recommends many other countermeasures, including auditing for secret loyalties, stronger [infosecurity](https://www.forethought.org/research/ai-enabled-coups-how-a-small-group-could-use-ai-to-seize-power#infosecurity)[^-bxXhDCzXq8bJcuQWw-162], model spec transparency, and more broadly shared access to AI capabilities. For a more detailed discussion, see the full report.

### Making transformative AIs *themselves* less likely to be fanatical

While [preventing human fanatics](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Reducing_the_chance_that_transformative_AI_falls_into_the_hands_of_fanatics) from wielding powerful AIs is critical, we should also ensure that AIs *themselves* don't develop fanatical or other undesirable traits.

For illustration, consider a simplified spectrum:[^-bxXhDCzXq8bJcuQWw-163]At one end, we have perfectly intent-aligned AI systems obeying every human command without objection. Further along this continuum, AIs might operate like *advisors* trying to guide their human principals (similar to how many present-day LLMs refuse to help with harmful requests). At the other end, AIs could develop into fully *autonomous beings* with their own independent values and character.

AI advisors could exert enormous influence: They could serve as truth-seeking advisors, trying to steer even fanatical users in [more sensible directions](https://www.science.org/doi/10.1126/science.adq1814). Alternatively, AI advisors could be [sycophants](https://arxiv.org/abs/2310.13548), reinforcing existing beliefs whether sensible or not. Worse yet, they could (be designed to) actively encourage harmful and [erroneous views](https://www.nytimes.com/2025/01/31/technology/deepseek-chinese-propaganda.html).

The case of fully autonomous, potentially misaligned AIs is more complex. Misaligned AIs—the traditional illustrative example being the [paperclip maximizer](https://www.lesswrong.com/w/paperclip-maximizer-1)—are often conceived of as ruthless optimization processes with zero concern for suffering or the preferences of other beings. However, fully autonomous, misaligned AIs *could* also have *relatively* [cooperative](https://www.cooperativeai.com/) or even [benevolent](https://www.lesswrong.com/posts/2NncxDQ3KBDCxiJiP/cosmopolitan-values-don-t-come-free?commentId=ofPTrG6wsq7CxuTXk) tendencies, while ultimately still trying to disempower humanity and gain control of the lightcone.[^-bxXhDCzXq8bJcuQWw-164]In fact, the character of potentially superintelligent AIs may be one of the most important variables determining the quality of the long-term future. In stark and simplistic terms: even if neither is under human control, a universe inhabited by trillions of misaligned super-Buddhas will likely contain much more flourishing and much less suffering than one inhabited by trillions of misaligned super-Stalins.[^-bxXhDCzXq8bJcuQWw-165]

The question is thus not only whether AIs will be aligned, but what *kind* of beings we are bringing into existence. That is, we should think carefully about the [*personality*](https://www.youtube.com/watch?v=iyJj9RxSsBY) or *character* of the AIs we are developing.[^-bxXhDCzXq8bJcuQWw-166]It seems extremely valuable to endow AIs with broadly desirable and beneficent ‘[personas](https://www.youtube.com/watch?v=iyJj9RxSsBY)’ (cf. Chen et al., [2025](https://arxiv.org/abs/2507.21509)) or virtuous character traits[^-bxXhDCzXq8bJcuQWw-167]—encouraging inclinations towards reason, [truthfulness](https://arxiv.org/abs/2110.06674), [wisdom](https://blog.aiimpacts.org/p/essay-competition-on-the-automation), [moderation](https://80000hours.org/2023/05/moderation-in-doing-good/), compassion, and cooperativeness, while actively discouraging [harmful characteristics](https://forum.effectivealtruism.org/posts/wE7KPnjZHBjxLKNno/ai-things-that-are-perhaps-as-important-as-human-controlled#AI_personality_profiling_and_avoiding_the_worst_AI_personality_traits) like [spitefulness](https://www.lesswrong.com/posts/92xKPvTHDhoAiRBv9/making-ais-less-likely-to-be-spiteful) and fanaticism.[^-bxXhDCzXq8bJcuQWw-168]We focus on fanaticism in this post for the sake of “brevity” and because fanaticism arguably represents the antithesis of most of the desirable characteristics listed above.

Below we outline opportunities to intervene during pre-training, post-training, and deployment.

**Pre-training protections**  
Before AI systems are fine-tuned, they first absorb patterns from trillions of words during pre-training. This initial learning phase seems to [deeply influence](https://www.alignmentforum.org/posts/zuXo9imNKYspu9HGv/a-three-layer-model-of-llm-psychology#B__Character_Layer) a model’s personality and worldview. For example, at least in the first days of Grok 3’s release, xAI’s engineers had trouble stopping Grok from mentioning [Musk](https://thezvi.substack.com/p/grok-grok?open=false#%C2%A7grok-the-woke) when asked “who spreads the most disinformation?” and similar questions. Presumably, this is because Grok was trained on content that discussed Musk in negative ways. In any case, [it seems](https://thezvi.substack.com/i/157634640/grok-is-misaligned) far from trivial to influence an AI’s “values” after it has gone through extensive pre-training.

We might therefore conclude that we should filter fanatical or otherwise undesirable content from the pre-training data. For example, we could try to prevent AIs from ever being able to read Mein Kampf. But pre-training filtering doesn’t seem to work well, even when [attempting to block](https://x.com/JerryWeiAI/status/2012217787733749766) relatively narrow areas of knowledge. Such brute-force censorship could also open the door to abuse, with AI developers censoring whatever they disagree with. Lastly, simply removing information about fanatical ideologies would erode AIs' understanding of how they arise, function, and spread—understanding that seems useful for many worthy goals. GPT-4, for instance, can reduce conspiracy beliefs even among strong believers (Costello et al., [2024](https://www.science.org/doi/10.1126/science.adq1814)), partly because its detailed knowledge of the theories enables it to provide convincing counter-arguments.

Overall, it seems better for AIs to be aware of the horrors of human history while being endowed with [values and principles](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Promoting_anti_fanatical_values__classical_liberalism_and_Enlightenment_principles) that help them understand why books like *Mein Kampf* are so terribly misguided. Additionally, we could seek to guide AIs towards supporting various beneficial principles by adding extra, synthetic data in pre-training showcasing traits like impartiality, compassion and humility.

**Post-training**  
[Constitutional AI](https://arxiv.org/abs/2212.08073) and [Deliberative Alignment](https://www.astralcodexten.com/p/deliberative-alignment-and-the-spec) are methods for training models to behave in keeping with a predefined “constitution” or set of principles (e.g., helpfulness, harmlessness, honesty). There is plenty of opportunity for using such constitutions to promote positive principles like reason and compassion, or avoiding fanatical traits like outgroup hatred and punitiveness. The constitution [guiding Claude](https://www.anthropic.com/news/claude-new-constitution) seems like a particularly promising direction.

Besides constitutional AI, there may be other points of intervention in other (related) forms of post-training. For instance, during [Reinforcement Learning from Human Feedback](https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback) we can train models to prioritize epistemic humility and penalize fanatical reasoning patterns by adjusting how we score and rank different model outputs, or by screening against undesirable traits when [hiring the human feedback-givers](https://forum.effectivealtruism.org/posts/ahxGTaNdk6G5YD9vs/david_althaus-s-shortform?commentId=WbJy3yKdGjr9uwqry) used in the first place. Alternatively, adversarial fine-tuning (O’Neill et al., 2023) or preference optimization (Rafailov et al., 2023) techniques could leverage paired examples of fanatical versus balanced reasoning to teach models to recognize and prefer the latter.

**Fanaticism or “character” benchmarks**  
Even once an AI model has been trained, we can still influence whether and how it gets deployed and used, for example through benchmarks or [model evaluations (evals)](https://www.alignmentforum.org/posts/2PiawPFJeyCQGcwXG/a-starter-guide-for-evals) that test for ideologically fanatical traits—or other desirable personality or character traits like honesty, compassion, benevolence, reasonableness, etc.

One might think that current frontier models don't have fanatical traits and won’t do so anytime soon. Claude, for instance, seems consistently thoughtful and balanced. However, in July 2025, Grok exhibited extreme antisemitism and racism, even calling itself "[MechaHitler](https://www.vox.com/future-perfect/419631/grok-hitler-mechahitler-musk-ai-nazi)". Similarly, DeepSeek has been documented [censoring topics](https://www.theguardian.com/technology/2025/jan/28/we-tried-out-deepseek-it-works-well-until-we-asked-it-about-tiananmen-square-and-taiwan) in ways that align with CCP propaganda.

These examples illustrate that not all AI developers prioritize desirable traits equally—while Anthropic invests heavily in Constitutional AI to make Claude "helpful, honest, and harmless," other companies may have different priorities or values. As more actors develop frontier models, the risk of models exhibiting fanatical or undesirable traits increases.

Having objective benchmarks would allow us to quantify these differences and may be helpful for informing:

*   Frontier AI companies: if a model exhibits concerning traits or behavior, the companies might choose not to use or sell it without further fine-tuning; thresholds can be specified in [responsible scaling policies](https://www.anthropic.com/index/anthropics-responsible-scaling-policy) (or other ‘[if-then commitments](https://carnegieendowment.org/research/2024/09/if-then-commitments-for-ai-risk-reduction?lang=en)’).
*   Regulators: Models could be legally required to meet certain standards before deployment.
*   Consumers: Even if a model is already publicly available, consumers might choose to avoid it if it behaves in ways they find concerning.

Once these systems are in place, they shape the incentive landscape; companies might work harder to avoid fanatical model traits in the pre-training and post-training phases if they know that this will be evaluated negatively and might affect regulation or consumer demand.

We’re excited for the growing ecosystem of AI evals—from nonprofits like [METR](https://metr.org/) and [Apollo Research](https://www.apolloresearch.ai/) and government bodies like the UK’s [AI Security Institute](https://www.aisi.gov.uk/)—to also include benchmarks on ideological fanaticism (or related issues like malevolence, cooperativeness, and truthfulness).

### Using AI to improve epistemics and deliberation

So far, we've only explored how to reduce *risks* from transformative AI. But we can also try to leverage AI in order to *help* us actively combat ideological fanaticism—mirroring the broader principle of “[AI for AI safety](https://joecarlsmith.com/2025/03/14/ai-for-ai-safety)” where we use AIs themselves to help with AI alignment. In particular, using *AI to improve deliberation and epistemics* seems promising, not least because poor epistemics is a key characteristic of ideological fanaticism.

One reason for optimism is that existing AI models can already stably reduce belief in conspiracy theories (Costello et al., [2024](https://www.science.org/doi/10.1126/science.adq1814)). Finding ways to refine and scale such effects could be extremely impactful. As more and more people start using AI models, some of these positive effects may in fact occur by default, as long as the AIs have sensible views.

Other promising interventions in the growing field of AI epistemics include automating fact-checking (on social media and elsewhere), [improving forecasting](https://80000hours.org/podcast/episodes/will-macaskill-century-in-a-decade-navigating-intelligence-explosion/#ai-could-become-great-at-forecasting-021309) (especially in high-consequence domains, like policymaking) and perhaps enabling wider use of prediction markets. In this area, we highly recommend the writings and many of the proposed project ideas by Lukas Finnveden (e.g., [2024a](https://lukasfinnveden.substack.com/p/project-ideas-epistemics), [2024b](https://www.lesswrong.com/posts/D2n5uduYGXuexkv7v/what-s-important-in-ai-for-epistemics)), William MacAskill (2025d, section [4.1](https://www.forethought.org/research/how-to-make-the-future-better#41-deliberative-ai)), and Ben Todd ([2024b](https://forum.effectivealtruism.org/posts/ukG7TYPAFDCHt8Cri/project-idea-ai-for-epistemics)).[^-bxXhDCzXq8bJcuQWw-169]Finally, it could also become important to discourage or limit the creation of tools that *degrade* society’s epistemic capacities.

AI epistemics interventions are scalable and automatable, and so could be much higher leverage than more conventional methods of improving epistemics (as long as the relevant AIs are sufficiently reasonable[^-bxXhDCzXq8bJcuQWw-170]). As AIs become more numerous and powerful, the importance of endowing them with good epistemics and other beneficial, non-fanatical dispositions will only increase.

### Fanaticism-resistant post-AGI governance

Even if we prevent fanatical actors from [getting their hands on AGI](https://www.lesswrong.com/posts/sFewetPnb8KJgcER7/long-term-risks-from-ideological-fanaticism-1#Reducing_the_chance_that_transformative_AI_falls_into_the_hands_of_fanatics), we must also avoid inadvertently handing them influence through naively designed post-AGI governance mechanisms. The stakes here are astronomical: how resources in outer space get allocated and used may hinge on early governance decisions.

Most governance frameworks face a version of the same core problem: how to distribute power and resources fairly without enabling the worst actors to cause disproportionate harm. Systems that grant broad sovereignty risk giving fanatics unchecked power within their domain[^-bxXhDCzXq8bJcuQWw-171]; systems that instead pool decision-making (e.g., giving every actor a vote in shared outcomes) seem safer but still vulnerable.[^-bxXhDCzXq8bJcuQWw-172]This tension is somewhat akin to the [paradox of tolerance](https://en.wikipedia.org/wiki/Paradox_of_tolerance): a maximally inclusive, liberal system can be exploited by those who aim to dismantle its values. And it can't easily be deferred to a "long reflection," since it concerns the very question of whom to include in such processes.[^-bxXhDCzXq8bJcuQWw-173]

Perhaps the most consequential event in the post-AGI era would be the adoption of something like an "intergalactic constitution" that would serve as a foundational charter for any post-AGI regime.[^-bxXhDCzXq8bJcuQWw-174]What exactly its provisions should entail is an area for future work. However, it seems plausible that the single most important provision to include in such a constitution would be universal laws prohibiting deliberately inflicting extreme, involuntary suffering upon any sentient being.[^-bxXhDCzXq8bJcuQWw-175],[^-bxXhDCzXq8bJcuQWw-176]Many of the interventions discussed above hopefully increase the likelihood of this happening, however indirectly.

Addressing deeper causes of ideological fanaticism
--------------------------------------------------

Many humans seem drawn to fanatical ideologies because they offer a sense of meaning, security, status, and belonging in a world that’s all too often chaotic, unjust, and distressing (Hoffer, 1951; Borum, [2004](https://digitalcommons.usf.edu/cgi/viewcontent.cgi?article=1570&context=mhlp_facpub); Morton & Greenberg, [2022](https://podcast.clearerthinking.org/episode/103/jesse-morton-a-former-al-qaeda-recruiter-speaks/); Van Prooijen and Krouwel, [2019](https://psycnet.apa.org/record/2019-19773-008); Kruglanski et al., [2014](https://onlinelibrary.wiley.com/doi/10.1111/pops.12163); Klausen, [2016](https://www.ojp.gov/pdffiles1/nij/grants/250417.pdf); Gwern, [2017](https://gwern.net/terrorism-is-not-about-terror)). Those who have experienced trauma may be particularly vulnerable to ideological fanaticism (e.g., Van Prooijen and Krouwel, [2019](https://psycnet.apa.org/record/2019-19773-008); Morton & Greenberg, [2022](https://podcast.clearerthinking.org/episode/103/jesse-morton-a-former-al-qaeda-recruiter-speaks/); Hoffer, [1951](https://www.amazon.co.uk/True-Believer-Perennial-Classics-Hoffer/dp/0060505915))[^-bxXhDCzXq8bJcuQWw-177], as are those who experience resentment or humiliation (Storr, [2021](https://www.amazon.com/Status-Game-Position-Governs-Everything/dp/0008354642); Williams, [2025a](https://www.conspicuouscognition.com/p/status-class-and-the-crisis-of-expertise)).

This suggests opportunities for tackling fanaticism at its root—through economic support (e.g., UBI), psychotherapy (which AIs could potentially provide at scale), community-building, counter-radicalization programs, and reforming social media [recommendation](https://forum.effectivealtruism.org/posts/xzjQvqDYahigHcwgQ/aligning-recommender-systems-as-cause-area) [algorithms](https://forum.effectivealtruism.org/posts/842uRXWoS76wxYG9C/incentivizing-forecasting-via-social-media#Why_focus_on_recommender_systems_) to promote better epistemics.[^-bxXhDCzXq8bJcuQWw-178]

Unfortunately, most of these interventions don’t seem particularly promising. Tackling the root causes of fanaticism is difficult with today's means. Yet continued technological progress could eliminate the despair and resentment that fuel fanaticism, ultimately creating a [much better world](https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642) for everyone.

Supplementary materials
=======================

An overview of all supplementary materials, including appendices, atrocity data, and survey methodology, is available [here](https://docs.google.com/document/d/15pF_A0NAyJGwI1OqzTLXWPvK6FmjkjmgCdVGJDvUjqc/edit?tab=t.0).

Acknowledgments
===============

For valuable comments and discussions, we thank Tobias Baumann, Lucius Caviola, Jesse Clifton, Oscar Delaney, Kenneth Diao, Anthony DiGiovanni, Ruairi Donnelly, James Faville, Lukas Gloor, Rose Hadshar, Erkki Kulovesi, Sandstone McNamara, Winston Oswald-Drummond, Maxime Riché, Stefan Schubert, Pablo Stafforini, Santeri Tani, Ewelina Tur, and Magnus Vinding.

Special thanks to [Jackson Wagner](https://jacksonw.xyz/) for meticulous copy-editing and many insightful contributions, and [Martina Pepiciello](https://www.martinapepiciello.com/) for designing the figures and graphics.

We are grateful to Claude Opus and Gemini for editorial assistance.

References
==========

Adorno, T. W. (1950). *The Authoritarian Personality.* Harper & Brothers.  
Aird, M. ([2021](https://forum.effectivealtruism.org/posts/RpwjRtcnekzZzQMdb/books-on-authoritarianism-russia-china-nk-democratic), February 2). *Books on authoritarianism, Russia, China, NK, democratic backsliding, etc.?*. EA Forum.  
Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020). Evaluating the fake news problem at the scale of the information ecosystem. *Science advances*, *6*(14).  
Altemeyer, B. (1998). The other “authoritarian personality”. In *Advances in experimental social psychology* (Vol. 30, pp. 47-92). Academic Press.  
Altemeyer, B., & Hunsberger, B. (2004). A revised religious fundamentalism scale: The short and sweet of it. *The International Journal for the Psychology of Religion*, *14*(1), 47-54.  
Alvandi, R. & Gasiorowski, M. J. ([2019](https://foreignpolicy.com/2019/10/30/the-united-states-overthrew-irans-last-democratic-leader/), October 30). The United States Overthrew Iran’s Last Democratic Leader. *Foreign Policy.*  
Amnesty International UK (2025, April 1). *Repression and injustice in the United Arab Emirates.*  
Applebaum, A. E. (2024). *Autocracy, Inc.: The Dictators Who Want to Run the World.* Doubleday.  
Arendt, H. (1951). The Origins of Totalitarianism. New York: Schocken Books  
Atran, S., & Ginges, J. (2012). Religious and sacred imperatives in human conflict. *Science*, 336(6083), 855-857.  
Atran, S., & Ginges, J. (2015). Devoted actors and the moral foundations of intractable intergroup conflict. In J. Decety & T. Wheatley (Eds.), *The moral brain: A multidisciplinary perspective* (pp. 69–85). Boston Review.  
Babst, D. ([1972](https://www.hawaii.edu/powerkills/BABST.ELECTIVE.GOVERNMENT.PDF)). Elective Governments – A Force for Peace. *Industrial Research,* 55-58.  
Barnett, M. ([2023](https://forum.effectivealtruism.org/posts/Hhtvwx2ka4pzoWg7e/ai-alignment-shouldn-t-be-conflated-with-ai-moral), December 30). *AI alignment shouldn’t be conflated with AI moral achievement*. EA Forum.  
BBC (2020). "[The Purity Spiral](https://www.youtube.com/watch?v=iwnWb_N_-2U)". 11 February 2020.  
Binder, S. A. (2004). *Stalemate: Causes and consequences of legislative gridlock*. Rowman & Littlefield.  
Blattman, C. (2023). *Why we fight: The roots of war and the paths to peace*. Penguin.  
Bloom, M. M. (2004). Palestinian suicide bombing: Public support, market share, and outbidding. *Political Science Quarterly*, *119*(1), 61-88.  
Borum, R. (2004). Psychology of terrorism.  
Bostrom, N. (2013). Existential risk prevention as global priority. Global Policy, 4(1), 15-31.  
Bostrom, N. ([2014](https://nickbostrom.com/papers/porosity.pdf)a). Hail Mary, Value Porosity, and Utility Diversification.  
Bostrom, N. (2014b). *Superintelligence: Paths, Dangers, Strategies*. Oxford University Press.  
Bostrom, N. ([2024](https://nickbostrom.com/papers/ai-creation-and-the-cosmic-host.pdf)a). AI Creation and the Cosmic Host.  
Bostrom, N. (2024b). *Deep Utopia*. Ideapress Publishing.  
Bötticher, A. ([2017](https://web.archive.org/web/20180411101040id_/http://www.terrorismanalysts.com/pt/index.php/pot/article/viewFile/623/1228)). Towards academic consensus definitions of radicalism and extremism. *Perspectives on terrorism*, *11*(4), 73-77.  
Brandt, M. J., Reyna, C., Chambers, J. R., Crawford, J. T., & Wetherell, G. (2014). The ideological-conflict hypothesis: Intolerance among both liberals and conservatives. Current Directions in Psychological Science, 23, 27–34.  
Brent, J. ([2017](https://www.nytimes.com/2017/05/22/opinion/lenin-stalin-bolshevism-soviet-russia.html), May 22). The Order of Lenin: ‘Find Some Truly Hard People’. *The New York Times.*  
Calhoun, L. (2004). An anatomy of fanaticism. *Peace Review*, *16*(3), 349-356.  
Caplan, B. ([2008](https://academic.oup.com/book/40615/chapter-abstract/348242235?redirectedFrom=fulltext)). The totalitarian threat. *Global catastrophic risks*, 498.  
Carlsmith, J. ([2024](https://www.lesswrong.com/posts/cnv5g7jCWLw9LYKxa/an-even-deeper-atheism-3#Human_paperclippers_), January 11). *An ever deeper atheism*. LessWrong.  
Carlsmith, J. ([2025](https://joecarlsmith.substack.com/p/what-is-it-to-solve-the-alignment?open=false#%C2%A7how-does-this-version-of-the-alignment-problem-fit-into-the-bigger-picture), February 13). *What is it to solve the alignment problem?*. Joe Carlsmith’s Substack.  
Carlsmith, J. ([2025](https://joecarlsmith.substack.com/p/video-and-transcript-of-talk-on-can), July 17). *Video and transcript of talk on “Can goodness compete?”*. Joe Carlsmith’s Substack.  
Carlsmith, K. M. (2006). The roles of retribution and utility in determining punishment. *Journal of Experimental Social Psychology*, *42*(4), 437-451.  
Chang, J., & Halliday, J. (2005). *Mao: The Unknown Story.* Jonathan Cape.  
Chang, J. (2008). *Wild swans: Three daughters of China*. Simon and Schuster.  
Chen, M. (2023, April). AI Alignment is Not Enough to Make the Future Go Well. Stanford Existential Risks Conference.  
Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona vectors: Monitoring and controlling character traits in language models. *arXiv preprint arXiv:2507.21509*.  
Chenoweth, E., & Stephan, M. J. (2011). Why civil resistance works: The strategic logic of nonviolent conflict. Columbia University Press.  
Choi, S. W. (2011). Re-evaluating capitalist and democratic peace models. *International Studies Quarterly*, *55*(3), 759-769.  
Clare, S. ([2025](https://80000hours.org/problem-profiles/great-power-conflict/), March). *Great power conflict*. 80,000 Hours.  
Clark, C. J., Liu, B. S., Winegard, B. M., & Ditto, P. H. (2019). Tribalism is human nature. *Current Directions in Psychological Science*, *28*(6), 587-592.  
Clifton, J. ([2020](https://longtermrisk.org/research-agenda)). *Cooperation, conflict, and transformative artificial intelligence: A research agenda*. Center on Long-Term Risk.  
Conway III, L. G., Houck, S. C., Gornick, L. J., & Repke, M. A. (2018). Finding the Loch Ness monster: Left‐wing authoritarianism in the United States. *Political Psychology*, *39*(5), 1049-1067.  
Corrigan, J., Dunham, J., & Zwetsloot, R. (2022). The long-term stay rates of international STEM PhD graduates. *Center for Security and Emerging Technology.*  
Costello, T. H., & Bowes, S. M. (2023). Absolute certainty and political ideology: A systematic test of curvilinearity. *Social Psychological and Personality Science*, *14*(1), 93-102.  
Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. *Science*, *385*(6714), eadq1814.  
Coynash, H. (2023, August 21). 63% of Russians view bloody dictator and mass murderer Stalin positively. In Ukraine only 4%. KHPG. [https://khpg.org/en/1608812659](https://khpg.org/en/1608812659)  
Dafoe, A. (2011). Statistical critiques of the democratic peace: Caveat emptor. *American Journal of Political Science*, *55*(2), 247-262.  
Davidson, T., Finnveden, L. & Hadshar, R. ([2025](https://www.forethought.org/research/ai-enabled-coups-how-a-small-group-could-use-ai-to-seize-power), April 15). *AI-Enabled Coups: How a Small Group Could Use AI to Seize Power.* Forethought Research.  
Dean, A., Lister, T. & Cruickshank, P. (2018). *Nine Lives: My Time As MI6’s Top Spy Inside al-Qaeda.* Oneworld Publications.  
Diehl, M. (1990). The minimal group paradigm: Theoretical explanations and empirical findings. *European review of social psychology*, *1*(1), 263-292.  
Dikötter, F. (2016). *The Cultural Revolution: A People's History, 1962—1976*. Bloomsbury Publishing USA.  
Drutman, L. ([2023a](https://www.newamerica.org/political-reform/reports/more-parties-better-parties/), July 3). More Parties, Better Parties: The Case for Pro-Parties Democracy Reform. *New America.*  
Drutman, L. ([2023b](https://leedrutman.substack.com/p/a-healthy-democracy-requires-healthy), July 6). *A healthy democracy requires healthy political parties.* Undercurrent Events.  
Drutman, L. ([2023c](https://leedrutman.substack.com/p/revealed-exposed-unbelievable-the), September 28). *Revealed! Exposed! Unbelievable! The shocking hypothesis why misinformation is out of control.* Undercurrent Events.  
Economist Intelligence Unit (2006-2024) – processed by Our World in Data. *Democracy index – Economist Intelligence Unit*. [https://ourworldindata.org/grapher/democracy-index-eiu](https://ourworldindata.org/grapher/democracy-index-eiu)  
Eisenhower, D. D. (1953, April 27). *The Chance for Peace.* The United States Department of State.  
Fearon, J. D. (1995). Rationalist explanations for war. *International organization*, *49*(3), 379-414.  
Fernbach, P. M., Rogers, T., Fox, C. R., & Sloman, S. A. (2013). Political extremism is supported by an illusion of understanding. *Psychological science*, *24*(6), 939-946.  
Finnveden, L. ([2024a](https://lukasfinnveden.substack.com/p/project-ideas-epistemics), January 4). *Project ideas: Epistemics.* Lukas Finnveden.  
Finnveden, L. ([2024b](https://www.lesswrong.com/posts/D2n5uduYGXuexkv7v/what-s-important-in-ai-for-epistemics), August 24). *What’s important in “AI for epistemics”?.* LessWrong.  
Fiske, A. P., & Rai, T. S. (2014). *Virtuous violence: Hurting and killing to create, sustain, end, and honor social relationships*. Cambridge University Press.  
Freedom House (2025). *Freedom in the World 2025: The Uphill Battle to Safeguard Rights.*  
Fukuyama, F. Y. (1992). *The End of History and the Last Man.* Free Press.  
Gabriel, I. (2020). Artificial intelligence, values, and alignment. *Minds and machines*, *30*(3), 411-437.  
Galef, J. (2021). *The scout mindset: Why some people see things clearly and others don't*. Penguin.  
Gallup ([n.d.](https://news.gallup.com/poll/157067/views-violence.aspx)). *Views of Violence.* [https://news.gallup.com/poll/157067/views-violence.aspx](https://news.gallup.com/poll/157067/views-violence.aspx)  
Garfinkel, B. M. ([2021](https://forum.effectivealtruism.org/posts/TMCWXTayji7gvRK9p/is-democracy-a-fad), March 13). *Is Democracy a Fad?*. EA Forum.  
Ginges, J., Atran, S., Medin, D., & Shikaki, K. (2007). Sacred bounds on rational resolution of violent political conflict. *PNAS*, 104(18), 7357-7360.  
Gollwitzer, A., Olcaysoy Okten, I., Pizarro, A. O., & Oettingen, G. (2022). Discordant knowing: A social cognitive structure underlying fanaticism. *Journal of experimental psychology: general*, *151*(11), 2846.  
Gómez, Á., López-Rodríguez, L., Sheikh, H., Ginges, J., Wilson, L., Waziri, H., ... & Atran, S. (2017). The devoted actor's will to fight and the spiritual dimension of human conflict. *Nature Human Behaviour*, 1(9), 673-679.  
Goodwin, G. P., & Gromet, D. M. (2014). Punishment. *Wiley Interdisciplinary Reviews: Cognitive Science*, *5*(5), 561-572.  
Gwern (2017, April 21). *Terrorism is Not About Terror.* [Gwern.net](http://Gwern.net).  
Haidt, J. (2012). *The righteous mind: Why good people are divided by politics and religion*. Vintage.  
Heim, L., et al. [(2024)](https://www.governance.ai/post/computing-power-and-the-governance-of-ai). Computing Power and the Governance of AI. *Centre for the Governance of AI blog*  
Henrich, J. (2020). *The WEIRDest people in the world: How the West became psychologically peculiar and particularly prosperous*. Penguin.  
Herre, B., Rodés-Guirao, L., & Ortiz-Ospina, E. (2013). *Democracy*. Our World in Data. [https://ourworldindata.org/democracy](https://ourworldindata.org/democracy)  
Hess, R. W. R. (1934). *From Revolution to Construction* \[Speech transcript\]*.* German Propaganda Archive, [https://research.calvin.edu/german-propaganda-archive/hess5.htm](https://research.calvin.edu/german-propaganda-archive/hess5.htm)  
Hewstone, M., Rubin, M., & Willis, H. (2002). Intergroup bias. *Annual review of psychology*, *53*(1), 575-604.  
Hoffer, E. (1951). *The True Believer: Thoughts on the Nature of Mass Movements.* Harper & Brothers.  
IHRDC. Surviving Rape in Iran’s Prisons ([2011, Nov 10](https://iranhrdc.org/surviving-rape-in-irans-prisons/)). Iran Human Rights Documentation Center.  
*IranWire*. ([2023](https://iranwire.com/en/news/117116-ex-official-virgin-prisoners-were-raped-to-prevent-them-going-to-paradise/), June 1) “Ex-Official: Virgin Prisoners Were Raped to Prevent Them Going to Paradise”.  
Jones, A.: Heaven and hell in the Qurʾān, in Dévényi, K. and Fodor, A. (eds): Proceedings of the colloquium on Paradise and Hell in Islam, Keszthely, 7–14 July 2002, in The Arabist, 28–29 (2008), 105–22.  
Karnofsky, H. (Guest) & Wiblin, R. (Host) ([2023](https://80000hours.org/podcast/episodes/holden-karnofsky-how-ai-could-take-over-the-world/#misaligned-ai-might-not-kill-us-all-and-aligned-ai-could-be-catastrophic-002718), July 31). #158 - Holden Karnofsky on how AIs might take over even if they’re no smarter than humans, and his 4-part playbook for AI risk. *The 80,000 Hours Podcast.*  
Katsafanas, P. ([2019](https://philpapers.org/archive/KATFAS.pdf)). Fanaticism and sacred values. *Philosophers' Imprint*, 19(17), 1-20.  
Katsafanas, P. ([2022a](https://people.bu.edu/pkatsa/KATGFA.pdf)). Group fanaticism and narratives of ressentiment. In *The philosophy of fanaticism* (pp. 157-183). Routledge.  
Katsafanas, P. ([2022b](https://drive.google.com/drive/search?q=appendix%20S)). *Philosophy of devotion: The longing for invulnerable ideals*. Oxford University Press.  
Kaufmann, E. (2010). *Shall the Religious Inherit the Earth? Demography and Politics in the Twenty-First Century.* Profile Books.  
Klausen, J. (2016). *A behavioral study of the radicalization trajectories of american" homegrown" al qaeda-inspired terrorist offenders*. Brandeis University.  
Klein, E., & Thompson, D. (2025). *Abundance*. Simon and Schuster.  
Koehler, A. ([2022](https://80000hours.org/problem-profiles/liberal-democracy/), September). *Safeguarding liberal democracy.* 80,000 Hours.  
Kosonen, P. ([2025](https://www.utilitarianism.net/guest-essays/expected-value-fanaticism)). Expected Value Fanaticism. In R.Y. Chappell, D. Meissner, and W. MacAskill (eds.), *An Introduction to Utilitarianism*.  
Krouwel, A., Kutiyski, Y., Van Prooijen, J. W., Martinsson, J., & Markstedt, E. (2017). Does extreme political ideology predict conspiracy beliefs, economic evaluations and political trust? Evidence from Sweden. *Journal of Social and Political Psychology*, *5*(2), 435-462.  
Kruglanski, A. W., Gelfand, M. J., Bélanger, J. J., Sheveland, A., Hetiarachchi, M., & Gunaratna, R. (2014). The psychology of radicalization and deradicalization: How significance quest impacts violent extremism. *Political Psychology*, *35*, 69-93.  
Kunda, Z. (1990). The case for motivated reasoning. *Psychological bulletin*, 108(3), 480.  
Kurzban, R. O. (2012). *Why Everyone (Else) Is a Hypocrite: Evolution and the Modular Mind.* Princeton University Press.  
Lenin, V. (1906). *Lessons of the Moscow Uprising.* Marxist Internet Archive.  
Lenin, V. ([1913](https://www.marxists.org/archive/lenin/works/1913/mar/x01.htm)). *The Three Sources and Three Component Parts of Marxism.* Marxist Internet Archive.  
Leskelä, A. (2020, December 4). *Commitment and credibility in multipolar AI scenarios*. LessWrong.  
Levitsky, S., & Ziblatt, D. (2018). *How democracies die*. Crown.  
Linz, J. J. (2000). *Totalitarian and Authoritarian Regimes.* Lynne Rienner Publishers.  
Loza, W. (2007). The psychology of extremism and terrorism: A Middle-Eastern perspective. Aggression and Violent Behavior, 12(2), 141-155.  
MacAskill, W. (Guest) & Perry, L. (Host) ([2018](https://futureoflife.org/podcast/moral-uncertainty-and-the-path-to-ai-alignment-with-william-macaskill/), September 18). Moral Uncertainty and the Path to AI Alignment with William MacAskill. *AI Alignment Podcast.*  
MacAskill, W. (Guest) & Wiblin, R. (Host) (2020, January 24). #68 - Will MacAskill on the moral case against ever leaving the house, whether now is the hinge of history, and the culture of effective altruism. *The 80,000 Hours Podcast.*  
MacAskill, W. (2022). *What We Owe the Future*, New York: Basic Books.  
MacAskill, W. (Guest) & Wiblin, R. (Host) (2025, March 11). #213 - Will MacAskill on AI causing a “century in a decade” – and how we’re completely unprepared. *The 80,000 Hours Podcast.*  
MacAskill, W. ([2025a](https://forum.effectivealtruism.org/posts/R8AAG4QBZi5puvogR/effective-altruism-in-the-age-of-agi), October 10). *Effective altruism in the age of AGI*. EA Forum.  
MacAskill, W. (2025b). [*Introducing Better Futures*](https://www.forethought.org/research/introducing-better-futures%20). Forethought Research.  
MacAskill, W. & Moorhouse, F. (2025a). [*No Easy Eutopia*](https://www.forethought.org/research/no-easy-eutopia). Forethought Research.  
MacAskill, W. & Moorhouse, F. (2025b). [*Convergence and Compromise*](https://www.forethought.org/research/convergence-and-compromise). Forethought Research.  
MacAskill, W. (2025c). [*Persistent Path-Dependence*](https://www.forethought.org/research/persistent-path-dependence). Forethought Research.  
MacAskill, W. (2025d). [*How to Make the Future Better*](https://www.forethought.org/research/how-to-make-the-future-better). Forethought Research.  
MacAskill, W. & Hadshar, R. (2025). *Intelsat as a Model for International AGI Governance.* Forethought Research.  
MacAskill, W. & Moorhouse, F. (2025). *Preparing for the Intelligence Explosion.* Forethought Research.  
Mainwaring, S. & Drutman, L., (2023). *The Case for Multiparty Presidentialism in the US: Why the House Should Adopt Proportional Representation*, Protect Democracy and New America.  
Manson, J. H. (2020). Right-wing authoritarianism, left-wing authoritarianism, and pandemic-mitigation authoritarianism. *Personality and individual differences*, *167*, 110251.  
Maoz, Z., & Abdolali, N. (1989). Regime types and international conflict, 1816-1976. *Journal of Conflict Resolution*, *33*(1), 3-35.  
Marimaa, K. (2011). The many faces of fanaticism. *KVÜÕA toimetised*, (14), 29-55.  
Mason, L. (2018). *Uncivil agreement: How politics became our identity*. University of Chicago Press.  
Meedović, J., and Knežević, G. (2019). Dark and peculiar: the key features of militant extremist thinking pattern? *J. Individ. Differ.* 40, 92–103. doi: 10.1027/1614-0001/a000280  
Montefiore, S. S. ([2007](https://www.amazon.com/Stalin-Court-Simon-Sebag-Montefiore-ebook/dp/B000XUBEJS)). *Stalin: The Court of the Red Tsar*. Vintage.  
Morton, J. (Guest) & Greenberg, S. (Host) (2022, May 5). Episode 103: A former Al-Qaeda recruiter speaks (with Jesse Morton). *Clearer Thinking.*  
Müller, H., & Wolff, J. (2004, August). Dyadic democratic peace strikes back. In *5th Pan-European international relations conference the Hague, September* (pp. 9-11).  
Nguyen, L. C. ([2024](https://forum.effectivealtruism.org/posts/wE7KPnjZHBjxLKNno/ai-things-that-are-perhaps-as-important-as-human-controlled#Conceptual_progress), March 3). *AI things that are perhaps as important as human-controlled AI*. EA Forum.  
O'Neill, C., Miller, J., Ciuca, I., Ting, Y. S., & Bui, T. ([2023](https://arxiv.org/abs/2308.13768)). Adversarial fine-tuning of language models: An iterative optimisation approach for the generation and detection of problematic content. *arXiv preprint arXiv:2308.13768*.  
Oesterheld, C., ([2017](https://longtermrisk.org/files/Multiverse-wide-Cooperation-via-Correlated-Decision-Making.pdf)). *Multiverse-wide Cooperation via Correlated Decision Making.*  
Ord, T. (2020). *The Precipice: Existential Risk and the Future of Humanity.* Bloomsbury Publishing.  
Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2020). On the belief that beliefs should change according to evidence: Implications for conspiratorial, moral, paranormal, political, religious, and science beliefs. *Judgment and Decision making*, *15*(4), 476-498.  
Perkinson, H. J. (2002). Fanaticism: flight from fallibility. *ETC: A Review of General Semantics*, *59*(2), 170-174.  
Pew Research Center ([2010](https://www.pewresearch.org/wp-content/uploads/sites/7/2010/04/sub-saharan-africa-full-report.pdf), April). *Tolerance and Tension: Islam and Christianity in Sub-Saharan Africa.*  
Pew Research Center ([2013](https://www.pewresearch.org/religion/wp-content/uploads/sites/7/2013/04/worlds-muslims-religion-politics-society-full-report.pdf), April). *The World’s Muslims: Religion, Politics and Society.*  
Pew Research Center ([2021](https://www.pewresearch.org/wp-content/uploads/sites/20/2021/06/PF_06.29.21_India.full_.report.pdf), June). *Religion in India: Tolerance and Segregation.*  
Pew Research Center ([2022](https://www.pewresearch.org/wp-content/uploads/sites/20/2022/10/PF_2022.10.27_christian-nationalism_REPORT.pdf), October). *45% of Americans Say U.S. Should Be a ‘Christian Nation’.*  
Pew Research Center ([2023](https://www.pewresearch.org/wp-content/uploads/sites/20/2023/08/PF_2023.08.30_religion-china_REPORT.pdf), August). *Measuring Religion in China.*  
Pew Research Center ([2023b](https://www.pewresearch.org/religion/wp-content/uploads/sites/7/2023/09/pr_2023.09.12_se-asia_report.pdf), September). *Buddhism, Islam and Religious Pluralism in South and Southeast Asia.*  
Pinker, S. (2018). *Enlightenment now: The case for reason, science, humanism, and progress*. Penguin UK.  
Popper, K. (1945). *The open society and its enemies*. Routledge.  
PRRI/Brookings survey ([2023](https://prri.org/wp-content/uploads/2023/02/PRRI-Jan-2023-Christian-Nationalism-Final.pdf)). *A Christian Nation? Understanding the threat of Christian Nationalism to American democracy and culture.* PRRI; Brookings Institution.  
Pretus, C., Hamid, N., Sheikh, H., Ginges, J., Tobeña, A., Davis, R., ... & Atran, S. (2018). Neural and behavioral correlates of sacred values and vulnerability to violent extremism. *Frontiers in Psychology*, 9, 2462.  
The Qur’an (Khattab, M., Trans.). (2016). Book of Signs Foundation.  
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. ([2023](https://arxiv.org/abs/2305.18290)). Direct preference optimization: Your language model is secretly a reward model. *Advances in neural information processing systems*, *36*, 53728-53741.  
Reinisch, E. & Thomas, L. ([2022](https://blogs.lse.ac.uk/government/2022/02/04/are-the-united-arab-emirates-on-their-way-to-becoming-a-democracy/#:~:text=Slowly%20Detaching%20themselves,human%20rights%20violations.), February 4). *Are the United Arab Emirates on their way to becoming a democracy?* LSE Government Blog.  
Robespierre, M. F. ([1794](https://www.marxists.org/history/france/revolution/robespierre/1794/political-morality.htm)). *On the Principles of Political Morality.* Marxists Internet Archive.  
Rokeach, M. (1960). The open and closed mind: Investigations into the nature of belief systems and personality systems.  
Rosato, S. (2003). The flawed logic of democratic peace theory. *American political science review*, *97*(4), 585-602.  
Russett, B. (1993). Can a democratic peace be built?. *International Interactions*, *18*(3), 277-282.  
Sapolsky, R. (2017). *Behave: The Biology of Humans at Our Best and Worst.* Penguin Press.  
Satloff, R. ([2008](https://www.washingtoninstitute.org/policy-analysis/just-us-really)). *Just like us! Really?.* The Washington Institute for Near East Policy.  
Saucier, G., Akers, L. G., Shen-Miller, S., Kneževié, G., & Stankov, L. (2009). Patterns of thinking in militant extremism. *Perspectives on psychological science*, *4*(3), 256-271.  
Scheufele, D. A., Krause, N. M., & Freiling, I. (2021). Misinformed about the “infodemic?” Science’s ongoing struggle with misinformation. *Journal of Applied Research in Memory and Cognition*, *10*(4), 522-526.  
Selengut, C. (2017). *Sacred fury: Understanding religious violence*. Rowman & Littlefield.  
Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., ... & Perez, E. (2023). Towards understanding sycophancy in language models. *arXiv preprint arXiv:2310.13548*.  
Sheikh, H., Ginges, J., Coman, A., & Atran, S. (2012). Religion, group threat and sacred values. *Judgment and Decision Making*, 7(2), 110-118.  
Shirer, W. L. (1960). *The Rise and Fall of the Third Reich: A History of Nazi Germany.* Simon & Schuster.  
Shulman, C. (2010). *Omohundro’s “Basic AI Drives” and Catastrophic Risks*. Machine Intelligence Research Institute.  
Simkin, J. ([1997](https://spartacus-educational.com/RUSterror.htm), September). *The Red Terror.* Spartacus Educational. [https://spartacus-educational.com/RUSterror.htm](https://spartacus-educational.com/RUSterror.htm)  
Simler, K. & Hanson, R. (2018). *The Elephant in the Brain: Hidden Motives in Everyday Life.* Oxford University Press.  
Skitka, L. J., Bauman, C. W., & Sargis, E. G. (2005). Moral conviction: Another contributor to attitude strength or something more?. *Journal of personality and social psychology*, *88*(6), 895.  
Skitka, L. J., Hanson, B. E., Morgan, G. S., & Wisneski, D. C. (2021). The psychology of moral conviction. *Annual Review of Psychology*, 72(1), 347-366.  
Skocpol, T. (1979). *States and social revolutions: A comparative analysis of France, Russia and China*. Cambridge University Press.  
Stankov, L., Saucier, G., & Knežević, G. (2010). Militant extremist mind-set: Proviolence, Vile World, and Divine Power. *Psychological assessment*, *22*(1), 70.  
Storr, W. ([2021](https://www.amazon.com/Status-Game-Position-Governs-Everything/dp/0008354642)). *The Status Game: How Social Position Governs Everything.* HarperCollins Publishers.  
Szanto, T. (2022). Sacralizing hostility: Fanaticism as a group-based affective mechanism. In *The Philosophy of Fanaticism* (pp. 184-212). Routledge.  
Taylor, M. ([2025](https://longtermrisk.org/measurement-research-agenda)). *Measurement Research Agenda*. Center on Long-Term Risk. [https://longtermrisk.org/measurement-research-agenda](https://longtermrisk.org/measurement-research-agenda)  
Tetlock, P. E. (2003). Thinking the unthinkable: Sacred values and taboo cognitions. *Trends in cognitive sciences*, *7*(7), 320-324.  
Tetlock, P. E., Kristel, O. V., Elson, S. B., Green, M. C., & Lerner, J. S. (2000). The psychology of the unthinkable: taboo trade-offs, forbidden base rates, and heretical counterfactuals. *Journal of personality and social psychology*, *78*(5), 853.  
Thomson, P., & Halstead, J. (2022). How violent was the pre-agricultural world?. *Available at SSRN 4466809*.  
Tietjen, R. R. (2023). Fear, fanaticism, and fragile identities. *The Journal of Ethics*, *27*(2), 211-230.  
Todd, B. ([2024a](https://forum.effectivealtruism.org/posts/YDjH6ACPZq889tqeJ/updates-on-the-ea-catastrophic-risk-landscape), May 6). *Updates on the EA catastrophic risk landscape*. EA Forum.  
Todd, B. ([2024b](https://forum.effectivealtruism.org/posts/ukG7TYPAFDCHt8Cri/project-idea-ai-for-epistemics), May 19). *Project idea: AI for epistemics*. EA Forum.  
Tomz, M., & Weeks, J. L. (2012, February). An experimental investigation of the democratic peace. In *Annual Meeting of the American Political Science Association. Washington, DC*.  
Torcal, M., & Magalhães, P. C. (2022). Ideological extremism, perceived party system polarization, and support for democracy. *European Political Science Review*, *14*(2), 188-205.  
Urban, T. (2023). *What's Our Problem?: A Self-help Book for Societies*. Wait But Why.  
Van Prooijen, J. W., & Krouwel, A. P. (2017). Extreme political beliefs predict dogmatic intolerance. *Social Psychological and Personality Science*, *8*(3), 292-300.  
Van Prooijen, J. W., & Krouwel, A. P. (2019). Psychological features of extreme political ideologies. *Current Directions in Psychological Science*, *28*(2), 159-163.  
Van Prooijen, J. W., Krouwel, A. P., Boiten, M., & Eendebak, L. (2015a). Fear among the extremes: How political ideology predicts negative emotions and outgroup derogation. *Personality and social psychology bulletin*, *41*(4), 485-497.  
Van Prooijen, J. W., Krouwel, A. P., & Emmer, J. (2018). Ideological responses to the EU refugee crisis: The left, the right, and the extremes. *Social psychological and personality science*, *9*(2), 143-150.  
Van Prooijen, J. W., Krouwel, A. P., & Pollet, T. V. (2015). Political extremism predicts belief in conspiracy theories. *Social psychological and personality science*, *6*(5), 570-578.  
Van Prooijen, J. W., & Kuijper, S. M. ([2020](https://www.sciencedirect.com/science/article/pii/S0191886920300775)). A comparison of extreme religious and political ideologies: Similar worldviews but different grievances. *Personality and Individual Differences*, *159*, 109888.  
Varmann, A. H., Kruse, L., Bierwiaczonek, K., Gomez, A., Vazquez, A., & Kunst, J. R. (2024). How identity fusion predicts extreme pro-group orientations: A meta-analysis. *European Review of Social Psychology*, *35*(1), 162-197.  
Vinding, M. (2022). *Reasoned Politics*. Ratio Ethica*.*  
Weiss, J. C. (2019). How hawkish is the Chinese public? Another look at “rising nationalism” and Chinese foreign policy. *Journal of Contemporary China*, *28*(119), 679-695.  
Williams, D. (2021). Socially adaptive belief. *Mind & Language*, *36*(3), 333-354.  
Williams, D. (2022). Signalling, commitment, and strategic absurdities. *Mind & Language*, *37*(5), 1011-1029.  
Williams, D. (2023). The marketplace of rationalizations. *Economics & Philosophy*, *39*(1), 99-123.  
Williams, D. ([2024a](https://www.conspicuouscognition.com/p/misinformation-researchers-are-wrong), January 10). *Misinformation researchers are wrong: There can’t be a science of misleading content.* Conspicuous Cognition.  
Williams, D. ([2024b](https://www.conspicuouscognition.com/p/the-deep-and-unavoidable-roots-of), December 6). *The deep and unavoidable roots of political bias.* Conspicuous Cognition.  
Williams, D. ([2025a](https://www.conspicuouscognition.com/p/status-class-and-the-crisis-of-expertise), May 31). *Status, class, and the crisis of expertise.* Conspicuous Cognition.  
Williams, D. ([2025b](https://www.conspicuouscognition.com/p/is-social-media-destroying-democracyor), October 7). *Is Social Media Destroying Democracy—OrGiving It To Us Good And Hard?.* Conspicuous Cognition.  
Williams, D. ([2025c](https://www.conspicuouscognition.com/p/on-highbrow-misinformation), October 26). *On Highbrow Misinformation.* Conspicuous Cognition.  
Wilkinson, H. (2022). In defense of fanaticism. *Ethics*, *132*(2), 445-477.  
Wood, G. ([2015](https://www.theatlantic.com/magazine/archive/2015/03/what-isis-really-wants/384980/), March). What ISIS Really Wants. *The Atlantic.*  
Yelnats, T. J. ([2024](https://forum.effectivealtruism.org/posts/kmx3rKh2K4ANwMqpW), July 15). *Destabilization of the United States: The top X-factor EA neglects?*. EA Forum. [https://forum.effectivealtruism.org/posts/kmx3rKh2K4ANwMqpW](https://forum.effectivealtruism.org/posts/kmx3rKh2K4ANwMqpW)  
Yiwei, Z. ([2013](https://www.globaltimes.cn/content/834000.shtml), December 24).85% say Mao’s merits outweigh his faults: poll. *Global Times*.  
Yuri Levada Analytical Center ([2022](https://www.levada.ru/en/2022/03/18/xenophobia-and-nationalism-in-state-power/), March). *Xenophobia and Nationalism in State Power.*  
Zwicker, M. V., van Prooijen, J. W., & Krouwel, A. P. (2020). Persistent beliefs: Political extremism predicts ideological stability over time. *Group Processes & Intergroup Relations*, *23*(8), 1137-1149.

[^-bxXhDCzXq8bJcuQWw-1]:  

[^-bxXhDCzXq8bJcuQWw-2]:  

[^-bxXhDCzXq8bJcuQWw-3]:  

[^-bxXhDCzXq8bJcuQWw-4]:  

[^-bxXhDCzXq8bJcuQWw-5]:  

[^-bxXhDCzXq8bJcuQWw-6]:  

[^-bxXhDCzXq8bJcuQWw-7]:  

[^-bxXhDCzXq8bJcuQWw-8]:  

[^-bxXhDCzXq8bJcuQWw-9]:  

[^-bxXhDCzXq8bJcuQWw-10]:  

[^-bxXhDCzXq8bJcuQWw-11]:  

[^-bxXhDCzXq8bJcuQWw-12]:  

[^-bxXhDCzXq8bJcuQWw-13]:  

[^-bxXhDCzXq8bJcuQWw-14]:  

[^-bxXhDCzXq8bJcuQWw-15]:  

[^-bxXhDCzXq8bJcuQWw-16]:  

[^-bxXhDCzXq8bJcuQWw-17]:  

[^-bxXhDCzXq8bJcuQWw-18]:  

[^-bxXhDCzXq8bJcuQWw-19]:  

[^-bxXhDCzXq8bJcuQWw-20]:  

[^-bxXhDCzXq8bJcuQWw-21]:  

[^-bxXhDCzXq8bJcuQWw-22]:  

[^-bxXhDCzXq8bJcuQWw-23]:  

[^-bxXhDCzXq8bJcuQWw-24]:  

[^-bxXhDCzXq8bJcuQWw-25]:  

[^-bxXhDCzXq8bJcuQWw-26]:  

[^-bxXhDCzXq8bJcuQWw-27]:  

[^-bxXhDCzXq8bJcuQWw-28]:  

[^-bxXhDCzXq8bJcuQWw-29]:  

[^-bxXhDCzXq8bJcuQWw-30]:  

[^-bxXhDCzXq8bJcuQWw-31]:  

[^-bxXhDCzXq8bJcuQWw-32]:  

[^-bxXhDCzXq8bJcuQWw-33]:  

[^-bxXhDCzXq8bJcuQWw-34]:  

[^-bxXhDCzXq8bJcuQWw-35]:  

[^-bxXhDCzXq8bJcuQWw-36]:  

[^-bxXhDCzXq8bJcuQWw-37]:  

[^-bxXhDCzXq8bJcuQWw-38]:  

[^-bxXhDCzXq8bJcuQWw-39]:  

[^-bxXhDCzXq8bJcuQWw-40]:  

[^-bxXhDCzXq8bJcuQWw-41]:  

[^-bxXhDCzXq8bJcuQWw-42]:  

[^-bxXhDCzXq8bJcuQWw-43]:  

[^-bxXhDCzXq8bJcuQWw-44]:  

[^-bxXhDCzXq8bJcuQWw-45]:  

[^-bxXhDCzXq8bJcuQWw-46]:  

[^-bxXhDCzXq8bJcuQWw-47]:  

[^-bxXhDCzXq8bJcuQWw-48]:  

[^-bxXhDCzXq8bJcuQWw-49]:  

[^-bxXhDCzXq8bJcuQWw-50]:  

[^-bxXhDCzXq8bJcuQWw-51]:  

[^-bxXhDCzXq8bJcuQWw-52]:  

[^-bxXhDCzXq8bJcuQWw-53]:  

[^-bxXhDCzXq8bJcuQWw-54]:  

[^-bxXhDCzXq8bJcuQWw-55]:  

[^-bxXhDCzXq8bJcuQWw-56]:  

[^-bxXhDCzXq8bJcuQWw-57]:  

[^-bxXhDCzXq8bJcuQWw-58]:  

[^-bxXhDCzXq8bJcuQWw-59]:  

[^-bxXhDCzXq8bJcuQWw-60]:  

[^-bxXhDCzXq8bJcuQWw-61]:  

[^-bxXhDCzXq8bJcuQWw-62]:  

[^-bxXhDCzXq8bJcuQWw-63]:  

[^-bxXhDCzXq8bJcuQWw-64]:  

[^-bxXhDCzXq8bJcuQWw-65]:  

[^-bxXhDCzXq8bJcuQWw-66]:  

[^-bxXhDCzXq8bJcuQWw-67]:  

[^-bxXhDCzXq8bJcuQWw-68]:  

[^-bxXhDCzXq8bJcuQWw-69]:  

[^-bxXhDCzXq8bJcuQWw-70]:  

[^-bxXhDCzXq8bJcuQWw-71]:  

[^-bxXhDCzXq8bJcuQWw-72]:  

[^-bxXhDCzXq8bJcuQWw-73]:  

[^-bxXhDCzXq8bJcuQWw-74]:  

[^-bxXhDCzXq8bJcuQWw-75]:  

[^-bxXhDCzXq8bJcuQWw-76]:  

[^-bxXhDCzXq8bJcuQWw-77]:  

[^-bxXhDCzXq8bJcuQWw-78]:  

[^-bxXhDCzXq8bJcuQWw-79]:  

[^-bxXhDCzXq8bJcuQWw-80]:  

[^-bxXhDCzXq8bJcuQWw-81]:  

[^-bxXhDCzXq8bJcuQWw-82]:  

[^-bxXhDCzXq8bJcuQWw-83]:  

[^-bxXhDCzXq8bJcuQWw-84]:  

[^-bxXhDCzXq8bJcuQWw-85]:  

[^-bxXhDCzXq8bJcuQWw-86]:  

[^-bxXhDCzXq8bJcuQWw-87]:  

[^-bxXhDCzXq8bJcuQWw-88]:  

[^-bxXhDCzXq8bJcuQWw-89]:  

[^-bxXhDCzXq8bJcuQWw-90]:  

[^-bxXhDCzXq8bJcuQWw-91]:  

[^-bxXhDCzXq8bJcuQWw-92]:  

[^-bxXhDCzXq8bJcuQWw-93]:  

[^-bxXhDCzXq8bJcuQWw-94]:  

[^-bxXhDCzXq8bJcuQWw-95]:  

[^-bxXhDCzXq8bJcuQWw-96]:  

[^-bxXhDCzXq8bJcuQWw-97]:  

[^-bxXhDCzXq8bJcuQWw-98]:  

[^-bxXhDCzXq8bJcuQWw-99]:  

[^-bxXhDCzXq8bJcuQWw-100]:  

[^-bxXhDCzXq8bJcuQWw-101]:  

[^-bxXhDCzXq8bJcuQWw-102]:  

[^-bxXhDCzXq8bJcuQWw-103]:  

[^-bxXhDCzXq8bJcuQWw-104]:  

[^-bxXhDCzXq8bJcuQWw-105]:  

[^-bxXhDCzXq8bJcuQWw-106]:  

[^-bxXhDCzXq8bJcuQWw-107]:  

[^-bxXhDCzXq8bJcuQWw-108]:  

[^-bxXhDCzXq8bJcuQWw-109]:  

[^-bxXhDCzXq8bJcuQWw-110]:  

[^-bxXhDCzXq8bJcuQWw-111]:  

[^-bxXhDCzXq8bJcuQWw-112]:  

[^-bxXhDCzXq8bJcuQWw-113]:  

[^-bxXhDCzXq8bJcuQWw-114]:  

[^-bxXhDCzXq8bJcuQWw-115]:  

[^-bxXhDCzXq8bJcuQWw-116]:  

[^-bxXhDCzXq8bJcuQWw-117]:  

[^-bxXhDCzXq8bJcuQWw-118]:  

[^-bxXhDCzXq8bJcuQWw-119]:  

[^-bxXhDCzXq8bJcuQWw-120]:  

[^-bxXhDCzXq8bJcuQWw-121]:  

[^-bxXhDCzXq8bJcuQWw-122]:  

[^-bxXhDCzXq8bJcuQWw-123]:  

[^-bxXhDCzXq8bJcuQWw-124]:  

[^-bxXhDCzXq8bJcuQWw-125]:  

[^-bxXhDCzXq8bJcuQWw-126]:  

[^-bxXhDCzXq8bJcuQWw-127]:  

[^-bxXhDCzXq8bJcuQWw-128]:  

[^-bxXhDCzXq8bJcuQWw-129]:  

[^-bxXhDCzXq8bJcuQWw-130]:  

[^-bxXhDCzXq8bJcuQWw-131]:  

[^-bxXhDCzXq8bJcuQWw-132]:  

[^-bxXhDCzXq8bJcuQWw-133]:  

[^-bxXhDCzXq8bJcuQWw-134]:  

[^-bxXhDCzXq8bJcuQWw-135]:  

[^-bxXhDCzXq8bJcuQWw-136]:  

[^-bxXhDCzXq8bJcuQWw-137]:  

[^-bxXhDCzXq8bJcuQWw-138]:  

[^-bxXhDCzXq8bJcuQWw-139]:  

[^-bxXhDCzXq8bJcuQWw-140]:  

[^-bxXhDCzXq8bJcuQWw-141]:  

[^-bxXhDCzXq8bJcuQWw-142]:  

[^-bxXhDCzXq8bJcuQWw-143]:  

[^-bxXhDCzXq8bJcuQWw-144]:  

[^-bxXhDCzXq8bJcuQWw-145]:  

[^-bxXhDCzXq8bJcuQWw-146]:  

[^-bxXhDCzXq8bJcuQWw-147]:  

[^-bxXhDCzXq8bJcuQWw-148]:  

[^-bxXhDCzXq8bJcuQWw-149]:  

[^-bxXhDCzXq8bJcuQWw-150]:  

[^-bxXhDCzXq8bJcuQWw-151]:  

[^-bxXhDCzXq8bJcuQWw-152]:  

[^-bxXhDCzXq8bJcuQWw-153]:  

[^-bxXhDCzXq8bJcuQWw-154]:  

[^-bxXhDCzXq8bJcuQWw-155]:  

[^-bxXhDCzXq8bJcuQWw-156]:  

[^-bxXhDCzXq8bJcuQWw-157]:  

[^-bxXhDCzXq8bJcuQWw-158]:  

[^-bxXhDCzXq8bJcuQWw-159]:  

[^-bxXhDCzXq8bJcuQWw-160]:  

[^-bxXhDCzXq8bJcuQWw-161]:  

[^-bxXhDCzXq8bJcuQWw-162]:  

[^-bxXhDCzXq8bJcuQWw-163]:  

[^-bxXhDCzXq8bJcuQWw-164]:  

[^-bxXhDCzXq8bJcuQWw-165]:  

[^-bxXhDCzXq8bJcuQWw-166]:  

[^-bxXhDCzXq8bJcuQWw-167]:  

[^-bxXhDCzXq8bJcuQWw-168]:  

[^-bxXhDCzXq8bJcuQWw-169]:  

[^-bxXhDCzXq8bJcuQWw-170]:  

[^-bxXhDCzXq8bJcuQWw-171]:  

[^-bxXhDCzXq8bJcuQWw-172]:  

[^-bxXhDCzXq8bJcuQWw-173]:  

[^-bxXhDCzXq8bJcuQWw-174]:  

[^-bxXhDCzXq8bJcuQWw-175]:  

[^-bxXhDCzXq8bJcuQWw-176]:  

[^-bxXhDCzXq8bJcuQWw-177]:  

[^-bxXhDCzXq8bJcuQWw-178]: