# What are information hazards?

_This post was written for [Convergence Analysis](https://www.convergenceanalysis.org/). Cross-posted to the [EA Forum](https://forum.effectivealtruism.org/posts/Nc5EjccDTfmcrG93j/what-are-information-hazards)._

The concept of _information hazards_ is highly relevant to many efforts to do good in the world (particularly, but not only, from the perspective of reducing [existential risks](https://forum.effectivealtruism.org/posts/skPFH8LxGdKQsTkJy/clarifying-existential-risks-and-existential-catastrophes)). I’m thus glad that many effective altruists and rationalists seem to know of, and refer to, this concept. However, it also seems that:

*   People referring to the concept often don’t clearly define/explain it
*   Many people (quite understandably) haven’t read Nick Bostrom’s [original (34 page) paper](https://nickbostrom.com/information-hazards.pdf) on the subject
*   Some people misunderstand or misuse the term “information hazards” in certain ways

Thus, this post seeks to summarise, clarify, and dispel misconceptions about the concept of information hazards. It doesn’t present any new ideas of my own.

* * *

In [his paper](https://nickbostrom.com/information-hazards.pdf), Bostrom defines an information hazard as:

> A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.  

He emphasises that **this concept is just about how _true_ information can cause harm, not how _false_ information can cause harm** (which is typically a more obvious possibility).

Bostrom’s paper outlines many different types of information hazards, and gives examples of each. The first two types listed are the following:

> _Data hazard_: Specific data, such as the genetic sequence of a lethal pathogen or a blueprint for making a thermonuclear weapon, if disseminated, create risk.  

> \[...\] _Idea hazard_: A general idea, if disseminated, creates a risk, even without a data-rich detailed specification.  

> For example, the idea of using a fission reaction to create a bomb, or the idea of culturing bacteria in a growth medium with an antibiotic gradient to evolve antibiotic resistance, may be all the guidance a suitably prepared developer requires; the details can be figured out. Sometimes the mere demonstration that something (such as a nuclear bomb) is possible provides valuable information which can increase the likelihood that some agent will successfully set out to replicate the achievement.  

He further writes:

> Even if the relevant ideas and data are already “known”, and published in the open literature, an increased risk may nonetheless be created by drawing attention to a particularly potent possibility.  

This leads him to his third type:

> _Attention hazard_: mere drawing of attention to some particularly potent or relevant ideas or data increases risk, even when these ideas or data are already “known”.  

> Because there are countless avenues for doing harm, an adversary faces a vast search task in finding out which avenue is most likely to achieve his goals. Drawing the adversary’s attention to a subset of especially potent avenues can greatly facilitate the search. For example, if we focus our concern and our discourse on the challenge of defending against viral attacks, this may signal to an adversary that viral weapons—as distinct from, say, conventional explosives or chemical weapons—constitute an especially promising domain in which to search for destructive applications.

The possibility of these and other types of information hazards means that it will sometimes be morally best to avoid creating or spreading even _true_ information. (Exactly _when_ and _how_ to attend to and reduce potential information hazards is beyond the scope of this post; for thoughts on that, see my [later post](https://www.lesswrong.com/posts/6ur8vDX6ApAXrRN3t/information-hazards-why-you-should-care-and-what-you-can-do).)

Context and scale
=================

Those quoted examples all relate to fairly large-scale risks (perhaps even existential risks). Some also relate to risks from advancing the development of potentially dangerous technologies (contrary to the principle of [differential progress](https://forum.effectivealtruism.org/posts/XCwNigouP88qhhei2/differential-progress-intellectual-progress-technological)). It seems to me that the concept of information hazards is most often used in relation to such large-scale, existential, and/or technological risks, and indeed that that’s where the concept is most useful.

However, it’s also worth noting that information hazards don’t _have_ to relate to these sorts of risks. **Information hazards can occur in a wide variety of contexts, and can sometimes be very mundane or minor.** Some of Bostrom’s types and examples highlight that. For example:

> Spoilers constitute a special kind of disappointment. Many forms of entertainment depend on the marshalling of ignorance. Hide-and-seek would be less fun if there were no way to hide and no need to seek. For some, knowing the day and the hour of their death long in advance might cast shadow over their existence.  

> \[Thus, it is also possible to have a\] _Spoiler hazard_: Fun that depends on ignorance and suspense is at risk of being destroyed by premature disclosure of truth.  

Who’s at risk?
==============

Spoiler hazards are a type of information hazards that risk harm only to the _knower_ of the true information themselves, and as a _direct result of them knowing the information_. (In contrast, in Bostrom’s earlier examples, although the knower might _eventually_ be harmed, this would be (1) _along with many other people_, and (2) a result of catastrophic or existential risks that were made more likely by the knowledge the knower spread, rather than as a _direct result_ of the knower having that knowledge.)

Bostrom also lists additional types of information hazards where the risk of harm is to the knower themselves, and directly results from their knowledge. But there appears to be no established term for the entire subset of information hazards that fit that description. Proposed terms I’m partial to include [“knowledge hazards” and “direct information hazards”](https://www.lesswrong.com/posts/Rut5wZ7qyHoj3dj4k/a-point-of-clarification-on-infohazard-terminology?commentId=kSGy6yorQmraSKNWT). (Further discussion can be found in that comments section. If you have thoughts on that, please comment here or there.)

But I should emphasise that that is indeed _only a subset_ of all information hazards; **information hazards _can_ harm people other than the knower themselves**, and, as mentioned above, this will be the case in the contexts where information hazards are perhaps most worrisome and worth attending to. (I emphasise this because some people misunderstand or misuse the term “information hazards” as referring _only_ to what we might call “knowledge hazards”; this misuse is apparent [here](https://www.lesswrong.com/posts/HyW62cRA9rgHvt9zP/information-hazards-and-community-hazards), and is discussed [here](https://www.lesswrong.com/posts/Rut5wZ7qyHoj3dj4k/a-point-of-clarification-on-infohazard-terminology).)

Information hazards are _risks_
===============================

As noted, an information hazard is “A **risk** **that** **arises** from the dissemination or the potential dissemination of (true) information that **may** cause harm or enable some agent to cause harm” (emphasis added). Thus, as best I can tell:

*   Something can create an information hazard even if no harm has yet occurred, and there’s no guarantee it will ever occur.

*   E.g., writing a paper on a plausibly dangerous technology can create an information hazard, even if the technology turns out to be safe after all.

*   But if we don’t have any _specific_ reason to believe that there’s even a “risk” of harm from some true information, and we just think that it’d be worth bearing in mind that there _might_ be a risk, it may be best to say there’s a “_potential_ information hazard”.

*   So it’s probably not _helpful_ to, for example, slap the label of “information hazard” on _all_ of biological research.

Closing remarks
===============

I hope you’ve found this post clear and useful. To recap:

*   The concept of information hazards relates to risks of harm from creating or spreading _true_ information (not from creating or spreading _false_ information).
*   The concept is definitely very useful in relation to existential risks and risks from technological development, but can also apply in a wide range of other contexts, and at much smaller scales.
*   _Some_ information hazards risk harm only to the knower of the true information themselves, and as a direct result of them knowing the information. But _many_ information hazards harm other people, or harm in other ways.
*   Information hazards are _risks_ of harm, not necessarily _guaranteed_ harms.

In my next posts, I'll:

*   [discuss and visually represent](https://www.lesswrong.com/posts/RY9XYoqPeMc8W8zbH/mapping-downside-risks-and-information-hazards) how information hazards and downside risks relate to each other and to other types of effects
*   [suggest some rules-of-thumb](https://www.lesswrong.com/posts/6ur8vDX6ApAXrRN3t/information-hazards-why-you-should-care-and-what-you-can-do) regarding why, when, and how one should deal with potential information hazards (aiming for more nuance than just “Always pursue truth!” or “Never risk it!”).

_My thanks to [David Kristoffersson](https://www.lesswrong.com/users/david_kristoffersson) and [Justin Shovelain](https://www.lesswrong.com/users/justinshovelain) for feedback on this post._

_Other sources relevant to this topic are listed_ _[here](https://forum.effectivealtruism.org/posts/EMKf4Gyee7BsY2RP8/michaela-s-shortform?commentId=dTghHNHmc5qf5znMQ)._