# Do yourself a FAVAR: security mindset

(We're always in search of a concise and actionable way to communicate security mindset. Here's my attempt.)

Just as a sprocket you don't use never fails and code you don't write has no bugs, an input constraint you don't require is never violated.

Want a secure system? Then DO yourself a FAVAR:

*   **D**escribe **O**bjectives: How is the system supposed to function? What constraints does its operation and output have?
*   **F**ind **A**ssumptions: What is the system assuming about its inputs, its environment, or itself?
*   **V**alidate **A**ssumptions: Are the assumptions we're making always correct? Even if they all are, then ↓
*   **R**emove the ones you can: Of course any incorrect assumptions have to be removed, but it is important to remove as many assumptions as possible. Helicopters [land](https://youtu.be/hkQbVvqmRoI?t=29) even if the engine dies.

Let's take this `transferMoney` procedure in a token program or an online bank as a case study:

    transferMoney(fromId, fromSig, amount, destId):
    	if amount < 0: return
    	if not validate(fromId, fromSig): return
    	curBal := db.get(fromId)
    	if curBal < amount: return
    	db.set(fromId, curBal - amount)
    	db.set(destId, db.get(destId) + amount)

*   Our objective is for total balance to be conserved, and for transactions to only be initiated by the sender.
*   Finding assumptions is the hard part. This code actually assumes at least this much:
    *   Addition and subtraction do not overflow the numbers
    *   (Signature validation is trusted)
    *   Nothing else updates the db during `transfer`'s execution
    *   The second-to-last line never throws an error
*   Validating: We scour the codebase. It seems, hopefully, the numbers come from a trusted source, the function caller locks the records, the database is very reliable, etc. But we notice that if we're wrong about any of these, even once, then a user could probably create or destroy unlimited money. Simply double-clicking the transfer button might actually send 2x the money but only remove 1x!!
    *   Note, we could add more checks, try/catches, do more static analysis, etc. Instead, we'll try this:
*   Remove assumptions: Instead of being so careful in everything about the function's environment, we can eliminate most of these assumption by writing the code differently!

    transferMoneyBetter(fromId, fromSig, amount, destId):
    	if amount < 0: return
    	if not validate(fromId, fromSig): return
    	with db.transaction as tx:
    		curBal := tx.get(fromId)
    		if (curBal < amount): return
    		db.set(fromId, safeSub(curBal, amount))
    		db.set(destId, safeAdd(db.get(destId), amount))

Now the system can tolerate untrusted input, a database connection that fails, simultaneous calls to `transfer`, etc. We reduced complexity instead of increasing it.

When we're trying to make safe AI, it is so so much higher impact to remove an assumption than to add a check or balance.

The hardest parts are **f**inding and **r**emoving assumptions, so the full message is "DO yourself a FAVAR, FR!"

More examples
-------------

*   You can try to make absolutely certain that the back door always has a security guard posted, or you can remove it.
*   You can try to make absolutely sure that your password file is protected, or you can just use hashes instead.
*   Whitelisting instead of blacklisting is (typically, broadly) about picking things you want from a known set instead of  trying to think of everything you might not want from an unknown set.
*   A system that is literally, physically unable to do X requires less careful analysis than one which is somehow otherwise prevented or trained to avoid  X.

### See also

*   [Godzilla Strategies](https://www.lesswrong.com/posts/DwqgLXn5qYC7GqExF/godzilla-strategies)
*   [Security Mindset and Ordinary Paranoia](https://intelligence.org/2017/11/25/security-mindset-ordinary-paranoia/)