# Domain Theory and the Prisoner's Dilemma: FairBot

*Wishes it were crossposted from the* [*AI Alignment Forum*](https://alignmentforum.org/)*. Contains more technical jargon than usual.*

Recall [Robust Cooperation in the Prisoner's Dilemma](https://arxiv.org/abs/1401.5577) and a hint of [domain theory](https://www.lesswrong.com/posts/4C4jha5SdReWgg7dF/a-brief-intro-to-domain-theory#The_Basics_).

In the Prisoner's Dilemma, players have the opportunity to harm the opponent for minor gain. To aid decision, players may be granted some knowledge about the matchup. In decreasing order of fidelity:

1.  Both player's source codes/both player's behavior in all possible environments. No strategy can be better against every opponent than any distinguishable strategy, since some opponent will punish one for not being the other.
2.  Both player's behavior in the current matchup. PrudentBot, who cooperates iff he knows both players will act alike[^39sk6ymdzwp], operates here.
3.  The opponent's behavior in the current matchup. FairBot, who cooperates iff he knows the opponent will cooperate, operates here.
4.  Nothing. Classical game theory says that, of the few possible strategies, defection is optimal.

This post focuses on the third case.

In domain theory, we partially order our sets by "information content". Suppose every player can cooperate, defect, or fail to have decided. The latter case helps to model algorithmic nontermination. Both cooperation and defection would have more information content than indecision.

Decision := {Cooperate, Defect, ⊥}  
Query := {"other cooperates", "both act alike", ...}  
Answer := {"I know this.", "I don't know this."}  
Knowledge := Query -> Answer  
Player := Knowledge -> Decision

Answer is ordered by knowing something having more content than not knowing it. I will order Query by implication, because then, the domain-theoretic custom of considering only monotonic functions means that Knowledge must respect implication!

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/e2b86d28e9ec75d088a455fc12d1af084ad121e912f090ac.png)

Possible queries involving only the opponent.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/b72a6f52126a5fc7ed29e552c6e9b675b1df5c7fbaab2309.png)

Possible states of knowledge about the opponent's decision. Those marked with an eye are logically omniscient - they have drawn all possible inferences. The green boundary separates where FairBot cooperates from where he doesn't.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/6c41c9681983c6a29c4941c1f43636db19122c315dff8e4f.png)

Shrinking Decision to {Cooperate, ⊥} for tractability, possible players that can't depend on knowledge about their own decision. Recall that it's custom in programmatic PD tournaments to treat indecision as defection.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/6fc4f0470b53092bc7f27641d563a2102f47f24b347e12e0.png)

Complete tournament between every pair of the above 8 players, given that each player knows what PA says about the matchup. Green is mutual cooperation, black is mutual defection, gold has left exploit up, red has up exploit left.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/9b3249346388e5156eeb4a78a2467aec55d95700c65d7514.png)

Same thing, except now up trusts PA: Up knows that if PA says a thing then that thing is true.

Shrinking Decision to {Cooperate, ⊥} restricts players to only ever start cooperating as their knowledge increases, never stop. Of course, someone who isn't going to cooperate might still eventually figure this out. That might be equivalent.

Across these 24 encounters a player might have, of the 8 players the only ones whose strategies aren't dominated are FairBot and ⊥Bot. After deleting the dominated players, FairBot dominates ⊥Bot. So in this setting where one can only know the opponent's decision and knowledge can only increase cooperation, FairBot is optimal.

This seems like a good time to pause for feedback. And can you recommend software for doing this reasoning at scale?

[^39sk6ymdzwp]: No, wait. Then it would cooperate with CooperateBot. Hmm.