Store player references for greater efficiency, make sure we clear all
references when the player quits.