Make sure to only free allocated TOF structures