Fix attaching multiple key handles on a refresh