Attempt to improve handle ref counting