Only call AddHandle when we know the handle is valid