Close worker socket when done with it