Removal of gitlab.socket should be in stop function too. Fix #4313