Delete socket notifier instance when remote client is deleted

signed-off-by: Simon Busch <morphis@gravedo.de>