use List<Socket> instead of ArrayList and removed redundant HashSet.Contains checks

HashSet.Add does nothing if item is already in HashSet (not like
Dictionary where ArgumentException is thrown)