Reorganize how we store sockets