Use one connection for each master/worker pair

It doesn't change performance. It just reduces the number of file
descriptors.