Revert "Limit number of known addresses per peer"

This reverts commit 6443711c4a384ed5bd54191d8f35befbf42339a9.