Resolves #2540. Fetch addresses present in a table when starting to avoid removing entries not configured from us