Fixed removing policies from listeners.

Fixes #1708.

Review please? @danielgtaylor