fix filtering

use pointers in the IdTable, not the actual objects.  This allows
us to revoke them, etc.