Remove filter terms if they no longer exist in the data.

Fixes #424