Gracefully handle filter keys that we don't know about

/cc @hampelm