Improve validation for filter sort and direction

Fixes #17841