psymodel: dont apply lowpass filters with a cutoff close to the nyquist

The IIR filter numerically diverges in such cases, this could easily be
fixed but would make the filter slower on some platforms

Fixes Ticket2246

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>