libffi: disable tests on FreeBSD as well

There are strange build failures of expect. Ikwildrpepper says the bash
eats all CPU, but the build is stuck.