Check for the open_memstream() function in configure.ac

If it's not available we need to skip some tests.

I also fixed some code to use config.h properly, rather than passing things
like -DHAS_UINT128 on the CLI to the compiler.