moved the bulk of the integration tests into a header file to work around the spark IDE preprocessor.