Backporting the integration test form v1.5 to v1.4

Back port of #2150 to v1.4, so we have integration testing for all version.