[Integration] Fix issue when a build configuration contains a slash

Fixes #2767 

/cc @alloy @segiddins