Reapply fix for issue 2943, and fix the failing test

It failed because the bundle result was cached, but this didn't happen if the file already existed.