[jenkins only] Add ultra-slim hack for incognito mode

jenkins PR for #12923