  The treatment and criteria for development of unstable Roche lobe overflow
(RLOF) that leads to the common envelope (CE) phase have hindered the
evolutionary predictions for decades. In particular, the formation of black
hole-black hole (BH-BH), black hole-neutron star (BH-NS), and neutron
star-neutron star (NS-NS) merging binaries depends sensitively on the CE phase
in classical isolated binary evolution model. All these mergers are now
reported as LIGO/Virgo sources or source candidates. CE is even considered by
some as a mandatory phase in the formation of BH-BH, BH-NS or NS-NS mergers in
binary evolution. At the moment, there is no full first-principles model for
development of CE. We employ the Startrack population synthesis code to test
the current advancements in studies on stability of RLOF for massive donors to
assess their effect on LIGO/Virgo source population. In particular, we allow
for more restrictive CE development criteria for massive donors. We also test a
modified condition for switching between different types of stable mass
transfer. We show how the revised CE development criteria and modified
condition to switch from thermal timescale mass transfer to nuclear timescale
mass transfer affect BH-BH, BH-NS and NS-NS local merger rate density, fraction
of unequal mass BH-BH and BH-NS mergers, and the shape of the mass distribution
of BH-BH mergers. We find that the changes in highly uncertain assumptions on
RLOF physics may significantly affect (i) local merger rate density, (ii) shape
of the mass and mass ratio distributions, and (iii) dominant evolutionary
formation (withand without CE) scenarios of LIGO/Virgo sources. Our results
demonstrate that without sufficiently strong constraints on RLOF physics, one
is not able to draw fully reliable conclusions about the population of double
compact object systems based on population synthesis studies.
