The hydrogen would have escaped to space rather than accumulating. Any methane and ammonia produced would have been decomposed by the Sun's UV radiation. Organic molecules would be hard to form in this neutral or mildly reducing atmosphere. The entire ocean circulates through the vents about once every 10 million years, limiting the accumulation time for organic molecules. If the atmosphere was neutral or mildly reducing, the concentration would have been about 1000 times more dilute.
Catalysts assist in bonding by actively removing water molecules, greatly speeding up reactions.
Conditions on early Earth certainly less than ideal. We are close to having things figured out, but there are still a lot of details that we don't understand. Because of copying mistakes, many versions of original molecule formed. Some variants replicated faster than others, or were more stable, and progressively crowded out less advantaged molecules. The RNA strands were not making conscious decisions to cooperate in order to survive.