Displacement of such water molecules is only favorable when the ligand can form strong compensating hydrogen bonds.
Water molecules in hydrophobic regions of protein binding sites make only weak interactions, and the requirements for favorable displacement are less stringent.