A process for producing an aqueous polymer dispersion of polymerized ethylenically unsaturated monomers M, where the monomers M comprise a) at least one monomer M0 which is selected from compounds of formula 1, wherein X is Cl or Br, Y is H, C 1- C 10 -alkyl, Cl, Br, OS 2 R 4, or C(=O)OR 5, R 1 is H or C 1 -C 10 -alkyl, R 2 is H or C 1 -C 10 -alkyl, R 3 is C 1- C 10 -alkyl, R 4 is C 1- C 10 -alkyl, and R 5 is C 1 -C 20 -alkyl; b) at least one monomer M1 which is selected from nonionic monoethylenically unsaturated monomers having a solubility in deionized water of at most 60 g/L at 25°C and 1 bar, and being different from monomers M0, c) at least one monomer M2 which is selected from ethylenically unsaturated monomers having a carboxylic acid group, d) optionally one or more ethylenically unsaturated monomers M3 that are different from monomers M0, M1, and M2, where the process comprises i) a first polymerization step of an aqueous radical emulsion polymerization of a first monomer composition M.a, which comprises at least 0.1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M0, at least 1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M2. ii) neutralizing at least 50% of the carboxylic acid groups derived from monomers M2 of the polymer of step i) by addition of a base, and subsequently iii) at least one further polymerization step of an aqueous radical emulsion polymerization of a second monomer composition M.b in the presence of the polymer of step ii). a) at least one monomer M0 which is selected from compounds of formula 1, wherein X is Cl or Br, Y is H, C 1- C 10 -alkyl, Cl, Br, OS 2 R 4, or C(=O)OR 5, R 1 is H or C 1 -C 10 -alkyl, R 2 is H or C 1 -C 10 -alkyl, R 3 is C 1- C 10 -alkyl, R 4 is C 1- C 10 -alkyl, and R 5 is C 1 -C 20 -alkyl; X is Cl or Br, Y is H, C 1- C 10 -alkyl, Cl, Br, OS 2 R 4, or C(=O)OR 5, R 1 is H or C 1 -C 10 -alkyl, R 2 is H or C 1 -C 10 -alkyl, R 3 is C 1- C 10 -alkyl, R 4 is C 1- C 10 -alkyl, and R 5 is C 1 -C 20 -alkyl; b) at least one monomer M1 which is selected from nonionic monoethylenically unsaturated monomers having a solubility in deionized water of at most 60 g/L at 25°C and 1 bar, and being different from monomers M0, c) at least one monomer M2 which is selected from ethylenically unsaturated monomers having a carboxylic acid group, d) optionally one or more ethylenically unsaturated monomers M3 that are different from monomers M0, M1, and M2, i) a first polymerization step of an aqueous radical emulsion polymerization of a first monomer composition M.a, which comprises at least 0.1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M0, at least 1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M2. at least 0.1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M0, at least 1% by weight, based on the total weight of the monomer composition M.a, of at least one monomer M2. ii) neutralizing at least 50% of the carboxylic acid groups derived from monomers M2 of the polymer of step i) by addition of a base, and subsequently iii) at least one further polymerization step of an aqueous radical emulsion polymerization of a second monomer composition M.b in the presence of the polymer of step ii).