A chemically amplified negative resist composition comprising a resist polymer having a leaving group and an acid generator, the resist polymer becoming insoluble in alkaline developer as a result of the leaving group undergoing an elimination reaction under the catalysis of an acid generated by the acid generator upon exposure to high-energy radiation, the resist polymer comprising recurring units having formula, recurring units of at least one type selected from formulae and, and recurring units of at least one type selected from formulae and, wherein at least one of the recurring units other than the recurring units having formula has at least one phenolic hydroxyl group and/or fluoroalcohol group, and the total of the recurring units other than the recurring units having formula accounts for 25 to 95 mol% based on the overall recurring units of the polymer, wherein A, B, and C each are a single bond, or a C 1 -C 10 alkylene group which may contain an ethereal oxygen atom at an intermediate of the chain, D is a single bond or a divalent, straight, branched or cyclic, aliphatic C 1 -C 10 hydrocarbon group which may be substituted with fluorine and which may contain an ethereal oxygen atom, carbonyl group or carbonyloxy group at an intermediate of the chain, R 1 is hydrogen, fluorine, methyl, or trifluoromethyl, R 2, R 3, R 4, R 5, and R 6 are each independently hydrogen, halogen, an optionally halo-substituted, straight, branched or cyclic C 2 -C 8 acyloxy group, an optionally halo-substituted, straight, branched or cyclic C 1 -C 6 alkyl group, or an optionally halo-substituted, straight, branched or cyclic C 1 -C 6 alkoxy group, L is hydrogen, a monovalent, straight, branched or cyclic, aliphatic C 1 -C 10 hydrocarbon group which may contain an ethereal oxygen atom, carbonyl group or carbonyloxy group at an intermediate of the chain, or an optionally substituted monovalent aromatic group, Rx and Ry each are hydrogen or a C 1 -C 15 alkyl group which may be substituted with hydroxy or alkoxy, Rx and Ry may bond together to form a ring with the carbon atom to which they are attached, excluding the case where Rx and Ry are hydrogen at the same time, Rf 1 and Rf 2 each are a C 1 -C 6 alkyl group having at least one fluorine atom, Rf 2 may bond with D to form a ring with the carbon atom to which they are attached, f is an integer of 1 to 3, g is an integer of 0 to 3, h is 1 or 2, i and j each are an integer of 0 to 3, r is 0 or 1, v is 1 or 2, s, t and u each are an integer of 0 to 2, a is an integer (5+2s-f), b is an integer (5+2t-g), c is an integer (5+2u-h), d is an integer of 0 to 5, e is an integer of 0 to 3, m, n and p are each independently 0 or 1, with the proviso that p is 1 when r is 0. wherein A, B, and C each are a single bond, or a C 1 -C 10 alkylene group which may contain an ethereal oxygen atom at an intermediate of the chain, D is a single bond or a divalent, straight, branched or cyclic, aliphatic C 1 -C 10 hydrocarbon group which may be substituted with fluorine and which may contain an ethereal oxygen atom, carbonyl group or carbonyloxy group at an intermediate of the chain, R 1 is hydrogen, fluorine, methyl, or trifluoromethyl, R 2, R 3, R 4, R 5, and R 6 are each independently hydrogen, halogen, an optionally halo-substituted, straight, branched or cyclic C 2 -C 8 acyloxy group, an optionally halo-substituted, straight, branched or cyclic C 1 -C 6 alkyl group, or an optionally halo-substituted, straight, branched or cyclic C 1 -C 6 alkoxy group, L is hydrogen, a monovalent, straight, branched or cyclic, aliphatic C 1 -C 10 hydrocarbon group which may contain an ethereal oxygen atom, carbonyl group or carbonyloxy group at an intermediate of the chain, or an optionally substituted monovalent aromatic group, Rx and Ry each are hydrogen or a C 1 -C 15 alkyl group which may be substituted with hydroxy or alkoxy, Rx and Ry may bond together to form a ring with the carbon atom to which they are attached, excluding the case where Rx and Ry are hydrogen at the same time, Rf 1 and Rf 2 each are a C 1 -C 6 alkyl group having at least one fluorine atom, Rf 2 may bond with D to form a ring with the carbon atom to which they are attached, f is an integer of 1 to 3, g is an integer of 0 to 3, h is 1 or 2, i and j each are an integer of 0 to 3, r is 0 or 1, v is 1 or 2, s, t and u each are an integer of 0 to 2, a is an integer (5+2s-f), b is an integer (5+2t-g), c is an integer (5+2u-h), d is an integer of 0 to 5, e is an integer of 0 to 3, m, n and p are each independently 0 or 1, with the proviso that p is 1 when r is 0.