A chemically amplified negative resist composition comprising a silicone structure-bearing polymer comprising recurring units of the general formula and having a weight average molecular weight of 3,000 to 500,000, wherein R 1 to R 4 are each independently a monovalent C 1 -C 8 hydrocarbon group, m is an integer of 1 to 100, a, b, c and d each are 0 or a positive number, with the proviso that a, b, c and d are not equal to 0 at the same time, and a+b+c+d = 1, X is a divalent organic group of the general formula: wherein Z is a divalent organic group selected from the following: n is 0 or 1, R 5 and R 6 are each independently C 1 -C 4 alkyl or alkoxy, and k is 0, 1, or 2, and Y is a divalent organic group of the general formula: wherein V is a divalent organic group selected from the following: p is 0 or 1, R 7 and R 8 are each independently C 1 -C 4 alkyl or alkoxy, and h is 0, 1, or 2, and at least one crosslinker selected from the group consisting of an amino condensate modified with formaldehyde or formaldehyde-alcohol, a phenol compound having on the average at least two methylol or alkoxymethylol groups in the molecule, and a polyhydric phenol compound having a hydroxyl group substituted by a glycidoxy group, a photoacid generator which is decomposed to generate an acid upon exposure to light of a wavelength of 190 to 500 nm, a solvent, and at least one amine compound having a fluoroalkyl group. a silicone structure-bearing polymer comprising recurring units of the general formula and having a weight average molecular weight of 3,000 to 500,000, wherein R 1 to R 4 are each independently a monovalent C 1 -C 8 hydrocarbon group, m is an integer of 1 to 100, a, b, c and d each are 0 or a positive number, with the proviso that a, b, c and d are not equal to 0 at the same time, and a+b+c+d = 1, X is a divalent organic group of the general formula: wherein Z is a divalent organic group selected from the following: n is 0 or 1, R 5 and R 6 are each independently C 1 -C 4 alkyl or alkoxy, and k is 0, 1, or 2, and Y is a divalent organic group of the general formula: wherein V is a divalent organic group selected from the following: p is 0 or 1, R 7 and R 8 are each independently C 1 -C 4 alkyl or alkoxy, and h is 0, 1, or 2, and wherein R 1 to R 4 are each independently a monovalent C 1 -C 8 hydrocarbon group, m is an integer of 1 to 100, a, b, c and d each are 0 or a positive number, with the proviso that a, b, c and d are not equal to 0 at the same time, and a+b+c+d = 1, X is a divalent organic group of the general formula: wherein Z is a divalent organic group selected from the following: n is 0 or 1, R 5 and R 6 are each independently C 1 -C 4 alkyl or alkoxy, and k is 0, 1, or 2, and Y is a divalent organic group of the general formula: wherein V is a divalent organic group selected from the following: p is 0 or 1, R 7 and R 8 are each independently C 1 -C 4 alkyl or alkoxy, and h is 0, 1, or 2, and wherein Z is a divalent organic group selected from the following: n is 0 or 1, R 5 and R 6 are each independently C 1 -C 4 alkyl or alkoxy, and k is 0, 1, or 2, and Y is a divalent organic group of the general formula: wherein V is a divalent organic group selected from the following: p is 0 or 1, R 7 and R 8 are each independently C 1 -C 4 alkyl or alkoxy, and h is 0, 1, or 2, and wherein V is a divalent organic group selected from the following: p is 0 or 1, R 7 and R 8 are each independently C 1 -C 4 alkyl or alkoxy, and h is 0, 1, or 2, and at least one crosslinker selected from the group consisting of an amino condensate modified with formaldehyde or formaldehyde-alcohol, a phenol compound having on the average at least two methylol or alkoxymethylol groups in the molecule, and a polyhydric phenol compound having a hydroxyl group substituted by a glycidoxy group, a photoacid generator which is decomposed to generate an acid upon exposure to light of a wavelength of 190 to 500 nm, a solvent, and at least one amine compound having a fluoroalkyl group.