A rolling bearing unit with a rotating speed detector, comprising: an outer ring which has double row outer ring raceways (4a, 4b) on an inner peripheral surface and is not rotated in use, a hub which has double row inner ring raceways (8a, 8b) on an outer peripheral surface and is supported in a inner side of the outer ring concentrically with the outer ring and provides a portion projected to an axially outboard side beyond an axially outboard end of the outer ring in the outer peripheral surface with a rotation side flange for supporting a wheel, plural rolling elements rollably formed between both of the outer ring raceways and both of the inner ring raceways every both rows, respectively, an annular encoder formed by changing magnetic characteristics of an axially inboard surface alternately with respect to a circumferential direction, the encoder supported in an axially inboard end of the hub concentrically with the hub, a synthetic resin-made cap which is attached to an axially inboard end of the outer ring and closes an opening of the axially inboard end of the outer ring, a bottomed cylindrical sensor insertion case made of a non-magnetic material, the sensor insertion case internally fitted to an inside of a through hole formed in a portion axially opposed to the encoder in the cap, and a sensor unit comprising a sensor which is opposed to a detected surface of the encoder through a bottom plate part of the sensor insertion case and changes an output signal according to a change in characteristics of the detected surface of the encoder, and a sensor holder having an insertion part which holds the sensor and is inserted into an inside of the sensor insertion case, and a mounting flange part which is formed on a proximal end of the insertion part and is coupled and fixed to an axially inboard surface of the cap, characterized in that the sensor insertion case is internally fitted to the inside of the through hole by being axially inserted into the inside of the through hole previously formed in the cap, and an annular elastic member is pinched between an outer peripheral surface of the sensor insertion case and an axially inboard end side portion in an inner peripheral surface of the through hole in a state having a radial interference. an outer ring which has double row outer ring raceways (4a, 4b) on an inner peripheral surface and is not rotated in use, a hub which has double row inner ring raceways (8a, 8b) on an outer peripheral surface and is supported in a inner side of the outer ring concentrically with the outer ring and provides a portion projected to an axially outboard side beyond an axially outboard end of the outer ring in the outer peripheral surface with a rotation side flange for supporting a wheel, plural rolling elements rollably formed between both of the outer ring raceways and both of the inner ring raceways every both rows, respectively, an annular encoder formed by changing magnetic characteristics of an axially inboard surface alternately with respect to a circumferential direction, the encoder supported in an axially inboard end of the hub concentrically with the hub, a synthetic resin-made cap which is attached to an axially inboard end of the outer ring and closes an opening of the axially inboard end of the outer ring, a bottomed cylindrical sensor insertion case made of a non-magnetic material, the sensor insertion case internally fitted to an inside of a through hole formed in a portion axially opposed to the encoder in the cap, and a sensor unit comprising a sensor which is opposed to a detected surface of the encoder through a bottom plate part of the sensor insertion case and changes an output signal according to a change in characteristics of the detected surface of the encoder, and a sensor holder having an insertion part which holds the sensor and is inserted into an inside of the sensor insertion case, and a mounting flange part which is formed on a proximal end of the insertion part and is coupled and fixed to an axially inboard surface of the cap, characterized in that the sensor insertion case is internally fitted to the inside of the through hole by being axially inserted into the inside of the through hole previously formed in the cap, and an annular elastic member is pinched between an outer peripheral surface of the sensor insertion case and an axially inboard end side portion in an inner peripheral surface of the through hole in a state having a radial interference.