Some of these rocks evidently contained Uranium and Thorium ores, because there are some deep deposits which show Uranium and Thorium radioactivity. That doesn't mean it is good, or that we can ignore it. But in their radioactive decay sequence, they generate other radioactive elements which have more dangerous implications for humans. Thorium 232, the most common Thorium isotope, decays directly to the isotope Radium 228. Uranium 238, the most common Uranium isotope, decays through five steps to the isotope Radium 226.
Radium is in the same chemical family as Calcium, so has similar solubility and reactions.
Since radium reacts like calcium, it will tend to go to the same places in the body as calcium, that is, to the bones and teeth. So when the Radium decays, the particles released will be in a potentially dangerous spot, with the possibility of damaging bone marrow cells. It is believed that radium pollution is related to leukemia. The half-lifes of the Radium isotopes are 1600 years for radium 226 and 5.8 years for radium 228. It is probably important for you to know the radium content of your city drinking water, in picocuries per liter.