Positive reinforcement during your visit is crucial. Cats with little human contact may be leery of having their backs petted because they feel vulnerable, so it's best not to rush into anything. If the cat seems too antsy to be out of the cage, keep petting her inside the cage. Rushing the cat into a situation for which she's not ready may cause a set-back. In my experience, but with exceptions, male cats are easier to socialize.
Females seem to be a bit more uptight and harder nuts to crack.
Biting, swatting and scratching are merely defensive mechanisms used to combat fear. Seek out a seasoned cat volunteer if you'd like to work with the difficult cats but are feeling a little shy about it. Try not to let the cat see your fear. Show her that you are kind and gentle, but confident about who's in control. It is important to forgive.