Also, timing is everything. You may have to do some experimentation to determine the proper time point for measurement. Sometimes you must come to the realization that AnnexinV is not the best apoptosis assay for your system. I find, as someone else pointed out, that adherent cells tend to give a high percent of background. Physical perturbation of the cells can cause PS flipping and false positives.
Controls are paramount.
You need to have a non-apoptotic control that is indeed AnnexinV negative, or at least dim. You also need to have a positive apoptotic control that is indeed Annexin V positive. Your actual test samples may be somewhere in between and you will have to decide how to interpret them based on your control. It is not always black and white. And finally, try to correlate your results with another assay.