Batteries have a positive and negative end.
Sometimes these two ends can swap over, but only with old batteries that have been treated badly - ie run down a lot. This often happens with transistor radios where you have 6 or so batteries in a line. The weakest battery in the line is being stressed much more than any other battery in the series and can suddenly switch polarity. This is why you should never mix old and new batteries. You want all the batteries to be as similar as possible so that each battery is placed under the same strain.