A similar situation is a ball hanging on a string. There is the force mg down and tension T up. They are equal and opposite if there is no acceleration. Now whirl it around in a circle. No new force acts, but T adjusts itself so that the net force is in the direction of acceleration and is equal to ma.
T is very smart.
If feels funny because you are so accustomed to the normal force in everyday life being equal and opposite to mg. Note that i did not mention centripetal or centrifugal force. These are downright misleading terms. When you get to the essense of what is going on, you will not need to mention them. People use these words in order to simplify a discussion of circular motion or to discuss it without getting into the details, but then the truth of what is really happening tends to get hidden.