The answer to this question is complicated and simple at the same time. To give you a one-word answer, no, you do not necessarily have to take the breathalyzer test if a police officer stopped you. However, depending on your condition, you may or may not want to take the test. If you take the test, and it shows that you are intoxicated, the prosecutor could build a strong case against you, causing you to face jail time and a hefty amount as a penalty.
On the other hand, if you decide to not take the test, thinking that it is your right, you may not be doing the right thing. In the US, driving is a privilege and not a right. Thus, when you get your license, you agree to a term called implied consent. This means that in exchange for getting the privilege of driving a car, you will abide by certain principles issued by the state. These also include one for agreeing to any tests for checking the BAC level.
You might get saved from a DUI conviction if you refuse to take the test, but you will surely experience a license suspension for over 12 months. This has nothing to do with your driving, but it is an administrative fault because you refused to do something you signed on paper when you got your license.