Courses in Discrete and Combinatorial Math and Mathematical Statistics will help you count possible quantum states and compute probabilities. They will also help you understand statistical methods used to analyze experimental data. A course in complex variables strengthens your ability to work with complex functions which are ubiquitous in physics. Since mathematics incorporates universal logic, it is relevant to more than physics. People with physics backgrounds often end up applying their analytical skills to other disciplines ranging from the world of finance to biological research.
Computer skills are also vital for a physics major.
Computers control experiments, gather data, and analyze data. Somebody has to set them up and program them. Experimental physicists often do this. Theoretical physicists also frequently use computers because consequences of the fundamental laws of physics can usually be determined only by approximate computer calculations. So physics majors should take as many computer science courses as possible.