Like CPAP machines, not all CPAP batteries can be used when flying. The FAA provides guidelines that CPAP batteries must adhere to in order to be considered safe for air travel. Even if you do not plan to use your battery while flying, it must still be packed in your carry-on baggage rather than checked luggage.
If you have an FAA-compliant CPAP machine and plan to use it while flying, look for a CPAP battery that is also FAA-compliant. This information will be listed on manufacturer websites and online retailer product pages.