Saltwater and chlorine have a tendency to ruin a lot of things. The material of your bag and the duration to which it would be exposed to saltwater and chlorine determines whether your bag will be damaged or the extent of the damage. If your bag is waterproof then chances are it’s likely not going to get damaged at all and if it is, then the damage would be minimal.
Canvas bags are the ones that are most likely to get damaged by saltwater and chlorine so you should always be careful with them. If the colors of the fabric tend to bleed when in contact with water, you should definitely avoid exposing your bag to saltwater and chlorine.