Yes, you can go swimming at Rocky Neck State Park. The park has a beach on Long Island Sound that is popular in summer.