They may appear to have similar characteristics like having 75 ohm impedance and similar sizes, there are few significant differences that could affect the quality of your audio. RCA cables are thinner than coaxial ones, due to their smaller core size which helps to mitigate interference.
Unlike coaxial cables, RCA cables do not contain enough insulation to provide a 5.1 channel digital transfer over one single line.
Yet despite the overall difference between the two types of cable, coaxial and RCA cables remain interchangeable and possess the same bandwidth regardless of what type you use.