Vegeta can't go SSJ3 because he hasn't trained in the otherworld, unlike Goku. A Saiyan can achieve SSJ3 through intense training an average mortal body isn't capable of, and Goku only achieved SSJ3 through years of training in the otherworld. Also, SSJ3 is a very limited transformation, so Vegeta probably hasn't even tried.