Latex paint is more flexible, whereas oil-based paint has a more brittle nature. So when you apply the oil-based paint over the latex primer, there will be binding issues, and your painting surface will have cracking or splitting. Therefore, if you want a good finish for your project, it is not advisable to cover latex primer with oil-based paint.