Question: is on purpose grammatically correct?
Answer: "on purpose" is grammatically correct because that's how people talk. "by accident" is grammatically correct because that's how people talk.