If you don't prime the drywall before painting, there is a risk of stripping the paint. Having a glue-like base, a primer helps the paint attach perfectly. Also, the lack of bonding can make cleaning more difficult. It even worsens with time as in several months after that paint has been applied.