No, you should not paint directly on drywall. You need to sand it first.  Then, you have to ensure that you mud as well as prime it. If you leave the wall unprimed, the drywall begins to yellow within a couple of months. Then you can apply any coating of paint.