Revert "build: Add more cflags to the Makefile"

This reverts commit 4c6e7e086a3541f8d7db77cd6023856b782b33d5.