Cast instead of #pragma clang diagnostic ignored "-Wassign-enum"

Removes some `#pragma clang diagnostic ignored` pollution in the master branch.