Prevent pointer -> int casts in constexprs

These cause issues, as addresses aren't fixed at compile-time.

Fixes #18294