Go via uintptr_t when casting away constness, otherwise GCC will warn
when using -Wcast-qual.