Clean up assume intrinsic pattern matching, no need to check that the argument is a value.

Also make it const safe and remove superfluous casting. NFC.