Fix ICE when casting `&[T]` to `*const [T]`

Closes #16947 

r?