Allow *-pointers in PtrTy (fixes #16781)

@huonw , r? :)