Avoid invalid *&ex -> ex optimizations.

This is a back-port of D-Programming-Language/dmd#3662
(commit 4da4da1), and fixes compilation of:
---
struct Foo { char[1] bar; }
bool checkBar(Foo* foo) { return true && *foo.bar.ptr; }
---