Avoid invalid *&ex -> ex optimizations.

This is a back-port of D-Programming-Language/dmd#3662 (commit 4da4da1), and fixes compilation of:
```d
struct Foo { char[1] bar; }
bool checkBar(Foo* foo) { return true && *foo.bar.ptr; }
```