Fix g_mindist with pbc=XY

Fixes #1189.

Change-Id: I7c8242df6f2644bebc49c13c86412f8bc1d37428