  The following combinatorial conjecture arises naturally from recent
ergodic-theoretic work of Ackelsberg, Bergelson, and Best. Let $M_1$, $M_2$ be
$k\times k$ integer matrices, $G$ be a finite abelian group of order $N$, and
$A\subseteq G^k$ with $|A|\ge\alpha N^k$. If $M_1$, $M_2$, $M_1-M_2$, and
$M_1+M_2$ are automorphisms of $G^k$, is it true that there exists a popular
difference $d \in G^k\setminus\{0\}$ such that \[\#\{x \in G^k: x, x+M_1d,
x+M_2d, x+(M_1+M_2)d \in A\} \ge (\alpha^4-o(1))N^k.\] We show that this
conjecture is false in general, but holds for $G = \mathbb{F}_p^n$ with $p$ an
odd prime given the additional spectral condition that no pair of eigenvalues
of $M_1M_2^{-1}$ (over $\overline{\mathbb{F}}_p$) are negatives of each other.
In particular, the "rotated squares" pattern does not satisfy this eigenvalue
condition, and we give a construction of a set of positive density in
$(\mathbb{F}_5^n)^2$ for which that pattern has no nonzero popular difference.
This is in surprising contrast to three-point patterns, which we handle over
all compact abelian groups and which do not require an additional spectral
condition.
