Before revision:  We show (i) a matching lower bound of (nm)^{1-o(1)} for L_1 and L _\inf assuming the Orthogonal Vectors Hypothesis and (ii) a matching lower bound of n^{2-o(1)} for L_2 in the imbalanced case of m = O(1) assuming the 3SUM Hypothesis.
After revision:  We show (i) a matching lower bound of (nm)^{1-o(1)} for L_1 and L _\infty assuming the Orthogonal Vectors Hypothesis and (ii) a matching lower bound of n^{2-o(1)} for L_2 in the imbalanced case of m = O(1) assuming the 3SUM Hypothesis.