Before revision:  For the specific problem of ReLU regression (equivalently, agnostically learning a ReLU), we show that any statistical-query algorithm with tolerance n^{- \Theta (  \epsilon ^{-1/2 )  } must use at least 2^{n^c} \epsilon queries for some constant  c > 0, where n is the dimension and \epsilon is the accuracy parameter.
After revision:  For the specific problem of ReLU regression (equivalently, agnostically learning a ReLU), we show that any statistical-query algorithm with tolerance n^{- \Theta ( 1/ \epsilon ^{-1/2 ) } must use at least 2^{n^c} \epsilon queries for some constant c > 0, where n is the dimension and \epsilon is the accuracy parameter.