Cache Scan.connection_pattern(). Hopefully, it will speed up theano.grad.