You are copying memory like crazy with all the calls to np.ascontiguousarray and this takes a lot of time. Intelligent looping over the arrays should avoid the necessity to do np.fliplr etc. and thus avoid to use np.ascontiguousarray and thus to copy memory and finally it should speed things up significantly.