Why do you sum out the image rows but take the mean over image columns? Looks like you just copy-pasted from a non-convolutional layer without thinking about what it does.