The kernels are written to check against src.rows in each loop iteration.  So this code should work even if the number of rows is not evenly divisible by 4.