Simplify the prediction mode search recursion.

No need to be so clever for something this simple.

I moved the cost initialization outside the recursing function because it
relied on the clever recursion to work. It should eventually be moved to its
own function that also initializes all the other fields just to be safe. I
didn't do that yet because I want to do it per-LCU and these functions are
still working on per-frame basis.
Modify search_tree to take a pixel coordinate instead of CU coordinate.

I need at least 4x4 resolution for NxN. It's simpler to just use pixel
coordinates.