Core: Mild optimization of Hole Filling

This fixes things where we simply just push back to fill a
vector. This is bad because it causes an malloc at 1, 2, 4, 8, 16, and
so on. If we can reserve or size the vector beforehand, we get quite a
speed up. Memory allocation is always slow.

This also linearize some things that were using list that should have
been vectors.

This also removes redundant structures.
Core: Draft of inpaint view that makes patches on demand