Separate the find oldest logic
Implement the "don't move to VRAM if score too low" logic