Apparatus for refinement of a lower-pel resolution motion estimation vector, comprising a higher-pel resolution vector ranker configured to determine a context out of a set of context candidates based on coding costs of a set of lower-pel resolution vectors within a neighborhood of the lower-pel resolution motion estimation vector, each context candidate being associated with a ranking among a set of higher-pel resolution candidate vectors surrounding the lower-pel resolution motion estimation vector; and refiner configured to refine the lower-pel resolution motion estimation vector so as to obtain a higher-pel resolution motion estimation vector by performing a sequential - according to the ranking associated with the context determined - testing of the set of higher-pel resolution candidate vectors whether a current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector, selecting the current higher-pel resolution candidate vector as the higher-pel resolution motion estimation vector if the current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector; and aborting the sequential testing according to a predetermined abort criterion, wherein the higher-pel resolution vector ranker is configured to perform the determination of the context by applying a vector-valued function onto the coding costs of the set of lower-pel resolution vectors to obtain a vector comprising a vector component per context candidate, wherein, for each context candidate, the respective vector component is a measure for a locally weighted average of the coding cost of the set of lower-pel resolution vectors, wherein a weighting centroid of the locally weighted average is spatially mutually separated for the vector components of the vector, and determining the minimum vector component of the vector so that the context candidate associated with the minimum vector component becomes the context. a higher-pel resolution vector ranker configured to determine a context out of a set of context candidates based on coding costs of a set of lower-pel resolution vectors within a neighborhood of the lower-pel resolution motion estimation vector, each context candidate being associated with a ranking among a set of higher-pel resolution candidate vectors surrounding the lower-pel resolution motion estimation vector; and refiner configured to refine the lower-pel resolution motion estimation vector so as to obtain a higher-pel resolution motion estimation vector by performing a sequential - according to the ranking associated with the context determined - testing of the set of higher-pel resolution candidate vectors whether a current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector, selecting the current higher-pel resolution candidate vector as the higher-pel resolution motion estimation vector if the current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector; and aborting the sequential testing according to a predetermined abort criterion, performing a sequential - according to the ranking associated with the context determined - testing of the set of higher-pel resolution candidate vectors whether a current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector, selecting the current higher-pel resolution candidate vector as the higher-pel resolution motion estimation vector if the current higher-pel resolution candidate vector has a lower coding cost associated therewith than a coding cost of the lower-pel resolution motion estimation vector and a coding cost of any preceding higher-pel resolution candidate vector; and aborting the sequential testing according to a predetermined abort criterion, wherein the higher-pel resolution vector ranker is configured to perform the determination of the context by applying a vector-valued function onto the coding costs of the set of lower-pel resolution vectors to obtain a vector comprising a vector component per context candidate, wherein, for each context candidate, the respective vector component is a measure for a locally weighted average of the coding cost of the set of lower-pel resolution vectors, wherein a weighting centroid of the locally weighted average is spatially mutually separated for the vector components of the vector, and determining the minimum vector component of the vector so that the context candidate associated with the minimum vector component becomes the context.