A method implemented in an encoder, the method comprising: determining, by a processor of the encoder, control points to encode an affine block from a video signal according to affine inter-prediction; determining, by the processor, a sub-block size of a sub-block of the affine block either a) based on a motion vector difference between the control points for the affine block, a width of the affine block, and a height of the affine block or b) from a three dimensional lookup table indexed by a motion vector difference between control points for the affine block, a size of the affine block, and a motion vector precision; determining, by the processor, a motion vector for the sub-block based on the control points; selecting, by the processor, an interpolation filter for application to a reference block corresponding to the sub-block, the interpolation filter selected based on the sub-block size; determining, by the processor, that a motion vector for a current sample points to a fractional position at the reference block; applying, by the processor, the interpolation filter to the reference block to determine a reference sample at the fractional position; determining, by the processor, residual information for the sub-block based on a difference between the current sample of the sub-block and the reference sample at the reference block; encoding, by the processor, the control points and the residual information in a bitstream; and transmitting, by a transmitter of the encoder, the bitstream toward a decoder to support reconstruction of the affine block as part of a video frame; and wherein the interpolation filter is selected to include a first number of taps when the sub-block size is greater than a threshold and is selected to include a second number of taps when the sub-block size is less than or equal to the threshold, and wherein the first number of taps is larger than the second number of taps. determining, by a processor of the encoder, control points to encode an affine block from a video signal according to affine inter-prediction; determining, by the processor, a sub-block size of a sub-block of the affine block either a) based on a motion vector difference between the control points for the affine block, a width of the affine block, and a height of the affine block or b) from a three dimensional lookup table indexed by a motion vector difference between control points for the affine block, a size of the affine block, and a motion vector precision; determining, by the processor, a motion vector for the sub-block based on the control points; selecting, by the processor, an interpolation filter for application to a reference block corresponding to the sub-block, the interpolation filter selected based on the sub-block size; determining, by the processor, that a motion vector for a current sample points to a fractional position at the reference block; applying, by the processor, the interpolation filter to the reference block to determine a reference sample at the fractional position; determining, by the processor, residual information for the sub-block based on a difference between the current sample of the sub-block and the reference sample at the reference block; encoding, by the processor, the control points and the residual information in a bitstream; and transmitting, by a transmitter of the encoder, the bitstream toward a decoder to support reconstruction of the affine block as part of a video frame; and