A video encoder comprising: a frequency converter; a quantizer; an encoding section; an intermediate buffer; a code amount estimation section; a code amount selector; and a rate controller, wherein the frequency converter is arranged to perform a frequency conversion process on an input signal, wherein the quantizer is arranged to perform a quantization process on the result of the frequency conversion process performed by the frequency converter, wherein the encoding section is arranged to generate a compressed video bit stream by performing an encoding process on the result of the quantization process performed by the quantizer, wherein the intermediate buffer is coupled between the output terminal of the quantizer and the input terminal of the encoding section, wherein the input terminal of the code amount estimation section is coupled to the output terminal of the quantizer, wherein an estimated code amount generated from the output terminal of the code amount estimation section is supplied to the first input terminal of the code amount selector, wherein an actual code amount of the compressed video bit stream encoded by the encoding section is supplied to the second input terminal of the code amount selector, wherein a determination signal indicative of whether the amount of data stored in the intermediate buffer is larger than a predetermined threshold or smaller than a predetermined threshold is supplied to a selection control terminal of the code amount selector, wherein the output terminal of the code amount selector is coupled to the input terminal of the rate controller, wherein an output signal of the rate controller is used to adjust the quantization scale of the quantizer, wherein, if the determination signal indicates that the amount of data stored in the intermediate buffer is larger than the predetermined threshold, the estimated code amount from the code amount estimation section, which is supplied to the first input terminal of the code amount selector, is output to the output terminal of the code amount selector, and the estimated code amount is used to adjust the quantization scale of the quantizer, and wherein, if the determination signal indicates that the amount of data stored in the intermediate buffer is smaller than the predetermined threshold, the actual code amount from the encoding section, which is supplied to the second input terminal of the code amount selector, is output to the output terminal of the code amount selector, and the actual code amount is used to adjust the quantization scale of the quantizer. a frequency converter; a quantizer; an encoding section; an intermediate buffer; a code amount estimation section; a code amount selector; and a rate controller, wherein the frequency converter is arranged to perform a frequency conversion process on an input signal, wherein the quantizer is arranged to perform a quantization process on the result of the frequency conversion process performed by the frequency converter, wherein the encoding section is arranged to generate a compressed video bit stream by performing an encoding process on the result of the quantization process performed by the quantizer, wherein the intermediate buffer is coupled between the output terminal of the quantizer and the input terminal of the encoding section, wherein the input terminal of the code amount estimation section is coupled to the output terminal of the quantizer, wherein an estimated code amount generated from the output terminal of the code amount estimation section is supplied to the first input terminal of the code amount selector, wherein an actual code amount of the compressed video bit stream encoded by the encoding section is supplied to the second input terminal of the code amount selector, wherein a determination signal indicative of whether the amount of data stored in the intermediate buffer is larger than a predetermined threshold or smaller than a predetermined threshold is supplied to a selection control terminal of the code amount selector, wherein the output terminal of the code amount selector is coupled to the input terminal of the rate controller, wherein an output signal of the rate controller is used to adjust the quantization scale of the quantizer, wherein, if the determination signal indicates that the amount of data stored in the intermediate buffer is larger than the predetermined threshold, the estimated code amount from the code amount estimation section, which is supplied to the first input terminal of the code amount selector, is output to the output terminal of the code amount selector, and the estimated code amount is used to adjust the quantization scale of the quantizer, and wherein, if the determination signal indicates that the amount of data stored in the intermediate buffer is smaller than the predetermined threshold, the actual code amount from the encoding section, which is supplied to the second input terminal of the code amount selector, is output to the output terminal of the code amount selector, and the actual code amount is used to adjust the quantization scale of the quantizer.