A stereo acoustic sound signal encoding apparatus comprising: a peak tracking section that detects peaks in waveforms of a plurality of sub frames obtained by dividing a frame of a right channel signal and a left channel signal, checks on a validity of a first frame time delay of the frame of the right channel signal and the left channel signal by comparing the first frame time delay with sub-frame time delays of the plurality of sub frames, and obtains a second frame time delay on the basis of the checked result, the detected peaks being positions of a maximum absolute values of the right channel signal and the left channel signal in each of the sub frames, the sub-frame time delays being estimated using differences in the positions of the detected peaks of the right channel signal and the left channel signal in each of the sub frames, and the first frame time delay being calculated using the sub-frame time delays of the sub frames in the frame; a time alignment section that changes a phase of either the right channel signal or the left channel signal on the basis of the second frame time delay; and an encoding section (403, 404, 405) that encodes a monaural signal, a side signal and the second frame time delay, the monaural signal and the side signal being generated by down mixing the right channel signal and the left channel signal, characterized in that the peak tracking section regards the first frame time delay as invalid in a case where the number of sub frames, in each of which a difference between the first frame time delay and the sub-frame time delay is equal to or more than a predetermined value, is equal to or more than a first threshold value. a peak tracking section that detects peaks in waveforms of a plurality of sub frames obtained by dividing a frame of a right channel signal and a left channel signal, checks on a validity of a first frame time delay of the frame of the right channel signal and the left channel signal by comparing the first frame time delay with sub-frame time delays of the plurality of sub frames, and obtains a second frame time delay on the basis of the checked result, the detected peaks being positions of a maximum absolute values of the right channel signal and the left channel signal in each of the sub frames, the sub-frame time delays being estimated using differences in the positions of the detected peaks of the right channel signal and the left channel signal in each of the sub frames, and the first frame time delay being calculated using the sub-frame time delays of the sub frames in the frame; a time alignment section that changes a phase of either the right channel signal or the left channel signal on the basis of the second frame time delay; and an encoding section (403, 404, 405) that encodes a monaural signal, a side signal and the second frame time delay, the monaural signal and the side signal being generated by down mixing the right channel signal and the left channel signal, characterized in that the peak tracking section regards the first frame time delay as invalid in a case where the number of sub frames, in each of which a difference between the first frame time delay and the sub-frame time delay is equal to or more than a predetermined value, is equal to or more than a first threshold value.