A playout delay adjustment method comprising: adjusting a playout delay surplus based on a difference value between a first playout delay due to a de-jitter buffer and a second playout delay estimated based on an E-model; determining an adaptation type of a current frame according to whether a previous frame is an active frame; and when the determined adaptation type is signal-based adaptation, performing time scale modification,TSM, according to an adaptation scheme determined according to a comparison result between the first playout delay and the second playout delay and a comparison result between a target delay and the first playout delay, wherein the performing of the TSM comprises: determining a reduced search range, reduced from an original search range, for a current frame, based on a location of a matching candidate segment determined in a previous frame; searching for a first matching candidate segment within the reduced search range by estimating similarity; determining whether to search within a remaining range except for the reduced search range from the original search range; searching for a second matching candidate segment within the remaining range by estimating similarity; determining, as a final location of a matching candidate segment, a location having a higher similarity between the location of the first matching candidate segment and the location of the second matching candidate segment; and performing time scale shrinking to shrink a frame size or time scale stretching to stretch the frame size based on the final location. adjusting a playout delay surplus based on a difference value between a first playout delay due to a de-jitter buffer and a second playout delay estimated based on an E-model; determining an adaptation type of a current frame according to whether a previous frame is an active frame; and when the determined adaptation type is signal-based adaptation, performing time scale modification,TSM, according to an adaptation scheme determined according to a comparison result between the first playout delay and the second playout delay and a comparison result between a target delay and the first playout delay, wherein the performing of the TSM comprises: determining a reduced search range, reduced from an original search range, for a current frame, based on a location of a matching candidate segment determined in a previous frame; searching for a first matching candidate segment within the reduced search range by estimating similarity; determining whether to search within a remaining range except for the reduced search range from the original search range; searching for a second matching candidate segment within the remaining range by estimating similarity; determining, as a final location of a matching candidate segment, a location having a higher similarity between the location of the first matching candidate segment and the location of the second matching candidate segment; and performing time scale shrinking to shrink a frame size or time scale stretching to stretch the frame size based on the final location. determining a reduced search range, reduced from an original search range, for a current frame, based on a location of a matching candidate segment determined in a previous frame; searching for a first matching candidate segment within the reduced search range by estimating similarity; determining whether to search within a remaining range except for the reduced search range from the original search range; searching for a second matching candidate segment within the remaining range by estimating similarity; determining, as a final location of a matching candidate segment, a location having a higher similarity between the location of the first matching candidate segment and the location of the second matching candidate segment; and performing time scale shrinking to shrink a frame size or time scale stretching to stretch the frame size based on the final location.