A method for operating a hearing device (100, 200), the method comprising: • receiving samples of a digital audio signal; • feeding received samples as a digital input signal (x(n)) to an input buffer; • processing the buffered samples (x(n)) to provide samples of a digital output signal (y(m)) such that the digital output signal (y(m)) is a sample-rate converted representation of the digital input signal (x(n)) with a predetermined target sample rate (F y); • estimating an average or smoothed input-buffer level (l(t)) indicating the number of unprocessed samples (x(n)) in the input buffer; and • controlling the processing of the buffered samples (x(n)) in a manner aiming at minimising the difference (e(k)) between the estimated average or smoothed input-buffer level (l(t)) and a target buffer level; • comparing a parameter representative of a difference between the estimated average or smoothed input buffer level and the target buffer level, or a speed of variation in the difference between the average input buffer level and the target level, to a predetermined threshold • controlling a conversion ratio between the input rate and the sample rate according to one of a first mode and a second mode, which is selected according to a result of the comparison between the parameter and the predetermined threshold, wherein: • in the first mode, the conversion ratio is controlled by performing skip and/or repeat operation in order to minimise the difference between the average input buffer level and the target buffer level, • in the second mode, the conversion ratio is controlled by changing an upsampling factor and/or downsampling factor; • the first mode is selected for controlling the conversion ratio when the parameter is below the predetermined threshold, and • the second mode is selected for controlling the conversion rate when the parameter exceeds the predetermined threshold thus indicating that the difference between the average input buffer level and the target buffer level is too large or changing too fast allow minimizing the difference by skip and/or repeat operations. • receiving samples of a digital audio signal; • feeding received samples as a digital input signal (x(n)) to an input buffer; • processing the buffered samples (x(n)) to provide samples of a digital output signal (y(m)) such that the digital output signal (y(m)) is a sample-rate converted representation of the digital input signal (x(n)) with a predetermined target sample rate (F y); • estimating an average or smoothed input-buffer level (l(t)) indicating the number of unprocessed samples (x(n)) in the input buffer; and • controlling the processing of the buffered samples (x(n)) in a manner aiming at minimising the difference (e(k)) between the estimated average or smoothed input-buffer level (l(t)) and a target buffer level; • comparing a parameter representative of a difference between the estimated average or smoothed input buffer level and the target buffer level, or a speed of variation in the difference between the average input buffer level and the target level, to a predetermined threshold • controlling a conversion ratio between the input rate and the sample rate according to one of a first mode and a second mode, which is selected according to a result of the comparison between the parameter and the predetermined threshold, wherein: • in the first mode, the conversion ratio is controlled by performing skip and/or repeat operation in order to minimise the difference between the average input buffer level and the target buffer level, • in the second mode, the conversion ratio is controlled by changing an upsampling factor and/or downsampling factor; • the first mode is selected for controlling the conversion ratio when the parameter is below the predetermined threshold, and • the second mode is selected for controlling the conversion rate when the parameter exceeds the predetermined threshold thus indicating that the difference between the average input buffer level and the target buffer level is too large or changing too fast allow minimizing the difference by skip and/or repeat operations.