Fix weird behavior in input buffer refill function