It's unquestionably a good idea to do so, since calcium can accumulate inside your tea kettle, altering your tea’s flavor as a result.