Yes, you can use teabags. Dip them into the milk in the saucepan as you simmer the milk, and then press on them to drain them thoroughly as you remove them.