You can. The old grout can be cut out using a church key or utility knife, and then new grout can be applied.