Using a plastic putty knife will be a good idea to remove the old caulk. But you have to make sure you're not damaging the surface.