You need to remove the old caulking so that you get proper installation and seal with your new caulk.