If you have a gap that's wider than 1/4", you'll want to add a backer rod like this or you can also stuff in some paper towels or toilet paper to fill the gap before caulking.