A family interview is always one of the best places to start. Maybe a relative remembers being told a story about grandpa's arrival that could provide you with the clue you need to start your search. Ask for family documents. Maybe someone has an old passport, or ship's ticket, or even a newspaper obituary that states when Uncle Joe arrived, and you know that Grandpa came just after him. Many State Census records also asked these questions.
City directories can help narrow down a year.
Don't over look them. Check these records for clues to your ancestor's arrival. Naturalization document are a good source for finding the passenger arrival document, as well as where your ancestor was born. The Declaration of Intention or Petition for Naturalization can provide you with the exact birthplace, and the ship, date, and port of arrival. Most of this information was provided on the documents from after 1906, but some of the earlier documents have this information and should be researched.