Harlem proper stretches from river to river, beginning at 125th Street on the West Side and 96th Street on the East Side. In the '20s and '30s, great bands like the Count Basie and Duke Ellington orchestras played at the Cotton Club and Sugar Cane Club, and literary giants like Langston Hughes and James Baldwin soaked up the scene. Some houses date back to a time when the area was something of a country retreat, and represent some of the best brownstone mansions in the city. For cultural visits, there's the Morris-Jumel Mansion, the Schomburg Center, the Studio Museum, and the Apollo Theater. But your best bet is to take a guided tour.
Sights tend to be far apart, and neighborhoods change quickly.
Don't wander thoughtlessly through Harlem, especially at night. Fort Tryon Park and the Cloisters are the two big reasons to come up this way.