The building was constructed from 1833 through 1840 and is now a National Historic Landmark. Tours are available. Head a couple of blocks east and a couple north, you can visit the N.C. Executive Mansion. Tours are available and admission is free. If you head south from the Capitol, look for the Raleigh City Museum in the Historic Briggs building at 220 Fayetteville Street Mall.
Admission is free though donations are encouraged.
With all the walking you've done by now, you should be. Whether it's a lunch or an evening meal, you can't go wrong if you wander a couple of blocks east of the Fayetteville Street Mall to City Market, at the intersection of Blount and Martin streets. Prices are decent and the food excellent at both. There are several other restaurant choices nearby as well. The City Market area was home to the first farmers market in Raleigh and is now a hot spot for galleries and specialty shops as well as being the center of downtown night life.