The building achieved fame when it was John Brown's refuge during his 1859 raid on Harper's Ferry. It is the only surviving building of the Armory; the others were destroyed during the Civil War.