Montgomery's brother, Andrew Montgomery, was a prominent Methodist minister in Atlanta's black community. Lemuel Grant gifted the Montgomeries land on which to build a church. The church and his home were burned down in the 1864 Battle of Atlanta during Sherman's March to the Sea. Following the war, he rebuilt his house, and the Montgomery brothers purchased another plot of land on Sweet Auburn (now Auburn Avenue), and founded the Big Bethel AME Church, the oldest African-American church in Atlanta at the site. The Montgomery brothers became respected leaders in their community in the 19th century.