Elizabethton, Tennessee
Brownlow married Eliza O'Brien (1819–1914) during 1836 in Elizabethton, Tennessee. They had seven children: Susan, John Bell, James Patton, Mary, Fannie, Annie, and Caledonia Temple.