It served as a penal colony for four decades after 1864 .