Port Madison, Washington territory, 1854-1889