The largest white oak in the US used to stand inside the wrought iron fence, but the tree came down in 2002