Much of the work on the house was executed by highly skilled artisan slaves. The grounds had been the site of a notable historic event while owned by George Gaines. When Gaines was serving as the US Indian Agent, he is said to have met with the famous chief Pushmataha, of the Choctaw Nation, under an old post oak tree on what would become the Gaineswood estate. They were negotiating the terms of the treaty that would lead to the Choctaw removal to Indian Territory. The tree became known as the Pushmataha Oak.