When the First Men arrived in Westeros, they fought with the Children of the Forest, and a pact got signed. Later, the Others came from the North. To save humanity, the Last Hero sought out the help of the Children. After the Wall got constructed, the First Men built the Fist as a rendevous point for the men of the night's Watch.