Beersel Castle was rebuilt from 1357 with a circular "enceinte". It had three large and one small tower. During the 14th century, ownership of the castle passed to the Seigneurs of Wittem (Witthem). Henry III of Wittem supported Emperor Maximilian during the revolts of 1483–1492, placing him at odds with Brussels which supported the rebellion. Beersel Castle was besieged by Brussels militias twice in 1488 and 1489. During the second siege, the castle was attacked with cannons provided to the rebels by Louis XI of France and fell after several days, sustaining significant damage especially at its southern side.