The area takes its name from the eponymous Castle Magner, a Norman-era tower house which is located approximately east of the village itself on the eastern boundary of Castlemagner townland. Owned by a Richard Magner during the Irish Rebellion of 1641, the tower house was largely destroyed in the late 16th century. The lands surrounding Castle Magner were confiscated during the Cromwellian conquest of Ireland. The castle is largely in ruin, though the stair turret and parts of the surrounding bawn wall remain.