It was ironic that he was the prisoner and not the jailer. His prison was made of more than mere stone and metal, though. It was much stronger. He was bound perpetually. Walls are meant to hold one in, subdue them in a sense.
Chains bind, walls enclose.
But, these are mere material things. One does not need to be shackled in chains to be bound. Such bindings are even more strong than those of stone or metal. Metal and stone merely imprisoned the flesh, not the heart and soul. For one to be bound heart and soul, they were irretrievably lost.