The Haunted Palace

In 1765, the inhabitants of Arkham are suspicious of the strange phenomena surrounding the grand "palace" that overlooks the town. They suspect the palace's owner, Joseph Curwen, is a warlock.
A young girl wanders up to the palace in a trance-like state. She is led by Curwen and his mistress, Hester, down into the dungeons. The girl is subjected to a strange ritual, in which an unseen creature rises up from a covered pit. The townspeople observe the girl wandering off, and they storm the palace to confront its owner. Though the girl appears unharmed, the townspeople surmise that she has been bewitched to forget what happened to her. They drag Curwen out to a tree where they intend to burn him. The mob leader, Ezra Weeden, insists that they do not harm Hester (to whom he had been previously engaged to marry). Before dying, Curwen puts a curse on Arkham and its inhabitants and their descendants, promising to rise from the grave to take his revenge.
In 1875, 110 years later, Curwen's great-great-grandson, Charles Dexter Ward, and his wife Anne arrive in Arkham after inheriting the palace. They find the townsfolk hostile towards them and are disturbed by the horrific deformities that afflict many of Arkham's inhabitants. Charles is surprised by how well he seems to know the palace and struck by his strong resemblance to a portrait of Curwen. He and Anne meet Simon, the palace caretaker, who persuades them to stay at the palace and to forget the townspeoples' hostility. Charles becomes more and more obsessed with the portrait of Curwen, and at times seems to change in his personality.
Charles and Anne befriend the local doctor, Dr. Marinus Willet. He explains the circumstances surrounding Curwen's death, and that the townspeople blame the deformities on the curse. He tells them of a black magic book, the Necronomicon, believed to have been in Curwen's possession, and which Curwen used to summon the Elder Gods Cthulhu and Yog-Sothoth. Curwen's plan was to mate mortal women with these beings in order to create a race of super-humans, which led to the deformities. The townspeople are terrified that Curwen has come back in the form of Charles to seek his revenge. Dr. Willet advises Charles and Anne to leave the town.
Charles seems to be falling under the control of something and insists that they stay in Arkham. One night, Charles is possessed by the spirit of Joseph Curwen. Curwen reunites with two other warlocks, Simon and Jabez, who also have possessed their descendants. They make plans to continue their work and resurrect Hester. Curwen's hold on Charles is limited, and he tells Simon and Jabez that Charles is fighting him.
Curwen begins his revenge on the descendants. He kills Weeden's descendant by releasing Weeden's monstrously deformed son from his locked room and attacks Leach's descendant with fire. Curwen takes complete control of Charles and he rapes Anne. Anne seeks help from Dr. Willet. Curwen and his associates succeed in resurrecting Hester. Curwen attempts to persuade Willet that Anne is insane.
The townspeople discover Leach's charred corpse and storm the palace. Dr. Willet and Anne try to rescue Charles and discover a secret entrance to the dungeons. They are ambushed by Curwen, Simon, Jabez, and Hester. Anne is offered as a sacrifice to the creature in the pit, while the residents break in and begin to raze the palace. The portrait of Curwen is destroyed, breaking Curwen's hold over Charles. Charles releases Anne, then urges Dr. Willet to take her away from the palace. While Curwen's associates seize Charles, Dr. Willet shepherds Anne from the burning palace. He returns to rescue Charles, and finds that Simon, Jabez, and Hester have escaped and left him to die. Charles and Willet barely escape the flames. Charles and Anne fervently thank Willet for saving their lives. However, it is apparent that Joseph Curwen still inhabits Charles' mind.
The film ends with the final verse of Poe's poem: "...While, like a ghastly rapid river, through the pale door, a hideous throng rush out forever and laugh - But smile no more."

Question: How is Curwen's hold over Charles broken?
Answer: his portrait is destroyed