Frankly, Caroline's arguments hold the most weight. She argues that if Lucas left her, she would be destitute, at the mercy of her brother's charity. Lucas argues that she would be poor but her reputation would be intact, whereas if the truth about him becomes known, she will be an outcast. Since Caroline lives year-round on the sparsely populated Cragmere Moor, I couldn't see that banishment from the ton would mean much to her, whereas life as Prudence's dependent would be sheer misery. Perhaps there are natural explanations for what seems to be supernatural phenomena.
Clues abound for either interpretation.
Rather than adding an interesting level of complexity to the story, the mystery about the role of the supernatural merely brings more confusion to an already perplexing narrative. Perhaps readers who can ignore My Fair Lord's logical inconsistencies may enjoy it more than I did.