Fanny Nelson fell ill in 1805, the year of her husband's death at Trafalgar, but recovered. From then on she was in indifferent health. She moved to Paris for a time to live with her son, where her eldest grandchild, also named Fanny, recalled her good nature and her devotion to her husband's memory. She would often kiss a miniature of him, once telling the younger Fanny 'When you are older little Fan, you may know what it is to have a broken heart.' Fanny Nelson returned to England and settled at Exmouth. She died at Harley Street on 4 May 1831 and was buried beside her son in the churchyard of St Margaret and St Andrew, Littleham-cum-Exmouth, Devon.