The Duke and Duchess and their two daughters lived a relatively sheltered life at their London residence, 145 Piccadilly