To delete OneDrive, you need to close the program first. After closing the program by pressing the Close OneDrive button in the bottom right Settings, remove OneDrive through Add/Remove Programs.