You can quite easily delete completed tasks in Microsoft To Do. All you have to do is expand Completed, right-click on the task you want to delete and select Delete task. This way, your task will be deleted for good.