Deleted Outlook attachments that have been downloaded to the desktop folder (or any other folder) can be deleted using the same techniques you would use to recover deleted desktop files in general.