Yes, it is fine to delete Icon cache files to fix the issue going on with your icons. Icon cache files help to retrieve icons faster as it takes time to retrieve directly from the app. When you will restart your PC, this folder will rebuild itself.