Use setlocal rather than set to set the filetype.