Try to use git rm --cached  filename