Track the ignore files, ignore everything except themselves