Remove and ignore the 'diffs' directory