Check ignore paths based on correct baton level

Fix #287 but doesn't fix #144