Do not return nil pointers when loading ignores (fixes #846)

Not sure, perhaps we should check for error, and respect that instead.
But then in the walker we'll have to check for a nil pointer anyway.