Video info scanner: fix incorrect skipping of directories

Continues #5332. This fixes directories incorrectly being skipped when changes only occurring more than 1 level down. Something the hash won't see.

It only (kinda) worked in the first place via the GUI because the hash was forcefully removed from the db before scanning.