Fix unreferencing ByLine.file once empty