Fix the assert regression in fileman