[refactor]Remove DirectoryInfo related code

Signed-off-by: Ethan Shan <shan0xiao0xi@gmail.com>