Remove recursive manifest directory

Moved .gitignore from manifests directory, and removed
the directory.