Add files and directories generated by the class at runtime

They're gonna be added anyway sooner or later because someone will
forget to delete them after debbugging so I might as well add them now.