Only create the module dir when necessary