[WIP] Add directory as option for createall command

Related with #1958 
JSON/YAML distinction should be updated here as well.