Create directory for each benchmark