Package: cmind
AboutCollective Mind (CM) is acollection of portable, extensible, technology-agnostic and ready-to-use automation recipes
with a human-friendly interface (aka CM scripts)to automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications
on any platform with any software and hardware.
They require Python 3.7+ with minimal dependencies and can run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
and any other operating system, in a cloud or inside automatically generated containers.
Furthermore, CM scripts arecontinuously extended by the communityto encode new knowledge and best practices about AI systems while keeping backward compatibility!CM scripts were originally developed based on the following requirements from theMLCommons engineers and researchersto help them automatically build, benchmark and optimize complex MLPerf benchmarks
across diverse and continuously changing models, data sets, software and hardware
from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:must work out of the box with the default options and without the need to edit some paths, environment variables and configuration files;must be non-intrusive, easy to debug and must reuse existing
user scripts and automation tools (such as cmake, make, ML workflows,
python poetry and containers) rather than substituting them;must have a very simple and human-friendly command line with a Python API and minimal dependencies;must require minimal or zero learning curve by using plain Python, native scripts, environment variables
and simple JSON/YAML descriptions instead of inventing new workflow languages;must have the same interface to run all automations natively, in a cloud or inside containers.Below you can find and try a few examples of the most-commonly usedautomation recipesthat can be chained into more complex automation workflowsusing simple JSON or YAML.Note that MLCommons CM is a collaborative engineering effort to gradually improve portability and functionality
across continuously changing models, data sets, software and hardware based on your feedback -
please check thisinstallation guide, report  encountered issueshereand contact us viapublic Discord Serverto help this community effort!CM human-friendly command linepipinstallcmind

cmpullrepomlcommons@ck

cmrunscript"python app image-classification onnx"cmr"python app image-classification onnx"cmr"download file _wget"--url=https://cKnowledge.org/ai/data/computer_mouse.jpg--verify=no--env.CM_DOWNLOAD_CHECKSUM=45ae5c940233892c2f860efdf0b66e7e
cmr"python app image-classification onnx"--input=computer_mouse.jpg
cmr"python app image-classification onnx"--input=computer_mouse.jpg--debug

cmfindscript"python app image-classification onnx"cmloadscript"python app image-classification onnx"--yaml

cmr"get python"--version_min=3.8.0--name=mlperf-experiments
cmr"install python-venv"--version_max=3.10.11--name=mlperf

cmr"get ml-model stable-diffusion"cmr"get ml-model huggingface zoo _model-stub.alpindale/Llama-2-13b-ONNX"--model_filename=FP32/LlamaV2_13B_float32.onnx--skip_cache
cmr"get dataset coco _val _2014"cmr"get dataset openimages"-j

cmshowcache
cmshowcache"get ml-model stable-diffusion"cmr"get generic-python-lib _package.onnxruntime"--version_min=1.16.0
cmr"python app image-classification onnx"--input=computer_mouse.jpg

cmr"python app image-classification onnx"--input=computer_mouse.jpg--debug

cmrmcache-f
cmr"python app image-classification onnx"--input=computer_mouse.jpg--adr.onnxruntime.version_max=1.16.0


cmr"get cuda"--version_min=12.0.0--version_max=12.3.1
cmr"python app image-classification onnx _cuda"--input=computer_mouse.jpg

cmguiscript"python app image-classification onnx"cmdockerscript"python app image-classification onnx"--input=computer_mouse.jpg
cmdockerscript"python app image-classification onnx"--input=computer_mouse.jpg-j-docker_it

cmdockerscript"get coco dataset _val _2017"--to=d:\Downloads\COCO-2017-val--store=d:\Downloads--docker_cm_repo=ctuning@mlcommons-ck

cmr"run common mlperf inference"--implementation=nvidia--model=bert-99--category=datacenter--division=closed
cmfindscript"run common mlperf inference"cmr"get generic-python-lib _package.torch"--version=2.1.2
cmr"get generic-python-lib _package.torchvision"--version=0.16.2
cmr"python app image-classification torch"--input=computer_mouse.jpg

cmrmrepomlcommons@ck
cmpullrepo--url=https://zenodo.org/records/10581696/files/cm-mlops-repo-20240129.zip

cmr"install llvm prebuilt"--version=17.0.6
cmr"app image corner-detection"cmrunexperiment--tags=tuning,experiment,batch_size--echo--batch_size={{VAR1{range(1,8)}}}cmreplayexperiment--tags=tuning,experiment,batch_size

cmr"get conda"cmpullrepoctuning@cm-reproduce-research-projects
cmr"reproduce paper micro-2023 victima _install_deps"cmr"reproduce paper micro-2023 victima _run"CM unified Python APIimportcmindoutput=cmind.access({'action':'run','automation':'script','tags':'python,app,image-classification,onnx','input':'computer_mouse.jpg'})ifoutput['return']==0:print(output)Examples of modular containers and GitHub actions with CM commandshttps://github.com/mlcommons/inference/blob/master/.github/workflows/test-bert.ymlhttps://github.com/mlcommons/ck/blob/master/cm-mlops/script/app-mlperf-inference/dockerfiles/bert-99.9/ubuntu_22.04_python_onnxruntime_cpu.DockerfileCM scriptswere successfully used tomodularize MLPerf inference benchmarksand help the community automate more than 95% of all performance and power submissions in the v3.1 round
across more than 120 system configurations (models, frameworks, hardware)
while reducing development and maintenance costs.Besides automating MLCommons projects, the community also started started using
and extendingCM scriptsto modularize, run and benchmark other software projects and make it
easier to rerun, reproduce and reuseresearch projects from published papers
at Systems and ML conferences.Please check theGetting Started Guideto understand how CM automation recipes work, how to use them to automate your own projects,
and how to implement and share new automations in your public or private projects.LicenseApache 2.0Copyright2022-2024MLCommons