Package: mini-lightning
Mini-LightningIntroductionMini-Lightningis alightweightmachine learning training library, which is a mini version ofPytorch-Lightningwith only1k lines of code. It has the advantages offaster, more concise and more flexible.Existing features: support for DDP(multi-node and multi-gpu), Sync-BN, DP, MP(model parallelism), AMP, gradient accumulation, warmup and lr_scheduler, grad clip, tensorboard, huggingface, peft, LLM, torchmetrics, model and result saving, beautiful console log, etc.Only theminimal interfacesare exposed, keeping the features ofsimplicity, easy to read, use and extend.examplescan be found inexamples/If you have any problems or bug finding, pleaseraise issue, Thank you.InstallationCreate a virtual environment and install Python (>= 3.8)Download the latest version (>=1.12) of Torch(corresponding CUDA version) from theofficial websiteof PyTorch.Install mini-lightning# from pypipipinstallmini-lightning-U# Or download the files from the repository to local,# and go to the folder where setup.py is located, and run the following command# (Recommended) You can enjoy the latest features and functions (including bug fixes)pipinstall-e.# -e: editable modeExamplesFirst, you need to install the Mini-LightningRun the following examples### test environmentpythonexamples/test_env.py### cvpipinstall"torchvision>=0.13"pythonexamples/cv.py# cv+dp (not recommended, please use DDP)pythonexamples/cv.py# setting device_ids=[0, 1]### nlp: bert gptpipinstall"transformers>=4.25""datasets>=2.7""peft>=0.3"pythonexamples/nlp_bert_mlm.py
pythonexamples/nlp_bert_seq_cls.py
pythonexamples/nlp_gpt_lm.py
pythonexamples/nlp_gpt_seq_cls.py# sftpythonexamples/nlp_gpt_zh_sft_adapter.py
pythonexamples/nlp_gpt_zh_sft_lora.py# llm (model parallelism)#   Ref: https://modelscope.cn/models/baichuan-inc/baichuan-7B/summarypythonexamples/nlp_baichuan_sft_lora.py#   Ref: https://modelscope.cn/models/ZhipuAI/chatglm2-6b/summarypythonexamples/nlp_chatglm2_sft_lora.py### dqnpipinstall"gym>=0.26.2""pygame>=2.1.2"pythonexamples/dqn.py### ganpipinstall"torchvision>=0.13"pythonexamples/gan.py### contrastive learningpipinstall"torchvision>=0.13""scikit-learn>=1.2"pythonexamples/cl.py# cl+ddptorchrun--nproc_per_node2examples/cl_ddp.py--device_ids01### gnn# download torch_geometric#   Ref: https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.htmlpythonexamples/gnn_node.py
pythonexamples/gnn_edge.py
pythonexamples/gnn_graph.py### aepipinstall"torchvision>=0.13""scikit-learn>=1.2"pythonexamples/ae.py### vaepipinstall"torchvision>=0.13"pythonexamples/vae.py### meta learningpipinstall"torchvision>=0.13"pythonexamples/meta_learning.py########## ddp# torchrun (Recommended)#   Ref: https://pytorch.org/docs/stable/elastic/run.html# spawn#   Ref: https://pytorch.org/docs/stable/notes/ddp.html## single-node, multi-gputorchrun--nproc_per_node2examples/cv_ddp.py--device_ids01pythoncv_ddp_spawn.py# setting device_ids=[0, 1]## multi-node# default: --master_port 29500, or set master_port to prevents port conflicts.torchrun--nnodes2--node_rank0--master_addr127.0.0.1--nproc_per_node4examples/cv_ddp.py--device_ids0123torchrun--nnodes2--node_rank1--master_addrxxx.xxx.xxx.xxx--nproc_per_node4examples/cv_ddp.py--device_ids0123TODOAutomatic parameter adjustmentExamples: Audio, Meta-learning, Diffusion, Auto-regressive, Reinforcement LearningSupport multi-gpu testOutput .log file