Package: mmpretrain
OpenMMLab websiteHOTOpenMMLab platformTRY IT OUT📘 Documentation|🛠️ Installation|👀 Model Zoo|🆕 Update News|🤔 Reporting IssuesEnglish |简体中文IntroductionMMPreTrain is an open source pre-training toolbox based on PyTorch. It is a part of theOpenMMLabproject.Themainbranch works withPyTorch 1.8+.Major featuresVarious backbones and pretrained modelsRich training strategies (supervised learning, self-supervised learning, multi-modality learning etc.)Bag of training tricksLarge-scale training configsHigh efficiency and extensibilityPowerful toolkits for model analysis and experimentsVarious out-of-box inference tasks.Image ClassificationImage CaptionVisual Question AnsweringVisual GroundingRetrieval (Image-To-Image, Text-To-Image, Image-To-Text)https://github.com/open-mmlab/mmpretrain/assets/26739999/e4dcd3a2-f895-4d1b-a351-fbc74a04e904What's new🌟 v1.2.0 was released in 04/01/2023Support LLaVA 1.5.Implement of RAM with a gradio interface.🌟 v1.1.0 was released in 12/10/2023Support Mini-GPT4 training and provide a Chinese model (based on Baichuan-7B)Support zero-shot classification based on CLIP.🌟 v1.0.0 was released in 04/07/2023Support inference of moremulti-modalalgorithms, such asLLaVA,MiniGPT-4,Otter, etc.Support around10 multi-modaldatasets!AddiTPN,SparKself-supervised learning algorithms.Provide examples ofNew ConfigandDeepSpeed/FSDP with FlexibleRunner. Here are the documentation links ofNew ConfigandDeepSpeed/FSDP with FlexibleRunner.🌟 Upgrade from MMClassification to MMPreTrainIntegrated Self-supervised learning algorithms fromMMSelfSup, such asMAE,BEiT, etc.SupportRIFormer, a simple but effective vision backbone by removing token mixer.Refactor dataset pipeline visualization.SupportLeViT,XCiT,ViG,ConvNeXt-V2,EVA,RevViT,EfficientnetV2,CLIP,TinyViTandMixMIMbackbones.This release introduced a brand new and flexible training & test engine, but it's still in progress. Welcome
to try according tothe documentation.And there are some BC-breaking changes. Please checkthe migration tutorial.Please refer tochangelogfor more details and other release history.InstallationBelow are quick steps for installation:condacreate-nopen-mmlabpython=3.8pytorch==1.10.1torchvision==0.11.2cudatoolkit=11.3-cpytorch-y
condaactivateopen-mmlab
pipinstallopenmim
gitclonehttps://github.com/open-mmlab/mmpretrain.gitcdmmpretrain
miminstall-e.Please refer toinstallation documentationfor more detailed installation and dataset preparation.For multi-modality models support, please install the extra dependencies by:miminstall-e".[multimodal]"User GuidesWe provided a series of tutorials about the basic usage of MMPreTrain for new users:Learn about ConfigsPrepare DatasetInference with existing modelsTrainTestDownstream tasksFor more information, please refer toour documentation.Model zooResults and models are available in themodel zoo.OverviewSupported BackbonesSelf-supervised LearningMulti-Modality AlgorithmsOthersVGGResNetResNeXtSE-ResNetSE-ResNeXtRegNetShuffleNet V1ShuffleNet V2MobileNet V2MobileNet V3Swin-TransformerSwin-Transformer V2RepVGGVision-TransformerTransformer-in-TransformerRes2NetMLP-MixerDeiTDeiT-3ConformerT2T-ViTTwinsEfficientNetEdgeNeXtConvNeXtHRNetVANConvMixerCSPNetPoolFormerInception V3MobileOneEfficientFormerMViTHorNetMobileViTDaViTRepLKNetBEiTMixMIMEfficientNet V2RevViTConvNeXt V2ViGXCiTLeViTRIFormerGLIPViT SAMEVA02DINO V2HiViTMoCo V1 (CVPR'2020)SimCLR (ICML'2020)MoCo V2 (arXiv'2020)BYOL (NeurIPS'2020)SwAV (NeurIPS'2020)DenseCL (CVPR'2021)SimSiam (CVPR'2021)Barlow Twins (ICML'2021)MoCo V3 (ICCV'2021)BEiT (ICLR'2022)MAE (CVPR'2022)SimMIM (CVPR'2022)MaskFeat (CVPR'2022)CAE (arXiv'2022)MILAN (arXiv'2022)BEiT V2 (arXiv'2022)EVA (CVPR'2023)MixMIM (arXiv'2022)iTPN (CVPR'2023)SparK (ICLR'2023)MFF (ICCV'2023)BLIP (arxiv'2022)BLIP-2 (arxiv'2023)OFA (CoRR'2022)Flamingo (NeurIPS'2022)Chinese CLIP (arxiv'2022)MiniGPT-4 (arxiv'2023)LLaVA (arxiv'2023)Otter (arxiv'2023)Image Retrieval Task:ArcFace (CVPR'2019)Training&Test Tips:RandAugAutoAugRepeatAugSamplerTTA...ContributingWe appreciate all contributions to improve MMPreTrain.
Please refer toCONTRUBUTINGfor the contributing guideline.AcknowledgementMMPreTrain is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks.
We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and supporting their own academic research.CitationIf you find this project useful in your research, please consider cite:@misc{2023mmpretrain,title={OpenMMLab's Pre-training Toolbox and Benchmark},author={MMPreTrain Contributors},howpublished={\url{https://github.com/open-mmlab/mmpretrain}},year={2023}}LicenseThis project is released under theApache 2.0 license.Projects in OpenMMLabMMEngine: OpenMMLab foundational library for training deep learning models.MMCV: OpenMMLab foundational library for computer vision.MIM: MIM installs OpenMMLab packages.MMEval: A unified evaluation library for multiple machine learning libraries.MMPreTrain: OpenMMLab pre-training toolbox and benchmark.MMDetection: OpenMMLab detection toolbox and benchmark.MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.MMRotate: OpenMMLab rotated object detection toolbox and benchmark.MMYOLO: OpenMMLab YOLO series toolbox and benchmark.MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.MMPose: OpenMMLab pose estimation toolbox and benchmark.MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.MMSelfSup: OpenMMLab self-supervised learning toolbox and benchmark.MMRazor: OpenMMLab model compression toolbox and benchmark.MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.MMTracking: OpenMMLab video perception toolbox and benchmark.MMFlow: OpenMMLab optical flow toolbox and benchmark.MMagic: OpenMMLabAdvanced,Generative andIntelligentCreation toolbox.MMGeneration: OpenMMLab image and video generative models toolbox.MMDeploy: OpenMMLab model deployment framework.Playground: A central hub for gathering and showcasing amazing projects built upon OpenMMLab.