Package: zoobot
ZoobotZoobot classifies galaxy morphology with deep learning.Zoobot is trained using millions of answers by Galaxy Zoo volunteers. This code will let youretrainZoobot to accurately solve your own prediction task.InstallQuickstartWorked ExamplesPretrained WeightsDatasetsDocumentation(for understanding/reference)InstallationYou can retrain Zoobot in the cloud with a free GPU using thisGoogle Colab notebook. To install locally, keep reading.Download the code using git:git clone git@github.com:mwalmsley/zoobot.gitAnd then pick one of the three commands below to install Zoobot and either PyTorch (recommended) or TensorFlow:# Zoobot with PyTorch and a GPU. Requires CUDA 11.3.
pip install -e "zoobot[pytorch_cu113]" --extra-index-url https://download.pytorch.org/whl/cu113

# OR Zoobot with PyTorch and no GPU
pip install -e "zoobot[pytorch_cpu]" --extra-index-url https://download.pytorch.org/whl/cpu

# OR Zoobot with PyTorch on Mac with M1 chip
pip install -e "zoobot[pytorch_m1]"

# OR Zoobot with TensorFlow. Works with and without a GPU, but if you have a GPU, you need CUDA 11.2. 
pip install -e "zoobot[tensorflow]This installs the downloaded Zoobot code using pipeditable modeso you can easily change the code locally. Zoobot is also available directly from pip (pip install zoobot[option]). Only use this if you are sure you won't be making changes to Zoobot itself. For Google Colab, usepip install zoobot[pytorch_colab]To use a GPU, you mustalreadyhave CUDA installed and matching the versions above.
I share my install stepshere. GPUs are optional - Zoobot will run retrain fine on CPU, just slower.QuickstartTheColab notebookis the quickest way to get started. Alternatively, the minimal example below illustrates how Zoobot works.Let's say you want to find ringed galaxies and you have a small labelled dataset of 500 ringed or not-ringed galaxies. You can retrain Zoobot to find rings like so:importpandasaspdfromgalaxy_datasets.pytorch.galaxy_datamoduleimportGalaxyDataModulefromzoobot.pytorch.trainingimportfinetune# csv with 'ring' column (0 or 1) and 'file_loc' column (path to image)labelled_df=pd.read_csv('/your/path/some_labelled_galaxies.csv')datamodule=GalaxyDataModule(label_cols=['ring'],catalog=labelled_df,batch_size=32)# load trained Zoobot modelmodel=finetune.FinetuneableZoobotClassifier(checkpoint_loc,num_classes=2)# retrain to find ringstrainer=finetune.get_trainer(save_dir)trainer.fit(model,datamodule)Then you can make predict if new galaxies have rings:fromzoobot.pytorch.predictionsimportpredict_on_catalog# csv with 'file_loc' column (path to image). Zoobot will predict the labels.unlabelled_df=pd.read_csv('/your/path/some_unlabelled_galaxies.csv')predict_on_catalog.predict(unlabelled_df,model,label_cols=['ring'],# only used forsave_loc='/your/path/finetuned_predictions.csv')Zoobot includes many guides and working examples - see theGetting Startedsection below.Getting StartedI suggest starting with theColab notebookor the worked examples below, which you can copy and adapt.For context and explanation, see thedocumentation.For pretrained model weights, precalculated representations, catalogues, and so forth, see thedata notesin particular.Worked ExamplesPyTorch (recommended):pytorch/examples/finetuning/finetune_binary_classification.pypytorch/examples/finetuning/finetune_counts_full_tree.pypytorch/examples/representations/get_representations.pypytorch/examples/train_model_on_catalog.py(only necessary to train from scratch)TensorFlow:tensorflow/examples/train_model_on_catalog.py(only necessary to train from scratch)tensorflow/examples/make_predictions.pytensorflow/examples/finetune_minimal.pytensorflow/examples/finetune_advanced.pyThere is more explanation and an API reference on thedocs.I alsoincludethe scripts used to create and benchmark our pretrained models. Many pretrained models are availablealready, but if you need one trained on e.g. different input image sizes or with a specific architecture, I can probably make it for you.When trained with a decision tree head (ZoobotTree, FinetuneableZoobotTree), Zoobot can learn from volunteer labels of varying confidence and predict posteriors for what the typical volunteer might say. Specifically, this Zoobot mode predicts the parameters for distributions, not simple class labels! For a demonstration of how to interpret these predictions, see thegz_decals_data_release_analysis_demo.ipynb.(Optional) Install PyTorch or TensorFlow, with CUDAIf you're not using a GPU, skip this step. Use the pytorch_cpu or tensorflow_cpu options in the section below.Install PyTorch 1.12.1 or Tensorflow 2.10.0 and compatible CUDA drivers. I highly recommend usingcondato do this. Conda will handle both creating a new virtual environment (conda create) and installing CUDA (cudatoolkit,cudnn)CUDA 11.3 for PyTorch:conda create --name zoobot38_torch python==3.8
conda activate zoobot38_torch
conda install -c conda-forge cudatoolkit=11.3CUDA 11.2 and CUDNN 8.1 for TensorFlow 2.10.0:conda create --name zoobot38_tf python==3.8
conda activate zoobot38_tf
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/  # add this environment variableLatest minor features (v1.0.4)Now supports multi-class finetuning. Seepytorch/examples/finetuning/finetune_multiclass_classification.pyRemovedsimplejpegdependency due to M1 install issue.Pinnedtimmversion to ensure MaX-ViT models load correctly. Models supporting the latesttimmwill follow.(internal until published) GZ Evo v2 now includes Cosmic Dawn (HSC). Significant performance improvement on HSC finetuning.Latest major features (v1.0.0)v1.0.0 recognises that most of the complexity in this repo is training Zoobot from scratch, but most non-GZ users will probably simply want to load the pretrained Zoobot and finetune it on their data.Adds new finetuning interface (finetune.run_finetuning()), examples.Refocuses docs on finetuning rather than training from scratch.Rework installation process to separate CUDA from Zoobot (simpler, easier)Better wandb logging throughout, to monitor trainingRemove need to make TFRecords. Now TF directly uses images.Refactor out augmentations and datasets togalaxy-datasetsrepo. TF and Torch now use identical augmentations (via albumentations).Many small quality-of-life improvementsContributions are very welcome and will be credited in any future work. Please get in touch! SeeCONTRIBUTING.mdfor more.Benchmarks and Replication - Training from ScratchThebenchmarksfolder contains slurm and Python scripts to train Zoobot from scratch. We use these scripts to make sure new code versions work well, and that TensorFlow and PyTorch achieve similar performance.Training Zoobot using the GZ DECaLS dataset option will create models very similar to those used for the GZ DECaLS catalogue and shared with the early versions of this repo. The GZ DESI Zoobot model is trained on additional data (GZD-1, GZD-2), as the GZ Evo Zoobot model (GZD-1/2/5, Hubble, Candels, GZ2).CitingIf you use this software, or otherwise wish to cite Zoobot as a software package, please use theJOSS paper:@article{Walmsley2023, doi = {10.21105/joss.05312}, url = {https://doi.org/10.21105/joss.05312}, year = {2023}, publisher = {The Open Journal}, volume = {8}, number = {85}, pages = {5312}, author = {Mike Walmsley and Campbell Allen and Ben Aussel and Micah Bowles and Kasia Gregorowicz and Inigo Val Slijepcevic and Chris J. Lintott and Anna M. m. Scaife and Maja Jabłońska and Kosio Karchev and Denise Lanzieri and Devina Mohan and David O’Ryan and Bharath Saiguhan and Crisel Suárez and Nicolás Guerra-Varas and Renuka Velu}, title = {Zoobot: Adaptable Deep Learning Models for Galaxy Morphology}, journal = {Journal of Open Source Software} }You might be interested in reading papers using Zoobot:Galaxy Zoo DECaLS: Detailed visual morphology measurements from volunteers and deep learning for 314,000 galaxies(2022)A Comparison of Deep Learning Architectures for Optical Galaxy Morphology Classification(2022)Practical Galaxy Morphology Tools from Deep Supervised Representation Learning(2022)Towards Foundation Models for Galaxy Morphology(2022)Harnessing the Hubble Space Telescope Archives: A Catalogue of 21,926 Interacting Galaxies(2023)Astronomaly at Scale: Searching for Anomalies Amongst 4 Million Galaxies(2023)Galaxy Zoo DESI: Detailed morphology measurements for 8.7M galaxies in the DESI Legacy Imaging Surveys(2023)Galaxy mergers in Subaru HSC-SSP: A deep representation learning approach for identification, and the role of environment on merger incidence(2023)Many other works use Zoobot indirectly via theGalaxy Zoo DECaLScatalog (and now via the newGalaxy Zoo DESIcatalog).