Package: rocks-classifier
Whats-this-rockThis project deploys a telegram bot that classifies rock images into 1
of 7 types.This package usestensorflowto accelerate deep learning experimentation.MLOps workflow likeExperiment TrackingModel ManagementHyperparameter Tuningwas all done usingWeights & BiasesAdditionally,nbdevwas used todevelop the packageproduce documentation based on a series of notebooks.CIpublishing toPyPiInspirationThe common complaint that you need massive amounts of data to do deep
learning  can be a very long way from the
truth!You very often don’t need much data at all, a lot of people are
looking for ways to share data and aggregate data, but that’s
unnecessary.They assume they need more data than they do, cause
they’re not familiar with the basics of transfer learning which is
this critical technique for needing orders of magnitudes less data.Jeremy
HowardsDocumentationDocumentation for the project has been created usingnbdev, and is available
atudaylunawat.github.io/Whats-this-rock.nbdevis a
notebook-driven development platform. Simply write notebooks with
lightweight markup and get high-quality documentation, tests, continuous
integration, and packaging for free!Once I discovered nbdev, I couldn’t help myself but redo the whole
project from scratch.It’s just makes me 10x more productive and makes the whole process
streamlined and more enjoyable.InstallationYou can directly install using pippipinstallrocks_classifierInstall - Directly from Github (latest beta version)pipinstallgit+https://github.com/udaylunawat/Whats-this-rock.gitDownload and process data%%bash
rocks_process_data--config-dirconfigs\remove_bad=True\remove_misclassified=True\remove_duplicates=True\remove_corrupted=True\remove_unsupported=True\sampling=None\train_split=0.8\Train ModelTrain model using default parameters inconfigs/config.yaml.rocks_train_model--config-dirconfigsYou can try different models and parameters by editingconfigs/config.yaml, or you can override it by passing arguments like
this:-By using Hydra it’s now much more easier to override parameters like
this:-rocks_train_model--config-dirconfigs\wandb.project=Whats-this-rock\wandb.mode=offline\wandb.use=False\dataset_id=[1,2]\epochs=30\lr=0.005\augmentation=None\monitor=val_loss\loss=categorical_crossentropy\backbone=resnet\lr_schedule=cosine_decay_restarts\lr_decay_steps=300\trainable=False\Wandb Sweeps (Hyperparameter Tuning)Editconfigs/sweep.yamlwandbsweep\--projectWhats-this-rock\--entityudaylunawat\configs/sweep.yamlThis will return a command with $sweepid, run it to start running
sweeps!wandbagentudaylunawat/Whats-this-rock/$sweepidTelegram BotYou can try the bothereon Telegram.Type/helpto get instructions in chat.Or deploy it yourselfrocks_deploy_botDemoColabGitHubDownloadRun in ColabView Source on GitHubDownload NotebookFeatures& Things I’ve Experimented withFeatureFeatureWandb- Experiment Tracking- System Tracking- Model Tracking- Hyperparameter TuningDatasets- Dataset 1- Dataset 2Augmentation- Keras-CV- MixUp- CutMix- NormalModels- ConvNextTiny- Efficientnet- Resnet101- MobileNetv1- MobileNetv2- XceptionOptimisers- Adam- Adamax- SGD- RMSPropLR Scheduler-CosineDecay-ExponentialDecay-CosineDecayRestartsRemove Images- Duplicate Images- Corrupt Images- Bad Images- MisclassifiedConfiguration Management- hydra- ml-collections- argparse-google-fireGenerators- tf.data.DataSet- ImageDataGeneratorDeployment- Heroku- RailwayEvaluation- Classification Report- Confusion MatrixGitHub Actions(CICD)- GitHub Super Linter- Deploy to Telegram- Deploy to Railway- nbdev tests CI- GitHub Pages(Documentation)Linting- Flake8- PydocstyleTelegram Bot- Greet- Info- Predict ImageFormatting- Black- yapfDocumentation- Code Description- Code comments- Source link- DoclinksBadges- Build- Issues- Lint CodebaseDockerPublishing- PyPiPlanned FeaturesFeatureFeatureDeploy- HuggingFacesBackend- FastAPICoding Style- Object OrientedFrontend- StreamlitWandB- Group Runs-WandB TablesBadges- RailwayTechnologies Used.Directory Tree├── imgs                              <- Images for skill banner, project banner and other images
│
├── configs                           <- Configuration files
│   ├── configs.yaml                  <- config for single run
│   └── sweeps.yaml                   <- confguration file for sweeps hyperparameter tuning
│
├── data
│   ├── corrupted_images              <- corrupted images will be moved to this directory
│   ├── misclassified_images          <- misclassified images will be moved to this directory
│   ├── bad_images                    <- Bad images will be moved to this directory
│   ├── duplicate_images              <- Duplicate images will be moved to this directory
│   ├── sample_images                 <- Sample images for inference
│   ├── 0_raw                         <- The original, immutable data dump.
│   ├── 1_external                    <- Data from third party sources.
│   ├── 2_interim                     <- Intermediate data that has been transformed.
│   └── 3_processed                   <- The final, canonical data sets for modeling.
│
├── notebooks                         <- Jupyter notebooks. Naming convention is a number (for ordering),
│                                        the creator's initials, and a short `-` delimited description, e.g.
│                                        1.0-jqp-initial-data-exploration`.
│
│
├── rocks_classifier                  <- Source code for use in this project.
│   │
│   ├── data                          <- Scripts to download or generate data
│   │   ├── download.py
│   │   ├── preprocess.py
│   │   └── utils.py
│   │
│   ├── callbacks                     <- functions that are executed during training at given stages of the training procedure
│   │   └── callbacks.py
│   │
│   ├── models                        <- Scripts to train models and then use trained models to make
│   │   │                                predictions
│   │   ├── evaluate.py
│   │   ├── models.py
│   │   ├── predict.py
│   │   ├── train.py
│   │   └── utils.py
│   │
│   │
│   └── visualization                 <- Scripts for visualizations
│
├── .dockerignore                     <- Docker ignore
├── .gitignore                        <- GitHub's excellent Python .gitignore customized for this project
├── LICENSE                           <- Your project's license.
├── README.md                         <- The top-level README for developers using this project.
├── CHANGELOG.md                      <- Release changes.
├── CODE_OF_CONDUCT.md                <- Code of conduct.
├── CONTRIBUTING.md                   <- Contributing Guidelines.
├── settings.ini                      <- configuration.
├── README.md                         <- The top-level README for developers using this project.
├── requirements.txt                  <- The requirements file for reproducing the analysis environment, e.g.
│                                        generated with `pip freeze > requirements.txt`
└── setup.py                          <- makes project pip installable (pip install -e .) so src can be importedBug / Feature RequestIf you find a bug (the site couldn’t handle the query and / or gave
undesired results), kindly open an issuehereby
including your search query and the expected result.If you’d like to request a new function, feel free to do so by opening
an issuehere.
Please include sample queries and their corresponding results.ContributingContributions make the open source community such an amazing place to
learn, inspire, and create.Any contributions you make aregreatly appreciated.Check out ourcontribution guidelinesfor more
information.LicenseLinkFree is licensed under the MIT License - see theLICENSEfile for details.CreditsDataset 1 - by Mahmoud
AlforawiDataset 2 - by
salmaneunusnbdev inspiration -tmabrahamSupportThis project needs a ⭐️ from you. Don’t forget to leave a star ⭐️Walt might be the one who knocksbut Hank is the one who rocks.