Package: glasses-detector
Glasses DetectorEyeglassesandsunglassesclassifier+glassesand theirframessegmenter. This project provides a quick way to use the pre-trained models via python script or terminal. Based on selected task, an image or a directory of images will be processed and corresponding labels or masks will be generated.Note: the project is BETA stage. Currently, only sunglasses classification models (excepthuge) and full glasses segmentation models (exceptmedium) are available.InstallationMinimum version ofPython 3.10is required. Also, you may want to installPytorchin advance for your device to enable GPU support. Note thatCUDAis backwards compatible, thus even if you have the newest version ofCUDA Toolkit,Pytorchshould work just fine.To install the package, simply run:pipinstallglasses-detectorYou can also install it from source:gitclonehttps://github.com/mantasu/glasses-detectorcdglasses-detector&&pipinstall.If you want to train your own models on the same datasets (or your custom ones), check theGitHub repository.FeaturesThere are 2 kinds of classifiers and 2 kinds of segmenters (terminology is a bit off but easier to handle with unique names):Eyeglasses classifier- identifies only transparent glasses, i.e., prescription spectacles.Sunglasses classifier- identifies only occluded glasses, i.e., sunglasses.Full glasses segmenter- segments full glasses, i.e., their frames and actual glasses (regardless of the glasses type).Glasses frames segmenter- segments glasses frames (regardless of the glasses type).Each kind has 5 different model architectures with naming conventions set fromtinytohuge.ClassificationA classifier only identifies whether a corresponding category of glasses (transparent eyeglasses or occluded sunglasses)  is present:Model typeEyeglasses classifierwearsdoesn't weardoesn't wearSunglasses classifierdoesn't wearwearsdoesn't wearAny glasses classifierwearswearsdoesn't wearThese are the performances ofeyeglassesandsunglassesmodels and their sizes. Note that the jointglassesclassifier would have an average accuracy and a combined model size of botheyeglassesandsunglassesmodels.Eyeglasses classification models (performance & weights)Model typeBCE loss $\downarrow$F1 score $\uparrow$ROC-AUC score $\uparrow$Num params $\downarrow$Model size $\downarrow$Eyeglasses classifier tinyTBATBATBATBATBAEyeglasses classifier smallTBATBATBATBATBAEyeglasses classifier mediumTBATBATBATBATBAEyeglasses classifier largeTBATBATBATBATBAEyeglasses classifier hugeTBATBATBATBATBASunglasses classification models (performance & weights)Model typeBCE loss $\downarrow$F1 score $\uparrow$ROC-AUC score $\uparrow$Num params $\downarrow$Model size $\downarrow$Sunglasses classifier tiny0.11490.91370.996727.53 k0.11 MbSunglasses classifier small0.06450.94340.9987342.82 k1.34 MbSunglasses classifier medium0.04910.96510.99921.52 M5.84 MbSunglasses classifier large0.05320.96850.99904.0 M15.45 MbSunglasses classifier hugeTBATBATBATBATBASegmentationA full-glasses segmenter generates masks of people wearing corresponding categories of glasses and their frames, whereas frames-only segmenter generates corresponding masks but only for glasses frames:Model typeFull/frames eyeglasses segmenterFull/frames sunglasses segmenterFull/frames any glasses segmenterThere is only one model group for eachfull-glassesandframes-onlysegmentationtasks. Each group is trained for botheyeglassesandsunglasses. Although you can use it as is, it is only one part of the finalfull-glassesorframes-onlysegmentationmodel - the other part is a specificclassifier, therefore, the accuracy and the model size would be a combination of the generic (base)segmenterand aclassifierof a specific glasses category.Full glasses segmentation models (performance & weights)Model typeBCE loss $\downarrow$F1 score $\uparrow$Dice score $\uparrow$Num params $\downarrow$Model size $\downarrow$Full glasses segmenter tiny0.05800.90540.9220926.07 k3.54 MbFull glasses segmenter small0.06030.89900.91313.22 M12.37 MbFull glasses segmenter mediumTBATBATBATBATBAFull glasses segmenter large0.05150.91520.927932.95 M125.89 MbFull glasses segmenter huge0.05160.91470.927258.63 M224.06 MbGlasses frames segmentation models (performance & weights)Model typeBCE loss $\downarrow$F1 score $\uparrow$Dice score $\uparrow$Num params $\downarrow$Model size $\downarrow$Glasses frames segmenter tinyTBATBATBATBATBAGlasses frames segmenter smallTBATBATBATBATBAGlasses frames segmenter mediumTBATBATBATBATBAGlasses frames segmenter largeTBATBATBATBATBAGlasses frames segmenter hugeTBATBATBATBATBAExamplesCommand LineYou can run predictions via the command line. For example, classification of a single or multiple images, can be performed viaglasses-detector-ipath/to/img--kindsunglasses-classifier# Prints 1 or 0glasses-detector-ipath/to/dir--kindsunglasses-classifier# Generates CSVRunning segmentation is similar, just change thekindargument:glasses-detector-ipath/to/img-kglasses-segmenter# Generates img_mask fileglasses-detector-ipath/to/dir-kglasses-segmenter# Generates dir with masksNote: you can also specify things like--output-path,--label-type,--size,--deviceetc. Use--glasses-detector -hfor more details or check thedocumentation page.Python ScriptYou can import the package and its models via the python script for more flexibility. Here is an example of how to classify people wearing sunglasses (will generate an output file where each line will contain the name of the image and the predicted label, e.g.,some_image.jpg,1):fromglasses_detectorimportSunglassesClassifierclassifier=SunglassesClassifier(model_type="small",pretrained=True).eval()classifier.predict(input_path="path/to/dir",output_path="path/to/output.csv",label_type="int",)Using a segmenter is similar, here is an example of using a sunglasses segmentation model:fromglasses_detectorimportFullSunglassesSegmenter# model_type can also be a tuple: (classifier size, base glasses segmenter size)segmenter=FullSunglassesSegmenter(model_type="small",pretrained=True).eval()segmenter.predict(input_path="path/to/dir",output_path="path/to/dir_masks",mask_type="img",)Note: there is much more flexibility that you can do with the given models, for instance, you can use only base segmenters without accompanying classifiers, or you can define your own prediction methods without resizing images to256x256(as what is done in the background). For more details refer to thedocumentation page, for instance at how segmenterprediction methodworks.DemoFeel free to play around with somedemo image files. For example, after installing throughpip, you can run:gitclonehttps://github.com/mantasu/glasses-detector&&cdglasses-detector/data
glasses-detector-idemo-odemo_labels.csv--kindsunglasses-classifier--labelstrReferencesThe following model architectures were used fromTorchvisionlibrary:Classifier small-ShuffleNet V2 (x0.5)based onShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignpaperClassifier medium-MobileNet V3 (small)based onSearching for MobileNetV3paperClassifier large-EfficientNet B0based onEfficientNet: Rethinking Model Scaling for Convolutional Neural NetworkspaperSegmenter small-LRASPPbased onSearching for MobileNetV3paperSegmenter medium-FCN (ResNet-50)based onFully Convolutional Networks for Semantic SegmentationpaperSegmenter large-DeepLab V3 (ResNet-101)based onRethinking Atrous Convolution for Semantic Image SegmentationpaperTiny classifiersandtiny segmentersare the custom models created by me with the aim to have as few parameters as possible while still maintaining a reasonable accuracy.Citation@misc{glasses-detector,author={Mantas Birškus},title={Glasses Detector},year={2023},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/mantasu/glasses-detector}},doi={TBA}}