Update README.md

README.md

@@ -1,3 +1,86 @@
 ---
 license: apache-2.0
 ---
+![Logo](https://github.com/city96/CityClassifiers/assets/125218114/0413003a-851d-42fc-b795-eae525b7b2e5)
+
+Experimental opinionated aesthetic score models.
+
+## CityAesthetics - Anime
+
+[Training/inference code](https://github.com/city96/CityClassifiers) | [Live Demo](https://huggingface.co/spaces/city96/CityAesthetics-demo)
+
+### Design goals
+
+The goal was to create an aesthetic predictor that can work well on one specific type of image (in this case, anime) while filtering out everything else. To achieve this, the model was trained on a set of 3080 hand-scored images with multiple refinement steps, where false positives and negatives would be added to the training set with corrected scores after each test run.
+
+This model focuses on as few false positives as possible. Only having one type of media seems to help with this, as predictors that attempt to do both real life and 2D images tend to produce false positives. If one were to have a mixed dataset with both types of images, then the simplest solution would be to use two separate aesthetic score models and a classifier to pick the appropriate one to use.
+
+#### Intentional biases
+
+- Completely negative towards real life photos (ideal score of 0%)
+- Strongly Negative towards text (subtitles, memes, etc) and manga panels
+- Fairly negative towards 3D and to some extent 2.5D images
+- Negative towards western cartoons and stylized images (chibi, parody)
+
+#### Issues
+
+- Tends to filter male characters due to being underrepresented in the training set
+- Requires at least 1 subject to be present in the image - doesn't work for scenery/landscapes
+- Noticeable positive bias towards anime characters with animal ears
+- Hit-or-miss with AI generated images due to style/quality not being correlated
+
+#### Out-of-scope
+
+- This model is not meant for moderation/live filtering/etc
+- The demo code is not meant to work with large-scale datasets and is therefore only single-threaded. If you're working on something that requires an optimized version that can work on pre-computed CLIP embeddings for faster iteration, feel free to [contact me](mailto:city@eruruu.net).
+
+### Usecases
+
+The main usecase will be to provide baseline filtering on large datasets (i.e. a high pass filter). For this, the score brackets were decided as follows:
+
+- <10% - Real life photos, noise, excessive text (subtitles, memes, etc)
+- 10-20% - Manga panels, images with no subject, non-human subjects
+- 20-40% - Sketches, oekaki, rough lineart (score depends on quality)
+- 40-50% - Flat shading, TV anime screenshots, average images
+- \>50% - "High quality" images based on my personal style preferences
+
+The \>60% score range is intended to help pick out the "best" images from a dataset. One could use it to filter by score (i.e. using it as a band pass filter), but the scores above 50% are a lot more vague. Instead, I'd recommend sorting the dataset by score instead and setting a limit on the total number of images to select.
+
+Top 100 images from a subset of danbooru2021:
+
+![AesPredv17_T100C](https://github.com/city96/CityClassifiers/assets/125218114/b7d8a167-a53a-46bb-8737-6c6c2a04f50f)
+
+### Training
+
+The training script provided is initialized with the current model settings as the defaults (7e-6 LR, cosine scheduler, 100K steps).
+
+![loss](https://github.com/city96/CityClassifiers/assets/125218114/611ae144-1390-48d3-988d-59a03c4a2f26)
+
+Final dataset score distribution for v1.8:
+```
+3080 images in dataset.
+  0 -   31 |
+  1 -  162 |||||
+  2 -  533 |||||||||||||||||
+  3 -  675 |||||||||||||||||||||
+  4 -  690 ||||||||||||||||||||||
+  5 -  576 ||||||||||||||||||
+  6 -  228 |||||||
+  7 -   95 |||
+  8 -   54 |
+  9 -   29
+ 10 -    7
+raw -    0
+```
+
+Version history:
+
+- v1.0 - Initial test model with ~150 images to test viability
+- v1.1 - Initialized top 5 score brackets with ~250 hand-picked images
+- v1.2 - Manually scored ~2500 danbooru images for the main training set
+- v1.3-v1.7 - Repeatedly ran the model against various datasets, adding the false negatives/positives to the training set to try and correct for various edgecases
+- v1.8 - Added 3D and 2.5D images to the negative brackets to filter these as well
+
+### Architecture
+
+The model itself is fairly simple. It takes embeddings from a CLIP model (in this case, `openai/clip-vit-large-patch14`) and expands them to 1024 dimensions. From there, a single block with residuals is followed by a few linear layers which converge down to the final output - a single float between 0.0 and 1.0.