🤝 Open to Collab

AbstractPhila PRO

AbstractPhil

19 6 26

https://civitai.com/user/AbstractPhila

AbstractEyes

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

updated a model about 3 hours ago

AbstractPhil/geolip-vit-base-x3

updated a model about 3 hours ago

AbstractPhil/geolip-vit-large-x3

repliedto their post about 4 hours ago

Direct pivot to distillation. I've accumulated enough experimental information to directly pivot my long term structure plan to distillation. This is to begin forming entire collectives of cooperative systems; differentiated expert distillation for generative behavior utilizing aleph addressed bottlenecks. With this I've also heavily begun experimenting with aleph competitions and cooperation using multiple pretrained frozen codebooks established from the SVAE system. The idea here is simple in theory; use InfoNCE and address independent experts to build a manifest of unique gated experts utilizing a multitude of distilled systems from many other models. Such as SigLIP 16B + LAION CLIPB as a pair. The experimentation in the past showed this process is potent and with that merits additional experimentation using the newly established paradigms. There are quite a bit of experiments to compare these to, so I have no shortage of comparators. After we train our baseline TinyViT with our gated system, we will know which experts are better at what and why they are better. As a direct continuation from the earlier CLIP distillation experiments I'm directly comparing InfoNCE anchoring with multiple industry standard distillations from multiple papers. First comparison is InfoNCE anchoring in comparison to raw features using CoCo and CLIP_B, which seemed like a fair experiment to train a student with. The upcoming series of experiments will provide the necessary information for how effective or ineffective this process is. https://huggingface.co/datasets/AbstractPhil/bulk-coco-features The first experiments will be based on multiple clips from the bulk-coco-features extractions. First we start with some clips, then some berts, then some smaller qwens, then some larger models, then some much much larger models. All meant to be compacted into selection mechanisms.

View all activity

Organizations

AbstractPhil 's models 206

AbstractPhila PRO

AI & ML interests

Recent Activity

Organizations

AbstractPhil 's models 206 Sort: Recently updated

AbstractPhil 's models 206