rocca

/

openai-clip-js

like 6

yes indeed, I thought it could be a cool experiment to run your converted ONNX to try it as a mobile application.
Also enjoying that you're loading it direct from our hub!! https://github.com/josephrocca/openai-clip-js/blob/7dc25c8c1ffd9b4dddb323426b4e9291eaeccaca/onnx-image-demo.html#L32-L33

Maybe we can also host your cool application https://github.com/josephrocca/clip-image-sorter on our Spaces?

@victor @radames Yep, I actually got y'all to change your headers just so I could load directly from the hub like that 😁

It wouldn't make sense to replicate the demos between huggingface spaces and github pages unless there's a way to have a single "master" copy from which both sites pull their code. It also wouldn't make sense to take down my github pages demos and just have everything on huggingface since I've linked to the github demos from many places at this point.

I wonder if there's a way for you to avoid trying to competing "head-on" with Github by just allow people to "symlink" a Github repo as a folder in a huggingface repo. Maybe git already has some sort of mechanism to do this sort of thing? That would be pretty neat! Then I'd just link the Github repo, and use that folder to as the source folder for the huggingface space, and it should work out of the box. It would potentially help solve the "chicken and egg" problem in helping people move across to huggingface while still allowing them to use Github stuff.

Also, I haven't used huggingface spaces yet, but here are two important things that would help set it apart from Github Pages in my opinion:

• Allow setting custom headers on the files - look at this service worker hack that I have to use to set COEP/COOP headers on Github Pages so that I can use Wasm threads / SharedArrayBuffers. This wouldn't be necessary if I could set headers for each file.
• Custom subdomain (like my-cool-space.rocca.huggingface.co) so that localStorage and stuff like that are isolated per app. Currently github has the equivalent of rocca.huggingface.co/my-cool-space (i.e. subdomain per account, not per space), which isn't good enough because localStorage can then conflict between apps (which can be a security issue, beyond just a cause of bugs). Actually, looking at a space now, it looks like spaces are served from hf.space/embed/rocca/my-cool-space? If so, and if localStorage/IndexedDB/etc. are enabled by the iframe, then that could be a significant security issue - private data from all apps could be exfiltrated by getting the user to simply click a link to a malicious Space.

All that said, what browser ML people like myself need most right now is for ML frameworks and tools to have web bindings/converters. It's often not that much extra work to add JS bindings to ML tools, thanks to how mature the Wasm ecosystem is becoming. For example, we've been working on getting tokenizers working in the browser, and the number of changes required to the code-base it quite small.

Anyway, thanks for your interest in this work. With WebGPU likely releasing later this year, it's a pretty exciting time for browser ML, and I hope Hugging Face takes the lead in helping push it forward! :)