Hugging Face's logo
Rajat Arya's picture
10

Rajat Arya

rajatarya
julien-c's profile picture sugatoray's profile picture paigelewis01's profile picture
·
https://rajatarya.com

AI & ML interests

None yet

Recent Activity

upvoted an article 4 days ago
From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub
reacted to jsulz's post with 👀 2 months ago
Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)
reacted to jsulz's post with 👍 2 months ago
Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)
View all activity

Organizations

Hugging Face's profile picture Xet Team's profile picture

rajatarya's activity

upvoted an article 4 days ago
view article
Article

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

• 42
reacted to jsulz's post with 👀👍🔥 2 months ago
view post
Post
1411
Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure.

To help, I built a Space jsulz/repo-info that lets you search for any repo and get back:

- Treemap of the repository, color coded by file/directory size
- Repo branches and their size
- Cumulative size of different file types (e.g., the total size of all the safetensors in the repo)

And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub
- https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes.

Some of the treemaps are pretty cool. Attached are black-forest-labs/FLUX.1-dev and for fun laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

  • 2 replies
·
upvoted 2 articles 3 months ago
view article
Article

Introducing Observers: AI Observability with Hugging Face datasets through a lightweight SDK

By davidberenstein1957 and 1 other •
• 35
replied to cfahlgren1's post 3 months ago
view reply

Also - easily reference your datasets in your research, giving conference committees greater confidence in the reproducibility of your results (ex. share the dataset, the model, the paper, and the analysis - all on the Hub).

Research needs reproducibility, use HF Hub for collaboration & dissemination:
upvoted an article 4 months ago
view article
Article

Improving Parquet Dedupe on Hugging Face Hub

• 32
upvoted 2 articles 6 months ago
view article
Article

The 5 Most Under-Rated Tools on Hugging Face

• 86
view article
Article

XetHub is joining Hugging Face!

• 82