scripts/USING_LOCAL_MODELS.md

Using Local Pre-Cached Models

Option 1: Download Models & Commit to Git (RECOMMENDED for your setup)

This approach stores models directly in the repo, so they're always available without any network dependency.

Step 1: Download Lightweight Models

python3 scripts/download_lightweight_models.py

This downloads smaller models (~500MB total) and saves them to models/ directory.

Step 2: Commit Models to Git

cd /Users/shouryaangrish/Documents/Work/HugginFaceInfy/infy
git add models/
git commit -m "Add pre-cached models for offline use"
git push origin main

Step 3: Update App to Use Local Models

Option A - Modify your app to use local models:

# In app.py, change:
import config
# To:
from scripts.config_local import SENTIMENT_MODEL, NER_MODEL, ...

Option B - Replace config.py entirely:

cp scripts/config_local.py config.py
git add config.py
git commit -m "Switch to local model loading"
git push origin main

Step 4: Test Locally

python3 app.py

Then click buttons - models will load from models/ directory (instant, no download!)

---

Benefits of This Approach

✅ No network dependency — Models stored locally in repo
✅ Bypasses HF whitelist — Company firewall won't block
✅ Instant loading — Models already on disk
✅ Consistent deployments — Same models for everyone
✅ Reproducible — Models don't change versions
✅ Works on Spaces — If you push to Spaces, models go with it

---

What Models Are Included

Model	Size	Task
DistilBERT (Sentiment)	~260 MB	Sentiment Analysis
BERT (Tokenizer)	~440 MB	Tokenization
Total	~500-700 MB

Note: NER, QA, Summarization still download from HF (too large for repo), but can be added if needed

---

How It Works

When you load models:

# config.py checks if local models exist
if Path("models/sentiment").exists():
    SENTIMENT_MODEL = "models/sentiment/model"  # Load locally
else:
    SENTIMENT_MODEL = "distilbert-base-uncased-..."  # Download from HF

So if models are in the repo, they load instantly. If not, they download from HF as fallback.

---

Step-by-Step Setup

For Your Laptop (Quick Demo Prep)

# 1. Download lightweight models (~500MB)
python3 scripts/download_lightweight_models.py

# 2. Test locally
python3 app.py
# Click "Analyze Sentiment" - should be instant (models loaded from "models/" dir)

# 3. Ready for demo!

For Spaces Deployment

# 1. Models already in repo from above
# 2. Push to Spaces
git push origin main

# 3. Spaces auto-deploys with pre-cached models
# 🎉 Demos run instantly!

---

File Structure After Setup

infy/
├── models/                          ← Pre-downloaded models
│   ├── sentiment/
│   │   ├── model/                   ← Model files
│   │   └── tokenizer/               ← Tokenizer files
│   └── tokenizer/
│       ├── model/
│       └── tokenizer/
├── app.py                           ← Uses local models
├── config.py                        ← Loads from "models/"
├── utils.py
├── requirements.txt
└── scripts/
    ├── download_lightweight_models.py
    ├── config_local.py
    └── README.md

---

Troubleshooting

Models directory too large for git?

Git has limits on file size. If you exceed them:

# Install Git LFS (Large File Storage)
brew install git-lfs
git lfs install

# Then add models to LFS
git lfs track "models/**/*.bin"
git lfs track "models/**/*.safetensors"
git add .gitattributes models/
git commit -m "Use Git LFS for large model files"
git push origin main

Note: Repo already has .gitattributes set up for this!

"Models still downloading during demo"?

• Make sure python3 scripts/download_lightweight_models.py completed
• Check models/ directory exists: ls -la models/
• Verify config.py is using local paths
• Restart app: python3 app.py

Want offline-only (no HF fallback)?

Edit scripts/config_local.py:

# Change this (current):
NER_MODEL = "dslim/bert-base-NER"

# To this (local only):
NER_MODEL = str(MODELS_DIR / "ner" / "model")
# Then download it: python3 scripts/download_lightweight_models.py

---

Estimated File Sizes

Component	Size
DistilBERT (sentiment)	~260 MB
BERT base (tokenizer)	~440 MB
Config/tokenizer files	~5 MB
Total for 2 models	~700 MB
Git repo (with models)	~750 MB

Git can handle this fine. For many more models, use Git LFS (already configured in .gitattributes)

---

Next Steps

1. Run: python3 scripts/download_lightweight_models.py
2. Test: python3 app.py → click a button → instant loading ✅
3. Commit: git add models/ → git push origin main
4. Demo: Perfect for your session!

---

Why This Solves Your Problem

Issue	Solution
Company firewall blocks HF	✅ Models stored locally, no external download
Slow network during demo	✅ Instant loading from disk
Attendees can't download	✅ Everything in repo, cloneable
Spaces issues	✅ Models come with Spaces push
Repeatability	✅ Same models for everyone

---

Ready? Run this on your laptop now:

python3 scripts/download_lightweight_models.py

Then let me know what the size is and we can decide if we add more models! 🚀