manisharma494 commited on
Commit
5f3ff4a
Β·
verified Β·
1 Parent(s): ac2c55a

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -371
README.md DELETED
@@ -1,371 +0,0 @@
1
- ---
2
- title: Visual Search System
3
- emoji: πŸ”
4
- colorFrom: blue
5
- colorTo: green
6
- sdk: streamlit
7
- sdk_version: "1.37.0"
8
- app_file: app.py
9
- pinned: false
10
- license: mit
11
- ---
12
-
13
-
14
-
15
-
16
- # Modern Visual Search System (CLIP-based)
17
-
18
- ## Overview
19
-
20
- This project implements a modern, scalable visual search system using OpenAI's CLIP model. It provides both a REST API (FastAPI) and a web interface (Streamlit) for searching large image datasets using natural language queries and image queries, returning the top 5 most visually relevant images.
21
-
22
- ## Features
23
-
24
- - **REST API**: FastAPI backend with comprehensive endpoints
25
- - **Web Interface**: Streamlit frontend with intuitive UI
26
- - **Text Search**: Search images using natural language descriptions
27
- - **Image Search**: Find similar images by providing an image as query
28
- - **Image Upload**: Upload images to find similar images in the dataset
29
- - **Top 5 Results**: Always returns the 5 most similar images
30
- - **Fast Search**: Uses precomputed embeddings for optimal performance
31
- - **GPU Acceleration**: Automatic GPU detection and utilization
32
- - **API Documentation**: Auto-generated Swagger/OpenAPI docs
33
-
34
- ## Architecture
35
-
36
- ```
37
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
38
- β”‚ Streamlit β”‚ β”‚ FastAPI β”‚ β”‚ CLIP Model β”‚
39
- β”‚ Frontend │◄──►│ Backend │◄──►│ (Encoder) β”‚
40
- β”‚ (Port 8501) β”‚ β”‚ (Port 8000) β”‚ β”‚ β”‚
41
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
42
- β”‚
43
- β–Ό
44
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
45
- β”‚ Image β”‚
46
- β”‚ Database β”‚
47
- β”‚ (images/) β”‚
48
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
49
- ```
50
-
51
- ## Technical Details
52
-
53
- - **Image & Text Features**: CLIP (ViT-B-32, openai weights)
54
- - **Search Algorithm**: Cosine similarity in CLIP embedding space
55
- - **Embedding Dimension**: 512 (default for ViT-B-32)
56
- - **Performance**: Fast, GPU-accelerated if available
57
- - **Dependencies**: `open-clip-torch`, `torch`, `fastapi`, `streamlit`, `Pillow`, `numpy`
58
-
59
- ## Quick Start
60
-
61
- ### 1. Install Dependencies
62
-
63
- ```bash
64
- pip install -r requirements.txt
65
- ```
66
-
67
- ### 2. Prepare Your Images
68
-
69
- The application will automatically check and download images for you:
70
-
71
- ```bash
72
- # Check image status
73
- python start_app.py --check-only
74
-
75
- # Download missing images (if any)
76
- python start_app.py --download-images
77
-
78
- # Or let the app handle everything automatically
79
- python start_app.py
80
- ```
81
-
82
- **Automatic Features:**
83
- - βœ… Checks if images are already downloaded
84
- - βœ… Downloads missing images in parallel (up to 20 workers)
85
- - βœ… Skips existing images to save time
86
- - βœ… Shows progress and statistics
87
- - βœ… Handles errors gracefully
88
- - βœ… **Smart startup**: Allows application to start with sufficient images (default: 1000+)
89
- - βœ… **Flexible requirements**: Can adjust minimum image requirements
90
- - βœ… **Graceful degradation**: Works even with partial image database
91
-
92
- ### 3. Start the Application
93
-
94
- #### Option A: Start Both Services (Recommended)
95
- ```bash
96
- python start_app.py
97
- ```
98
-
99
- This will:
100
- 1. Check dependencies
101
- 2. Verify/download images automatically
102
- 3. Start both the FastAPI backend and Streamlit frontend
103
-
104
- #### Command Line Options
105
-
106
- ```bash
107
- # Basic startup (recommended)
108
- python start_app.py
109
-
110
- # Skip automatic image download
111
- python start_app.py --skip-download
112
-
113
- # Force download images even if some exist
114
- python start_app.py --download-images
115
-
116
- # Use custom number of parallel workers
117
- python start_app.py --max-workers 10
118
-
119
- # Use custom images directory
120
- python start_app.py --images-dir my_images
121
-
122
- # Set custom minimum image requirement
123
- python start_app.py --min-images 500
124
-
125
- # Only check image status, don't start services
126
- python start_app.py --check-only
127
- ```
128
-
129
- #### Option B: Start Services Separately
130
-
131
- **Start FastAPI Backend:**
132
- ```bash
133
- cd api && python main.py
134
- ```
135
-
136
- **Start Streamlit Frontend (in another terminal):**
137
- ```bash
138
- streamlit run streamlit_app.py
139
- ```
140
-
141
- #### Option C: Download Images Only
142
-
143
- ```bash
144
- # Download all images
145
- python download_images.py
146
-
147
- # Download with custom settings
148
- python download_images.py --max-workers 15 --output-dir images
149
-
150
- # Check status only
151
- python download_images.py --check-only
152
- ```
153
-
154
- ### 4. Access the Application
155
-
156
- - **Web Interface**: http://localhost:8501
157
- - **API Documentation**: http://localhost:8000/docs
158
- - **API Health Check**: http://localhost:8000/api/health
159
-
160
- ## API Endpoints
161
-
162
- ### Core Endpoints
163
-
164
- | Endpoint | Method | Description |
165
- |----------|--------|-------------|
166
- | `/` | GET | Root endpoint with API info |
167
- | `/api/health` | GET | Health check |
168
- | `/api/info` | GET | System information |
169
- | `/api/images` | GET | List available images |
170
-
171
- ### Search Endpoints
172
-
173
- | Endpoint | Method | Description |
174
- |----------|--------|-------------|
175
- | `/api/search/text` | POST | Text-based search |
176
- | `/api/search/image` | POST | Image-based search |
177
- | `/api/search/image/upload` | POST | Upload image for search |
178
-
179
- ### Example API Usage
180
-
181
- #### Text Search
182
- ```bash
183
- curl -X POST "http://localhost:8000/api/search/text" \
184
- -H "Content-Type: application/json" \
185
- -d '{"query": "red car", "top_k": 5}'
186
- ```
187
-
188
- #### Image Search
189
- ```bash
190
- curl -X POST "http://localhost:8000/api/search/image" \
191
- -H "Content-Type: application/json" \
192
- -d '{"image_path": "images/0001.jpg", "top_k": 5}'
193
- ```
194
-
195
- #### Image Upload Search
196
- ```bash
197
- curl -X POST "http://localhost:8000/api/search/image/upload" \
198
- -F "file=@your_image.jpg" \
199
- -F "top_k=5"
200
- ```
201
-
202
- ## Web Interface Features
203
-
204
- ### Text Search
205
- - Enter natural language descriptions
206
- - Examples: "a cat sitting", "red car", "person walking", "building with windows"
207
- - Returns images that best match the text description
208
-
209
- ### Image Search
210
- - Select an image from your dataset
211
- - Finds images visually similar to the selected image
212
- - Excludes the query image from results
213
-
214
- ### Image Upload Search
215
- - Upload any image file (JPG, PNG, BMP, TIFF)
216
- - Find similar images in your dataset
217
- - Real-time processing and results
218
-
219
- ### Results Display
220
- - **Ranking**: 1-5 (top 5 results)
221
- - **Filename**: Name of the image file
222
- - **Path**: Full path to the image
223
- - **Similarity Score**: Percentage indicating how well the image matches the query
224
- - **Image Preview**: Visual display of results
225
-
226
- ## Testing
227
-
228
- ### Test the API
229
- ```bash
230
- python test_api.py
231
- ```
232
-
233
- ### Test the CLI (Legacy)
234
- ```bash
235
- python test_search.py
236
- ```
237
-
238
- ### Run Demo
239
- ```bash
240
- python demo_search.py
241
- ```
242
-
243
- ## Development
244
-
245
- ### Project Structure
246
- ```
247
- visual-search-system/
248
- β”œβ”€β”€ api/
249
- β”‚ └── main.py # FastAPI backend
250
- β”œβ”€β”€ src/
251
- β”‚ └── models/
252
- β”‚ └── multimodal_encoder.py # CLIP encoder
253
- β”œβ”€β”€ images/ # Image database
254
- β”œβ”€β”€ models/
255
- β”‚ └── clip_index/ # Precomputed embeddings
256
- β”œβ”€β”€ streamlit_app.py # Streamlit frontend
257
- β”œβ”€β”€ start_app.py # Startup script
258
- β”œβ”€β”€ test_api.py # API tests
259
- β”œβ”€β”€ run_search.py # CLI interface (legacy)
260
- └── requirements.txt # Dependencies
261
- ```
262
-
263
- ### Adding New Features
264
-
265
- 1. **New API Endpoints**: Add to `api/main.py`
266
- 2. **New UI Components**: Add to `streamlit_app.py`
267
- 3. **New Search Methods**: Extend `src/models/multimodal_encoder.py`
268
-
269
- ## Performance
270
-
271
- - **Precomputed Embeddings**: Uses existing embeddings in `models/clip_index/` for faster search
272
- - **Real-time Fallback**: Falls back to real-time encoding if precomputed embeddings aren't available
273
- - **GPU Acceleration**: Automatically uses GPU if available for faster processing
274
- - **Memory Efficient**: Processes images in batches to manage memory usage
275
- - **Async Processing**: FastAPI handles concurrent requests efficiently
276
-
277
- ## Deployment
278
-
279
- ### Hugging Face Spaces (Recommended)
280
-
281
- This application is designed to work seamlessly on Hugging Face Spaces:
282
-
283
- 1. **Fork this repository** to your Hugging Face account
284
- 2. **Create a new Space** on Hugging Face Spaces
285
- 3. **Connect your forked repository** to the Space
286
- 4. **The application will automatically**:
287
- - Check for downloaded images
288
- - Download missing images in parallel
289
- - Start the Streamlit interface
290
- - Handle all dependencies automatically
291
-
292
- **Features for Hugging Face Spaces:**
293
- - βœ… Automatic image downloading with progress tracking
294
- - βœ… Optimized for Spaces environment (reduced parallel workers)
295
- - βœ… Built-in error handling and recovery
296
- - βœ… No manual setup required
297
- - βœ… Works with the provided `photos_url.csv` dataset
298
-
299
- **Space Configuration:**
300
- - **SDK**: Streamlit
301
- - **Hardware**: CPU Basic (free tier) or GPU for faster processing
302
- - **App File**: `app.py`
303
-
304
- ### Docker Deployment
305
-
306
- Create a `Dockerfile`:
307
-
308
- ```Dockerfile
309
- FROM python:3.9-slim
310
-
311
- WORKDIR /app
312
-
313
- # Install system dependencies
314
- RUN apt-get update && apt-get install -y \
315
- libgl1-mesa-glx \
316
- libglib2.0-0 \
317
- && rm -rf /var/lib/apt/lists/*
318
-
319
- # Copy requirements and install Python dependencies
320
- COPY requirements.txt .
321
- RUN pip install --no-cache-dir -r requirements.txt
322
-
323
- # Copy application code
324
- COPY . .
325
-
326
- # Expose ports
327
- EXPOSE 8000 8501
328
-
329
- # Start the application
330
- CMD ["python", "start_app.py"]
331
- ```
332
-
333
- Build and run:
334
- ```bash
335
- docker build -t visual-search .
336
- docker run -p 8000:8000 -p 8501:8501 -v $(pwd)/images:/app/images visual-search
337
- ```
338
-
339
- ### Production Deployment
340
-
341
- For production, consider:
342
- - Using a production ASGI server like Gunicorn
343
- - Setting up proper CORS configuration
344
- - Implementing authentication
345
- - Using a reverse proxy (nginx)
346
- - Setting up monitoring and logging
347
-
348
- ## Troubleshooting
349
-
350
- ### Common Issues
351
-
352
- - **API Connection Failed**: Make sure the FastAPI server is running on port 8000
353
- - **No images found**: Ensure images are in the `images/` directory with `.jpg` extension
354
- - **Model loading errors**: Check that all dependencies are installed correctly
355
- - **GPU issues**: Install appropriate CUDA version for your GPU
356
- - **Memory errors**: Reduce batch size or use CPU-only mode
357
- - **Port conflicts**: Change ports in `start_app.py` if needed
358
-
359
- ### Debug Mode
360
-
361
- Run with debug information:
362
- ```bash
363
- # API with debug logging
364
- cd api && uvicorn main:app --reload --log-level debug
365
-
366
- # Streamlit with debug
367
- streamlit run streamlit_app.py --logger.level debug
368
- ```
369
-
370
- ## License
371
- MIT