README.md

metadata

title: IoT Sensor Data RAG for Smart Buildings
emoji: 🏢
colorFrom: blue
colorTo: indigo
sdk: streamlit
sdk_version: 1.42.1
app_file: app.py
pinned: false

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

IoT Sensor Data RAG for Smart Buildings

🏢 Problem Statement

Create a RAG system that processes IoT sensor data, maintenance manuals, and building specifications to provide predictive maintenance insights and operational optimization.

🎯 Key Requirements

• ✅ IoT sensor data ingestion and real-time processing
• ✅ Maintenance manual and building specification integration
• ✅ Predictive maintenance algorithm implementation
• ✅ Operational efficiency optimization recommendations
• ✅ Anomaly detection and alert systems

🚀 Technical Challenges Solved

• ✅ Real-time sensor data streaming and processing
• ✅ Multi-sensor data fusion and correlation
• ✅ Predictive modeling for equipment failure
• ✅ Building system integration and compatibility
• ✅ Energy efficiency optimization algorithms

🏗️ System Architecture

Core Components

• RAG Engine: Vector database (ChromaDB) with Sentence-Transformers embeddings
• IoT Data Processor: Real-time sensor data streaming and anomaly detection
• Predictive Analytics: Equipment failure prediction and maintenance recommendations
• Document Intelligence: PDF/TXT processing with smart chunking strategies
• Web Interface: Modern Streamlit dashboard with Material design theme

Technology Stack

• Backend: Python, Streamlit, ChromaDB
• Embeddings: Sentence-Transformers (all-MiniLM-L6-v2)
• Vector Database: ChromaDB with cosine similarity
• LLM Integration: Local Transformers + OpenAI API (optional)
• Data Processing: Pandas, NumPy, Scikit-learn
• Visualization: Plotly for real-time sensor monitoring

📊 Features

1. Real-Time IoT Monitoring

• Live sensor data streaming simulation
• Multi-sensor data fusion (temperature, humidity, power consumption)
• Real-time anomaly detection using rolling z-score analysis
• Interactive time-series visualizations

2. Intelligent Document RAG

• PDF and TXT document ingestion
• Smart text chunking (500 tokens with 50 token overlap)
• Context-aware retrieval using vector similarity
• Source attribution and relevance scoring

3. Predictive Maintenance

• Equipment failure prediction algorithms
• Maintenance schedule optimization
• Energy efficiency recommendations
• Anomaly-based alert systems

4. Evaluation & Analytics

• Retrieval accuracy metrics
• Response latency measurement
• Document relevance scoring
• System performance monitoring

🚀 Quick Start

Prerequisites

• Python 3.8+
• 8GB+ RAM (for local LLM models)
• Internet connection (for initial model downloads)

Installation

# Clone the repository
git clone https://github.com/itsnewcoder/iot-smart-building-rag.git
cd iot-smart-building-rag

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate  # Windows
# source .venv/bin/activate  # Linux/Mac

# Install dependencies
pip install -r requirements.txt

Configuration

Create a .env file in the root directory (optional):

OPENAI_API_KEY=your_openai_api_key_here

Run Locally

streamlit run app.py

Access your app at: http://localhost:8501

📁 Project Structure

iot-smart-building-rag/
├── app.py                      # Main Streamlit application
├── requirements.txt            # Python dependencies
├── README.md                   # This file
├── .streamlit/
│   └── config.toml            # Streamlit theme configuration
├── rag/                       # RAG system core
│   ├── __init__.py
│   ├── ingest.py              # Document ingestion & vector store
│   ├── retrieval.py           # Context retrieval engine
│   ├── generate.py            # LLM response generation
│   └── evaluate.py            # System evaluation metrics
├── models/                     # Predictive models
│   ├── __init__.py
│   └── predictive.py          # Anomaly detection & maintenance
├── data/                       # Sample data
│   ├── manuals/               # Maintenance manuals (PDF/TXT)
│   ├── specs/                 # Building specifications
│   └── sensors/               # IoT sensor data (CSV)
└── .chroma/                   # Vector database storage

🔧 Usage Guide

1. Dashboard Tab

• Start Stream: Begin real-time sensor data simulation
• Live Monitoring: View real-time sensor readings and trends
• Anomaly Detection: See detected anomalies with z-score analysis
• Maintenance Tips: Get AI-powered maintenance recommendations

2. RAG QA Tab

• Ask Questions: Query maintenance procedures and building specs
• Context Retrieval: View relevant document chunks and sources
• AI Responses: Get context-aware answers from local or OpenAI models

3. Evaluation Tab

• Retrieval Testing: Test system with custom queries
• Performance Metrics: View latency and relevance scores
• Quality Assessment: Evaluate RAG system effectiveness

4. Data Manager Tab

• Document Index: View indexed documents and sources
• File Upload: Add new PDFs/TXTs to the knowledge base
• Vector Store: Manage document embeddings and storage

📈 Sample Queries

Try these example questions in the RAG QA tab:

• "How to reset chiller pump?"
• "What are the fault codes for HVAC systems?"
• "How to maintain building temperature sensors?"
• "What are the power consumption optimization tips?"
• "How to troubleshoot humidity sensor issues?"

🎯 Evaluation Metrics

Retrieval Quality

• Relevance Scoring: Cosine similarity-based ranking
• Source Attribution: Document source tracking
• Context Retrieval: Top-k document retrieval

Performance Metrics

• Response Latency: End-to-end query processing time
• Throughput: Queries processed per second
• Memory Usage: Vector database storage efficiency

RAG Effectiveness

• Context Relevance: Retrieved document quality
• Answer Accuracy: Response relevance to queries
• Source Diversity: Multiple document source utilization

🌐 Deployment

HuggingFace Spaces (Recommended)

1. Create new Space at huggingface.co/spaces
2. Choose Streamlit as SDK
3. Upload project files
4. Set environment variables in Space settings

Streamlit Cloud

1. Push code to GitHub
2. Connect repository at share.streamlit.io
3. Deploy automatically

Local Deployment

# Production server
streamlit run app.py --server.port 8501 --server.address 0.0.0.0

🔍 Technical Implementation Details

Embedding Strategy

• Model: sentence-transformers/all-MiniLM-L6-v2
• Dimensions: 384
• Normalization: L2 normalization for cosine similarity
• Chunking: 500 tokens with 50 token overlap

Vector Database

• Database: ChromaDB
• Similarity: Cosine distance
• Persistence: Local file storage (.chroma directory)
• Indexing: HNSW algorithm for fast retrieval

Anomaly Detection

• Method: Rolling z-score analysis
• Window Size: 50 data points
• Threshold: Z-score > 3.0
• Metrics: Temperature, humidity, power consumption

Predictive Maintenance

• Algorithm: Rule-based heuristics + statistical analysis
• Input: Sensor data + anomaly patterns
• Output: Maintenance recommendations + efficiency tips
• Real-time: Continuous monitoring and updates

🧪 Testing

Local Testing

# Test RAG modules
python -c "from rag.ingest import ensure_vector_store; print('✅ RAG Ready')"

# Test predictive models
python -c "from models.predictive import detect_anomalies; print('✅ Models Ready')"

# Test full application
streamlit run app.py

Sample Data

The system includes sample data for testing:

• HVAC Sensor Data: Temperature, humidity, power readings
• Chiller Manual: Maintenance procedures and fault codes
• Building Specs: System specifications and requirements

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🎓 Academic Use

This project was developed as part of an academic RAG system implementation course. It demonstrates:

• RAG Architecture: Complete retrieval-augmented generation system
• IoT Integration: Real-time sensor data processing
• Predictive Analytics: Machine learning for maintenance
• Vector Databases: ChromaDB implementation
• Modern Web UI: Streamlit-based dashboard

📞 Support

For questions or issues:

• GitHub Issues: Create an issue
• Documentation: Check this README and code comments
• Community: Streamlit and HuggingFace communities

🚀 Future Enhancements

• Real-time IoT device integration
• Advanced ML models for failure prediction
• Multi-modal document support (images, audio)
• API endpoints for external systems
• Mobile-responsive interface
• Advanced analytics dashboard
• Integration with building management systems

---

Built with ❤️ for Smart Building Intelligence

Last updated: January 2025