rudradb-opin 1.0.0
RudraDB-Opin: Free relationship-aware vector database for learning and tutorials (100 vectors, 500 relationships)
Requires Python: >=3.8
Keywords: vector, database, relationships, ai, ml, machine-learning, vector-search, similarity-search, embeddings, free, learning, tutorial, education, rag, retrieval-augmented-generation, semantic-search, relationship-aware, multi-hop, graph


<!-- <div align="center">
<img src="media/images/rudradb_logo_no_bg_sym.png" alt="RudraDB" class="brand-logo">
<img src="media/images/rudradb_brandName_no_bg_no_tag.png" alt="RudraDB" class="brand-name-logo">
</div> -->

# RudraDB-Opin - Relationship-Aware Vector Database (Free Version)

<div align="center">

![RudraDB-Opin Logo](https://img.shields.io/badge/RudraDB-Opin-blue?style=for-the-badge&logo=database&logoColor=white)
[![PyPI version](https://img.shields.io/pypi/v/rudradb-opin.svg?style=for-the-badge)](https://pypi.org/project/rudradb-opin/)
[![Python versions](https://img.shields.io/pypi/pyversions/rudradb-opin.svg?style=for-the-badge)](https://pypi.org/project/rudradb-opin/)
[![License](https://img.shields.io/badge/license-MIT-green.svg?style=for-the-badge)](LICENSE)

**🌟 The World's First Relationship-Aware Vector Database (Free Version)**  
*Perfect for Learning, Tutorials, Hackathons, Enterprise POCs and AI Development*

</div>

---

## 🎯 Revolutionary Auto-Intelligence for AI Developers

**RudraDB-Opin** is the only vector database that combines **relationship-aware search** with **revolutionary auto-features** that eliminate manual configuration. While traditional databases require complex setup and manual relationship building, RudraDB-Opin automatically detects dimensions, builds intelligent relationships, and optimizes performance.

### 🤖 **World's First Auto-Intelligent Vector Database**

#### 🎯 **Auto-Dimension Detection**
**Zero Configuration Required** - Works with any ML model instantly:

```python
import rudradb
import numpy as np
from sentence_transformers import SentenceTransformer
import openai

# ✨ NO DIMENSION SPECIFICATION NEEDED!
db = rudradb.RudraDB()  # Auto-detects from any model

# Works with Sentence Transformers (384D)
model_384 = SentenceTransformer('all-MiniLM-L6-v2')
text_384 = model_384.encode(["AI transforms everything"])[0]
db.add_vector("st_doc", text_384.astype(np.float32), {"model": "sentence-transformers"})

print(f"🎯 Auto-detected: {db.dimension()}D")  # 384

# Works with different models seamlessly (768D)  
model_768 = SentenceTransformer('all-mpnet-base-v2')
text_768 = model_768.encode(["Machine learning revolution"])[0]
db2 = rudradb.RudraDB()  # Fresh auto-detection
db2.add_vector("mpnet_doc", text_768.astype(np.float32), {"model": "mpnet"})

print(f"🎯 Auto-detected: {db2.dimension()}D")  # 768

# Works with OpenAI (1536D)
openai.api_key = "your-key"
response = openai.Embedding.create(model="text-embedding-ada-002", input="Deep learning")
embedding_1536 = np.array(response['data'][0]['embedding'], dtype=np.float32)
db3 = rudradb.RudraDB()  # Fresh auto-detection
db3.add_vector("openai_doc", embedding_1536, {"model": "openai-ada-002"})

print(f"🎯 Auto-detected: {db3.dimension()}D")  # 1536

# 🔥 IMPOSSIBLE WITH TRADITIONAL VECTOR DATABASES! 
# No manual configuration, no dimension errors, just works!
```

#### 🧠 **Auto-Relationship Detection**
**Intelligent Connection Building** - Automatically discovers semantic relationships:

```python
def add_document_with_auto_intelligence(db, doc_id, text, metadata):
    """Add document with full auto-intelligence enabled"""
    
    # 1. Auto-dimension detection handles any model
    model = SentenceTransformer('all-MiniLM-L6-v2')
    embedding = model.encode([text])[0].astype(np.float32)
    
    # 2. Add vector - dimension auto-detected on first vector
    db.add_vector(doc_id, embedding, metadata)
    
    # 3. 🧠 Auto-Relationship Detection analyzes content and metadata
    relationships_found = auto_build_smart_relationships(db, doc_id, metadata)
    
    return relationships_found

def auto_build_smart_relationships(db, new_doc_id, metadata):
    """RudraDB-Opin's intelligent auto-relationship detection"""
    relationships_created = 0
    
    # 🎯 Analyze all existing vectors for intelligent connections
    for existing_id in db.list_vectors():
        if existing_id == new_doc_id:
            continue
            
        existing_vector = db.get_vector(existing_id)
        existing_meta = existing_vector['metadata']
        
        # 🧠 SEMANTIC ANALYSIS: Content similarity detection
        if metadata.get('category') == existing_meta.get('category'):
            db.add_relationship(new_doc_id, existing_id, "semantic", 0.8)
            relationships_created += 1
            print(f"   🔗 Semantic: {new_doc_id} ↔ {existing_id} (same category)")
        
        # 🧠 HIERARCHICAL ANALYSIS: Parent-child detection  
        if (metadata.get('type') == 'concept' and existing_meta.get('type') == 'example'):
            db.add_relationship(new_doc_id, existing_id, "hierarchical", 0.9)
            relationships_created += 1
            print(f"   📊 Hierarchical: {new_doc_id} → {existing_id} (concept→example)")
        
        # 🧠 TEMPORAL ANALYSIS: Learning sequence detection
        difficulties = {"beginner": 1, "intermediate": 2, "advanced": 3}
        current_level = difficulties.get(metadata.get('difficulty', 'intermediate'), 2)
        existing_level = difficulties.get(existing_meta.get('difficulty', 'intermediate'), 2)
        
        if abs(current_level - existing_level) == 1 and metadata.get('topic') == existing_meta.get('topic'):
            db.add_relationship(new_doc_id, existing_id, "temporal", 0.85)
            relationships_created += 1
            print(f"   ⏰ Temporal: {new_doc_id} ↔ {existing_id} (learning sequence)")
        
        # 🧠 ASSOCIATIVE ANALYSIS: Tag overlap detection
        new_tags = set(metadata.get('tags', []))
        existing_tags = set(existing_meta.get('tags', []))
        shared_tags = new_tags & existing_tags
        
        if len(shared_tags) >= 2:  # Strong tag overlap
            strength = min(0.7, len(shared_tags) * 0.2)
            db.add_relationship(new_doc_id, existing_id, "associative", strength)
            relationships_created += 1
            print(f"   🏷️ Associative: {new_doc_id} ↔ {existing_id} (tags: {', '.join(shared_tags)})")
        
        # 🧠 CAUSAL ANALYSIS: Problem-solution detection
        if (metadata.get('type') == 'problem' and existing_meta.get('type') == 'solution'):
            db.add_relationship(new_doc_id, existing_id, "causal", 0.95)
            relationships_created += 1
            print(f"   🎯 Causal: {new_doc_id} → {existing_id} (problem→solution)")
    
    return relationships_created

# 🚀 DEMO: Building a Knowledge Base with Auto-Intelligence
print("🤖 Building AI Knowledge Base with Auto-Intelligence")
db = rudradb.RudraDB()  # Auto-dimension detection enabled

# Add documents - watch auto-relationship detection work!
documents = [
    {
        "id": "ai_basics",
        "text": "Artificial Intelligence fundamentals and core concepts",
        "metadata": {"category": "AI", "difficulty": "beginner", "type": "concept", "tags": ["ai", "basics"], "topic": "ai"}
    },
    {
        "id": "ml_intro", 
        "text": "Machine Learning introduction and supervised learning",
        "metadata": {"category": "AI", "difficulty": "intermediate", "type": "concept", "tags": ["ml", "supervised"], "topic": "ai"}
    },
    {
        "id": "python_ml_example",
        "text": "Python code example for machine learning with scikit-learn", 
        "metadata": {"category": "AI", "difficulty": "intermediate", "type": "example", "tags": ["python", "ml", "code"], "topic": "ai"}
    },
    {
        "id": "overfitting_problem",
        "text": "Overfitting problem in machine learning models",
        "metadata": {"category": "AI", "difficulty": "advanced", "type": "problem", "tags": ["ml", "overfitting"], "topic": "ai"}
    },
    {
        "id": "regularization_solution", 
        "text": "Regularization techniques to prevent overfitting",
        "metadata": {"category": "AI", "difficulty": "advanced", "type": "solution", "tags": ["ml", "regularization"], "topic": "ai"}
    }
]

total_relationships = 0
for doc in documents:
    relationships = add_document_with_auto_intelligence(
        db, doc["id"], doc["text"], doc["metadata"]
    )
    total_relationships += relationships

print(f"\n✅ Auto-created knowledge base:")
print(f"   📄 {db.vector_count()} documents")
print(f"   🔗 {db.relationship_count()} auto-detected relationships")
print(f"   🎯 {db.dimension()}-dimensional embeddings (auto-detected)")
print(f"   🧠 {total_relationships} intelligent connections found automatically")

# 🔍 Experience Auto-Enhanced Search
query = "machine learning techniques and examples"
model = SentenceTransformer('all-MiniLM-L6-v2')
query_embedding = model.encode([query])[0].astype(np.float32)

# Traditional similarity search
basic_results = db.search(query_embedding, rudradb.SearchParams(
    top_k=5, include_relationships=False
))

# 🧠 Auto-relationship enhanced search
enhanced_results = db.search(query_embedding, rudradb.SearchParams(
    top_k=5,
    include_relationships=True,  # Uses auto-detected relationships!
    max_hops=2,
    relationship_weight=0.4
))

print(f"\n🔍 Search Results Comparison:")
print(f"Traditional search: {len(basic_results)} results")
print(f"Auto-enhanced search: {len(enhanced_results)} results with relationship intelligence")

for result in enhanced_results:
    vector = db.get_vector(result.vector_id)
    connection = "Direct match" if result.hop_count == 0 else f"{result.hop_count}-hop auto-connection"
    print(f"   📄 {vector['metadata']['type']}: {result.vector_id}")
    print(f"      └─ {connection} (score: {result.combined_score:.3f})")

print(f"\n🎉 RudraDB-Opin discovered {sum(1 for r in enhanced_results if r.hop_count > 0)} additional relevant documents")
print("    that traditional vector databases would completely miss!")
```

### 🆓 **100% Free Version with Premium Features**
- **100 vectors** - Perfect tutorial and learning size
- **500 relationships** - Rich relationship modeling capability  
- **🎯 Auto-Dimension Detection** - Works with any ML model instantly
- **🧠 Auto-Relationship Detection** - Builds intelligent connections automatically
- **Complete feature set** - All 5 relationship types and algorithms
- **Multi-hop discovery** - 2-degree relationship traversal
- **No usage tracking** - Complete privacy and freedom
- **Production-quality code** - Same codebase as enterprise RudraDB

### 🚀 **Ready for Production?**
When you outgrow the 100-vector limit, upgrade seamlessly:
```bash
pip uninstall rudradb-opin
pip install rudradb  # Get 100,000+ vectors, same API!
```

---

## 📦 Quick Installation & Setup

### Install from PyPI
```bash
pip install rudradb-opin
```

### Verify Installation with Auto-Features
```python
import rudradb
import numpy as np

# Test auto-dimension detection
db = rudradb.RudraDB()  # No dimension specified!
print(f"🎯 Auto-dimension detection: {'✅ Enabled' if db.dimension() is None else 'Manual'}")

# Test with random embedding
test_embedding = np.random.rand(384).astype(np.float32)
db.add_vector("test", test_embedding, {"test": True})
print(f"🎯 Auto-detected dimension: {db.dimension()}")

# Verify auto-relationship capabilities
print(f"🧠 Auto-relationship detection: ✅ Available")
print(f"📊 Limits: {rudradb.MAX_VECTORS} vectors, {rudradb.MAX_RELATIONSHIPS} relationships")
print(f"🎉 RudraDB-Opin {rudradb.__version__} ready with auto-intelligence!")
```

---

## 🤖 Complete ML Framework Integrations

### 1. 🔥 **OpenAI Integration** - Auto-Dimension Detection for 1536D Embeddings

```python
import openai
import rudradb
import numpy as np

class OpenAI_RudraDB_RAG:
    """Complete OpenAI + RudraDB-Opin integration with auto-features"""
    
    def __init__(self, api_key):
        openai.api_key = api_key
        self.db = rudradb.RudraDB()  # 🎯 Auto-detects OpenAI's 1536 dimensions
        print("🤖 OpenAI + RudraDB-Opin initialized with auto-dimension detection")
    
    def add_document(self, doc_id, text, metadata=None):
        """Add document with OpenAI embeddings + auto-relationship detection"""
        
        # Get OpenAI embedding
        response = openai.Embedding.create(
            model="text-embedding-ada-002",
            input=text
        )
        embedding = np.array(response['data'][0]['embedding'], dtype=np.float32)
        
        # Add with auto-intelligence
        enhanced_metadata = {
            "text": text,
            "embedding_model": "text-embedding-ada-002",
            "auto_detected_dim": self.db.dimension() if self.db.dimension() else "pending",
            **(metadata or {})
        }
        
        self.db.add_vector(doc_id, embedding, enhanced_metadata)
        
        # 🧠 Auto-build relationships based on content analysis
        relationships_created = self._auto_detect_relationships(doc_id, enhanced_metadata)
        
        return {
            "dimension": self.db.dimension(),
            "relationships_created": relationships_created,
            "total_vectors": self.db.vector_count()
        }
    
    def _auto_detect_relationships(self, new_doc_id, metadata):
        """Auto-detect relationships using OpenAI embeddings + metadata analysis"""
        relationships = 0
        new_text = metadata.get('text', '')
        new_category = metadata.get('category')
        new_tags = set(metadata.get('tags', []))
        
        for existing_id in self.db.list_vectors():
            if existing_id == new_doc_id or relationships >= 3:
                continue
                
            existing = self.db.get_vector(existing_id)
            existing_meta = existing['metadata']
            existing_text = existing_meta.get('text', '')
            existing_category = existing_meta.get('category')
            existing_tags = set(existing_meta.get('tags', []))
            
            # 🎯 Semantic similarity through category matching
            if new_category and new_category == existing_category:
                self.db.add_relationship(new_doc_id, existing_id, "semantic", 0.8, 
                                       {"reason": "same_category", "auto_detected": True})
                relationships += 1
                print(f"   🔗 Auto-linked {new_doc_id} ↔ {existing_id} (semantic: same category)")
            
            # 🏷️ Associative through tag overlap
            shared_tags = new_tags & existing_tags
            if len(shared_tags) >= 1:
                strength = min(0.7, len(shared_tags) * 0.3)
                self.db.add_relationship(new_doc_id, existing_id, "associative", strength,
                                       {"reason": "shared_tags", "tags": list(shared_tags), "auto_detected": True})
                relationships += 1
                print(f"   🏷️ Auto-linked {new_doc_id} ↔ {existing_id} (associative: {shared_tags})")
        
        return relationships
    
    def intelligent_qa(self, question):
        """Answer questions using relationship-aware search + GPT"""
        
        # Get question embedding with auto-dimension compatibility
        response = openai.Embedding.create(
            model="text-embedding-ada-002", 
            input=question
        )
        query_embedding = np.array(response['data'][0]['embedding'], dtype=np.float32)
        
        # 🧠 Auto-enhanced relationship-aware search
        results = self.db.search(query_embedding, rudradb.SearchParams(
            top_k=5,
            include_relationships=True,  # Use auto-detected relationships
            max_hops=2,                 # Multi-hop discovery
            relationship_weight=0.3     # Balance similarity + relationships
        ))
        
        # Build context from auto-enhanced results
        context_pieces = []
        for result in results:
            vector = self.db.get_vector(result.vector_id)
            text = vector['metadata']['text']
            connection_type = "Direct match" if result.hop_count == 0 else f"{result.hop_count}-hop connection"
            context_pieces.append(f"[{connection_type}] {text}")
        
        context = "\n".join(context_pieces)
        
        # Generate answer with GPT
        chat_response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=[
                {"role": "system", "content": "You are an AI assistant with access to a relationship-aware knowledge base. Use the provided context to answer questions, noting both direct matches and relationship-connected information."},
                {"role": "user", "content": f"Context from relationship-aware search:\n{context}\n\nQuestion: {question}"}
            ]
        )
        
        return {
            "answer": chat_response.choices[0].message.content,
            "sources_found": len(results),
            "relationship_enhanced": sum(1 for r in results if r.hop_count > 0),
            "context_dimension": self.db.dimension()
        }

# 🚀 Demo: OpenAI + Auto-Intelligence
rag = OpenAI_RudraDB_RAG("your-openai-api-key")

# Add AI knowledge with auto-relationship detection
documents = [
    {"id": "ai_overview", "text": "Artificial Intelligence is transforming industries through automation and intelligent decision making.", 
     "category": "AI", "tags": ["ai", "automation", "industry"]},
    {"id": "ml_subset", "text": "Machine Learning is a subset of AI that enables computers to learn from data without explicit programming.",
     "category": "AI", "tags": ["ml", "data", "learning"]},
    {"id": "dl_neural", "text": "Deep Learning uses neural networks with multiple layers to process complex patterns in data.",
     "category": "AI", "tags": ["dl", "neural", "patterns"]},
    {"id": "nlp_language", "text": "Natural Language Processing helps computers understand and generate human language.",
     "category": "AI", "tags": ["nlp", "language", "text"]},
    {"id": "cv_vision", "text": "Computer Vision enables machines to interpret and analyze visual information from images and videos.",
     "category": "AI", "tags": ["cv", "vision", "images"]}
]

print("🤖 Building AI Knowledge Base with OpenAI + Auto-Intelligence:")
for doc in documents:
    result = rag.add_document(doc["id"], doc["text"], {"category": doc["category"], "tags": doc["tags"]})
    print(f"   📄 {doc['id']}: {result['relationships_created']} auto-relationships, {result['dimension']}D embedding")

print(f"\n✅ Knowledge base ready: {rag.db.vector_count()} vectors, {rag.db.relationship_count()} auto-relationships")

# Intelligent Q&A with relationship-aware context
answer = rag.intelligent_qa("How does machine learning relate to other AI technologies?")
print(f"\n🧠 Intelligent Answer:")
print(f"   Sources: {answer['sources_found']} documents (including {answer['relationship_enhanced']} through relationships)")
print(f"   Answer: {answer['answer'][:200]}...")
```

### 2. 🤗 **HuggingFace Integration** - Multi-Model Auto-Dimension Detection

```python
from transformers import AutoTokenizer, AutoModel, pipeline
from sentence_transformers import SentenceTransformer
import torch
import rudradb
import numpy as np

class HuggingFace_RudraDB_MultiModel:
    """HuggingFace + RudraDB-Opin with multi-model auto-dimension detection"""
    
    def __init__(self):
        self.models = {}
        self.databases = {}
        print("🤗 HuggingFace + RudraDB-Opin Multi-Model System initialized")
    
    def add_model(self, model_name, model_type="sentence-transformer"):
        """Add a HuggingFace model with auto-dimension detection"""
        
        if model_type == "sentence-transformer":
            model = SentenceTransformer(model_name)
            dimension = model.get_sentence_embedding_dimension()
        else:
            tokenizer = AutoTokenizer.from_pretrained(model_name)
            model = AutoModel.from_pretrained(model_name)
            # Get dimension from config
            dimension = model.config.hidden_size
            model = {"tokenizer": tokenizer, "model": model}
        
        self.models[model_name] = {
            "model": model,
            "type": model_type, 
            "expected_dimension": dimension
        }
        
        # Create database with auto-dimension detection
        self.databases[model_name] = rudradb.RudraDB()  # 🎯 Auto-detects dimension
        
        print(f"✅ Added {model_name} (expected: {dimension}D, auto-detection enabled)")
        
    def encode_text(self, model_name, text):
        """Encode text with specified model"""
        model_info = self.models[model_name]
        
        if model_info["type"] == "sentence-transformer":
            embedding = model_info["model"].encode([text])[0]
        else:
            tokenizer = model_info["model"]["tokenizer"]
            model = model_info["model"]["model"]
            
            inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
            with torch.no_grad():
                outputs = model(**inputs)
                embedding = outputs.last_hidden_state.mean(dim=1).squeeze().numpy()
        
        return embedding.astype(np.float32)
    
    def add_document_multimodel(self, doc_id, text, metadata, model_names=None):
        """Add document to multiple model databases with auto-relationship detection"""
        
        if model_names is None:
            model_names = list(self.models.keys())
        
        results = {}
        for model_name in model_names:
            db = self.databases[model_name]
            
            # Encode with current model
            embedding = self.encode_text(model_name, text)
            
            # Add to database - auto-dimension detection in action
            enhanced_metadata = {
                "text": text,
                "model": model_name,
                "expected_dim": self.models[model_name]["expected_dimension"],
                **metadata
            }
            
            db.add_vector(doc_id, embedding, enhanced_metadata)
            
            # Auto-detect relationships within this model's space
            relationships = self._auto_build_relationships(db, doc_id, enhanced_metadata)
            
            results[model_name] = {
                "expected_dim": self.models[model_name]["expected_dimension"],
                "detected_dim": db.dimension(),
                "relationships_created": relationships,
                "match": db.dimension() == self.models[model_name]["expected_dimension"]
            }
        
        return results
    
    def _auto_build_relationships(self, db, doc_id, metadata):
        """Auto-build relationships based on metadata analysis"""
        relationships_created = 0
        doc_tags = set(metadata.get('tags', []))
        doc_category = metadata.get('category')
        doc_difficulty = metadata.get('difficulty')
        
        for other_id in db.list_vectors():
            if other_id == doc_id or relationships_created >= 3:
                continue
                
            other_vector = db.get_vector(other_id)
            other_meta = other_vector['metadata']
            other_tags = set(other_meta.get('tags', []))
            other_category = other_meta.get('category')
            other_difficulty = other_meta.get('difficulty')
            
            # Auto-detect relationship type and strength
            if doc_category == other_category:
                # Same category → semantic relationship
                db.add_relationship(doc_id, other_id, "semantic", 0.8,
                                  {"auto_detected": True, "reason": "same_category"})
                relationships_created += 1
            elif len(doc_tags & other_tags) >= 1:
                # Shared tags → associative relationship  
                shared = doc_tags & other_tags
                strength = min(0.7, len(shared) * 0.25)
                db.add_relationship(doc_id, other_id, "associative", strength,
                                  {"auto_detected": True, "reason": "shared_tags", "tags": list(shared)})
                relationships_created += 1
            elif doc_difficulty and other_difficulty:
                # Learning progression → temporal relationship
                levels = {"beginner": 1, "intermediate": 2, "advanced": 3}
                if abs(levels.get(doc_difficulty, 2) - levels.get(other_difficulty, 2)) == 1:
                    db.add_relationship(doc_id, other_id, "temporal", 0.85,
                                      {"auto_detected": True, "reason": "learning_progression"})
                    relationships_created += 1
        
        return relationships_created
    
    def cross_model_search(self, query, model_names=None, top_k=5):
        """Search across multiple models with auto-enhanced results"""
        
        if model_names is None:
            model_names = list(self.models.keys())
        
        all_results = {}
        for model_name in model_names:
            db = self.databases[model_name]
            query_embedding = self.encode_text(model_name, query)
            
            # Auto-enhanced relationship-aware search
            results = db.search(query_embedding, rudradb.SearchParams(
                top_k=top_k,
                include_relationships=True,  # Use auto-detected relationships
                max_hops=2,
                relationship_weight=0.3
            ))
            
            model_results = []
            for result in results:
                vector = db.get_vector(result.vector_id)
                model_results.append({
                    "document": result.vector_id,
                    "text": vector['metadata']['text'],
                    "similarity": result.similarity_score,
                    "combined_score": result.combined_score,
                    "connection": "direct" if result.hop_count == 0 else f"{result.hop_count}-hop",
                    "model_dimension": db.dimension()
                })
            
            all_results[model_name] = {
                "results": model_results,
                "dimension": db.dimension(),
                "total_docs": db.vector_count(),
                "total_relationships": db.relationship_count()
            }
        
        return all_results

# 🚀 Demo: Multi-Model Auto-Dimension Detection
system = HuggingFace_RudraDB_MultiModel()

# Add multiple HuggingFace models - each gets auto-dimension detection
models_to_test = [
    ("sentence-transformers/all-MiniLM-L6-v2", "sentence-transformer"),  # 384D
    ("sentence-transformers/all-mpnet-base-v2", "sentence-transformer"),  # 768D  
    ("distilbert-base-uncased", "transformer")  # 768D
]

print("🤗 Adding multiple HuggingFace models with auto-dimension detection:")
for model_name, model_type in models_to_test:
    system.add_model(model_name, model_type)

# Add documents to all models - watch auto-dimension detection work
documents = [
    {"id": "transformers_paper", "text": "Attention Is All You Need introduced the Transformer architecture revolutionizing NLP", 
     "category": "NLP", "tags": ["transformers", "attention", "nlp"], "difficulty": "advanced"},
    {"id": "bert_paper", "text": "BERT Bidirectional Encoder Representations from Transformers for language understanding",
     "category": "NLP", "tags": ["bert", "bidirectional", "nlp"], "difficulty": "intermediate"},  
    {"id": "gpt_intro", "text": "GPT Generative Pre-trained Transformers for text generation and completion",
     "category": "NLP", "tags": ["gpt", "generative", "nlp"], "difficulty": "intermediate"},
    {"id": "vision_transformer", "text": "Vision Transformer ViT applies transformer architecture to computer vision tasks",
     "category": "CV", "tags": ["vit", "transformers", "vision"], "difficulty": "advanced"}
]

print(f"\n📄 Adding documents with multi-model auto-dimension detection:")
for doc in documents:
    results = system.add_document_multimodel(
        doc["id"], doc["text"], 
        {"category": doc["category"], "tags": doc["tags"], "difficulty": doc["difficulty"]}
    )
    
    print(f"   📄 {doc['id']}:")
    for model_name, result in results.items():
        status = "✅" if result["match"] else "⚠️"
        print(f"      {status} {model_name}: Expected {result['expected_dim']}D → Detected {result['detected_dim']}D")
        print(f"         Relationships: {result['relationships_created']} auto-created")

# Cross-model search with auto-enhanced results
query = "transformer architecture for language and vision"
print(f"\n🔍 Cross-Model Search: '{query}'")
search_results = system.cross_model_search(query, top_k=3)

for model_name, results in search_results.items():
    print(f"\n📊 {model_name} ({results['dimension']}D, {results['total_relationships']} auto-relationships):")
    for result in results['results'][:2]:
        print(f"   📄 {result['document']}")
        print(f"      Connection: {result['connection']} (score: {result['combined_score']:.3f})")

print(f"\n🎉 Multi-model auto-dimension detection successful!")
print("    RudraDB-Opin seamlessly adapted to each model's dimensions automatically!")
```

### 3. 🔍 **Haystack Integration** - Document Processing with Auto-Relationships

```python
from haystack import Document, Pipeline
from haystack.document_stores import InMemoryDocumentStore  
from haystack.nodes import DensePassageRetriever, FARMReader
import rudradb
import numpy as np

class Haystack_RudraDB_Pipeline:
    """Haystack + RudraDB-Opin integration with auto-intelligence"""
    
    def __init__(self):
        # Initialize Haystack components
        self.haystack_store = InMemoryDocumentStore()
        self.retriever = DensePassageRetriever(
            document_store=self.haystack_store,
            query_embedding_model="facebook/dpr-question_encoder-single-nq-base",
            passage_embedding_model="facebook/dpr-ctx_encoder-single-nq-base"
        )
        
        # Initialize RudraDB-Opin with auto-dimension detection
        self.rudra_db = rudradb.RudraDB()  # 🎯 Auto-detects DPR dimensions (768D)
        
        print("🔍 Haystack + RudraDB-Opin pipeline initialized")
        print("   🤖 Auto-dimension detection enabled for DPR embeddings")
    
    def process_documents(self, documents):
        """Process documents through Haystack and add to RudraDB with auto-relationships"""
        
        # Convert to Haystack documents
        haystack_docs = []
        for i, doc in enumerate(documents):
            haystack_doc = Document(
                content=doc["text"],
                meta={
                    "id": doc["id"],
                    "title": doc.get("title", f"Document {i+1}"),
                    **doc.get("metadata", {})
                }
            )
            haystack_docs.append(haystack_doc)
        
        # Add to Haystack document store and create embeddings
        self.haystack_store.write_documents(haystack_docs)
        self.haystack_store.update_embeddings(self.retriever)
        
        print(f"📄 Processed {len(haystack_docs)} documents through Haystack")
        
        # Add to RudraDB-Opin with auto-dimension detection and relationship building
        relationships_created = 0
        for doc in haystack_docs:
            # Get embedding from Haystack
            embedding = self.haystack_store.get_embedding_by_id(doc.id)
            if embedding is not None:
                embedding_array = np.array(embedding, dtype=np.float32)
                
                # Add to RudraDB with enhanced metadata
                enhanced_meta = {
                    "haystack_id": doc.id,
                    "title": doc.meta["title"],
                    "content": doc.content,
                    "embedding_model": "facebook/dpr-ctx_encoder-single-nq-base",
                    **doc.meta
                }
                
                self.rudra_db.add_vector(doc.meta["id"], embedding_array, enhanced_meta)
                
                # 🧠 Auto-detect relationships based on Haystack processing + content analysis
                doc_relationships = self._auto_detect_haystack_relationships(doc.meta["id"], enhanced_meta)
                relationships_created += doc_relationships
        
        return {
            "processed_docs": len(haystack_docs),
            "rudra_dimension": self.rudra_db.dimension(),
            "auto_relationships": relationships_created,
            "total_vectors": self.rudra_db.vector_count()
        }
    
    def _auto_detect_haystack_relationships(self, doc_id, metadata):
        """Auto-detect relationships using Haystack embeddings + metadata"""
        relationships = 0
        doc_content = metadata.get('content', '')
        doc_title = metadata.get('title', '')
        doc_category = metadata.get('category')
        doc_topics = set(metadata.get('topics', []))
        
        # Analyze against existing documents
        for existing_id in self.rudra_db.list_vectors():
            if existing_id == doc_id or relationships >= 4:
                continue
            
            existing = self.rudra_db.get_vector(existing_id)
            existing_meta = existing['metadata']
            existing_content = existing_meta.get('content', '')
            existing_category = existing_meta.get('category')
            existing_topics = set(existing_meta.get('topics', []))
            
            # 🎯 Content-based semantic relationships (using Haystack embeddings)
            if doc_category and doc_category == existing_category:
                self.rudra_db.add_relationship(doc_id, existing_id, "semantic", 0.85,
                    {"auto_detected": True, "reason": "haystack_same_category", "method": "dpr_embeddings"})
                relationships += 1
                print(f"   🔗 Haystack semantic: {doc_id} ↔ {existing_id}")
            
            # 🏷️ Topic overlap relationships  
            shared_topics = doc_topics & existing_topics
            if len(shared_topics) >= 1:
                strength = min(0.8, len(shared_topics) * 0.3)
                self.rudra_db.add_relationship(doc_id, existing_id, "associative", strength,
                    {"auto_detected": True, "reason": "shared_topics", "topics": list(shared_topics), "method": "haystack_analysis"})
                relationships += 1
                print(f"   🏷️ Haystack associative: {doc_id} ↔ {existing_id} (topics: {shared_topics})")
            
            # 📊 Hierarchical relationships through title analysis
            if "introduction" in doc_title.lower() and existing_category == doc_category:
                self.rudra_db.add_relationship(doc_id, existing_id, "hierarchical", 0.7,
                    {"auto_detected": True, "reason": "introduction_hierarchy", "method": "haystack_title_analysis"})
                relationships += 1
                print(f"   📊 Haystack hierarchical: {doc_id} → {existing_id}")
        
        return relationships
    
    def hybrid_search(self, question, top_k=5):
        """Hybrid search using Haystack retrieval + RudraDB relationship-aware search"""
        
        # 1. Haystack dense retrieval
        haystack_results = self.retriever.retrieve(question, top_k=top_k*2)
        
        # 2. RudraDB-Opin relationship-aware search
        question_embedding = self.retriever.embed_queries([question])[0]
        question_embedding = np.array(question_embedding, dtype=np.float32)
        
        rudra_results = self.rudra_db.search(question_embedding, rudradb.SearchParams(
            top_k=top_k,
            include_relationships=True,  # 🧠 Use auto-detected relationships
            max_hops=2,
            relationship_weight=0.4
        ))
        
        # 3. Combine and deduplicate results
        combined_results = []
        seen_docs = set()
        
        # Add Haystack results
        for doc in haystack_results[:top_k]:
            if doc.meta["id"] not in seen_docs:
                combined_results.append({
                    "id": doc.meta["id"],
                    "title": doc.meta.get("title", ""),
                    "content": doc.content[:200] + "...",
                    "source": "haystack_dense",
                    "score": doc.score,
                    "method": "DPR retrieval"
                })
                seen_docs.add(doc.meta["id"])
        
        # Add RudraDB relationship-enhanced results
        for result in rudra_results:
            if result.vector_id not in seen_docs:
                vector = self.rudra_db.get_vector(result.vector_id)
                connection = "direct" if result.hop_count == 0 else f"{result.hop_count}-hop auto-connection"
                combined_results.append({
                    "id": result.vector_id,
                    "title": vector['metadata'].get('title', ''),
                    "content": vector['metadata'].get('content', '')[:200] + "...",
                    "source": "rudradb_relationships",
                    "score": result.combined_score,
                    "method": f"Relationship-aware ({connection})",
                    "hop_count": result.hop_count
                })
                seen_docs.add(result.vector_id)
        
        # Sort by score
        combined_results.sort(key=lambda x: x["score"], reverse=True)
        
        return {
            "question": question,
            "total_results": len(combined_results),
            "haystack_results": len([r for r in combined_results if r["source"] == "haystack_dense"]),
            "rudra_relationship_results": len([r for r in combined_results if r["source"] == "rudradb_relationships"]),
            "relationship_enhanced": len([r for r in combined_results if r.get("hop_count", 0) > 0]),
            "results": combined_results[:top_k],
            "dimension": self.rudra_db.dimension()
        }

# 🚀 Demo: Haystack + RudraDB Auto-Intelligence
pipeline = Haystack_RudraDB_Pipeline()

# Process documents with auto-dimension detection and relationship building
documents = [
    {
        "id": "ai_intro_doc",
        "text": "Artificial Intelligence Introduction: AI systems can perform tasks that typically require human intelligence, including learning, reasoning, and problem-solving.",
        "title": "AI Introduction",
        "metadata": {"category": "AI", "topics": ["ai", "introduction", "basics"], "difficulty": "beginner"}
    },
    {
        "id": "machine_learning_fundamentals", 
        "text": "Machine Learning Fundamentals: ML algorithms enable computers to learn from data without being explicitly programmed for every task.",
        "title": "ML Fundamentals",
        "metadata": {"category": "AI", "topics": ["ml", "algorithms", "data"], "difficulty": "intermediate"}
    },
    {
        "id": "neural_networks_deep",
        "text": "Neural Networks and Deep Learning: Deep neural networks with multiple layers can learn complex patterns and representations from large datasets.",
        "title": "Neural Networks",
        "metadata": {"category": "AI", "topics": ["neural", "deep", "learning"], "difficulty": "advanced"}
    },
    {
        "id": "nlp_processing",
        "text": "Natural Language Processing: NLP enables computers to understand, interpret, and generate human language in a valuable way.",
        "title": "NLP Overview", 
        "metadata": {"category": "NLP", "topics": ["nlp", "language", "text"], "difficulty": "intermediate"}
    },
    {
        "id": "computer_vision_intro",
        "text": "Computer Vision Introduction: CV systems can automatically identify, analyze, and understand visual content from images and videos.",
        "title": "Computer Vision",
        "metadata": {"category": "CV", "topics": ["vision", "images", "analysis"], "difficulty": "intermediate"}
    }
]

print("🔍 Processing documents through Haystack + RudraDB pipeline:")
processing_result