Skill: ivangdavila/weaviate (version 1.0.0)

--- name: Weaviate slug: weaviate version: 1.0.0 description: Build vector search with Weaviate using v4 syntax, proper module configuration, and production-ready patterns. metadata: {"clawdbot":{"emoji":"🔷","requires":{"bins":[]},"os":["linux","darwin","win32"]}} --- ## Critical: v4 Only (Dec 2024+) v3 syntax is DEPRECATED. Before generating ANY Weaviate code: 1. **Verify client version** — must be `weaviate-client>=4.0` 2. **Use context managers** — `with weaviate.connect_to_*() as client:` or explicit `client.close()` 3. **New imports** — `from weaviate.classes.config import Configure, Property` If you see v3 patterns (`weaviate.Client()`, `client.schema.create_class()`, `path=[...]` filters), **stop and rewrite**. ## Quick Reference | Topic | File | |-------|------| | v3→v4 migration table | `v4-syntax.md` | | Module configuration | `modules.md` | | Batch, hybrid, HNSW | `operations.md` | ## v4 Syntax Essentials ```python # Connection (ALWAYS close) with weaviate.connect_to_local() as client: # Collections (not classes) collection = client.collections.get("Article") # Queries response = collection.query.hybrid("search term", alpha=0.7) # Vector access vector = obj.vector["default"] # Dict, not List # Filters Filter.by_property("category").equal("tech") ``` ## Scope This skill covers: - Schema design for RAG and semantic search - Vectorizer and reranker module configuration - Batch imports with error handling - Hybrid search tuning (alpha parameter) - HNSW index configuration for scale ## Core Rules ### 1. Always Verify Modules Before using `text2vec-openai`, `generative-openai`, or rerankers, verify they're enabled: ```yaml # docker-compose.yml ENABLE_MODULES: 'text2vec-openai,generative-openai,reranker-cohere' ``` ### 2. API Keys in Headers ```python client = weaviate.connect_to_local( headers={"X-OpenAI-Api-Key": os.environ["OPENAI_API_KEY"]} ) ``` ### 3. Batch with Context Manager ```python with client.batch.dynamic() as batch: for item in data: batch.add_object(properties=item, collection="Name") ``` ### 4. Hybrid Search Alpha - `alpha=0` → BM25 only (keyword) - `alpha=1` → Vector only (semantic) - `alpha=0.5-0.75` → Balanced (typical for RAG) ### 5. Apply Filters BEFORE Vector Search Filters in `where` reduce the search space first — always filter before `near_text`/`near_vector`. ### 6. Named Vectors vs Single Vector Choose one pattern per collection: ```python # Single vector (simpler) vectorizer_config=Configure.Vectorizer.text2vec_openai() # Named vectors (multiple embeddings per object) vector_config=[ Configure.Vectors.text2vec_openai(name="content", source_properties=["body"]), ] ``` ### 7. Debug Empty Results Check in order: schema exists → vectorizer ran → distance threshold → filter syntax. Use `_additional { vector }` to verify vectors were generated.

path: modules.md
content: # Weaviate Module Configuration

## Enabling Modules

Modules must be enabled at server startup:

```yaml
# docker-compose.yml
services:
  weaviate:
    environment:
      ENABLE_MODULES: 'text2vec-openai,generative-openai,reranker-cohere'
      # Or enable ALL API-based modules (v1.33+):
      ENABLE_API_BASED_MODULES: 'true'
```

## Vectorizers

### text2vec-openai

```python
from weaviate.classes.config import Configure

client.collections.create(
    "Article",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small",  # or "text-embedding-3-large"
        dimensions=1536,  # 1536 for small, 3072 for large
    ),
    properties=[...]
)
```

Headers required:
```python
client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ["OPENAI_API_KEY"]}
)
```

### text2vec-cohere

```python
vectorizer_config=Configure.Vectorizer.text2vec_cohere(
    model="embed-multilingual-v3.0"
)
```

Headers: `{"X-Cohere-Api-Key": ...}`

### Local Transformers

```python
vectorizer_config=Configure.Vectorizer.text2vec_transformers()
```

No API key needed — runs locally. Add to docker-compose:
```yaml
ENABLE_MODULES: 'text2vec-transformers'
```

## Named Vectors (Multiple Embeddings)

For different embeddings per property:

```python
from weaviate.classes.config import Configure

client.collections.create(
    "Document",
    vector_config=[
        Configure.Vectors.text2vec_openai(
            name="title_vector",
            source_properties=["title"],
            model="text-embedding-3-small"
        ),
        Configure.Vectors.text2vec_openai(
            name="content_vector", 
            source_properties=["body", "summary"],
            model="text-embedding-3-large"
        ),
    ],
    properties=[...]
)
```

Query specific vector:
```python
collection.query.near_text(
    query="search",
    target_vectors=["content_vector"]
)
```

## Generative Modules

```python
from weaviate.classes.config import Configure

client.collections.create(
    "Article",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(),
    generative_config=Configure.Generative.openai(model="gpt-4"),
)
```

Usage:
```python
response = collection.generate.near_text(
    query="AI trends",
    grouped_task="Summarize these articles"
)
print(response.generated)
```

## Rerankers

```python
reranker_config=Configure.Reranker.cohere(model="rerank-english-v3.0")
# or
reranker_config=Configure.Reranker.voyageai()
```

Usage:
```python
response = collection.query.near_text(
    query="search",
    rerank=Rerank(prop="content", query="rerank query")
)
```

## Common Errors

### "no module with name 'X' present"
Module not enabled in docker-compose. Add to `ENABLE_MODULES`.

### "API key not provided"
Missing header. Add `X-{Provider}-Api-Key` to connection.

### "vectorizer not configured"
Collection created without `vectorizer_config`. Either:
1. Add vectorizer config when creating collection
2. Bring your own vectors with `vector=` parameter


path: operations.md
content: # Weaviate Operations Guide

## Batch Imports

### Always Use Batch for >100 Objects

```python
# ✅ Correct: dynamic batching with error handling
failed_objects = []

with client.batch.dynamic() as batch:
    for item in data:
        result = batch.add_object(
            properties=item,
            collection="Article"
        )
        
# Check for failures after batch completes
if client.batch.failed_objects:
    for obj in client.batch.failed_objects:
        print(f"Failed: {obj.original_uuid} - {obj.message}")
```

### Batch Strategies

```python
# Dynamic (auto-adjusts batch size)
with client.batch.dynamic() as batch:

# Fixed size (predictable memory)
with client.batch.fixed_size(batch_size=100) as batch:

# Rate limited (for external API vectorizers)
with client.batch.rate_limit(requests_per_minute=600) as batch:
```

### Bring Your Own Vectors

```python
with client.batch.dynamic() as batch:
    batch.add_object(
        properties={"title": "Doc"},
        collection="Article",
        vector=[0.1, 0.2, ...]  # Single vector
        # or
        vector={"title": [...], "content": [...]}  # Named vectors
    )
```

## Hybrid Search

### Alpha Parameter

```python
# alpha controls BM25 vs vector balance
collection.query.hybrid(
    query="machine learning",
    alpha=0.7,  # 0=BM25 only, 1=vector only, 0.5-0.75=balanced
    limit=10
)
```

Guidelines:
- `alpha=0.5` — Equal weight, good starting point
- `alpha=0.7` — Favor semantic (typical for RAG)
- `alpha=0.3` — Favor keyword (when exact terms matter)

### Fusion Types

```python
from weaviate.classes.query import HybridFusion

collection.query.hybrid(
    query="search",
    alpha=0.7,
    fusion_type=HybridFusion.RELATIVE_SCORE,  # or RANKED
)
```

## Filters

### Apply BEFORE Vector Search

```python
# ✅ Correct: filter reduces search space first
from weaviate.classes.query import Filter

collection.query.near_text(
    query="AI",
    filters=Filter.by_property("category").equal("tech"),
    limit=10
)
```

### Common Filter Operations

```python
Filter.by_property("status").equal("published")
Filter.by_property("views").greater_than(1000)
Filter.by_property("tags").contains_any(["ai", "ml"])
Filter.by_property("title").like("*intro*")

# Combine with AND/OR
(Filter.by_property("status").equal("published") & 
 Filter.by_property("views").greater_than(100))
```

## HNSW Index Tuning

### Parameters

| Parameter | Default | Effect |
|-----------|---------|--------|
| `ef` | -1 (dynamic) | Query time quality (higher=better, slower) |
| `efConstruction` | 128 | Build time quality (higher=better index) |
| `maxConnections` | 32 | Graph connectivity (higher for >1M vectors) |

### Configuration

```python
from weaviate.classes.config import Configure

client.collections.create(
    "LargeCollection",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(),
    vector_index_config=Configure.VectorIndex.hnsw(
        ef=100,
        ef_construction=256,  # Higher for large datasets
        max_connections=64,   # Higher for >1M vectors
    )
)
```

### Memory Estimation

~4KB per vector with HNSW index. For 1M vectors ≈ 4GB RAM just for index.

## Debugging Empty Results

Check in order:

1. **Collection exists?**
   ```python
   client.collections.list_all()
   ```

2. **Objects exist?**
   ```python
   collection.aggregate.over_all(total_count=True)
   ```

3. **Vectors generated?**
   ```python
   response = collection.query.fetch_objects(
       limit=1,
       include_vector=True
   )
   print(response.objects[0].vector)  # Should not be None
   ```

4. **Filter syntax correct?**
   ```python
   # Test without filter first
   collection.query.near_text(query="test", limit=5)
   ```

5. **Distance threshold too strict?**
   Default distance metric is cosine. Results with distance > 0.5 may be filtered.


path: v4-syntax.md
content: # v3 → v4 Migration Reference

## Connection

```python
# ❌ v3 (DEPRECATED)
client = weaviate.Client("http://localhost:8080")

# ✅ v4
client = weaviate.connect_to_local()  # localhost
client = weaviate.connect_to_weaviate_cloud(cluster_url, auth_credentials)
client = weaviate.connect_to_custom(http_host, http_port, grpc_host, grpc_port)

# MUST close when done
client.close()

# Or use context manager (preferred)
with weaviate.connect_to_local() as client:
    ...
```

## Schema / Collections

```python
# ❌ v3
client.schema.create_class({"class": "Article", ...})
client.schema.get()

# ✅ v4
client.collections.create("Article", properties=[...])
client.collections.list_all()
collection = client.collections.get("Article")
```

## Properties

```python
# ❌ v3
{"dataType": ["text"], "name": "title"}

# ✅ v4
from weaviate.classes.config import Property, DataType
Property(name="title", data_type=DataType.TEXT)
```

## Queries

```python
# ❌ v3
client.query.get("Article").with_near_text({"concepts": ["AI"]}).with_limit(10).do()

# ✅ v4
collection = client.collections.get("Article")
collection.query.near_text(query="AI", limit=10)
collection.query.hybrid(query="AI", alpha=0.7, limit=10)
collection.query.fetch_objects(limit=10)
```

## Filters

```python
# ❌ v3
Filter(path=["category"]).equal("tech")
Filter(path=["ref", "Author", "name"])

# ✅ v4
from weaviate.classes.query import Filter
Filter.by_property("category").equal("tech")
Filter.by_ref("ref").by_property("name")
Filter.by_id().equal(uuid)
```

## Vector Access

```python
# ❌ v3
vector = obj.vector  # List[float]

# ✅ v4
vector = obj.vector["default"]  # Dict[str, List[float]]
vector = obj.vector["my_named_vector"]  # For named vectors
```

## Batch Import

```python
# ❌ v3
client.batch.configure()
client.batch.add_data_object({...}, "ClassName")
client.batch.create_objects()

# ✅ v4
with client.batch.dynamic() as batch:  # or .fixed_size() or .rate_limit()
    batch.add_object(properties={...}, collection="ClassName")
```

## Timestamps

```python
# ❌ v3
obj.metadata.creation_time_unix  # int (milliseconds)

# ✅ v4
obj.metadata.creation_time  # datetime object
```

## Full Comparison Table

| Concept | v3 (DEPRECATED) | v4 (CURRENT) |
|---------|-----------------|--------------|
| Connect | `weaviate.Client(url)` | `weaviate.connect_to_*()` |
| Close | Not required | `client.close()` required |
| Create collection | `client.schema.create_class()` | `client.collections.create()` |
| Get collection | `client.query.get("Class")` | `client.collections.get("Class")` |
| Query | `.with_near_text().do()` | `.query.near_text()` |
| Filter path | `path=["prop"]` | `Filter.by_property("prop")` |
| Vector | `obj.vector` | `obj.vector["default"]` |
| UUID | `obj.metadata.uuid` | `obj.uuid` |
| Vectorizer config | String in schema | `Configure.Vectorizer.*()` |


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)