kycli 0.3.0
**kycli** is a high-performance Python CLI toolkit built with Cython for speed.
Requires Python: >=3.9

<p align="center">
    <img src="https://raw.githubusercontent.com/balakrishna-maduru/kycli/master/assets/logo.png" alt="KyCLI Logo" width="300">
</p>

# kycli — The Microsecond-Fast Key-Value Toolkit

`kycli` is a high-performance, developer-first key-value storage engine. It bridges the gap between the simplicity of a flat-file database and the blazing speed of in-memory caches like Redis, all while remaining completely serverless and lightweight.

Built with **Cython** and linked directly to the **Raw SQLite C API (`libsqlite3`)**, `kycli` is optimized for local development, CLI productivity, and high-throughput Python backends.

---

## ⚡ Performance: Real-World Stats

`kycli` is designed to be the fastest local storage option available for Python. By bypassing standard abstraction layers and moving critical logic to C, we achieve microsecond-level latency.

### Benchmark Results (Average of 1,000 calls)

| **Operation** | **Implementation** | **Avg Latency** | **vs. Standard Python** |
| :--- | :--- | :--- | :--- |
| **L1 Cache Hit** | **Cython LRU** | **1.5 µs** | **Near-In-Memory** |
| **Key Retrieval (Get)** | **Direct C API** | **2.8 µs** | **150x Faster** |
| **Batch Save** | **Atomic C-Loop** | **15 µs / item** | **Extreme Throughput** |
| **History Lookup** | **Indexed C API** | 5.0 µs | Instant Auditing |

> **Why so fast?** Standard Python storage tools use network sockets (Redis) or heavy wrappers (SQLAlchemy). `kycli` uses direct memory pointers to an embedded C engine, removing 99% of the overhead.
>
> **Scaling**: In a benchmark of **10,000 class records** with Pydantic validation, `kycli` maintains a sub-2ms write latency and microsecond-fast reads. See [PERFORMANCE.md](PERFORMANCE.md) for full details.

---

## 🚀 Installation

Install the latest version from PyPI:
```bash
pip install kycli
```

---

## 💻 CLI Command Reference

`kycli` provides a set of ultra-short commands for maximum terminal productivity.

### 📂 Workspace Management
| Command | Description | Example |
| :--- | :--- | :--- |
| **`kyuse`** | Switch/Create workspace | `kyuse project_alpha` |
| **`kyws`** | List workspaces | `kyws` |
| **`kydrop`**| Delete workspace | `kydrop old_ws` |
| **`kymv`** | Move key to workspace | `kymv api_key prod` |

### 📝 Basic Operations
| Command | Description | Example |
| :--- | :--- | :--- |
| **`kys`** | Save Key/Value | `kys session "active" --ttl 360` <br> `kys secret "pass" --key "k1"` |
| **`kyg`** | Get Value | `kyg session` <br> `kyg secret --key "k1"` |
| **`kypatch`**| Patch JSON | `kypatch user '{"age": 30}'` |
| **`kyl`** | List Keys | `kyl "user.*"` |
| **`kyd`** | Delete Key | `kyd host` |
| **`kypush`**| Push to List | `kypush logs "error"` |
| **`kyrem`** | Remove from List | `kyrem tags "old"` |
| **`kyrem`** | Remove from List | `kyrem tags "old"` |
### 🧱 Queues & Stacks Operations
| Command | Description | Example |
| :--- | :--- | :--- |
| **`kypush`** | Push to Queue/Stack | `kypush "task" --priority 10` |
| **`kypop`** | Pop item (Atomic) | `kypop` |
| **`kypeek`** | Peek next item | `kypeek` |
| **`kycount`** | Count items | `kycount` |
| **`kyclear`** | Clear implementation | `kyclear` |
| **`kyws`** | Create Typed WS | `kyws create q --type queue` |

### 🔍 Search & Utility
| Command | Description | Example |
| :--- | :--- | :--- |
| **`kyg -s`**| Search Values | `kyg -s "db_pass"` |
| **`kyshell`** | Open Interactive TUI | `kyshell` |
| **`init`** | Shell Setup | `kycli init` |
| **`kyfo`** | Optimize Index | `kyfo` |
| **`kyh`** | Help | `kyh` |

### 🛠️ Advanced & Recovery
| Command | Description | Example |
| :--- | :--- | :--- |
| **`kye`** | Export Data | `kye backup.json json` |
| **`kyi`** | Import Data | `kyi data.csv` |
| **`kyr`** | Restore Deleted Key | `kyr host` |
| **`kyrt`** | Point-in-Time Recovery | `kyrt "2023-10-27 10:00:00"` |
| **`kyc`** | Execute Command | `kyc hello` |
| **`kyco`** | Compact DB | `kyco 7` |
| **`kyrotate`** | Rotate Master Key | `kyrotate --new-key "newpass" --old-key "oldpass" --backup` |

---

### 📂 Workspace Management (Multi-Tenancy)
`kycli` supports isolated contexts called **Workspaces**. Each workspace is a separate SQLite database.

#### `kyuse` — Switch / Create Workspace
```bash
kyuse project_alpha
```

#### `kyws` — List Workspaces
Shows all available workspaces. The active one is marked with `✨`.
```bash
kyws
# Result: 📂 Workspaces:
#    default
# ✨ project_alpha
#    temp_test
```

#### `kymv` — Move Key
Moves a key (and its value) from the current workspace to another.
- **Safety**: Asks for confirmation if the key exists in the target.
```bash
kymv my_api_key project_beta
# Result: ✅ Moved 'my_api_key' to 'project_beta'.
```

#### `kydrop` — Delete Workspace
Permanently deletes an entire workspace and its database file.
- **Safety**: Requires explicit confirmation (`y/N`) in CLI.
- **Restriction**: You cannot drop the currently active workspace.
```bash
kydrop temp_test
# Result: ⚠️ DANGER: Are you sure you want to PERMANENTLY delete workspace 'temp_test'? (y/N): y
# Result: ✅ Workspace 'temp_test' deleted.
```

---

### `kyh` — The Help Center
Shows the available commands and basic usage instructions.
```bash
kyh
# Or use the -h flag on specific commands
```

### `kys <key> <value> [--ttl <sec>]` — Save Data
Saves a value to a key.
- **Auto-Normalization**: Keys are lowercased and trimmed.
- **Safety**: Asks `(y/n)` before overwriting an existing key.
- **TTL (Time To Live)**: Set an expiration time in seconds.
```bash
kys username "balakrishna"
# Result: ✅ Saved: username (New)

kys username "maduru"
# Result: ⚠️ Key 'username' already exists. Overwrite? (y/n):

kys session_id "data" --ttl 1h
# Result: ✅ Saved: session_id (New) (Expires in 1 hour)
```

### 🧱 Typed Workspaces: Queues & Stacks
Optimize your workspace for specific data structures with strict concurrency guarantees (`BEGIN IMMEDIATE` locking).

#### 1. Queue (FIFO)
```bash
# Create
kyws create jobs --type queue
kyuse jobs

# Push & Pop (Atomic)
kypush "job_1"
kypop
# Result: job_1
```

#### 2. Stack (LIFO)
```bash
kyws create undo_log --type stack
kypush "cmd1"
kypush "cmd2"
kypop
# Result: cmd2
```

#### 3. Priority Queue
```bash
kyws create triage --type priority_queue
kypush "low" --priority 1
kypush "high" --priority 100
kypop
# Result: high
```
See [docs/QUEUES_STACKS.md](docs/QUEUES_STACKS.md) for full details.

### 📂 Advanced JSONPath & Dot-Notation
`kycli` allows you to treat your key-value store like a document database. You can query and update deep nested structures without retrieving the entire object.

#### Nested Retrieval:
```bash
# Get a specific field
kyg user.profile.email

# Access list items by index
kyg logs[0]

# List slicing (e.g., first 5 logs)
kyg logs[0:5]
```

#### Atomic Patching (Partial Updates):
Instead of rewriting a large JSON object, you can update just one field.
```bash
# Update just the 'age' field inside the 'user' object
kys user.profile.age 25
```

### 📦 Collection Operations (Lists & Sets)
Manage lists stored in keys efficiently without manual read-modify-write loops.

```bash
# Append to a list
kypush my_list "new_item"

# Append only if the item doesn't exist (Set behavior)
kypush my_tags "python" --unique

# Remove from a list
kyrem my_list "old_item"
```

---

### 🔐 Enterprise-Grade Security: Zero-Trust Encryption
`kycli` takes a **Zero-Trust** approach by encrypting the **Entire Database File**.
- **Full Database Encryption**: The workspace file (`.db`) is stored as an encrypted binary blob (AES-GCM).
- **Opaque File**: It **cannot** be opened by `sqlite3`, DB Browser, or any other tool. It appears as random noise without the key.
- **In-Memory Speed**: On access, `kycli` decrypts the workspace into secure RAM for microsecond-fast operations.

#### Via CLI:
```bash
# Save with encryption (Encrypts entire workspace file)
kys secret_token "ghp_secure" --key "my-master-password"

# Retrieve with encryption
kyg secret_token --key "my-master-password"
```

#### Via Environment Variable (Recommended):
This works for the CLI, TUI (`kyshell`), and Python Library usage.
```
export KYCLI_MASTER_KEY="my-master-password"
```

#### Master Key Rotation
Rotate all stored values to a new master key.
```bash
kyrotate --new-key "newpass" --old-key "oldpass" --backup

# Dry-run to see how many values would rotate
kyrotate --new-key "newpass" --old-key "oldpass" --dry-run
```
```bash
export KYCLI_MASTER_KEY="my-master-password"
kyg secret_token
```

#### 🔑 Using a Custom Key per Command
You can use a specific key for individual commands without setting it for the whole session. This is useful for storing records with different keys in the same workspace.

```bash
# Save 'rec1' with Key A
kys rec1 "secret" --key "KeyA"

# Save 'rec2' with Key B
KYCLI_MASTER_KEY="KeyB" kys rec2 "secret"

# To read 'rec1', you must provide Key A
kyg rec1 --key "KeyA"
```

> [!IMPORTANT]
> If you attempt to retrieve an encrypted key without the correct `master_key`, `kycli` will return a masked message: `[ENCRYPTED: Provide a master key to view this value]` instead of raw ciphertext.

### ⏳ Value-Level TTL (Time To Live)
Like Redis, you can set keys to expire automatically. `kycli` implements **Soft Expiration**: when a key hits its TTL, it is moved to the **Archive** table (not deleted) and can be recovered within 15 days using the `kyr` command.
```bash
# Expire in 60 seconds
kys temp_code "1234" --ttl 60

# Expire in 1 day
kys daily_report "data" --ttl 1d

# Expire in 1 month (30 days)
kys monthly_archive "data" --ttl 1M
```

---

### `kyg [-s] <key_or_query>` — Get & Search
- **Get Key**: `kyg <key>` retrieves a value.
- **Search**: `kyg -s <query>` performs a Google-like search across all values using FTS5.

```bash
# Get exact key
kyg username
# Result: maduru

# Search for "admin" anywhere in the database
kyg -s "admin"
# Result:
# {
#   "user_profile": { "name": "balu", "role": "admin" }
# }
```

### `kyl [pattern]` — List Keys
Lists all keys or those matching a pattern.
```bash
kyl
# Result: 🔑 Keys: username, user_id, env

kyl "user.*"
# Result: 🔑 Keys: username, user_id
```

### `kyv [key | -h]` — View History (Audit Log)
`kycli` never deletes your old values; it archives them.
- **`kyv -h`**: Shows the full history of ALL keys in a formatted table.
- **`kyv <key>`**: Shows the latest value from the history for that specific key.
```bash
kyv -h
# Result: 📜 Full Audit History (All Keys)
# Timestamp            | Key             | Value
# -----------------------------------------------------
# 2026-01-03 13:20:01  | username        | maduru
# 2026-01-03 13:10:00  | username        | balakrishna
```

### `kyd <key>` — Delete Key (Soft Delete)
Removes a key from the active store.
- **Double-Confirmation**: Requires you to re-type the key name to prevent accidental loss.
- **Tip**: Deletion is "soft" in terms of data—it stays in history and can be recovered.
```bash
kyd username
# Result: ⚠️ DANGER: To delete 'username', please re-enter the key name: username
# Result: Deleted
# Result: 💡 Tip: If this was accidental, use 'kyr username' to restore it.
```

### `kyr <key>` — Restore Key
Restores a key from its history back into the active store.
- **Note**: This works for keys in the **Archive** table. KyCLI keeps deleted data for **15 days** before permanent removal.
```bash
kyr username
# Result: ✅ Key 'username' restored from history.
```

### `kyrt <timestamp>` — Point-in-Time Recovery (PITR)
Reconstruct the entire database state at a specific moment in time. This is a "Time Machine" for your data.
- **Mechanism**: Clears the current store and repopulates it with the state as of the given timestamp using the audit log.
```bash
# Restore to New Year's Day
kyrt "2026-01-01 12:00:00"
# Result: 🕒 Database restored to 2026-01-01 12:00:00
```

### `kyco [retention_days]` — Database Compaction & Maintenance
Cleanup old history and optimize the database file.
- **Retention**: History older than `retention_days` (default 15) is purged.
- **Optimization**: Runs SQLite `VACUUM` and `ANALYZE` to reclaim space and optimize query paths.
```bash
# Cleanup everything older than 7 days
kyco 7
# Result: 🧹 Compaction complete: Space reclaimed and stale history purged.
```

### `kye <file> [format]` — Export Data
Exports your entire store to a file.
- **Format**: `csv` (default) or `json`.
```bash
kye backup.csv
kye data.json json
```

### `kyi <file>` — Import Data
Bulk imports data from a CSV or JSON file.
```bash
kyi backup.csv
```

### `kyc <key> [args...]` — Execute Mode
Run a stored value directly as a shell command.
- **Static Execution**: Run the command exactly as stored.
- **Dynamic Execution**: Pass additional arguments that get appended to the stored command.
```bash
# Store a command
kys list_files "ls -la"

# Execute it
kyc list_files

# Dynamic execution (appends /tmp)
kyc list_files /tmp
```

---

### `kyshell` — Interactive TUI Shell
Launch a multi-pane interactive shell to manage your data.
- **Auto-completion**: Tab-completion for all commands.
- **Split View**: Real-time audit trail in a separate pane.
- **Workspace Aware**: Semantic status bar showing active workspace and user.
```bash
kycli kyshell
```

---

## 📚 Documentation & Guides

| Topic | Description | Link |
| :--- | :--- | :--- |
| **Workspaces** | Managing multiple projects/tenants (`kyuse`, `kymv`). | [docs/WORKSPACES.md](docs/WORKSPACES.md) |
| **Data Management** | Import, Export, and Backups (`kye`, `kyi`). | [docs/DATA_MGMT.md](docs/DATA_MGMT.md) |
| **Recovery** | Time travel (PITR), Restore, and Compaction (`kyrt`, `kyco`). | [docs/RECOVERY.md](docs/RECOVERY.md) |

---


### ⚙️ Global Configuration & Env Vars
`kycli` is highly configurable. You can change the database location, export formats, and UI themes via environment variables or configuration files.

#### 🌍 Environment Variables (Highest Priority)
The most direct way to configure `kycli` is via shell environment variables.

- **`KYCLI_DB_PATH`**: Sets the root directory for storing usage data (workspace databases).
- **`KYCLI_MASTER_KEY`**: Sets the default master key for AES-256 encryption.
  ```bash
  export KYCLI_DB_PATH="/custom/path/to/data_dir/"
  export KYCLI_MASTER_KEY="your-secret-password"
  ```
  *Note: If `KYCLI_DB_PATH` is a directory, workspaces will be created inside it (e.g. `/custom/path/to/data_dir/default.db`). If it is a file path, it will be used as a single database overriding workspaces.*

#### 📁 Configuration Files
`kycli` looks for configuration in `.kyclirc` or `.kyclirc.json`.

**Example `.kyclirc` (JSON):**
```json
{
  "db_path": "~/.kydata.db",
  "export_format": "csv"
}
```

## 🐍 Python Library Interface

### 1. Dictionary-like Interface (Sync)
The easiest way to integrate into any Python script or class.
```python
from kycli import Kycore

# Use as a context manager for automatic cleanup
with Kycore() as kv:
    # Set and Get (Dict-style)
    kv['theme'] = 'dark'
    print(kv['theme'])  # dark

    # Check existence
    if 'theme' in kv:
        print("Settings loaded.")

    # Bulk count
    print(f"Items stored: {len(kv)}")

# 4. Encryption & TTL (Sync)
with Kycore(master_key="secret-pass") as kv:
    # Save with 10-minute expiration
    kv.save("temp_password", "123456", ttl="10m")
    
    # Save with 1-month expiration
    kv.save("persistent_secret", "data", ttl="1M")
    
    print(kv.getkey("temp_password")) # 123456

# 5. Batch Operations (save_many)
with Kycore() as kv:
    items = [("k1", "v1"), ("k2", "v2"), ("k3", "v3")]
    kv.save_many(items, ttl="1h")
    # Result: ⚡ Atomic transaction per batch (extremely fast)

# 5. Queues & Stacks (API)
use `set_type` to separate queues from KV stores.
```python
from kycli import Kycore

# Initialize and set type
with Kycore("jobs.db") as q:
    q.set_type("queue")
    
    # Push (Atomic & Durable)
    q.push("task_1")
    q.push({"id": 2, "action": "email"})
    
    # Pop (Thread-Safe)
    item = q.pop()
    print(item) # task_1
```

# 6. Maintenance & PITR
with Kycore() as kv:
    # Cleanup history older than 30 days
    kv.compact(retention_days=30)
    
    # Point-in-Time Recovery
    kv.restore_to("2026-01-01 00:00:00")
```

### 2. High-Throughput (Async)
Designed for `asyncio` applications like FastAPI.
```python
import asyncio
from kycli import Kycore

async def run_tasks():
    # Use encryption and TTL in async environments
    with Kycore(master_key="async-vault") as kv:
        await kv.save_async("session:active", "true", ttl=3600)
        current = await kv.getkey_async("session:active")
        print(f"Session active: {current}")

asyncio.run(run_tasks())
```

### 3. Schema Validation (Pydantic)
Enforce data integrity by attaching a Pydantic model to your store.
```python
from pydantic import BaseModel
from kycli import Kycore

class UserSchema(BaseModel):
    name: str
    age: int

# Initialize with schema validation
with Kycore(schema=UserSchema) as kv:
    # This will succeed and auto-serialize
    kv.save("user:101", {"name": "Balu", "age": 30})
    
    # This will raise a ValueError (Schema Validation Error)
    kv.save("user:102", {"name": "Invalid"}) 
```

### 4. Application / Class Integration
Wrap `Kycore` inside your classes for persistent state management.
```python
class UserManager:
    def __init__(self):
        self.db = Kycore()

    def update_profile(self, user_id, data):
        self.db.save(f"user:{user_id}", data)

    def close(self):
        self.db.__exit__(None, None, None)
```

### 4. FastAPI Web Server Integration
```python
from fastapi import FastAPI, Depends
from kycli import Kycore

app = FastAPI()

def get_db():
    with Kycore() as db:
        yield db

@app.get("/config/{key}")
async def fetch_config(key: str, db: Kycore = Depends(get_db)):
    return {"val": await db.getkey_async(key)}
```

---

## 🏗 Architecture & Internal Safety

- **SQLite Engine**: Running in `WAL` (Write-Ahead Logging) mode for concurrent reads/writes.
- **Atomic Operations**: Exports use a "temp-file then rename" strategy to prevent corruption.
- **Data Integrity**: Keys are automatically lowercased and stripped to prevent duplicate-but-slightly-different keys.
- **Auto-Purge Policy**: Deleted keys are moved to an **Archive** table and automatically purged after **15 days** to keep the database size optimized.
- **Embedded C**: Core operations are written in Cython, binding directly to native library pointers.

---

## 📊 Benchmarking

Want to test the speed on your own hardware?
```bash
PYTHONPATH=. python3 tests/performance/kycli_benchmark.py
```

 ---

## 👤 Author & Support

**Balakrishna Maduru**  
- [GitHub](https://github.com/balakrishna-maduru)  
- [LinkedIn](https://www.linkedin.com/in/balakrishna-maduru/in/balakrishna-maduru)  
- [Twitter](https://x.com/krishonlyyou)