dspyteach 0.1.10b5
CLI utility that summarizes single files into teaching briefs using DSPy, with optional MLflow tracking
Requires Python: <3.12,>=3.10
Keywords: dspy, teaching, summaries, cli, mlflow, tracking

# DSPyTeach

DSPyTeach is a CLI for running DSPy-based analysis over source files.

It supports two modes:
- `teach`: generate a teaching brief
- `refactor`: generate a refactor prompt/template

The CLI can target local or hosted OpenAI-compatible providers through DSPy/LiteLLM, with first-class defaults for LM Studio.

## Quick Start

### 1. Install

```bash
uv sync
source .venv/bin/activate
```

### 2. Verify CLI

```bash
dspyteach --help
```

### 3. Run a first analysis

```bash
dspyteach path/to/file.py -o ./out
```

## Important Output Behavior

If you do **not** pass `--output-dir`, the CLI writes results **in place** (overwrites the source file path being analyzed).

Use `-o/--output-dir` unless in-place overwrite is intentional.

## Runtime Defaults

Current defaults from `dspy_file/domain/models.py`:

- Default provider: `lmstudio`
- LM Studio base URL: `http://localhost:1234/v1`
- Default LM Studio model: `qwen_qwen3-vl-4b-instruct`
- OpenAI default model: `gpt-5`
- Ollama base URL: `http://localhost:11434`

## Requirements

- Python `>=3.10,<3.12`
- A reachable model provider:
  - LM Studio (`--provider lmstudio`)
  - Ollama (`--provider ollama`)
  - OpenAI-compatible endpoint (`--provider openai`)

## Provider Configuration

You can set provider options with flags or environment variables.

### Flags

```bash
dspyteach <path> \
  --provider lmstudio \
  --model qwen_qwen3-vl-4b-instruct \
  --api-base http://localhost:1234/v1 \
  --api-key lm-studio \
  --dspy-adapter auto \
  --cache-policy auto \
  --lmstudio-generation-path responses \
  --lmstudio-escalation-path chat-completions \
  -o ./out
```

### Environment variables

- `DSPYTEACH_PROVIDER`
- `DSPYTEACH_MODEL`
- `DSPYTEACH_API_BASE`
- `DSPYTEACH_API_KEY`
- `DSPYTEACH_DSPY_ADAPTER` (`auto|chat|json`)
- `DSPYTEACH_CACHE_POLICY` (`auto|off|on`)
- `DSPYTEACH_STREAMING_EXPERIMENTAL` (`1|true|yes|on` to enable)
- `OPENAI_API_KEY` (fallback for `--provider openai`)

Example:

```bash
export DSPYTEACH_PROVIDER=lmstudio
export DSPYTEACH_MODEL=qwen_qwen3-vl-4b-instruct
export DSPYTEACH_API_BASE=http://localhost:1234/v1
export DSPYTEACH_API_KEY=lm-studio
export DSPYTEACH_DSPY_ADAPTER=auto
export DSPYTEACH_CACHE_POLICY=auto
export DSPYTEACH_STREAMING_EXPERIMENTAL=0
```

## Runtime Policy Controls (V1)

Runtime precedence is: CLI flags > environment variables > defaults.

- Adapter policy (`--dspy-adapter`, default `auto`):
  - `auto`: DSPy-managed adapter behavior (recommended default)
  - `chat`: force `dspy.ChatAdapter`
  - `json`: force `dspy.JSONAdapter`
- Cache policy (`--cache-policy`, default `auto`):
  - `auto`: conservative off for structured-output reliability
  - `off`: disable LM cache
  - `on`: explicit opt-in for caching (may increase stale/format drift risk)
- Streaming (`--streaming-experimental`, default off):
  - Experimental opt-in only; default V1 behavior stays non-streaming.

Examples:

```bash
# Compatibility-first defaults (recommended)
dspyteach ./docs/example.md \
  --provider lmstudio \
  --dspy-adapter auto \
  --cache-policy auto \
  -o ./out

# Explicit JSON adapter + experimental streaming
dspyteach ./docs/example.md \
  --provider lmstudio \
  --dspy-adapter json \
  --cache-policy off \
  --streaming-experimental \
  -o ./out
```

## Compatibility and Migration (V1)

- No existing flags were removed or renamed in V1.
- Existing provider/model/api-base/api-key behavior remains unchanged unless new runtime policy options are explicitly enabled.
- Default runtime behavior remains compatibility-first:
  - adapter: `auto` (DSPy-managed)
  - cache: `auto` (conservative off)
  - streaming: off unless explicitly enabled

## Usage

### Analyze one file

```bash
dspyteach ./docs/example.md -o ./out
```

### Analyze a directory recursively

```bash
dspyteach ./repo --glob "**/*.py" --glob "**/*.md" -o ./out
```

### Top-level only (non-recursive)

```bash
dspyteach ./repo --non-recursive -o ./out
```

### Confirm each file interactively

```bash
dspyteach ./repo --confirm-each -o ./out
```

### Exclude directories

```bash
dspyteach ./repo --exclude-dirs "build/,dist/,node_modules/" -o ./out
```

### Refactor mode

```bash
dspyteach --mode refactor ./repo -o ./out
```

### Raw prediction output

```bash
dspyteach ./file.py --raw -o ./out
```

## Prompt Templates (Refactor Mode)

Bundled prompt templates live in `dspy_file/prompts/`.

- Use `--prompt <name-or-path>` to pick a specific template.
- If multiple bundled templates exist and `--prompt` is omitted, the CLI prompts you to choose one.

## MLflow (Optional)

MLflow tracking is optional and enabled per run:

```bash
dspyteach ./repo --mlflow --mlflow-experiment dspyteach-local -o ./out
```

Install optional dependency when needed:

```bash
uv sync --extra mlflow
```

## Project Structure

```text
dspy-file/
├── dspy_file/
│   ├── analyze_file_cli.py      # Compatibility facade (stable import/entrypoint surface)
│   ├── cli/                     # CLI parser + entrypoint modules
│   ├── application/             # Orchestration services (runtime + analysis flow)
│   ├── domain/                  # Enums/constants/errors (framework-agnostic core)
│   ├── infrastructure/          # Provider HTTP, DSPy runtime, MLflow adapters
│   ├── file_analyzer.py         # Teaching analyzer pipeline (uses dspy.Refine)
│   ├── refactor_analyzer.py     # Refactor template generator
│   ├── file_helpers.py          # File discovery, IO, rendering helpers
│   ├── signatures.py            # DSPy signatures
│   └── prompts/                 # Bundled prompt templates
├── docs/
│   ├── ARCHITECTURE.md         # Runtime/data-flow architecture
│   ├── lm-studio-provider.md    # LM Studio setup and troubleshooting
│   └── RELEASING.md             # Release workflow
├── CHANGELOG.md                 # User-facing behavior changes
├── tests/
└── pyproject.toml
```

## Architecture Notes

See `docs/ARCHITECTURE.md` for a detailed module and runtime flow overview.

Extension points:
- Add providers/policies in `dspy_file/domain/models.py`.
- Add provider transport/runtime integrations in `dspy_file/infrastructure/`.
- Add orchestration behavior in `dspy_file/application/`.
- Keep `dspy_file/analyze_file_cli.py` as a stable facade for downstream imports.

## Troubleshooting

### LM Studio preflight failed

If LM Studio is selected, the CLI checks `GET /v1/models` before running inference.
It also probes the preferred generation path (default: `/v1/responses`) and can
escalate to a fallback path (`/v1/chat/completions`).

- Ensure LM Studio server is running (`lms server start` or Developer tab in app).
- Ensure `--api-base` points to the active host/port.
- If auth is enabled in LM Studio, pass `--api-key` (or set `DSPYTEACH_API_KEY`).

## LM Studio Endpoint Tracks

The CLI now supports three LM Studio tracks:

1. Retrieval/generation path:
- Default probe: `POST /v1/responses` (`--lmstudio-generation-path responses`)
- Optional fallback probe: `POST /v1/chat/completions`
  (`--lmstudio-escalation-path chat-completions`)

2. Embedding/index path:
- Build a local JSONL embedding index from target files via `POST /v1/embeddings`.
- Example:
```bash
dspyteach ./docs/temp \
  --provider lmstudio \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --embedding-index-out ./out/embeddings.jsonl \
  -o ./out
```
- Optional chunking for index quality:
```bash
dspyteach ./docs/temp \
  --provider lmstudio \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --embedding-index-out ./out/embeddings.jsonl \
  --embedding-chunk-size 1200 \
  --embedding-chunk-overlap 120 \
  --embedding-chunk-min-chars 80 \
  --embedding-include-metadata \
  -o ./out
```

## RAG (V1, Opt-In)

RAG is **not enabled by default**. Existing non-RAG behavior is preserved unless you pass `--rag-enabled`.

### Build a local RAG index

```bash
dspyteach ./repo \
  --provider lmstudio \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --rag-build-index \
  --rag-index-path ./out/rag-index.jsonl \
  --embedding-chunk-size 1200 \
  --embedding-chunk-overlap 120 \
  --embedding-include-metadata \
  -o ./out
```

### Run analysis with RAG augmentation

```bash
dspyteach ./repo \
  --provider lmstudio \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --rag-enabled \
  --rag-index-path ./out/rag-index.jsonl \
  --rag-top-k 4 \
  --rag-score-threshold 0.35 \
  --rag-query-mode current-file \
  --rag-max-context-chars 1600 \
  -o ./out
```

RAG-specific flags:
- `--rag-enabled`
- `--rag-build-index`
- `--rag-index-path`
- `--rag-top-k`
- `--rag-score-threshold`
- `--rag-query-mode` (`current-file|path-only`)
- `--rag-max-context-chars`

See `docs/rag-strategy.md` for architecture and operational guidance.

## RAG Evaluation Harness

Run the local harness to track retrieval + generation quality:

```bash
dspyteach-rag-eval \
  --dataset dspy_file/evaluation/rag_datasets/baseline_v1.json \
  --index-path ./out/rag-index.jsonl \
  --api-base http://localhost:1234/v1 \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --out-json ./out/rag_eval.json \
  --out-md ./out/rag_eval.md
```

Optional regression gate against prior baseline:

```bash
dspyteach-rag-eval \
  --dataset dspy_file/evaluation/rag_datasets/baseline_v1.json \
  --index-path ./out/rag-index.jsonl \
  --api-base http://localhost:1234/v1 \
  --embedding-model text-embedding-nomic-embed-text-v1.5 \
  --baseline-json ./out/rag_eval_baseline.json
```

3. Operational model lifecycle:
- `--lmstudio-load-model` → `POST /api/v1/models/load` **(experimental/opt-in)**
- `--lmstudio-unload-model` → `POST /api/v1/models/unload` **(experimental/opt-in)**
- `--lmstudio-download-model <id>` → `POST /api/v1/models/download` **(experimental/opt-in)**

### Native lifecycle support boundary (V1)

For V1, native `/api/v1/models/*` operations are supported as **experimental/opt-in**:

- **Guaranteed**:
  - Explicit operation flags remain the contract (`load`, `unload`, `download`)
  - `load` / `download` failures are **fail-fast** (non-zero exit + actionable error)
  - `unload` failures are **warning-only** (best-effort cleanup)
- **Best-effort**:
  - Endpoint behavior differences across LM Studio versions
  - Environment-specific host/network edge cases

Rollback path:
- Do not pass native lifecycle flags; continue with OpenAI-compatible-only flow.

### No files found

- Recheck path and glob patterns.
- Remove restrictive `--exclude-dirs` values.

### Ollama stop warning

If `ollama` is not on PATH, stopping the model at process end may warn.
Use `--keep-provider-alive` to skip stop behavior.

## Release

See `docs/RELEASING.md`.

For an end-user facing summary of behavioral changes, see `CHANGELOG.md`.