tengwar 0.3.8
The AI-Native Programming Language
License: MIT
Requires Python: >=3.9
Keywords: ai, programming-language, functional, lisp, mcp, binary-ast

# Tengwar

**The programming language built for AI.**

Tengwar is a functional programming language designed from the ground up for AI agents to write, reason about, and execute. It combines a minimal Lisp-like syntax with Unicode operators, Python interop, and a binary AST protocol that lets AI emit executable programs as structured opcodes — zero syntax errors, zero parsing overhead.

```
;; Tengwar: What AI-native code looks like

(pipe ⟦1 2 3 4 5 6 7 8 9 10⟧
  {filter {> _ 5} _}
  {map {* _ 2} _}
  (reduce + 0))                    ;; → 90

(let data (json-parse (dict-get (http-get "https://api.example.com/data") "body"))
     ids  (map {dict-get _ "id"} data)
  (json-encode ids))

(>> (py-import "pandas")
    (py-call pandas "read_csv" "data.csv") → df
    (py-call df "describe"))
```

## Install

```bash
pip install tengwar
```

## Why Tengwar?

| Feature | Python | JavaScript | Tengwar |
|---------|--------|------------|---------|
| AI syntax errors | Frequent | Frequent | **Impossible** (Binary AST) |
| Tokens per program | ~100 | ~120 | **~40** |
| Sandbox mode | Complex | Complex | **Built-in** |
| Python interop | N/A | None | **Native** |
| Functional stdlib | Limited | Limited | **80+ builtins** |
| Pattern matching | 3.10+ | No | **Native** |
| Tail recursion | No | No | **Yes** |

## Features

### Core Language
- **S-expression syntax** — minimal, unambiguous, zero-effort for AI to generate
- **Unicode operators** — `λ` `→` `⟦⟧` `⟨⟩` `∅` `≡` `↺` — or ASCII equivalents
- **Pattern matching** — `(match expr (pattern1 body1) (pattern2 body2))`
- **Let bindings** — `(let x 10 y 20 (+ x y))`
- **Pipe operator** — `(pipe value f1 f2 f3)` — thread data through transforms
- **Short lambdas** — `{+ _ 1}` = `(λ _ (+ _ 1))`
- **Closures** with lexical scope
- **Tail-call optimization** — 10,000+ recursive calls, no stack overflow
- **Parallel execution** — `(‖ expr1 expr2 expr3)` runs concurrently
- **Mutable state** — `(mut! x 0)` when you need it

### 80+ Standard Library Functions

**Collections:** `map` `filter` `reduce` `flat-map` `find` `take` `drop` `take-while` `drop-while` `zip-with` `group-by` `unique` `frequencies` `partition` `scan` `chunks` `interleave` `repeat` `iterate` `sort` `sort-by` `reverse` `concat` `len` `head` `tail` `nth` `range` `any?` `all?` `count`

**Dictionaries:** `dict` `dict-get` `dict-set` `dict-del` `dict-keys` `dict-vals` `dict-pairs` `dict-has?` `dict-merge` `dict-size`

**Strings:** `fmt` `starts-with?` `ends-with?` `chars` `char-at` `pad-left` `pad-right` `split` `join` `upper` `lower` `trim` `replace`

**Math:** `abs` `min` `max` `clamp` `floor` `ceil` `round` `pi` `e` `rand` `rand-int`

**Types:** `type` `int?` `float?` `str?` `bool?` `vec?` `tuple?` `dict?` `fn?` `nil?`

**I/O:** `read-file` `write-file` `append-file` `file-exists?` `http-get` `http-post` `json-parse` `json-encode`

**System:** `time-ms` `sleep` `uuid` `env-get`

### Python Interop

Call any Python library directly from Tengwar:

```
(py-import "numpy")
(>> (py-call numpy "array" ⟦1 2 3 4 5⟧) → arr
    (py-call arr "mean"))              ;; → 3.0

(py-import "requests")
(>> (py-call requests "get" "https://api.github.com") → resp
    (py-attr resp "status_code"))      ;; → 200

;; Dot access works too
(>> (py-import "math")
    math.pi)                           ;; → 3.14159...
```

### Binary AST Protocol (TBAP)

**The killer AI feature.** Instead of generating text and parsing it, emit compact binary opcodes that map directly to AST nodes.

```python
from tengwar.binary_ast import ASTBuilder, encode_b64, run_b64

# AI builds programs structurally — ZERO syntax errors possible
b = ASTBuilder()
program = b.program(
    b.define("double", b.lam(["x"], b.binop("*", b.sym("x"), b.int(2)))),
    b.apply(b.sym("double"), b.int(21))
)
result = b.run(program)  # → 42

# Or encode to base64 for embedding in tool-call JSON
b64 = encode_b64("(reduce + 0 (map {* _ 2} ⟦1 2 3 4 5⟧))")
result = run_b64(b64)    # → 30
```

### Sandbox Mode

Safe execution with resource limits:

```python
from tengwar.interpreter import Interpreter

i = Interpreter(sandbox=True)
i.run_source("(reduce + 0 ⟦1 2 3 4 5⟧)")  # ✓ Pure computation

i.run_source('(read-file "/etc/passwd")')    # ✗ File I/O blocked
i.run_source('(http-get "http://evil.com")') # ✗ Network blocked
i.run_source('(py-import "os")')             # ✗ Python imports blocked

i.max_steps = 10000  # Prevents infinite loops
```

### Error Handling

```
(catch (+ 1 2) (λ e (fmt "Error: {}" e)))         ;; → 3
(catch (throw "oops") (λ e (fmt "Caught: {}" e)))  ;; → "Caught: oops"
(try {+ 1 1} {_})                                  ;; → (true, ...)
```

## Quick Reference

```
;; Define & functions
(≡ name value)             (λ x y (+ x y))         {+ _ 1}

;; Conditional & pattern match
(? cond then else)         (match val (0 "z") (_ "other"))

;; Let & pipe
(let x 10 y 20 (+ x y))   (pipe data sort head)

;; Collections
⟦1 2 3⟧                   ⟨1 2 3⟩                  (dict "a" 1 "b" 2)

;; Bind & sequence
(>> expr → name next)      (↺ f (λ n (f (- n 1))))

;; Parallel
(‖ (http-get url1) (http-get url2) (http-get url3))
```

## ASCII Mode

| Unicode | ASCII | Unicode | ASCII |
|---------|-------|---------|-------|
| `λ` | `fn` | `≡` | `def` |
| `→` | `->` | `⟦⟧` | `[]` |
| `⟨⟩` | tuple | `∅` | `nil` |
| `?` | `if` | `>>` | `do` |
| `↺` | `rec` | `‖` | `par` |

## v0.3 Changelog

- **80+ new builtins** — dictionaries, extended functional, strings, math, types, I/O
- **Let bindings** — `(let x 1 y 2 body)` local scope
- **Pipe operator** — `(pipe value f1 f2 f3)` data threading
- **Error handling** — `throw`, `catch`, `try` forms
- **Python interop** — `py-import`, `py-call`, `py-attr`, `py-eval`, dot access
- **HTTP/JSON/File I/O** — agent primitives for real-world tasks
- **Binary AST Protocol** — AI emits opcodes, zero syntax errors
- **ASTBuilder** — programmatic AST construction API
- **Sandbox mode** — safe execution with I/O, network, and step limits
- **Deep recursion** — 10,000+ recursive calls supported
- **200 tests** — comprehensive test suite

## License

MIT