npm package: @kt3k/tku
Description: `@kt3k/tku` is a CLI tool that counts the total number of tokens in a git repository. It uses [tiktoken](https://www.npmjs.com/package/tiktoken) to tokenize file contents and reports the token count per file and in total.
Author: Yoshiya Hinosawa
Latest version: 1.2.4

# @kt3k/tku — Git Repository Token Counter

## Overview

`@kt3k/tku` is a CLI tool that counts the total number of tokens in a git
repository. It uses [tiktoken](https://www.npmjs.com/package/tiktoken) to
tokenize file contents and reports the token count per file and in total.

## Motivation

When working with LLMs, knowing the token size of a codebase helps estimate
context window usage, cost, and whether a repository (or subset) fits within
model limits. `tku` provides a quick, accurate measurement directly from the
command line.

## Usage

```sh
npx @kt3k/tku [options] [path]
```

- `path` — Path to a git repository (defaults to current directory).

### Options

| Flag                      | Description                                                                                                            | Default              |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `--encoding <encoding>`   | Tiktoken encoding (e.g. `o200k_base`, `cl100k_base`)                                                                   | `o200k_base`         |
| `-e, --exclude <glob...>` | Additional glob patterns to exclude                                                                                    | none                 |
| `--no-gitignore`          | Do not respect `.gitignore` rules                                                                                      | respect `.gitignore` |
| `--json`                  | Output results as JSON                                                                                                 | false                |
| `--top <n>`               | Show only the top N files by token count                                                                               | show all             |
| `--sort <field>`          | Sort by `tokens` or `path`                                                                                             | `tokens`             |
| `-t, --tree`              | Display results as a directory tree (ignores `--top`, `--json`, `--sort`)                                              | false                |
| `--dirs`                  | Summarize by directory. With `--tree`, shows only directories. Without `--tree`, shows a flat directory summary table. | false                |

### Examples

```sh
# Count tokens in the current repo
npx @kt3k/tku

# Count tokens using cl100k_base encoding, show top 20 files
npx @kt3k/tku --encoding cl100k_base --top 20

# JSON output for scripting
npx @kt3k/tku --json

# Exclude test files
npx @kt3k/tku --exclude "**/*.test.*" --exclude "**/fixtures/**"

# Directory tree view
npx @kt3k/tku --tree

# Directory tree, directories only
npx @kt3k/tku --tree --dirs

# Flat directory summary
npx @kt3k/tku --dirs
```

## Output

### Default (human-readable)

```
tokens  path
 1.2 K  src/index.ts
   892  src/utils.ts
   345  README.md
──────
 2.4 K  total (3 files)
```

### JSON (`--json`)

```json
{
  "encoding": "o200k_base",
  "files": [
    { "path": "src/index.ts", "tokens": 1204 },
    { "path": "src/utils.ts", "tokens": 892 },
    { "path": "README.md", "tokens": 345 }
  ],
  "totalTokens": 2441,
  "totalFiles": 3
}
```

### Tree (`--tree`)

```
tokens  path
 2.4 K  .
 2.1 K  ├── src/
 1.2 K  │   ├── index.ts
   892  │   └── utils.ts
   345  └── README.md
```

### Tree, directories only (`--tree --dirs`)

```
tokens  path
 2.4 K  .
 2.1 K  └── src/
```

### Directory summary (`--dirs`)

```
tokens  path
 2.1 K  src
────────
 2.4 K  total (3 files)
```

## License

MIT