wikly 0.2.1
A tool to export and analyze Wiki.js content with AI capabilities
Requires Python: >=3.8
Keywords: wiki, wiki.js, export, analyze, ai, style-guide, content-management

# Wikly

A command-line tool to export and analyze content from a Wiki.js instance using the GraphQL API. This is a **read-only** tool that will not make any changes to your Wiki.js content.

## Features

- Export pages with metadata and content from Wiki.js
- Multiple output formats (JSON, Markdown, HTML)
- Easy to use command-line interface
- Support for environment variables and configuration files
- Read-only operation (won't modify your wiki)
- Export content with original paths and hierarchy
- Content analysis with Gemini AI to ensure style guide compliance

## Installation

### Using pip

```bash
pip install wikly
```

### From source

```bash
git clone https://github.com/yourusername/wikly.git
cd wikly
pip install -e .
```

## Configuration

There are three ways to configure Wikly:

1. **Configuration File**: Use the `wikly init` command to generate a template configuration file.
2. **Command Line Options**: Pass options directly when running commands.
3. **Environment Variables**: Configure through environment variables.

The tool follows this precedence: Command Line > Config File > Environment Variables

### Configuration File

Run the following command to create a template configuration file:

```bash
wikly init
```

This creates a `wikly_config.yaml` file with the following structure:

```yaml
wikly:
  host: https://your-wiki-instance.com
  api_key: YOUR_API_KEY_HERE

export:
  default_format: markdown
  default_output: wiki_pages
  delay: 0.1
  metadata_file: .wikly_export_metadata.json

gemini:
  api_key: YOUR_GEMINI_API_KEY_HERE
```

You can specify a different path for the configuration file:

```bash
wikly init --path custom_config.yaml
```

### Environment Variables

Set the following environment variables:

* `WIKLY_HOST`: Base URL of your Wiki.js instance
* `WIKLY_API_KEY`: API token with appropriate permissions
* `GEMINI_API_KEY`: Google Gemini API key (optional, for analysis features)

A convenient way to manage these variables is to create a `.env` file in your working directory:

```
WIKLY_HOST=https://your-wiki-instance.com
WIKLY_API_KEY=your-api-token
GEMINI_API_KEY=your-gemini-api-key
```

The tool will automatically load these variables when run.

## Usage

All commands accept a `--config-file` option to specify a custom configuration file:

```bash
wikly <command> --config-file my_config.yaml
```

### Initialization

Generate a template configuration file:

```bash
wikly init
```

This command creates three files:
1. `wikly_config.yaml` - Main configuration file
2. `wiki_style_guide.md` - Sample style guide for content analysis
3. `ai_instructions.md` - AI-specific instructions for content analysis

Options:
* `--path`: Specify a custom location for the configuration file (default: wikly_config.yaml)
* `--force`: Force overwrite if the files already exist

The configuration file includes settings for Wiki.js connection, export options, and AI analysis parameters.

### Test Connection

Verify your Wiki.js connection:

```bash
wikly test
```

Options:
* `--url`: Override the Wiki.js URL
* `--token`: Override the API token
* `--config-file`: Path to custom configuration file

### List Pages

List all pages in your Wiki.js instance:

```bash
wikly list
```

Options:
* `--url`: Override the Wiki.js URL
* `--token`: Override the API token
* `--config-file`: Path to custom configuration file

### Export Pages

Export all pages from Wiki.js:

```bash
wikly export
```

Options:
* `--url`: Override the Wiki.js URL
* `--token`: Override the API token
* `--output`: Custom output location
* `--delay`: Adjust request delay (seconds)
* `--debug`: Enable debug output
* `--format`: Choose output format (json, markdown, html)
* `--incremental/--full`: Enable/disable incremental export (default: incremental)
* `--force-full`: Force a full export
* `--reset-hashes`: Reset all content hashes (forces recomputing)
* `--metadata-file`: Custom location for metadata file
* `--config-file`: Path to custom configuration file

### Analyze Content

Run semantic analysis on exported content:

```bash
wikly analyze
```

This command uses the Gemini AI to analyze Wiki.js content against a style guide, identifying issues and suggesting improvements.

Options:
* `--format`: Input format (json, markdown)
* `--output`: Custom output location
* `--input`: Input file or directory
* `--api-key`: Google Gemini API key
* `--style-guide`: Custom path to style guide file
* `--ai-guide`: Custom path to AI instructions file
* `--config-file`: Path to custom configuration file

The analysis process:
1. Loads content from the specified source (JSON or Markdown files)
2. Reads the style guide and AI instructions (created by `wikly init`)
3. Analyzes each page for style compliance
4. Generates a report with issues and suggestions

Results include:
- A summary of each page's compliance with the style guide
- Specific discrepancies with location and severity
- Suggested corrections for each issue
- An overall compliance score for each page

### Generate Report

Generate an HTML report from existing analysis results:

```bash
# Using an explicit input file
wikly report analysis_results.json

# Using configuration defaults
wikly report
```

This command takes previously generated analysis results (JSON) and creates a visual HTML report without needing to re-run the analysis.

Options:
* `--output`, `-o`: Custom output path for the HTML report (default: analysis_report.html)
* `--style-guide`: Path to style guide file to include in the report
* `--config-file`: Path to custom configuration file

If you don't specify an input file, the command will use the default paths from your configuration. This means you can simply run `wikly report` after running `wikly analyze` to generate an HTML report from the latest analysis results.

This is useful when:
- You want to generate a report with different formatting
- You need to share results with team members
- You've manually edited the analysis results
- You want to create multiple report versions from the same analysis

#### Style Guide and AI Instructions

The `wikly init` command creates two files for content analysis:

1. **Style Guide** (`wiki_style_guide.md`): Contains human-readable guidelines for writing wiki content. This is the primary reference for what "good" content looks like.

2. **AI Instructions** (`ai_instructions.md`): Contains instructions specifically for the AI analyzer, such as how to prioritize issues, what context to consider, and special analysis requirements.

You can customize both files to match your organization's style requirements and content standards.

### Listing Pages (Metadata Only)

To fetch and save a list of all pages (without content):

```bash
wikly list --output wiki_pages.json
```

### Exporting Pages with Content

To export all pages with their full content:

```bash
wikly export --output wiki_export.json
```

By default, the exporter uses incremental mode, which only fetches content for pages that have been updated since the last export. This significantly speeds up subsequent exports.

The incremental export also detects local changes to exported files. If you modify a file after exporting it, the exporter will detect the change and re-fetch the content from Wiki.js during the next export.

To force a full export of all pages:

```bash
wikly export --force-full
```

#### Export Formats

You can export in different formats using the `--format` option:

```bash
# Export as JSON (default)
wikly export --format json

# Export as Markdown files
wikly export --format markdown --output wiki_markdown

# Export as HTML files
wikly export --format html --output wiki_html
```

#### Additional Export Options

```bash
# Set delay between API requests
wikly export --delay 0.5

# Toggle between incremental and full exports
wikly export --incremental  # Default, only fetches updated content
wikly export --full         # Fetches all content

# Force a full export regardless of other settings
wikly export --force-full

# Reset all content hashes (useful if having issues with local change detection)
wikly export --reset-hashes

# Specify a custom metadata file location
wikly export --metadata-file /path/to/metadata.json

# Enable verbose debugging output
wikly export --debug
```

The exporter tracks metadata about previous exports in a `.wikly_export_metadata.json` file, including:
- The last update time for each page
- Content hashes to detect local modifications
- Original paths and titles from Wiki.js

This allows the exporter to intelligently decide which pages need to be re-fetched during incremental exports, based on both server-side updates and local file changes.

##### Handling Edited Files

When you edit a file locally after exporting it, the exporter will detect the changes during the next export by comparing content hashes. There are three possible outcomes:

1. **Re-fetch the page**: By default, the exporter will detect local changes and re-fetch the page from Wiki.js.
2. **Keep local changes**: You can manually update the metadata file to match your local changes.
3. **Force reset all hashes**: Use `--reset-hashes` option to force recomputing all content hashes.

For complex workflows with many local edits, you may want to set up version control on your exported files.

### Analyzing Content for Style Compliance

The `analyze` command lets you check your wiki content against a style guide using Google's Gemini AI:

```bash
wikly analyze path/to/exported/content style_guide.md
```

This will:
1. Process all Markdown and HTML files in the specified directory
2. Compare each file against the provided style guide
3. Generate a detailed report of discrepancies and suggestions
4. Save both raw results (JSON) and a readable HTML report

#### Incremental Analysis

By default, the analyze command uses incremental mode, which only analyzes pages that have changed since the last analysis. This significantly improves performance for large wikis:

```bash
# Incremental analysis (default)
wikly analyze --incremental

# Force full analysis of all pages
wikly analyze --full

# Force a full analysis regardless of other settings
wikly analyze --force-full

# Reset all content hashes (useful if having issues with local change detection)
wikly analyze --reset-hashes

# Specify a custom metadata file location
wikly analyze --metadata-file /path/to/metadata.json
```

The analyzer tracks metadata about previous analyses in a `.wikly_analysis_metadata.json` file, including:
- Content hashes to detect file changes
- Analysis timestamps for each page
- Issue counts and compliance scores

This allows the analyzer to intelligently decide which pages need to be re-analyzed, based on both content changes and previous analysis results.

#### Additional Options

```bash
# Set a custom output location for results
wikly analyze content_dir style_guide.md --output analysis.json --report report.html

# Use a specific Gemini model
wikly analyze content_dir style_guide.md --model gemini-1.5-pro

# Add delay between API calls to avoid rate limits
wikly analyze content_dir style_guide.md --delay 2.0

# Provide a separate AI-specific guidance file
wikly analyze content_dir style_guide.md --ai-guide ai_specific_guide.md

# Enable debug output
wikly analyze content_dir style_guide.md --debug
```

#### AI Guide

You can optionally provide an AI-specific guidance file that contains instructions specifically for the AI analyzer, separate from the human-readable style guide. This allows you to:

- Give more technical instructions to the AI without cluttering the human style guide
- Provide examples of correct and incorrect content for better AI understanding
- Add contextual information that helps the AI make better judgments

Example usage:
```bash
wikly analyze content_dir human_style_guide.md --ai-guide ai_specific_instructions.md
```

#### Rate Limiting Protection

The tool implements several strategies to handle Gemini API rate limits:

- Configurable delay between file processing (use `--delay` option)
- Random jitter added to delays to prevent synchronized requests
- Exponential backoff for 429 (Too Many Requests) errors
- Automatic retries when rate limits are hit (up to 5 attempts)

These features help ensure your analysis completes successfully even with large content sets.

#### Listing Available Models