Skill: mrchenkuan/large-document-reader (version 1.0.2)

--- name: large-document-reader description: | Intelligently splits long academic or technical documents into chapters, generates structured JSON summaries for each, and creates a file system with a global index. This enables efficient AI retrieval and analysis, perfectly solving context window limitations by enabling “overview via summaries, deep-dive on demand” workflows. version: 1.0.0 author: Document Assistant category: research tags: [document-processing, knowledge-management, summarization, ai-optimization] metadata: {} --- # Literature Structuring Expert Automatically decompose long documents (papers, reports, books) into a structured, AI-friendly knowledge base. Splits by chapter, generates machine-readable summaries, and builds a navigable index to overcome context limits. ## When to Use This Skill Use this skill when the user: - Has a document that is too long for the AI's context window. - Needs to perform cross-chapter analysis or get a high-level overview of a long text. - Wants to build a reusable, queryable knowledge base from a PDF, Markdown, or text file. - Asks: "How can I get my AI to read this whole book/paper?" ## Quick Reference | Situation | Action | |-----------|--------| | User provides a long document | 1. Analyze and split it into chapters.<br>2. Generate a JSON summary for each chapter.<br>3. Create a master index file. | | User asks a high-level, cross-chapter question | Provide the content of the `MASTER_INDEX.md` file to the AI. | | User asks a detailed, chapter-specific question | Provide the corresponding single file from the `./chapters/` directory to the AI. | | Task completed | Present the generated file tree and `MASTER_INDEX.md` preview to the user. | ## Core Workflow ### Phase 1: Intelligent Splitting 1. **Analyze Input**: Receive the long document text or file path. 2. **Identify Structure**: Automatically analyze the document to identify heading hierarchies (e.g., `#`, `##`, `1.`, `1.1`) to determine chapter boundaries. Prioritize user-specified splitting preferences. 3. **Execute Split**: Split the document into independent plain-text files by chapter. * **Naming Convention**: `{sequence_number}_{chapter_title}.md` (e.g., `01_Introduction.md`). * **Storage Location**: All chapter files are saved in the `./chapters/` directory. ### Phase 2: Summary Generation & Structuring 1. **Generate Summary per Chapter**: For each file in `./chapters/`, generate a corresponding JSON summary file. * **Structured Fields** (JSON format): ```json { "chapter_id": "Unique identifier matching the filename, e.g., 02_1", "chapter_title": "Chapter Title", "abstract": "Core summary of the chapter, 200-300 words.", "keywords": ["Keyword1", "Keyword2", "Keyword3"], "key_points": ["Key point one", "Key point two"], "related_sections": ["IDs of other chapters strongly related to this one"] } ``` * **Storage Location**: JSON summary files are saved in the `./summaries/` directory (e.g., `01_Introduction.summary.json`). ### Phase 3: Create Global Index 1. **Aggregate Information**: Collect data from all JSON files in `./summaries/`. 2. **Generate Index**: Create a global index file, `MASTER_INDEX.md`. * **Content**: Lists all chapters' IDs, titles, a short abstract preview, and keywords in a Markdown list or table. * **Purpose**: Provides a "bird's-eye view" for quick navigation and high-level Q&A. ## Final Deliverables & File Structure Upon completion, the following file tree is generated: ``` Project_Root/ ├── chapters/ # 【Source Repository】Contains all split chapter texts (.md files) │ ├── 01_Introduction.md │ ├── 02_1_Experimental_Methods.md │ └── ... ├── summaries/ # 【Summary Repository】Contains all structured JSON summaries │ ├── 01_Introduction.summary.json │ ├── 02_1_Experimental_Methods.summary.json │ └── ... └── MASTER_INDEX.md # 【Global Navigation】Core document summary index ``` ## Usage Instructions for the User **For Global, Cross-Chapter Queries** (e.g., “What is the paper's main thesis?”): * Provide the content of the `MASTER_INDEX.md` file to the AI. This is token-efficient. **For Specific, In-Depth Queries Within a Chapter** (e.g., “What were the parameters in the 'Methods' section?”): * Provide the corresponding single chapter file from the `chapters/` directory to the AI for full context.

path: _meta.json
content: ownerId: kn71sm4wd6xye7vdzs5b99n0bs81y6c6
slug: large-document-reader
version: 1.0.0
publishedAt: 1772181227260

path: scripts/extract_chapters.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
智能切分学术论文为章节文件
按照large-document-reader技能规范实现
"""

import re
import os
from pathlib import Path

def extract_chapters(file_path):
    """提取文档中的所有章节"""
    with open(file_path, 'r', encoding='utf-8') as f:
        lines = f.readlines()

    chapters = []
    current_section = None
    current_content = []

    for i, line in enumerate(lines):
        # 检查是否是章节标题
        # 匹配格式：1.1.1、1.1.2、一、1.1、1、(1)等
        stripped = line.strip()

        # 主章节：第1章、1、1.
        if re.match(r'^\s*(第\d+章|\d+\.\s+[^\s].*|^\d+\.\s*$)', stripped):
            # 保存前一个章节
            if current_section:
                chapters.append({
                    'start': current_section['start'],
                    'end': i,
                    'title': current_section['title'],
                    'content': current_section['content']
                })

            # 提取标题
            if re.match(r'^\s*(第\d+章)', stripped):
                title = re.sub(r'^\s*(第\d+章)\s*', '', stripped)
                title = re.sub(r'[\.\。、\s]+$', '', title)
            else:
                title = re.sub(r'^\s*(\d+)\.?\s*', '', stripped)
                title = re.sub(r'[\.\。、\s]+$', '', title)

            current_section = {
                'start': i,
                'title': title[:60],
                'content': []
            }

        # 子章节：1.1、1.2、1.1.1等
        elif re.match(r'^\s*\d+\.\d+\.\s+', stripped) or re.match(r'^\s*\d+\.\d+\.\d+\.\s+', stripped):
            # 保存前一个章节
            if current_section:
                chapters.append({
                    'start': current_section['start'],
                    'end': i,
                    'title': current_section['title'],
                    'content': current_section['content']
                })

            title = stripped
            parts = re.split(r'\s+', title)
            title = parts[0] if len(parts) > 0 else title
            title = re.sub(r'[\.\。、\s]+$', '', title)

            current_section = {
                'start': i,
                'title': title[:60],
                'content': []
            }

        # 中文小节：一、二、三、1.1等
        elif re.match(r'^\s*[一二三四五六七八九十]+[、\.]\s+', stripped) or re.match(r'^\s*(\d+\.\d+)\.?\s+', stripped):
            # 保存前一个章节
            if current_section:
                chapters.append({
                    'start': current_section['start'],
                    'end': i,
                    'title': current_section['title'],
                    'content': current_section['content']
                })

            title = re.sub(r'^\s*[一二三四五六七八九十]+[、\.]?\s*', '', stripped)
            title = re.sub(r'^\s*(\d+\.\d+)\.?\s*', r'\1', stripped)
            title = re.sub(r'[\.\。、\s]+$', '', title)

            current_section = {
                'start': i,
                'title': title[:60],
                'content': []
            }

        # 括号小节：(1)、(2)等
        elif re.match(r'^\s*\(\d+\)[\.、]\s*', stripped) or re.match(r'^\s*\(\d+\)\.?\s*', stripped):
            # 保存前一个章节
            if current_section:
                chapters.append({
                    'start': current_section['start'],
                    'end': i,
                    'title': current_section['title'],
                    'content': current_section['content']
                })

            title = re.sub(r'^\s*\(\d+\)[\.、]?\s*', '', stripped)
            title = re.sub(r'[\.\。、\s]+$', '', title)

            current_section = {
                'start': i,
                'title': title[:60],
                'content': []
            }

        # 继续添加内容
        elif current_section:
            current_section['content'].append(line)

    # 添加最后一个章节
    if current_section:
        chapters.append({
            'start': current_section['start'],
            'end': len(lines),
            'title': current_section['title'],
            'content': current_section['content']
        })

    return chapters

def main():
    file_path = "/Users/chenkuan/Desktop/毕业论文/规模与杠杆对银行系统性风险的影响研究_王怡涵.txt"

    print(f"正在读取文件: {file_path}")
    print("=" * 60)

    chapters = extract_chapters(file_path)

    print(f"\n总共找到 {len(chapters)} 个章节/小节")
    print("\n章节列表:")
    for idx, chapter in enumerate(chapters, 1):
        content_length = len(chapter['content'])
        print(f"{idx}. {chapter['title'][:50]} - 内容行数: {content_length}")

    return chapters

if __name__ == "__main__":
    chapters = main()

    # 保存到文件供后续使用
    with open('/Users/chenkuan/.openclaw/workspace/large-document-reader-1.0.0/chapters_info.json', 'w', encoding='utf-8') as f:
        import json
        json.dump(chapters, f, ensure_ascii=False, indent=2)

    print("\n章节信息已保存到 chapters_info.json")


path: scripts/save_chapters.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
保存章节文件
"""

import json
import os
import re

# 读取章节信息
with open('chapters_info.json', 'r', encoding='utf-8') as f:
    chapters = json.load(f)

# 读取原始文件
with open('/Users/chenkuan/Desktop/毕业论文/规模与杠杆对银行系统性风险的影响研究_王怡涵.txt', 'r', encoding='utf-8') as f:
    original_lines = f.readlines()

# 保存每个章节到chapters目录
for idx, chapter in enumerate(chapters, 1):
    # 清理标题
    title = chapter['title']
    title = re.sub(r'[^\w\u4e00-\u9fa5\-\s]', '', title)  # 只保留中文、英文、数字、连字符、空格
    title = re.sub(r'\s+', '_', title)  # 空格转为下划线
    title = title[:50]  # 限制长度

    # 生成文件名
    filename = f"{idx:02d}_{title}.md"
    filepath = os.path.join('chapters', filename)

    # 提取章节内容
    content_lines = chapter['content']
    content = ''.join(content_lines)

    # 保存文件
    with open(filepath, 'w', encoding='utf-8') as f:
        f.write(content)

    print(f"保存: {filename} ({len(content_lines)} 行)")

print(f"\n完成！共保存 {len(chapters)} 个章节文件到 chapters/ 目录")


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (HIGH)