cot-forge 0.1.0
A package for generation of chain-of-thought datasets.
License: MIT
Requires Python: >=3.10

<p align="center"><img src="./assets/cot-forge_logo.png" alt="CoT Forge Logo" width="500"></p>

# CoT Forge ✨

A Python library for generating high-quality Chain of Thought (CoT) ⛓️ reasoning data for training and fine-tuning large language models. 🧠

![Version](https://img.shields.io/badge/version-0.1.0-blue)
![Python](https://img.shields.io/badge/python-≥3.10-blue)
![License](https://img.shields.io/badge/license-MIT-green)

## Overview

CoT Forge helps you create synthetic training data that includes complex reasoning chains, enabling LLMs to learn more robust and transparent reasoning capabilities. 🚀 Inspired by research like [HuatuoGPT-o1](https://github.com/FreedomIntelligence/HuatuoGPT-o1), this library implements a flexible framework for:

* Creating verifiable question/answer pairs 🎯
* Finding optimal reasoning paths through tree search algorithms 🌳
* Reformatting reasoning chains into natural language ✍️

## Installation

CoT Forge is compatible with Python 3.10 and above. You can install it via pip or clone the repository directly. ⬇️

```bash
# Install using pip
pip install cot-forge

# Or directly from the repository
pip install git+https://github.com/MattSamach/cot-forge
```

## Quick Start 🏃‍♀️
### Example Usage
The following example demonstrates how to use CoT Forge to generate reasoning data for a complex scientific question. The library supports multiple LLM providers, including OpenAI, Anthropic Claude, and Google Gemini. It implements the Naive Linear Search strategy for finding reasoning paths and uses the LLM Judge Verifier to ensure the correctness of the answer generated by the search LLM.

```python
from cot_forge.llm import GeminiProvider, LMStudioProvider
from cot_forge.reasoning import CoTBuilder, NaiveLinearSearch
from cot_forge.reasoning.verifiers import LLMJudgeVerifier
from pprint import pprint

# Initialize LLM providers
gemini = GeminiProvider(api_key="your-api-key")
llama = LMStudioProvider(model_name="meta-llama-3-8b-instruct")

# Setup CoT builder with essential components
builder = CoTBuilder(
    search_llm=gemini,
    post_processing_llm=llama,
    search=NaiveLinearSearch(max_depth=3),
    verifier=LLMJudgeVerifier(gemini, strict=False)
)
# Example question with complex scientific reasoning
question = "In a quantum computing system with 8 qubits in a maximally entangled state, if environmental decoherence causes the system to lose quantum information at a rate of 10% per microsecond, how long will it take for the system's von Neumann entropy to reach 90% of its maximum possible value? Assume the system starts in a pure state with zero entropy."

# Example ground truth answer
ground_truth = "It would take approximately 23.03 microseconds for the system's von Neumann entropy to reach 90% of its maximum possible value."

search_result, reasoning = builder.process(
    question=question,
    ground_truth_answer=ground_truth
)
```
### Search Result
```python
# Print the search result
print(search_result)
```
```output
SearchResult(success=True, question=In a quantum computing system ..., num_terminal_nodes=1, num_successful_nodes=1, successful_answers=["Based on a simplified model where the off-diagonal elements of the density matrix decay exponentially with a characteristic time constant estimated from the 10% loss of quantum information, it will take approximately 21.95 microseconds for the system's von Neumann entropy to reach 90% of its maximum possible value. This result is still based on simplifying assumptions and approximations. A more accurate model would require a more detailed description of the decoherence process."])
```
We can see that the `SearchResult` is successful and has at least one successful reasoning path. Let's investigate this reasoning path further.

### Node Chain
Let's access the reasoning path for the first successful node. The `get_full_node_chain` method returns a list of nodes that represent the reasoning path taken to arrive at the answer.
```python
successful_node = search_result.get_successful_terminal_nodes()[0]

# Print the strategies for each node in the reasoning path
for idx, node in enumerate(successful_node.get_full_node_chain()):
    print(f"Node: {idx}: {node.strategy.name}")
```
```output
Node: 0: initialize
Node: 1: correction
Node: 2: explore_new_paths
```

### Full Chain of Thought
The `get_full_cot` method returns the full chain of thought for the successful node as a dictionary. This includes the reasoning steps taken to arrive at the answer. The full chain of thought is quite long in this case, so we'll just look at the first and last few steps.
```python
# Print the first two steps of the full chain of thought
pprint(successful_node.get_full_cot()[:2])
```
```output
[{'action': 'Inner Thinking',
  'content': 'For an 8-qubit system, the maximum possible von Neumann entropy '
             'occurs when the system is in a completely mixed state. In this '
             'case, each qubit is equally likely to be in state |0> or |1>. '
             'The maximum entropy is given by S_max = n * ln(2), where n is '
             'the number of qubits. So, S_max = 8 * ln(2) ≈ 5.545.',
  'title': 'Maximum Entropy Calculation'},
 {'action': 'Inner Thinking',
  'content': 'We want to find the time it takes for the entropy to reach 90% '
             'of its maximum value. So, the target entropy is S_target = 0.9 * '
             'S_max = 0.9 * 8 * ln(2) ≈ 4.990.',
  'title': 'Target Entropy Calculation'}]
```

```python
# Print the last two three of the full chain of thought
pprint(successful_node.get_full_cot()[-3:])
```

```output
[{'action': 'Inner Thinking',
  'content': 'We want to find t such that \\(S(t) = 0.9 * S_{max}\\). So, '
             '\\(0.9 * S_{max} = S_{max} * (1 - e^{-t/T})\\). This simplifies '
             'to \\(0.9 = 1 - e^{-t/T}\\), which means \\(e^{-t/T} = 0.1\\). '
             'Taking the natural logarithm of both sides, we get \\(-t/T = '
             'ln(0.1)\\), so \\(t = -T * ln(0.1)\\).',
  'title': 'Solving for Time'},
 {'action': 'Inner Thinking',
  'content': 'Substituting our estimate for T, we have \\(t = -(-1/ln(0.9)) * '
             'ln(0.1) = (ln(0.1) / ln(0.9)) \\approx 21.95\\) microseconds.',
  'title': 'Final Calculation'},
 {'action': 'Final Conclusion',
  'content': 'Based on a simplified model where the off-diagonal elements of '
             'the density matrix decay exponentially with a characteristic '
             'time constant estimated from the 10% loss of quantum '
             'information, it will take approximately 21.95 microseconds for '
             "the system's von Neumann entropy to reach 90% of its maximum "
             'possible value. This result is still based on simplifying '
             'assumptions and approximations. A more accurate model would '
             'require a more detailed description of the decoherence process.'}]
```

### Natural Language Reformatting
Finally, we can see the natural language reformatting of the reasoning. This is useful for training data that is more interpretable and easier to understand. The <thinking> tag indicates the start of the reasoning process. The final conclusion comes after the tag.
```python
# Print the reasoning in natural language
natural_language_reasoning = reasoning['chain_of_thought_responses'][0]
print(natural_language_reasoning[:650])
```
```output
<thinking>Okay, so we have 8 qubits, and they're all entangled, which is cool. Decoherence is messing things up at 10% per microsecond, and we want to know when the entropy hits 90% of its max.

First, let's figure out the maximum entropy. Each qubit can be completely mixed, meaning it's equally likely to be 0 or 1. The entropy for one completely mixed qubit is ln(2). Since we have 8 qubits, the maximum entropy is 8 * ln(2). Okay, got that.

Now, 90% of that max entropy is 0.9 * 8 * ln(2). That's our target entropy.

Hmm, how does decoherence increase the entropy? It's not super obvious. The system loses quantum information, so it becomes mor
```
Check the end of the natural language reasoning for the final conclusion.
```python
print(natural_language_reasoning[-500:])
```
```output
roseconds for the entropy to reach 90% of its maximum. Still a lot of approximations here though!</thinking>

Based on an exponential decay model for decoherence and the given information loss rate, it will take approximately 21.95 microseconds for the system's von Neumann entropy to reach 90% of its maximum possible value. This estimate relies on the assumption that the decoherence process leads to an exponential decay of the initial quantum information and a corresponding increase in entropy.
```

## Core Features ⚙️

* **Multiple LLM Providers**: Support for OpenAI, Anthropic Claude, and Google Gemini models
* **Flexible Search Strategies**: Choose from beam search, naive linear, and more
* **Quality Verification**: Built-in verifiers to ensure reasoning is correct
* **Result Scoring**: Various scoring methods to select the best reasoning paths
* **Natural Language Reformatting**: Convert structured reasoning into natural language for fine tuning
* **Persistence**: Save and resume reasoning generation for large datasets
* **Extensibility**: Easily add custom reasoning strategies, verifiers, and more

## Documentation 📚

For detailed documentation, see the [docs folder](./docs/):

- [Core Concepts](./docs/core-concepts.md)
- [Strategies](./docs/strategies.md)
- [Reasoning Nodes](./docs/reasoning-nodes.md)
- [Search Algorithms](./docs/search-algorithms.md)
- [Naive Linear Search](./docs/naivelinearsearch.md)
- [Beam Search](./docs/beamsearch.md)
- [LLM Providers](./docs/llm-providers.md)
- [Verification](./docs/verification.md)
- [Persistence](./docs/persistence.md)
- [Examples](./examples/README.md)

## License 📝
This project is licensed under the MIT License - see the [LICENSE](./LICENSE) file for details.

## Contributing 🤝

Please see [CONTRIBUTING](./CONTRIBUTING.md) for guidelines on contributing to this project.