Package: autologic
autologicautologic is a Python package that implements the SELF-DISCOVER framework proposed in the paper"SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures".It provides a way for large language models (LLMs) to automatically compose modular reasoning structures to tackle complex reasoning tasks, without the need for training data or labels.Key FeaturesImplements the full SELF-DISCOVER pipeline enabling LLMs to self-discover reasoning structuresWorks with Gemini Pro and local GGUF models via llama-cpp-pythonFlexible integration powered by a simple LLMConfigInteractive prompts or standalone executionCLI and Python API accessFramework OverviewThe SELF-DISCOVER framework consists of two key stages:Stage 1: Self-discover a reasoning structure for the task from a set of seed "reasoning modules"Stage 2: Solve instances by following the composed structureThe first stage has 3 steps guided by meta-prompting:SELECT relevant reasoning modulesADAPT modules to be task-specificIMPLEMENT structure with adapted modulesGetting StartedInstall from Pypipython3-mvenvvenvsourcevenv/bin/activateCMAKE_ARGS="-DLLAMA_METAL=on"python3-mpipinstallautologicInstallation as editable packageThe following instructions show the CMAKE arguments for compiling llama-cpp-python dependencies with support for metal.
Refer tohttps://github.com/abetlen/llama-cpp-pythonfor instructions regarding different GPUs.gitclonehttps://github.com/waszumteufel/autologic.gitcdautologic
python3-mvenvvenvsourcevenv/bin/activateCMAKE_ARGS="-DLLAMA_METAL=on"python3-mpipinstall-e.Directly install from Githubpython3-mvenvvenvsourcevenv/bin/activateCMAKE_ARGS="-DLLAMA_METAL=on"python3-mpipinstallgit+https://github.com/waszumteufel/autologic@main#egg=autologicNote on CMAKE_ARGSThe CMAKE_ARGS environment variable is used to direct llama-cpp-python to be compiled with specific support for GPU drivers for hardware acceleration.
Below is a non-exhaustive list of common values.Metal (MPS)For Macs with the M-series processor.CMAKE_ARGS="-DLLAMA_METAL=on"VulkanCMAKE_ARGS="-DLLAMA_VULKAN=on"OpenBLASCMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS"cuBLASCMAKE_ARGS="-DLLAMA_CUBLAS=onhipBLASFor AMD CardsCMAKE_ARGS="-DLLAMA_HIPBLAS=on"ConfigurationGemini API KeyTo use the Gemini model, set theGEMINI_PRO_API_KEYvariable:GEMINI_PRO_API_KEY="sk-..."If this is not set,solve()will automatically read it from the.envfile in the working directory or general environment.OpenAI API KeyTo use the OpenAI models, set theAUTOLOGIC_OPENAI_API_KEYvariable:AUTOLOGIC_OPENAI_API_KEY="sk-..."If this is not set,solve()will automatically read it from the.envfile in the working directory or general environment.Environment VariablesIt is recommended to set sensitive values like API keys in a.envfile.This will be automatically loaded from the current working directory if present, and used to populate any unset configuration values that have matching environment variable names (i.e.GEMINI_PRO_API_KEY,AUTOLOGIC_OPENAI_API_KEY).See the "Configuration" section of the documentation for the full set of options. Using.envand environment variables allows keeping credentials secure.This allows conveniently configuring API keys and other settings without exposing them directly in code. TheLLMConfigcan then be initialized without explicitly passing these values every time.UsageImport and call the top-level solve() method:Mixtral Example with the Python APIThe below code block illustrates how to use the Python API to use SELF-DISCOVER with a Mixtral GGUF.fromautologicimportreasoningEnginemy_config=reasoningEngine.LLMConfig(gguf_path="/tmp/mixtral-8x7b-instruct-v0.1.Q8_0.gguf",context_length=8000,model_type=reasoningEngine.ModelType.LOCAL,temp=0.2,chat_template=reasoningEngine.ChatTemplate.MIXTRAL_INSTRUCT,threads=12)result=reasoningEngine.solve(task="What is 2 + 2?",discover_config=my_config)print(result)Gemini Pro Example with the Python APIThe API Key for Gemini Pro is read from theGEMINI_PRO_API_KEYenvironment variable or .env file. It can optionally be passed in through theautologic.reasoningEngine.LLMConfig.api_keyfield.fromautologicimportreasoningEnginemy_config=reasoningEngine.LLMConfig(context_length=8000,model_type=reasoningEngine.ModelType.GEMINI,temp=0.2,)result=reasoningEngine.solve(task="What is 2 + 2?",discover_config=my_config)print(result)OpenAI Example with the Python APIThe API Key for OpenAI is read from theAUTOLOGIC_OPENAI_API_KEYenvironment variable or .env file. It can optionally be passed in through theautologic.reasoningEngine.LLMConfig.api_keyfield.fromautologicimportreasoningEnginellmConfig=reasoningEngine.LLMConfig(model_type=reasoningEngine.ModelType.OPENAI,model_name="gpt-3.5-turbo-0125",# You can use any model name available to you through openai - omitting the model_name will default to gpt-3.5-turbo-0125temp=0.2,context_length=2000)problem_task="Beth and Sam are 500 miles apart. If Beth travels at 60mph and leaves her house at 1pm, what time will she arrive at Sam's house?"# 9:20PManswer=reasoningEngine.solve(task=problem_task,verbose=True,discover_config=llmConfig)Mixtral Solving a prompt with a reasoning Structure generated by OpenAI's GPT4fromautologicimportreasoningEnginesolve_config=reasoningEngine.LLMConfig(gguf_path="/tmp/mixtral-8x7b-instruct-v0.1.Q8_0.gguf",context_length=8000,model_type=reasoningEngine.ModelType.LOCAL,temp=0.2,chat_template=reasoningEngine.ChatTemplate.MIXTRAL_INSTRUCT,threads=12)discover_config=reasoningEngine.LLMConfig(model_type=reasoningEngine.ModelType.OPENAI,# You can use any model name available to you through openai - omitting the model_name will default to gpt-3.5-turbo-0125model_name="gpt-4-turbo-preview",temp=0.2,context_length=2000)prompt="Beth and Sam are 500 miles apart. If Beth travels at 60mph and leaves her house at 1pm, what time will she arrive at Sam's house?"# 9:20PMresult=reasoningEngine.solve(task=prompt,discover_config=discover_config,solve_config=solve_config,verbose=True)print(result)Mixtral Solving a prompt with a reasoning Structure generated by Google's Gemini ProfromautologicimportreasoningEnginesolve_config=reasoningEngine.LLMConfig(gguf_path="/Users/kerekovskik/.cache/lm-studio/models/misc/manual_dl/mixtral-8x7b-instruct-v0.1.Q8_0.gguf",context_length=8000,model_type=reasoningEngine.ModelType.LOCAL,temp=0.2,chat_template=reasoningEngine.ChatTemplate.MIXTRAL_INSTRUCT,threads=12)discover_config=reasoningEngine.LLMConfig(model_type=reasoningEngine.ModelType.GEMINI,temp=0.2,context_length=2000)prompt="Beth and Sam are 500 miles apart. If Beth travels at 60mph and leaves her house at 1pm, what time will she arrive at Sam's house?"# 9:20PMresult=reasoningEngine.solve(task=prompt,discover_config=discover_config,solve_config=solve_config,verbose=True)print(result)Or use the CLI:CLI Usage with a promptGemini Pro example with CLIExample with a promptautologicgemini--temp0.2\--retries3\--prompt"What weighs more? A pound of feathers or a pound of lead?"Example outputThinking...
2024-02-14 18:50:34.054248 | Starting SELECT Phase
2024-02-14 18:50:37.008261 | SELECT Phase Complete
2024-02-14 18:50:37.008367 | Starting ADAPT Phase
2024-02-14 18:50:40.322604 | ADAPT Phase Complete
2024-02-14 18:50:40.322641 | Starting IMPLEMENT Phase
2024-02-14 18:50:45.567405 | IMPLEMENT Phase Complete
2024-02-14 18:50:45.567738 | Starting to Solve Problem using Reasoning Structure
2024-02-14 18:50:52.775653 | Solution has been found.


ANSWER: The weight of a pound of feathers is equal to the weight of a pound of lead.OpenAI example with CLIExample with a promptautologicopenai--temp0.2\--retries3\--prompt"What weighs more? A pound of feathers or a pound of lead?"Example outputThinking...
2024-02-14 18:50:34.054248 | Starting SELECT Phase
2024-02-14 18:50:37.008261 | SELECT Phase Complete
2024-02-14 18:50:37.008367 | Starting ADAPT Phase
2024-02-14 18:50:40.322604 | ADAPT Phase Complete
2024-02-14 18:50:40.322641 | Starting IMPLEMENT Phase
2024-02-14 18:50:45.567405 | IMPLEMENT Phase Complete
2024-02-14 18:50:45.567738 | Starting to Solve Problem using Reasoning Structure
2024-02-14 18:50:52.775653 | Solution has been found.


ANSWER: The weight of a pound of feathers is equal to the weight of a pound of lead.Qwen1.5 72B example with CLIThe below example shows how to specify a path to a GGUF model and how to specify the prompt format for the given model with the--formatflag.
The example also showcases the usage of the--verboseflag which shows detailed reasoning information as well as the reasoning structure that the LLM self-discovers for the given problem.autologiclocal--temp0.2\--retries5\--prompt"What weighs more? A pound of feathers or a pound of lead?"\--gguf_path/tmp/qwen1_5-72b-chat-q5_k_m.gguf\--threads12\--layers-1\--formatchatml\--verboseExample output:Thinking...
2024-02-14 20:05:23.116629 | Starting SELECT Phase
2024-02-14 20:06:41.595659 | SELECT Phase Complete
2024-02-14 20:06:41.595711 | Reasoning Modules Picked:
- 3. How can I simplify the problem so that it is easier to solve?
- 9. Critical Thinking: This style involves analyzing the problem from different perspectives, questioning assumptions, and evaluating the evidence or information available. It focuses on logical reasoning, evidence-based decision-making, and identifying potential biases or flaws in thinking.
- 16. What are the underlying causes or factors contributing to the problem?
- 22. How can progress or success in solving the problem be measured or evaluated?
- 23. What indicators or metrics can be used?

2024-02-14 20:06:41.595719 | Starting ADAPT Phase
2024-02-14 20:07:22.370683 | ADAPT Phase Complete
2024-02-14 20:07:22.370713 | Task-specific Reasoning Module verbiage:
- Clarify the units and equalities: Both items are measured in pounds, which means they weigh the same amount by definition; a pound is a unit of weight or mass.
- Understand the context and potential misconceptions: The question is a classic example of a trick question, as some may think lead is denser and therefore heavier, but in reality, both weigh the same.
- Address the physical properties without affecting the weight: While lead is denser than feathers, density does not affect the weight for the same volume; both weigh one pound, regardless of their density.

2024-02-14 20:07:22.370720 | Starting IMPLEMENT Phase
2024-02-14 20:08:46.753718 | IMPLEMENT Phase Complete
2024-02-14 20:08:46.753805 | Reasoning Structure:
{
  "Reasoning Structure": {
    "Step 1: Understand the Weighing Units": {
      "Description": "Both items are measured in the same unit.",
      "Action": "Confirm that both 'pound of feathers' and 'pound of lead' use the same unit (pound).",
      "Unit": "pound"
    },
    "Step 2: Recognize the Trick Question": {
      "Description": "The question is designed to prompt a misconception.",
      "Action": "Identify that some might think density affects weight, but it doesn't for the same volume.",
      "Misconception": "Density affects weight"
    },
    "Step 3: Define Density": {
      "Description": "Density is mass per unit volume.",
      "Action": "Explain that density does not change the weight for the same amount of mass.",
      "Density": "mass / volume"
    },
    "Step 4: Compare Weights Without Density": {
      "Description": "Focus on the weight aspect.",
      "Action": "Since both weigh one pound, compare them without considering density.",
      "Comparison": "Equal weight"
    },
    "Step 5: Conclusion": {
      "Description": "Both weigh the same amount.",
      "Action": "Both a pound of feathers and a pound of lead weigh the same.",
      "Final Conclusion": ""
    },
    "FINAL_ANSWER": ""
  }
}
2024-02-14 20:08:46.753961 | Starting to Solve Problem using Reasoning Structure
2024-02-14 20:10:07.283210 | Problem Solved
Completed Reasoning Structure:
{
  "Reasoning Structure": {
    "Step 1: Understand the Weighing Units": {
      "Description": "Both items are measured in the same unit.",
      "Action": "Confirm that both 'pound of feathers' and 'pound of lead' use the same unit (pound).",
      "Unit": "pound"
    },
    "Step 2: Recognize the Trick Question": {
      "Description": "The question is designed to prompt a misconception.",
      "Action": "Identify that some might think density affects weight, but it doesn't for the same volume.",
      "Misconception": "Density affects weight"
    },
    "Step 3: Define Density": {
      "Description": "Density is mass per unit volume.",
      "Action": "Explain that density does not change the weight for the same amount of mass.",
      "Density": "mass / volume"
    },
    "Step 4: Compare Weights Without Density": {
      "Description": "Focus on the weight aspect.",
      "Action": "Since both weigh one pound, compare them without considering density.",
      "Comparison": "Equal weight"
    },
    "Step 5: Conclusion": {
      "Description": "Both weigh the same amount.",
      "Action": "Both a pound of feathers and a pound of lead weigh the same.",
      "Final Conclusion": "A pound of feathers and a pound of lead weigh equally."
    },
    "FINAL_ANSWER": "A pound of feathers and a pound of lead weigh equally."
  }
}
2024-02-14 20:10:07.283302 | Solution has been found.


ANSWER: A pound of feathers and a pound of lead weigh equally.Mixed Inference mode via the CLI - Qwen1.5 solving a prompt with a Gemini Pro Reasoning Structureautologicmixed\--solve-temp0.2\--solve-gguf_path$HOME/hf/qwen1_5-72b-chat-q5_k_m.gguf\--solve-threads12\--solve-layers-1\--solve-formatchatml\--solve-model_typelocal\--discover-model_typegemini\--discover-temp0.2\--prompt"What weighs more? A pound of feathers or a pound of lead?"\--verbose\--retries5Mixed Inference mode via the CLI - Qwen1.5 solving a prompt with a GP4 Reasoning Structureautologicmixed\--solve-temp0.2\--solve-gguf_path$HOME/hf/qwen1_5-72b-chat-q5_k_m.gguf\--solve-threads12\--solve-layers-1\--solve-formatchatml\--solve-model_typelocal\--discover-model_typeopenai\--discover-model_name'gpt-4-turbo-preview'\--discover-temp0.2\--prompt"What weighs more? A pound of feathers or a pound of lead?"\--verbose\--retries5Interactive CLI ModeThe CLI can be invoked without passing a--promptargument to enter an interactive mode. This allows conveniently sending multi-line prompts for the model to reason over:%autologicgemini--temp0.2

EnteringInteractiveMode:CTRL+Ctosendmulti-lineinput.CTRL+Dtoexittheprogram.
>After starting interactive mode, you will be prompted to enter text:EnteringInteractiveMode:CTRL+Ctosendmulti-lineinput.CTRL+Dtoexittheprogram.
>Whatweighsmore?>Apoundoffeathers?
>Orapoundoflead?
>Enter your reasoning prompt spanning multiple lines.To submit the prompt, pressCTRL + Cto send the input while staying in interactive mode.This will trigger the model to start reasoning over the prompt:> What weighs more?
> a pound of feathers or 
> a pound of lead?
> ^C
Thinking...
2024-02-14 20:17:38.307588 | Starting SELECT Phase
2024-02-14 20:17:42.403366 | SELECT Phase Complete
2024-02-14 20:17:42.403473 | Starting ADAPT Phase
2024-02-14 20:17:45.680310 | ADAPT Phase Complete
2024-02-14 20:17:45.680418 | Starting IMPLEMENT Phase
2024-02-14 20:17:52.029561 | IMPLEMENT Phase Complete
2024-02-14 20:17:52.029897 | Starting to Solve Problem using Reasoning Structure
2024-02-14 20:17:57.064476 | Solution has been found.


ANSWER: A pound of feathers weighs the same as a pound of lead.Once you are done, send theCTRL + Dsignal to exit interactive mode and quit the CLI program.> Goodbye!This interactive workflow allows you to conveniently test long, complex reasoning without having to put all the prompt in quotes or escape newlines.TODOExpose information on Reasoning Structure and Reasoning Module selection via Python API. Currently, it is only visible when using verbose=True in CLI and API.Add support for other prompt formats (Llama2, airoboros, etc.)Add REST support via Flask