What happens when you combine the Chain of Thought (CoT) reasoning capabilities of LLMs with a heuristic-guided tree search algorithm? In the Tree of Thoughts (ToT) paper, the authors (Yao et al.) have coupled GPT-4 with tree search algorithms to attack a few tasks on which left-to-right CoT struggles. And the results are impressive. For example, on the "Game of 24" task, while GPT-4 with CoT prompting only managed to solve 4% of tasks, ToT achieved a success rate of 74%.
If you are interested in the topics of algorithmic AI, tree search, reasoning, planning, or "System 2" thinking, then you may find this blog post useful.