Post
1037
📢 Having a massive amount of data to bulk the remotely accessed LLM 🤖 with Chain-of-Though (CoT) 🔗 might result in connection loss.
The latter may lead to Python Exception 💥 and challenges with generated content restoration.
To address on this problem, sharing the no-strings / tiny framework that exploits SQLite3 for caching each query.
Such caching allows smooth relaunch in the case of any data loss. ☕
With that, happy to share bulk-chain project and more on that within links below:
⭐ github: https://github.com/nicolay-r/bulk-chain
📦 PyPI: https://pypi.org/project/bulk-chain/
There are three steps to quickstart
(see them in attachment 👇):
✅ 1. Install library
✅ 2. Declare CoT-schema in json file 📄
✅ 3. Wrap your transformer or use existed adapters
https://github.com/nicolay-r/bulk-chain/tree/master/ext
For example, here is the provider for Replicate IO service (https://replicate.com/):
https://github.com/nicolay-r/bulk-chain/blob/master/ext/replicate.py
that supports one of the largers publicly available LLaMA-3.1-405B:
meta-llama/Llama-3.1-405B-Instruct
The latter may lead to Python Exception 💥 and challenges with generated content restoration.
To address on this problem, sharing the no-strings / tiny framework that exploits SQLite3 for caching each query.
Such caching allows smooth relaunch in the case of any data loss. ☕
With that, happy to share bulk-chain project and more on that within links below:
⭐ github: https://github.com/nicolay-r/bulk-chain
📦 PyPI: https://pypi.org/project/bulk-chain/
There are three steps to quickstart
(see them in attachment 👇):
✅ 1. Install library
✅ 2. Declare CoT-schema in json file 📄
✅ 3. Wrap your transformer or use existed adapters
https://github.com/nicolay-r/bulk-chain/tree/master/ext
For example, here is the provider for Replicate IO service (https://replicate.com/):
https://github.com/nicolay-r/bulk-chain/blob/master/ext/replicate.py
that supports one of the largers publicly available LLaMA-3.1-405B:
meta-llama/Llama-3.1-405B-Instruct
