Package: scalexi
Overviewscalexiis a versatile open-source Python library that focuses on facilitating low-code development and fine-tuning of diverse Large Language Models (LLMs). It extends beyond its initial OpenAI models integration, offering a scalable framework for various LLMs.Key toscalexiis its low-code approach, significantly reducing the complexity of dataset preparation and manipulation. It features advanced dataset conversion tools, adept at transforming raw contextual data into structured datasets fulfilling LLMs fine-tuning requirements. These tools support multiple question formats, like open-ended, closed-ended, yes-no, and reflective queries, streamlining the creation of customized datasets for LLM fine-tuning.A standout feature is the library's automated dataset generation, which eases the workload involved in LLM training.scalexialso provides essential utilities for cost estimation and token counting, aiding in effective resource management throughout the fine-tuning process.Developed by ScaleX Innovationscalexi.ai, the library leverages a robust specification to facilitate fine-tuning context-specific models with OpenAI API. Also,scalexiensures a user-friendly experience while maintaining high performance and error handling.Explore the full capabilities of Large Language Models withscalexi's intuitive and efficient Python API with minimal coding for easy LLM development and fine-tuning from dataset creation to LLM evaluation.DocumentationFor comprehensive guides, API references, and usage examples, visit thescalexiDocumentation. It provides an up-to-date information you need to effectively utilize thescalexilibrary for LLM development and fine-tuning.FeaturesLow-Code Interface:scalexioffers a user-friendly, low-code platform that simplifies interactions with LLMs. Its intuitive design minimizes the need for extensive coding, making LLM development accessible to a broader range of users.Automated Dataset Generation: The library excels in converting raw data into structured formats, aligning with specific LLM fine-tuning requirements. This automation streamlines the dataset preparation process, saving time and reducing manual effort.Versatile Dataset Format Support:scalexiis designed to handle various dataset formats including CSV, JSON, and JSONL. It also facilitates effortless conversion between these formats, providing flexibility in dataset management and utilization.Simplified Fine-Tuning Process: The library provides simplified interfaces for fine-tuning LLMs. These user-friendly tools allow for easy customization and optimization of models on specific datasets, enhancing model performance and applicability.Efficient Model Evaluation:scalexiincludes utilities for the automated evaluation of fine-tuned models. This feature assists in assessing model performance, ensuring the reliability and effectiveness of the fine-tuned models.Token Usage Estimation: The library incorporates functions to accurately estimate token usage and associated costs. This is crucial for managing resources and budgeting in LLM projects, providing users with a clear understanding of potential expenses.InstallationEasily installscalexiwith pip. Just run the following command in your terminal:pipinstallscalexiThis will install scalexi and its dependencies, making it ready for use with Python 3.11 and above (not tested on lower Python versions).TutorialsFor documentation and tutorials, visit thescalexiDocumentation. It provides an up-to-date information you need to effectively utilize thescalexilibrary for LLM development and fine-tuning.LicenseThis project is licensed under the ScaleXI 1.0 License - see the LICENSE file for details.