ai X plain SDK : A High-Level and Standardized Toolkit for AI Assets (2024)

Abstract: The aiXplain SDK is an open-source Python toolkit which aims to simplify the wide and complex ecosystem of AI resources. The toolkit enables access to a wide selection of AI assets, including datasets, models, and metrics, from both academic and commercial sources, which can be selected, executed and evaluated in one place through different services in a standardized format with consistent documentation provided. The study showcases the potential of the proposed toolkit with different code examples and by using it on a user journey where state-of-the-art Large Language Models are fine-tuned on instruction prompt datasets, outperforming their base versions.

## introduction a software development kit (sdk) is a collection of software development tools in one installable package @xcite . the popularity of these toolkits in ai stems from their powerful features, ease of use, and applications in diverse fields including deep learning @xcite @xcite , computer vision @xcite , natural language processing @xcite @xcite , and beyond. this wide range of options available, however, can make it difficult to combine services from different sdks into one application, since the integration requires a deep understanding of the usage, dependencies, and intricacies of each technology. to address this challenge, we introduce the aixplain sdk, a unified platform providing seamless access to a diverse collection of ai resources, including datasets, models, and metrics. by integrating both open-source and commercial options, this sdk abstracts complexities such as hosting and billing, streamlining the research process. the sdk's flexibility that allows for effortless swapping of components by just changing the asset id enables faster prototyping. furthermore, the standardization of metrics and datasets within the sdk creates a level playing field for comparative analysis by mitigating the influence of disparate evaluation methodologies. researchers can efficiently discover, utilize, and assess these resources in a standardized, well-documented environment. the aixplain sdk aims to help both artificial intelligence users and developers. figure 1 exemplifies how with a few lines of code users can embed a machine learning model from the aixplain marketplace into their application. for developers, the proposed sdk covers the entire machine learning development lifecycle, allowing them to select/onboard data as well as to train, evaluate and serve their models. the sdk's python code is released under the apache-2.0 license and is publicly accessible on github 1 , where comprehensive documentation and tutorials are also available. the getting started guide * , along with the tutorial series 1 is prepared to help new users get familiar with the toolkit. a demo 1 is also provided to see the capabilities of sdk in action for a real world use-case. this setup helps new users to get started quickly and facilitates easy contributions from the entire community to the project. ## modules figure 2 depicts the architecture of the proposed sdk. the toolkit was designed to handle different kinds of assets such as corpora, datasets, models and metrics. in this section, we delve into each of these core modules, detailing their functionalities and highlighting how they converge to enhance overall performance and streamline user interactions within the system. ## services inherent to the machine learning (ml) lifecycle, it is crucial to consider the multifaceted roles and needs of ai professionals who contribute to the successful development, deployment, and maintenance of ml models. as depicted in figure 2 , the design of the proposed sdk centers on forging a unified and collaborative ecosystem tailored for the wide spectrum of ai professionals engaging in the ml development lifecycle. in the following subsections we explain in detail each of these services. ## related work scikit learn @xcite is an example of a traditional machine learning sdk. the toolkit is known by its simplicity and accessibility to apply traditional machine learning algorithms for problems that involve structured data. pytorch @xcite and tensorflow @xcite are examples of more recent sdks used in the development of state-of-the-art deep learning models. on top of them, other highlevel frameworks were proposed such as hugging-face's transformers @xcite and keras @xcite . software development kits have also been proposed for specific machine learning tasks such as opencv @xcite for computer vision; and nltk @xcite and stanza @xcite for natural language processing. popular cloud services also make their own sdks available to manipulate their services programmatically, including the ai ones. this is the case for google * and aws * cloud services. within this wide and complex ecosystem, the sdk aims to be a marketplace where the ai assets and tools provided by other suppliers and sdk could be found into a single, standardized and welldocumented access point. ## conclusion this study demonstrates how complicated can be the creation of an ai application combining assets from the wide and complex range of software toolkits in the field. to solve this problem, we propose the aixplain sdk which enables access to ai corpora, datasets, models and metrics from different commercial and community sources in a standardized format. through straightforward, well-documented, and exemplified services, the toolkit enables onboarding data assets as well as finetuning, evaluating, serving, and using ai models. the toolkit's potential is demonstrated in a user journey where three state-of-the-art large language models are fine-tuned on instruction prompt question-answering datasets. after the fine-tuning process, an evaluation is conducted in the proposed sdk demonstrating how the trained models outperformed the base ones. finally, the toolkit is publicly available on github and released under an open-source license (apache-2.0) along with a demo, example notebooks and video tutorials. we hope the community engages in its use and development, contributing to its growth. 1 import pandas as pd 2 from aixplain . factories import datasetfactory 3 from aixplain . modules import metadata 4 from aixplain . enums import function , language , license 1 from aixplain . factories import datasetfactory , modelfactory , finetunefactory 2 3 @xmath0. get (" ... ") 4 @xmath1. get (" ... ") 5 @xmath2. create ( 6 " finetuned_model " , 7 [ dataset ], 8 model 9 ) 10 @xmath3. start () 11 finetuned_model . check_finetune_status () figure 8: model fine-tuning example on the sdk