Package: uniflow
🌊 uniflowuniflowprovides a unified LLM interface to extract and transform and raw documents.Document types: Uniflow enables data extraction fromPDFs,HTMLsandTXTs.LLM agnostic: Uniflow supports most common-used LLMs for text tranformation, includingOpenAI models (GPT3.5 and GPT4),Google Gemini models (Gemini 1.5,MultiModal),AWSBedRockmodels,Huggingface open source models includingMistral-7B,Azure OpenAI models, etc.:question: The Problems to TackleUniflow addresses two key challenges in preparing LLM training data for ML scientists:first, extracting legacy documents like PDFs and Word files into clean text, which LLMs can learn from, is tricky due to complex PDF layouts and missing information during extraction; andsecond, the labor-intensive process of transforming extracted data into a format suitable for training LLMs, which involves creating datasets with both preferred and rejected answers for each question to support feedback-based learning techniques.Hence, we built Uniflow, a unified LLM interface to extract and transform and raw documents.:seedling: Use CasesUniflow aims to help every data scientist generate their own privacy-perserved, ready-to-use training datasets for LLM finetuning, and hence make finetuning LLMs more accessible to everyone:rocket:.Check Uniflow hands-on solutions:Extract financial reports (PDFs) into summerrizationExtract financial reports (PDFs) and finetune financial LLMsExtract a math book (HTMLs) into your question answer datasetExtract PDFs into your question answer datasetBuild RLHF/RLAIF perference datasets for LLM finetuning:computer: InstallationInstallinguniflowtakes about 5-10 minutes if you follow the 3 steps below:Create a conda environment on your terminal using:conda create -n uniflow python=3.10 -y
conda activate uniflow  # some OS requires `source activate uniflow`Install the compatible pytorch based on your OS.If you are on a GPU, installpytorch based on your cuda version. You can find your CUDA version vianvcc -V.pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121  # cu121 means cuda 12.1If you are on a CPU instance,pip3 install torchInstalluniflow:pip3 install uniflow(Optional) If you are running one of the followingOpenAIflows, you will have to set up your OpenAI API key. To do so, create a.envfile in your root uniflow folder. Then add the following line to the.envfile:OPENAI_API_KEY=YOUR_API_KEY(Optional) If you are running theHuggingfaceModelFlow, you will also need to install thetransformers,accelerate,bitsandbytes,scipylibraries:pip3 install transformers accelerate bitsandbytes scipy(Optional) If you are running theLMQGModelFlow, you will also need to install thelmqgandspacylibraries:pip3 install lmqg spacyCongrats you have finished the installation!:man_technologist: Dev SetupIf you are interested in contributing to us, here are the preliminary development setups.conda create -n uniflow python=3.10 -y
conda activate uniflow
cd uniflow
pip3 install poetry
poetry install --no-rootAWS EC2 Dev SetupIf you are on EC2, you can launch a GPU instance with the following config:EC2g4dn.xlarge(if you want to run a pretrained LLM with 7B parameters)Deep Learning AMI PyTorch GPU 2.0.1 (Ubuntu 20.04)EBS: at least 100GAPI keysIf you are running one of the followingOpenAIflows, you will have to set up your OpenAI API key.To do so, create a.envfile in your root uniflow folder. Then add the following line to the.envfile:OPENAI_API_KEY=YOUR_API_KEY:scroll: Uniflow ManualOverviewTo useuniflow, follow of three main steps:Pick aConfigThis determines the LLM and the different configurable parameters.Construct yourPromptsConstruct the context that you want to use to prompt your model. You can configure custom instructions and examples using thePromptTemplateclass.Run yourFlowRun the flow on your input data and generate output from your LLM.Note: We're currently building havePreprocessingflows as well to help process data from different sources, such aspdf,html,Markdown, and more.1. ConfigTheConfigdetermines which LLM is used and how the input data is serialized and deserialized. It also has parameters that are specific to the LLM.Here is a table of the different pre-defined configurations you can use and their corresponding LLMs:ConfigLLMConfiggpt-3.5-turbo-1106OpenAIConfiggpt-3.5-turbo-1106HuggingfaceConfigmistralai/Mistral-7B-Instruct-v0.1LMQGConfiglmqg/t5-base-squad-qg-aeYou can run each config with the defaults, or you can pass in custom parameters, such astemperatureorbatch_sizeto the config for your use case. See theadvanced custom configurationsection for more details.2. PromptingBy default,uniflowis set up to generate questions and answers based on theContextyou pass in. To do so, it has a default instruction and few-shot examples that it uses to guide the LLM.Here is the default instruction:Generate one question and its corresponding answer based on the last context in the last example. Follow the format of the examples below to include context, question, and answer in the responseHere are the default few-shot examples:context="The quick brown fox jumps over the lazy brown dog.",
    question="What is the color of the fox?",
    answer="brown."

    context="The quick brown fox jumps over the lazy black dog.",
    question="What is the color of the dog?",
    answer="black."To run with these default instructions and examples, all you need to do is pass in a list ofContextobjects to the flow.uniflowwill then generate a custom prompt with the instructions and few-shot examples for eachContextobject to send to the LLM. See theRunning the flowsection for more details.ContextTheContextclass is used to pass in the context for the LLM prompt. AContextconsists of acontextproperty, which is a string of text.To rununiflowwith the default instructions and few-shot examples, you can pass in a list ofContextobjects to the flow. For example:from uniflow.op.prompt import Context

data = [
    Context(
        context="The quick brown fox jumps over the lazy brown dog.",
    ),
    ...
]

client.run(data)For a more detailed overview of running the flow, see theRunning the flowsection.PromptTemplateIf you want to run with a custom prompt instruction or few-shot examples, you can use thePromptTemplateobject. It hasinstructionandexampleproperties.PropertyTypeDescriptioninstructionstrDetailed instructions for the LLMexamplesList[Context]The few-shot examples.You can overwrite any of the defaults as needed.To see an example of how to use thePromptTemplateto rununiflowwith a custominstruction, few-shot examples, and customContextfields to generate a summary, check out theopenai_pdf_source_10k_summary notebookRunning the FlowOnce you've decided on yourConfigand prompting strategy, you can run the flow on the input data.Import theuniflowClient,Config, andContextobjects.from uniflow.flow.client import TransformClient
from uniflow.flow.config import TransformOpenAIConfig, OpenAIModelConfig
from uniflow.op.prompt import ContextPreprocess your data in to chunks to pass into the flow. In the future we will havePreprocessingflows to help with this step, but for now you can use a library of your choice, likepypdf, to chunk your data.raw_input_context = ["It was a sunny day and the sky color is blue.", "My name is bobby and I am a talent software engineer working on AI/ML."]Create a list ofContextobjects to pass your data into the flow.data = [
    Context(context=c)
    for c in raw_input_context
][Optional] If you want to use a customized instruction and/or examples, create aPromptTemplate.from uniflow.op.prompt import PromptTemplate

guided_prompt = PromptTemplate(
instruction="Generate a one sentence summary based on the last context below. Follow the format of the examples below to include context and summary in the response",
few_shot_prompt=[
    Context(
        context="When you're operating on the maker's schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in. Plus you have to remember to go to the meeting. That's no problem for someone on the manager's schedule. There's always something coming on the next hour; the only question is what. But when someone on the maker's schedule has a meeting, they have to think about it.",
        summary="Meetings disrupt the productivity of those following a maker's schedule, dividing their time into impractical segments, while those on a manager's schedule are accustomed to a continuous flow of tasks.",
    ),
],
)Create aConfigobject to pass into theClientobject.config = TransformOpenAIConfig(
    prompt_template=guided_prompt,
    model_config=OpenAIModelConfig(
        response_format={"type": "json_object"}
    ),
)
client = TransformClient(config)Use theclientobject to run the flow on the input data.output = client.run(data)Process the output data. By default, the LLM output will be a list of output dicts, one for eachContextpassed into the flow. Each dict has aresponseproperty which has the LLM response, as well as any errors. For exampleoutput[0]['output'][0]would look like this:{
    'response': [{'context': 'It was a sunny day and the sky color is blue.',
    'question': 'What was the color of the sky?',
    'answer': 'blue.'}],
    'error': 'No errors.'
}ExamplesFor more examples, see theexamplefolder.Advanced Custom ConfigurationYou can also configure the flows by passing custom configurations or arguments to theConfigobject if you want to further tune specific parameters like the the LLM model, number of threads, the temperature, and more.Every configuration has the following parameters:ParameterTypeDescriptionprompt_templatePromptTemplateThe template to use for the guided prompt.num_threadsintThe number of threads to use for the flow.model_configModelConfigThe configuration to pass to the model.You can further configure themodel_configby passing in one of theModel Configswith custom parameters.Model ConfigTheModel Configis a configuration that is passed to the baseConfigobject and determines which LLM model is used and has parameters that are specific to the LLM model.ModelConfigThe base config is calledModelConfigand has the following parameters:ParameterTypeDefaultDescriptionmodel_namestrgpt-3.5-turbo-1106OpenAI siteOpenAIModelConfigTheOpenAIModelConfiginherits from theModelConfigand has the following additional parameters:ParameterTypeDefaultDescriptionnum_callsint1The number of calls to make to the OpenAI API.temperaturefloat1.5The temperature to use for the OpenAI API.response_formatDict[str, str]{"type": "text"}The response format to use for the OpenAI API. Can be "text" or "json"HuggingfaceModelConfigTheHuggingfaceModelConfiginherits from theModelConfig, but overrides themodel_nameparameter to use themistralai/Mistral-7B-Instruct-v0.1model by default.ParameterTypeDefaultDescriptionmodel_namestrmistralai/Mistral-7B-Instruct-v0.1Hugging Face sitebatch_sizeint1The batch size to use for the Hugging Face API.LMQGModelConfigTheLMQGModelConfiginherits from theModelConfig, but overrides themodel_nameparameter to use thelmqg/t5-base-squad-qg-aemodel by default.ParameterTypeDefaultDescriptionmodel_namestrlmqg/t5-base-squad-qg-aeHugging Face sitebatch_sizeint1The batch size to use for the LMQG API.Custom Configuration ExampleHere is an example of how to pass in a custom configuration to theClientobject:from uniflow.flow.client import TransformClient
from uniflow.flow.config import TransformOpenAIConfig, OpenAIModelConfig
from uniflow.op.prompt import Context


contexts = ["It was a sunny day and the sky color is blue.", "My name is bobby and I am a talent software engineer working on AI/ML."]

data = [
    Context(
        context=c
    )
    for c in contexts
]

config = OpenAIConfig(
  num_threads=2,
  model_config=OpenAIModelConfig(
    model_name="gpt-4",
    num_calls=2,
    temperature=0.5,
  ),
)
client = TransformClient(config)
output = client.run(data)As you can see, we are passing in a custom parameters to theOpenAIModelConfigto theOpenAIConfigconfigurations according to our needs.