Package: prompt-optimizer
PromptOptimizerMinimize LLM token complexity to save API costs and model computations.DocsFeaturesPlug and Play Optimizers:Minimize token complexity using optimization methods without any access to weights, logits or decoding algorithm. Directly applicable to virtually all NLU systems.Protected Tags:Special protected tags to mark important sections of prompt that should not be removed/modified.Sequential Optimization:Chain different optimizers together sequentially.Optimization Metrics:Number of tokens reduced and semantic similarity before and after optimization.Langhcain and JSON Support:Supports langchain style prompt chains and OpenAI request JSON Object.Why?Minimize Token Complexity:Token Complexity is the amount of prompt tokens required to achieve a given task. Reducing token complexity corresponds to linearly reducing API costs and quadratically reducing computational complexity of usual transformer models.Save Money:For large businesses, saving 10% on token count can lead to saving 100k USD per 1M USD.Extend Limitations:Some models have small context lengths, prompt optimizers can help them process larger than context documents.Prompt# TokensCorrect Response?Who is the president of the United States of America?11✅Who president US3  (-72%)✅InstallationQuick Installationpip install prompt-optimizerInstall from sourcegitclonehttps://github.com/vaibkumr/prompt-optimizer.git;cdprompt-optimizer;pipinstall-e.DisclaimerThere is a compression vs performance tradeoff -- the increase in compression comes at the cost of loss in model performance. The tradeoff can be greatly mitigated by chosing the right optimize for a given task. There is no single optimizer for all cases. There is no Adam here.Getting startedfromprompt_optimizer.poptimimportEntropyOptimprompt="""The Belle Tout Lighthouse is a decommissioned lighthouse and British landmark located at Beachy Head, East Sussex, close to the town of Eastbourne."""p_optimizer=EntropyOptim(verbose=True,p=0.1)optimized_prompt=p_optimizer(prompt)print(optimized_prompt)EvaluationsFollowing are the results forlogiqaOpenAI evals task. It is only performed for a subset of first 100 samples. Please note the optimizer performance over this task should not be generalized to other tasks, more thorough testing and domain knowledge is needed to choose the optimal optimizer.Name% Tokens ReducedLogiQA AccuracyUSD Saved Per $100Default0.00.320.0Entropy_Optim_p_0.050.060.36.35Entropy_Optim_p_0.10.110.2811.19Entropy_Optim_p_0.250.260.2226.47Entropy_Optim_p_0.50.50.0849.65SynonymReplace_Optim_p_1.00.010.331.06Lemmatizer_Optim0.010.331.01NameReplace_Optim0.010.341.13Punctuation_Optim0.130.3512.81Autocorrect_Optim0.010.31.14Pulp_Optim_p_0.050.050.315.49Pulp_Optim_p_0.10.10.259.52Cost-Performance TradeoffThe reduction in cost often comes with a loss in LLM performance. Almost every optimizer have hyperparameters that control this tradeoff.For example, inEntropyOptimthe hyperparamterp, a floating point number between 0 and 1 controls the ratio of tokens to remove.p=1.0corresponds to removing all tokens whilep=0.0corresponds to removing none.The following chart shows the trade-off for different values ofpas evaluated on the OpenAI evalslogiqatask for a subset of first 100 samples.ContributingThere are several directions to contribute to. Please seeCONTRIBUTING.mdfor contribution guidelines and possible future directions.SocialContact us on twitterVaibhav KumarandVaibhav Kumar.Inspiration