Package: itune
i tune your parameters so you don't have toitune is a Python package for optimizing parameters using reinforcement learning with human feedback (RLHF).Example: Favourite number guessing gameThis code example demonstrates a simple guessing game usingitune. The algorithm will randomly choose a number from 1 to 5 (inclusive) and ask the user if that is their favourite number. If the user says yes, the algorithm will learn to choose that number more often in the future. If the user says no, the algorithm will continue to explore different numbers.fromituneimportMultiArmedBandit,TuneMAX_VALUE=5ITERATIONS=5itune=Tune(strategy=MultiArmedBandit())for_inrange(ITERATIONS):print(f"Your favourite number from 1 to{MAX_VALUE}(inclusive) is{itune.choose(fav_num=list(range(1,5+1)))}")user_input=input("Yes (y) / No (n)?")####################### reward functionitune.register_outcome(user_input=="y")######################OutputSuppose your favorite number is 5. The output of the code might look like this:Your favourite number from 1 to 5 (inclusive) is 3
Yes (y) / No (n)?n
Your favourite number from 1 to 5 (inclusive) is 4
Yes (y) / No (n)?n
Your favourite number from 1 to 5 (inclusive) is 2
Yes (y) / No (n)?n
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?y
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?yIt's worth noting thatituneretains its progress by loading and saving its state implicitly, enabling seamless continuation from previous sessions.Your favourite number from 1 to 5 (inclusive) is 1
Yes (y) / No (n)?n
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?y
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?y
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?y
Your favourite number from 1 to 5 (inclusive) is 5
Yes (y) / No (n)?yDuring subsequent runs,itunetends to favour the previously successful choice.Example: Retrieval Augmented Generation (RAG) applicationThis example requires features not yet available initune v0.1. It demonstrates where the library is headed.ProblemOptimizing a user-facing program with multiple parameters can be tedious, especially when the parameters are correlated.SolutionUse theitunelibrary to discover the most effective combination of parameters.ituneis a parameter optimizer for user-facing programs.ExampleThe following code shows how to useituneto optimize thechunk_sizeandllmparameters of a RAG model:fromllama_indeximportServiceContext,SimpleDirectoryReader,VectorStoreIndexfromllama_index.llmsimportOpenAI,PaLMfromituneimportTune,ContextualBanditdocuments=SimpleDirectoryReader("data").load_data()itune=Tune(strategy=ContextualBandit())service_context=ServiceContext.from_defaults(######################################################################## pass in acceptable list of values to these parameters and use RLHF to# optimize over combinations of themchunk_size=itune.choose(chunk_size=[250,500,1000,2000])llm=itune.choose(llm=[PaLM(),OpenAI()])#######################################################################)index=VectorStoreIndex.from_documents(documents,service_context=service_context)query_engine=index.as_query_engine()response=query_engine.query("What did the author do growing up?")print(response)user_input=input("Good response?")# reward functionitune.register_outcome(user_input=="y")BenefitsUsingitunecan save you time and effort when writing programs with multiple parameters. It also allows you to focus on developing your end-to-end solution whileitunefigures out the best combination of parameters.InstallingInstall and update usingpip:$ pip install itune