Package: pyaskit
AskIt(pyaskit)IntroductionAskItserves as a dedicated library or domain-specific language designed to streamline the utilization of Large Language Models (LLMs) such as GPT-4, Gemini, and LLama2. It simplifies the complexities of prompt engineering and eradicates the requirement for parsing responses from LLMs, making programming tasks smoother.UsingAskIt, you can deploy LLMs for a multitude of tasks, such as:Natural Language Processing: including translation, paraphrasing, and sentiment analysis.Problem Solving: resolving mathematical problems.Code Generation: creating new codes, and more.pyaskitcan use GPT, Gemini, or LLama2 as a backend.pyaskitoperates through theOpenAI APIorLLama2 API. Besides Python,AskIthas also been implemented in TypeScript. You can access the TypeScript version,ts-askit.Key FeaturesType-Guided Output Control: Get a response in the specified type.No need to specify the output format in the promptNo need to parse the response to extract the desired outputfrompyaskitimportask# Automatically parses the response to an integersum=ask(int,"add 1 + 1")# `sum` is an integer with a value of 2fromtypingimportTypedDict,Listfrompyaskitimportask# Define a typed dictionary for programming languagesclassPL(TypedDict):name:stryear_created:int# Automatically extracts structured information into a list of dictionarieslangs=ask(List[PL],"List the two oldest programming languages.")# `langs` holds information on the oldest programming languages in a structured format like# [{'name': 'Fortran', 'year_created': 1957},#  {'name': 'Lisp', 'year_created': 1958}]Template-Based Function Definition: Define functions using a prompt template.frompyaskitimportfunction@function(codable=False)deftranslate(s:str,lang:str)->str:"""Translate {{s}} into {{lang}} language."""s=translate("こんにちは世界。","English")# `s` would be "Hello, world."Code Generation: Generate functions from the unified interface.frompyaskitimportfunction@function(codable=True)defget_html(url:str)->str:"""Get the webpage from {{url}}."""# When `codable` is set to True, the body of the function is automatically coded by an LLM.html=get_html("https://github.com/katsumiok/pyaskit/blob/main/README.md")# `html` contains the HTML version of this README.mdProgramming by Example (PBE): Define functions using examples.
Refer to theProgramming by Example (PBE)section for further details.InstallationTo installAskIt, run this command in your terminal:pipinstallpyaskitorpipinstallgit+https://github.com/katsumiok/pyaskit.gitPreparation for OpenAI APIBefore usingAskIt, you need to set your OpenAI API key as an environment variableOPENAI_API_KEY:exportOPENAI_API_KEY=<yourOpenAIAPIkey><your OpenAI API key>is a string that looks like this:sk-<your key>.
You can find your API key in theOpenAI dashboard.You can also specify the model name as an environment variableASKIT_MODEL:exportASKIT_MODEL=<modelname><model name>is the name of the model you want to use.
The latest AskIt is tested withgpt-4andgpt-3.5-turbo-16k. You can find the list of available models in theOpenAI API documentation.Preparation for Gemini APIBefore usingAskIt, you need to set your Google API key as an environment variableGOOGLE_API_KEY:exportGOOGLE_API_KEY=<yourGoogleAPIkey>You can find your API key in theOpenAI dashboard.You need to specify the model name as an environment variableASKIT_MODEL:exportASKIT_MODEL=<modelname><model name>is the name of the model you want to use.
The latest AskIt is tested withgemini-pro.Preparation for Llama 2 API (Experimental)Before usingAskItwith Llama 2, you need to install it. To install Llama 2, run this command in your terminal:pipinstallgit+https://github.com/facebookresearch/llama.gitYou also need to download the tokenizer model and the checkpoint of the model you want to use. Please refer to the Llama 2 documentation for further details.We provide an example of usingAskItwith Llama 2 in theexamplesdirectory.
To run the example, run this command in your terminal:torchrun--nproc_per_node1examples/use_llama2.py\--ckpt_dirllama-2-7b-chat/\--tokenizer_pathtokenizer.model\--max_seq_len512--max_batch_size6Getting StartedHere are some basic examples to help you familiarize yourself withAskIt:Hello Worldimportpyaskitasais=ai.ask(str,'Paraphrase "Hello World!"')print(s)To utilizeAskIt, start by importing thepyaskitmodule. TheaskAPI, which takes two arguments - the output type and the prompt - produces the LLM's output in the designated format. In this case, the output type isstrand the prompt isParaphrase "Hello World!". A comprehensive explanation of types inAskItis provided in theTypessection. Executing this code will yield a paraphrase of the prompt, such as:Greetings, Planet!Defining a Function from a Prompt TemplateWith thefunctiondecoratorThefunctiondecorator allows defining a function with a prompt template. The parameters of a defined function can be used as parameters of a prompt template. For example,frompyaskitimportfunction@function(codable=False)defparaphrase(text:str)->str:"""Paraphrase {{text}}"""s=paraphrase('Hello World!')print(s)Where{{text}}represents a template parameter and corresponds to the function parameter.With thedefineAPIThedefineAPI allows for prompt parameterization using template syntax:importpyaskitasaiparaphrase=ai.define(str,'Paraphrase {{text}}')s=paraphrase(text='Hello World!')# s = paraphrase('Hello World!') # This is also validprint(s)In this instance, thedefineAPI creates a templated function that instructs the LLM to paraphrase specified text. Invoking theparaphrasefunction with 'Hello World!' will return a paraphrased version of this text. Running this code might output something like "Greetings, Planet!".ThedefineAPI allows for straightforward creation of custom functions to harness the capabilities of large language models for diverse tasks. Further examples can be found in theexamplesdirectory.Generating Functions for Codable TasksCertain tasks, such as those requiring real-time data, external resources like network access, file access, or database access, are unsuitable for LLM execution. However,AskItcan handle these tasks by converting the prompt into a Python program in the background.The subsequent example demonstrates usingAskItto tackle a task necessitating network access:importpyaskitasaiget_html=ai.define(str,'Get the webpage from {{url}}').compile()html=get_html(url='https://csail.mit.edu')print(html)In this scenario, you only need to callcompile()on the function returned by thedefineAPI. Thecompilefunction transforms the prompt into a Python program and returns a function that executes this code, behaving just like a regular Python function.While the above example does not specify the type of the parameterurl,AskItprovides thedefunAPI to do so. The following code demonstrates how to define a function in which the type of the parameterurlis specified asstr:importpyaskitasaiget_html=ai.defun(str,{"url":str},'Get the webpage from {{url}}').compile()html=get_html(url='https://csail.mit.edu')print(html)The second argument of thedefunAPI is a dictionary that maps parameter names to their types.We can the same thing with the following code:frompyaskitimportfunction@function(codable=True)defget_html(url:str)->str:"""Get the webpage from {{url}}"""html=get_html(url='https://csail.mit.edu')print(html)Programming by Example (PBE)Function Definition Using ExamplesLanguage Learning Models (LLMs) offer the advantage of few-shot learning, a capability thatAskItutilizes in programming tasks.AskItenables you to solve tasks using the Programming by Example (PBE) technique, where you provide examples of the desired input and output.Let's consider creating a function to add two binary numbers (represented as strings). This function accepts two binary numbers and returns their sum, also in binary form. The following code demonstrates defining such a function using illustrative examples.frompyaskitimportdefinetraining_examples=[{"input":{"x":"1","y":"0"},"output":"1"},{"input":{"x":"1","y":"1"},"output":"10"},{"input":{"x":"101","y":"11"},"output":"1000"},{"input":{"x":"1001","y":"110"},"output":"1111"},{"input":{"x":"1111","y":"1"},"output":"10000"},]add_binary_numbers=define(str,"Add {{x}} and {{y}}",training_examples=training_examples)sum_binary=add_binary_numbers(x="101",y="11")print(sum_binary)# Output: "1000"In this example, thedefineAPI takes three arguments: the output type, the prompt, and the training examples. Each entry in the training examples list is a dictionary containing an 'input' dictionary (with variable names and values) and an 'output' representing the expected function output given the input. ThedefineAPI then returns a function that accepts input variables as keyword arguments and outputs the LLM's output in the specified type.Theadd_binary_numbersfunction, which adds two binary numbers, behaves like any regular Python function.Testing the Generated FunctionYou can use thecompilefunction to test the generated function using an optional list of test examples.The following code demonstrates how to test the function defined above with new test examples:test_examples=[{"input":{"x":"0","y":"1"},"output":"1"},{"input":{"x":"10","y":"0"},"output":"10"},{"input":{"x":"110","y":"10"},"output":"1000"},]f=add_binary_numbers.compile(test_examples=test_examples)sum_binary=f(x="101",y="11")print(sum_binary)# Output: "1000"Here,fis the generated function that operates similarly toadd_binary_numbers. By comparing the output of the generated function with the expected output for each test example,AskItensures the generated function behaves as expected. If any discrepancy arises,AskItre-attempts the translation. After multiple unsuccessful translation attempts,AskItraises an exception.Specifying Types inAskItAskItoffers APIs to designate the output types for Language Learning Models (LLMs). By supplying these types as the first argument to theaskanddefineAPIs, you can manage the LLM's output format. You can also use type hints provided Python.The following table describes the various types supported byAskIt:TypeDescriptionType ExampleValue ExampleintIntegert.int123floatFloating Point Numbert.float1.23boolBooleant.boolTruestrStringt.str"Hello World!"literalLiteralt.literal(123)123listListt.list(t.int)[1, 2, 3]dictDictionaryt.dict({'a': t.int,'b': t.str}){'a': 1, 'b': "abc"}recordDictionaryt.record(t.str, t.int){'a': 1, 'b': 2}tupleTuplet.tuple(t.int, t.str)(1, "abc")unionUnion (Multiple Possible Values)t.union(t.literal('yes'), t.literal('no'))"yes" or "no"t.literal('yes') | t.literal('no')"yes" or "no"t.literal('yes', 'no')"yes" or "no"NoneNoneNoneNoneNote that each type declaration aidsAskItin parsing and understanding the desired output, ensuring your LLM returns data in the precise format you require.Prompt Template UsageThe prompt template is a string composed of placeholders for the parameters of the function being defined. Placeholders are denoted by double curly braces{{and}}and can only contain a variable name. This variable name is then used as a parameter in the defined function.Function parameters can be defined in two ways: either by keyword arguments or by positional arguments. For keyword arguments, the variable name within the placeholder serves as the keyword argument's name. For positional arguments, the sequence in which placeholders appear defines the order of the positional arguments.Consider the following example which demonstrates how to define a function,add, that accepts two argumentsxandyand returns their sum:frompyaskitimportdefineimportpyaskit.typesastadd=define(t.int,'{{x}} + {{y}}')print(add(x=1,y=2))# keyword argumentsprint(add(1,2))# positional argumentsIn this case, theaddfunction can be invoked using either keyword or positional arguments, with the sum ofxandyreturned as the output.Notably, if the same variable name is used multiple times in the prompt template, subsequent uses are mapped to the initial occurrence. Observe this behavior in the following example:frompyaskitimportdefineimportpyaskit.typesastadd=define(t.int,'{{x}} + {{y}} + {{x}} + {{z}}')print(add(x=1,y=2,z=3))print(add(1,2,3))Here,{{x}}appears twice in the prompt template. The second occurrence of{{x}}maps back to the first. Hence, even though{{z}}is the fourth placeholder in the template, it aligns with the third argument of the function.ContributingSeeCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.LicenseThis project is licensed under the MIT License - see theLICENSE.mdfile for detailsCitationIf you use our software in your research, please cite our paper:@misc{okuda2023askit,title={AskIt: Unified Programming Interface for Programming with Large Language Models},author={Katsumi Okuda and Saman Amarasinghe},year={2023},eprint={2308.15645},archivePrefix={arXiv},primaryClass={cs.PL}}