Package: kork
KorkKorkis anexperimentalLangchain chainthat helps build natural language APIs powered by LLMs.FeaturesAssemble a natural language API from a set of python functions.Generate a prompt to help the LLM write acorrectprogram.Execute the program generated by the LLMsafely.Program generation takes a single LLM call, as opposed to agents that use a call per action taken.Show me! 😹Image manipulation API using PILHowThe chain takes a user query, translates it into a program, and executes it using theKorkinterpreter.The interpreter is used to run code written in theKorkprogramming language.The language is limited by design to variable declarations, function invocations and arithmetic operations.This limitation allows controlling the kind of programs the LLMs can generate, making the output of the LLM more predictable and safer to execute in a production setting. (An agent with access to "constrained" tools has similar benefits, but requires a call per action taken.)Korkadds a few additional tricks (e.g., retriever interfaces for examples and foreign functions) to help guide the LLM to generate acorrectprogram.What?No loops, no conditionals, no file access, no network access, no arbitrary code execution. WHAT?!The ability to invoke custom functions goes a long way in terms of the kinds of programs that can be written! (You can always add awrite_to_filefunction!)LimitationsKorkcannot write arbitrarycode. If that's what you need, save yourself some time and usedockerand a real programming language.TheKorkchain isnotan Agent, so it cannot inspect intermediate steps, instead it generates a pre-defined set of function calls that need to be executed. (But remember that function calls can involve calls to an agent!)TheKorklanguage and interpreter are limited tofunction invocation,variable declarationandbasic arithmetic(no function declaration, loops etc.)Only supportingint,float,str,type(None),booltypes. No support forlistsorobjecttypes.Very limited type annotations.QualityThe short version: No benchmarks yet!The long version: Quality of generated programs depends on many factors.You can experiment with tweaking the prompt (e.g., explain the syntax of the language), changing the foreign function retriever (e.g., retrieve the most relevant foreign functions based on the user query), or providing examples in the form of (query, expected program).The prompt, examples and syntax of the language can trip up the LLM to assume
it's programming in a specific language (e.g., typescript or python) and assume
that it can use language features that are not supported byKorkor to import
libraries that are not available.SafetyNothing is bulletproof when humans are involved.Don't do silly things like exposingevalas a foreign function.If a foreign function allocates memory, the LLM could ask for more memory than is available crashing the process.Think carefully before using the output of an LLM. If the LLM behaves maliciously, is it safe to use an LLM in the given context?Future WorkLet us know if you're interested in contributing or have ideas for improvements!Allow enabling/disabling language featuresAllow changing underlying language syntaxAdd support for objectsAdd other language features (e.g., loops)Provide foreign function retriever implementations based on similarity to user queryCompatibilityKorkis tested against python 3.8, 3.9, 3.10, 3.11.Installationpipinstallkork🙏 AppreciationLark-- For making it easy to define a grammar and parse it!Bob Nystrom-- For writingCrafting Interpreters!© ContributingIf you have any ideas or feature requests, please open an issue and share!SeeCONTRIBUTING.mdfor more information.🎶 Why the name?Fast to type and maybe sufficiently unique.