To set it up, follow the instructions on https://database.guide/2-sample-databases-sqlite/, placing the .db file in a notebooks folder at the root of this repository.from langchain import OpenAI, SQLDatabase, SQLDatabaseChaindb = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")llm = OpenAI(temperature=0, verbose=True)NOTE: For data-sensitive projects, you can specify return_direct=True in the SQLDatabaseChain initialization to directly return the output of the SQL query without any additional formatting. This prevents the LLM from seeing any contents within the database. Note, however, the LLM still has access to the database scheme (i.e. dialect, table and key names) by default.db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)db_chain.run("How many employees are there?")            > Entering new SQLDatabaseChain chain...    How many employees are there?    SQLQuery:    /workspace/langchain/langchain/sql_database.py:191: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.      sample_rows = connection.execute(command)    SELECT COUNT(*) FROM "Employee";    SQLResult: [(8,)]    Answer:There are 8 employees.    > Finished chain.    'There are 8 employees.'Use Query Checkerâ€‹Sometimes the Language Model generates invalid SQL with small mistakes that can be self-corrected using the same technique used by the SQL Database Agent to try and fix the SQL using the LLM. You can simply specify this option when creating the


to try and fix the SQL using the LLM. You can simply specify this option when creating the chain:db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True, use_query_checker=True)db_chain.run("How many albums by Aerosmith?")            > Entering new SQLDatabaseChain chain...    How many albums by Aerosmith?    SQLQuery:SELECT COUNT(*) FROM Album WHERE ArtistId = 3;    SQLResult: [(1,)]    Answer:There is 1 album by Aerosmith.    > Finished chain.    'There is 1 album by Aerosmith.'Customize Promptâ€‹You can also customize the prompt that is used. Here is an example prompting it to understand that foobar is the same as the Employee tablefrom langchain.prompts.prompt import PromptTemplate_DEFAULT_TEMPLATE = """Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.Use the following format:Question: "Question here"SQLQuery: "SQL Query to run"SQLResult: "Result of the SQLQuery"Answer: "Final answer here"Only use the following tables:{table_info}If someone asks for the table foobar, they really mean the employee table.Question: {input}"""PROMPT = PromptTemplate(    input_variables=["input", "table_info", "dialect"], template=_DEFAULT_TEMPLATE)db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT, verbose=True)db_chain.run("How many employees are there in the foobar table?")            > Entering new SQLDatabaseChain chain...    How many employees are there in the foobar table?


chain...    How many employees are there in the foobar table?    SQLQuery:SELECT COUNT(*) FROM Employee;    SQLResult: [(8,)]    Answer:There are 8 employees in the foobar table.    > Finished chain.    'There are 8 employees in the foobar table.'Return Intermediate Stepsâ€‹You can also return the intermediate steps of the SQLDatabaseChain. This allows you to access the SQL statement that was generated, as well as the result of running that against the SQL Database.db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT, verbose=True, use_query_checker=True, return_intermediate_steps=True)result = db_chain("How many employees are there in the foobar table?")result["intermediate_steps"]            > Entering new SQLDatabaseChain chain...    How many employees are there in the foobar table?    SQLQuery:SELECT COUNT(*) FROM Employee;    SQLResult: [(8,)]    Answer:There are 8 employees in the foobar table.    > Finished chain.    [{'input': 'How many employees are there in the foobar table?\nSQLQuery:SELECT COUNT(*) FROM Employee;\nSQLResult: [(8,)]\nAnswer:',      'top_k': '5',      'dialect': 'sqlite',      'table_info': '\nCREATE TABLE "Artist" (\n\t"ArtistId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("ArtistId")\n)\n\n/*\n3 rows from Artist


KEY ("ArtistId")\n)\n\n/*\n3 rows from Artist table:\nArtistId\tName\n1\tAC/DC\n2\tAccept\n3\tAerosmith\n*/\n\n\nCREATE TABLE "Employee" (\n\t"EmployeeId" INTEGER NOT NULL, \n\t"LastName" NVARCHAR(20) NOT NULL, \n\t"FirstName" NVARCHAR(20) NOT NULL, \n\t"Title" NVARCHAR(30), \n\t"ReportsTo" INTEGER, \n\t"BirthDate" DATETIME, \n\t"HireDate" DATETIME, \n\t"Address" NVARCHAR(70), \n\t"City" NVARCHAR(40), \n\t"State" NVARCHAR(40), \n\t"Country" NVARCHAR(40), \n\t"PostalCode" NVARCHAR(10), \n\t"Phone" NVARCHAR(24), \n\t"Fax" NVARCHAR(24), \n\t"Email" NVARCHAR(60), \n\tPRIMARY KEY ("EmployeeId"), \n\tFOREIGN KEY("ReportsTo") REFERENCES "Employee" ("EmployeeId")\n)\n\n/*\n3 rows from Employee table:\nEmployeeId\tLastName\tFirstName\tTitle\tReportsTo\tBirthDate\tHireDate\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\n1\tAdams\tAndrew\tGeneral Manager\tNone\t1962-02-18 00:00:00\t2002-08-14 00:00:00\t11120 Jasper Ave NW\tEdmonton\tAB\tCanada\tT5K 2N1\t+1 (780) 428-9482\t+1 (780) 428-3457\tandrew@chinookcorp.com\n2\tEdwards\tNancy\tSales


Manager\t1\t1958-12-08 00:00:00\t2002-05-01 00:00:00\t825 8 Ave SW\tCalgary\tAB\tCanada\tT2P 2T3\t+1 (403) 262-3443\t+1 (403) 262-3322\tnancy@chinookcorp.com\n3\tPeacock\tJane\tSales Support Agent\t2\t1973-08-29 00:00:00\t2002-04-01 00:00:00\t1111 6 Ave SW\tCalgary\tAB\tCanada\tT2P 5M5\t+1 (403) 262-3443\t+1 (403) 262-6712\tjane@chinookcorp.com\n*/\n\n\nCREATE TABLE "Genre" (\n\t"GenreId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("GenreId")\n)\n\n/*\n3 rows from Genre table:\nGenreId\tName\n1\tRock\n2\tJazz\n3\tMetal\n*/\n\n\nCREATE TABLE "MediaType" (\n\t"MediaTypeId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("MediaTypeId")\n)\n\n/*\n3 rows from MediaType table:\nMediaTypeId\tName\n1\tMPEG audio file\n2\tProtected AAC audio file\n3\tProtected MPEG-4 video file\n*/\n\n\nCREATE TABLE "Playlist" (\n\t"PlaylistId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("PlaylistId")\n)\n\n/*\n3 rows from Playlist


KEY ("PlaylistId")\n)\n\n/*\n3 rows from Playlist table:\nPlaylistId\tName\n1\tMusic\n2\tMovies\n3\tTV Shows\n*/\n\n\nCREATE TABLE "Album" (\n\t"AlbumId" INTEGER NOT NULL, \n\t"Title" NVARCHAR(160) NOT NULL, \n\t"ArtistId" INTEGER NOT NULL, \n\tPRIMARY KEY ("AlbumId"), \n\tFOREIGN KEY("ArtistId") REFERENCES "Artist" ("ArtistId")\n)\n\n/*\n3 rows from Album table:\nAlbumId\tTitle\tArtistId\n1\tFor Those About To Rock We Salute You\t1\n2\tBalls to the Wall\t2\n3\tRestless and Wild\t2\n*/\n\n\nCREATE TABLE "Customer" (\n\t"CustomerId" INTEGER NOT NULL, \n\t"FirstName" NVARCHAR(40) NOT NULL, \n\t"LastName" NVARCHAR(20) NOT NULL, \n\t"Company" NVARCHAR(80), \n\t"Address" NVARCHAR(70), \n\t"City" NVARCHAR(40), \n\t"State" NVARCHAR(40), \n\t"Country" NVARCHAR(40), \n\t"PostalCode" NVARCHAR(10), \n\t"Phone" NVARCHAR(24), \n\t"Fax" NVARCHAR(24), \n\t"Email" NVARCHAR(60) NOT NULL, \n\t"SupportRepId" INTEGER, \n\tPRIMARY KEY ("CustomerId"), \n\tFOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")\n)\n\n/*\n3 rows from Customer


REFERENCES "Employee" ("EmployeeId")\n)\n\n/*\n3 rows from Customer table:\nCustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n1\tLuÃs\tGonÃ§alves\tEmbraer - Empresa Brasileira de AeronÃ¡utica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSÃ£o JosÃ© dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n2\tLeonie\tKÃ¶hler\tNone\tTheodor-Heuss-StraÃŸe 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n3\tFranÃ§ois\tTremblay\tNone\t1498 rue BÃ©langer\tMontrÃ©al\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n*/\n\n\nCREATE TABLE "Invoice" (\n\t"InvoiceId" INTEGER NOT NULL, \n\t"CustomerId" INTEGER NOT NULL, \n\t"InvoiceDate" DATETIME NOT NULL, \n\t"BillingAddress" NVARCHAR(70), \n\t"BillingCity" NVARCHAR(40), \n\t"BillingState" NVARCHAR(40), \n\t"BillingCountry" NVARCHAR(40), \n\t"BillingPostalCode" NVARCHAR(10), \n\t"Total" NUMERIC(10, 2) NOT NULL, \n\tPRIMARY


NUMERIC(10, 2) NOT NULL, \n\tPRIMARY KEY ("InvoiceId"), \n\tFOREIGN KEY("CustomerId") REFERENCES "Customer" ("CustomerId")\n)\n\n/*\n3 rows from Invoice table:\nInvoiceId\tCustomerId\tInvoiceDate\tBillingAddress\tBillingCity\tBillingState\tBillingCountry\tBillingPostalCode\tTotal\n1\t2\t2009-01-01 00:00:00\tTheodor-Heuss-StraÃŸe 34\tStuttgart\tNone\tGermany\t70174\t1.98\n2\t4\t2009-01-02 00:00:00\tUllevÃ¥lsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n3\t8\t2009-01-03 00:00:00\tGrÃ©trystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n*/\n\n\nCREATE TABLE "Track" (\n\t"TrackId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(200) NOT NULL, \n\t"AlbumId" INTEGER, \n\t"MediaTypeId" INTEGER NOT NULL, \n\t"GenreId" INTEGER, \n\t"Composer" NVARCHAR(220), \n\t"Milliseconds" INTEGER NOT NULL, \n\t"Bytes" INTEGER, \n\t"UnitPrice" NUMERIC(10, 2) NOT NULL, \n\tPRIMARY KEY ("TrackId"), \n\tFOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"), \n\tFOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"), \n\tFOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")\n)\n\n/*\n3 rows from Track


REFERENCES "Album" ("AlbumId")\n)\n\n/*\n3 rows from Track table:\nTrackId\tName\tAlbumId\tMediaTypeId\tGenreId\tComposer\tMilliseconds\tBytes\tUnitPrice\n1\tFor Those About To Rock (We Salute You)\t1\t1\t1\tAngus Young, Malcolm Young, Brian Johnson\t343719\t11170334\t0.99\n2\tBalls to the Wall\t2\t2\t1\tNone\t342562\t5510424\t0.99\n3\tFast As a Shark\t3\t2\t1\tF. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman\t230619\t3990994\t0.99\n*/\n\n\nCREATE TABLE "InvoiceLine" (\n\t"InvoiceLineId" INTEGER NOT NULL, \n\t"InvoiceId" INTEGER NOT NULL, \n\t"TrackId" INTEGER NOT NULL, \n\t"UnitPrice" NUMERIC(10, 2) NOT NULL, \n\t"Quantity" INTEGER NOT NULL, \n\tPRIMARY KEY ("InvoiceLineId"), \n\tFOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"), \n\tFOREIGN KEY("InvoiceId") REFERENCES "Invoice" ("InvoiceId")\n)\n\n/*\n3 rows from InvoiceLine table:\nInvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n1\t1\t2\t0.99\t1\n2\t1\t4\t0.99\t1\n3\t2\t6\t0.99\t1\n*/\n\n\nCREATE TABLE "PlaylistTrack" (\n\t"PlaylistId" INTEGER NOT NULL, \n\t"TrackId" INTEGER NOT NULL, \n\tPRIMARY KEY ("PlaylistId", "TrackId"), \n\tFOREIGN KEY("TrackId")


KEY ("PlaylistId", "TrackId"), \n\tFOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"), \n\tFOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")\n)\n\n/*\n3 rows from PlaylistTrack table:\nPlaylistId\tTrackId\n1\t3402\n1\t3389\n1\t3390\n*/',      'stop': ['\nSQLResult:']},     'SELECT COUNT(*) FROM Employee;',     {'query': 'SELECT COUNT(*) FROM Employee;', 'dialect': 'sqlite'},     'SELECT COUNT(*) FROM Employee;',     '[(8,)]']Choosing how to limit the number of rows returnedâ€‹If you are querying for several rows of a table you can select the maximum number of results you want to get by using the 'top_k' parameter (default is 10). This is useful for avoiding query results that exceed the prompt max length or consume tokens unnecessarily.db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True, use_query_checker=True, top_k=3)db_chain.run("What are some example tracks by composer Johann Sebastian Bach?")            > Entering new SQLDatabaseChain chain...    What are some example tracks by composer Johann Sebastian Bach?    SQLQuery:SELECT Name FROM Track WHERE Composer = 'Johann Sebastian Bach' LIMIT 3    SQLResult: [('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I.


for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude',)]    Answer:Examples of tracks by Johann Sebastian Bach are Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace, Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria, and Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude.    > Finished chain.    'Examples of tracks by Johann Sebastian Bach are Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace, Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria, and Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude.'Adding example rows from each tableâ€‹Sometimes, the format of the data is not obvious and it is optimal to include a sample of rows from the tables in the prompt to allow the LLM to understand the data before providing a final query. Here we will use this feature to let the LLM know that artists are saved with their full names by providing two rows from the Track table.db = SQLDatabase.from_uri(    "sqlite:///../../../../notebooks/Chinook.db",    include_tables=['Track'], # we include only one table to save tokens in the prompt :)    sample_rows_in_table_info=2)The sample rows are added to the prompt after each corresponding table's column information:print(db.table_info)        CREATE TABLE "Track" (        "TrackId" INTEGER NOT NULL,         "Name"


"TrackId" INTEGER NOT NULL,         "Name" NVARCHAR(200) NOT NULL,         "AlbumId" INTEGER,         "MediaTypeId" INTEGER NOT NULL,         "GenreId" INTEGER,         "Composer" NVARCHAR(220),         "Milliseconds" INTEGER NOT NULL,         "Bytes" INTEGER,         "UnitPrice" NUMERIC(10, 2) NOT NULL,         PRIMARY KEY ("TrackId"),         FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"),         FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"),         FOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")    )        /*    2 rows from Track table:    TrackId Name    AlbumId MediaTypeId GenreId Composer    Milliseconds    Bytes   UnitPrice    1   For Those About To Rock (We Salute You) 1   1   1   Angus Young, Malcolm Young, Brian Johnson   343719  11170334    0.99    2   Balls to the Wall   2   2   1   None    342562  5510424 0.99    */db_chain = SQLDatabaseChain.from_llm(llm, db, use_query_checker=True, verbose=True)db_chain.run("What are some example


db, use_query_checker=True, verbose=True)db_chain.run("What are some example tracks by Bach?")            > Entering new SQLDatabaseChain chain...    What are some example tracks by Bach?    SQLQuery:SELECT "Name", "Composer" FROM "Track" WHERE "Composer" LIKE '%Bach%' LIMIT 5    SQLResult: [('American Woman', 'B. Cummings/G. Peterson/M.J. Kale/R. Bachman'), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Johann Sebastian Bach'), ('Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria', 'Johann Sebastian Bach'), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude', 'Johann Sebastian Bach'), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata', 'Johann Sebastian Bach')]    Answer:Tracks by Bach include 'American Woman', 'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria', 'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude', and 'Toccata and Fugue in D Minor, BWV 565: I. Toccata'.    > Finished chain.    'Tracks by Bach include \'American Woman\', \'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\', \'Aria Mit 30


D Minor, BWV 1043: I. Vivace\', \'Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria\', \'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude\', and \'Toccata and Fugue in D Minor, BWV 565: I. Toccata\'.'Custom Table Infoâ€‹In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first sample_rows_in_table_info sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns. This information can be provided as a dictionary with table names as the keys and table information as the values. For example, let's provide a custom definition and sample rows for the Track table with only a few columns:custom_table_info = {    "Track": """CREATE TABLE Track (    "TrackId" INTEGER NOT NULL,     "Name" NVARCHAR(200) NOT NULL,    "Composer" NVARCHAR(220),    PRIMARY KEY ("TrackId"))/*3 rows from Track table:TrackId Name    Composer1   For Those About To Rock (We Salute You) Angus Young, Malcolm Young, Brian Johnson2   Balls to the Wall   None3   My favorite song ever   The coolest composer of all time*/"""}db = SQLDatabase.from_uri(    "sqlite:///../../../../notebooks/Chinook.db",    include_tables=['Track', 'Playlist'],


include_tables=['Track', 'Playlist'],    sample_rows_in_table_info=2,    custom_table_info=custom_table_info)print(db.table_info)        CREATE TABLE "Playlist" (        "PlaylistId" INTEGER NOT NULL,         "Name" NVARCHAR(120),         PRIMARY KEY ("PlaylistId")    )        /*    2 rows from Playlist table:    PlaylistId  Name    1   Music    2   Movies    */        CREATE TABLE Track (        "TrackId" INTEGER NOT NULL,         "Name" NVARCHAR(200) NOT NULL,        "Composer" NVARCHAR(220),        PRIMARY KEY ("TrackId")    )    /*    3 rows from Track table:    TrackId Name    Composer    1   For Those About To Rock (We Salute You) Angus Young, Malcolm Young, Brian Johnson    2   Balls to the Wall   None    3   My favorite song ever   The coolest composer of all time    */Note how our custom table definition and sample rows for Track overrides the sample_rows_in_table_info parameter. Tables that are not overridden by custom_table_info, in this example Playlist, will have their table info gathered automatically as usual.db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)db_chain.run("What are some example tracks by Bach?")            > Entering new


tracks by Bach?")            > Entering new SQLDatabaseChain chain...    What are some example tracks by Bach?    SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;    SQLResult: [('American Woman',), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude',), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata',)]    Answer:text='You are a SQLite expert. Given an input question, first create a syntactically correct SQLite query to run, then look at the results of the query and return the answer to the input question.\nUnless the user specifies in the question a specific number of examples to obtain, query for at most 5 results using the LIMIT clause as per SQLite. You can order the results to return the most informative data in the database.\nNever query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.\nPay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.\n\nUse the following format:\n\nQuestion: "Question here"\nSQLQuery: "SQL Query to run"\nSQLResult: "Result of the SQLQuery"\nAnswer: "Final answer here"\n\nOnly use the following


of the SQLQuery"\nAnswer: "Final answer here"\n\nOnly use the following tables:\n\nCREATE TABLE "Playlist" (\n\t"PlaylistId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("PlaylistId")\n)\n\n/*\n2 rows from Playlist table:\nPlaylistId\tName\n1\tMusic\n2\tMovies\n*/\n\nCREATE TABLE Track (\n\t"TrackId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(200) NOT NULL,\n\t"Composer" NVARCHAR(220),\n\tPRIMARY KEY ("TrackId")\n)\n/*\n3 rows from Track table:\nTrackId\tName\tComposer\n1\tFor Those About To Rock (We Salute You)\tAngus Young, Malcolm Young, Brian Johnson\n2\tBalls to the Wall\tNone\n3\tMy favorite song ever\tThe coolest composer of all time\n*/\n\nQuestion: What are some example tracks by Bach?\nSQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE \'%Bach%\' LIMIT 5;\nSQLResult: [(\'American Woman\',), (\'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\',), (\'Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria\',), (\'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude\',), (\'Toccata and Fugue in D Minor, BWV 565: I. Toccata\',)]\nAnswer:'    You are a SQLite expert. Given an input question, first create a syntactically correct SQLite query to run, then look at the results of the


first create a syntactically correct SQLite query to run, then look at the results of the query and return the answer to the input question.    Unless the user specifies in the question a specific number of examples to obtain, query for at most 5 results using the LIMIT clause as per SQLite. You can order the results to return the most informative data in the database.    Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.    Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.        Use the following format:        Question: "Question here"    SQLQuery: "SQL Query to run"    SQLResult: "Result of the SQLQuery"    Answer: "Final answer here"        Only use the following tables:        CREATE TABLE "Playlist" (        "PlaylistId" INTEGER NOT NULL,         "Name" NVARCHAR(120),         PRIMARY KEY ("PlaylistId")    )        /*    2 rows from Playlist table:    PlaylistId  Name    1   Music    2   Movies    */        CREATE TABLE Track (        "TrackId" INTEGER NOT NULL,         "Name" NVARCHAR(200) NOT NULL,        "Composer"


NVARCHAR(200) NOT NULL,        "Composer" NVARCHAR(220),        PRIMARY KEY ("TrackId")    )    /*    3 rows from Track table:    TrackId Name    Composer    1   For Those About To Rock (We Salute You) Angus Young, Malcolm Young, Brian Johnson    2   Balls to the Wall   None    3   My favorite song ever   The coolest composer of all time    */        Question: What are some example tracks by Bach?    SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;    SQLResult: [('American Woman',), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude',), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata',)]    Answer:    {'input': 'What are some example tracks by Bach?\nSQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE \'%Bach%\' LIMIT 5;\nSQLResult: [(\'American Woman\',), (\'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\',), (\'Aria Mit 30 VerÃ¤nderungen, BWV 988 "Goldberg Variations": Aria\',), (\'Suite for Solo Cello No.


"Goldberg Variations": Aria\',), (\'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude\',), (\'Toccata and Fugue in D Minor, BWV 565: I. Toccata\',)]\nAnswer:', 'top_k': '5', 'dialect': 'sqlite', 'table_info': '\nCREATE TABLE "Playlist" (\n\t"PlaylistId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(120), \n\tPRIMARY KEY ("PlaylistId")\n)\n\n/*\n2 rows from Playlist table:\nPlaylistId\tName\n1\tMusic\n2\tMovies\n*/\n\nCREATE TABLE Track (\n\t"TrackId" INTEGER NOT NULL, \n\t"Name" NVARCHAR(200) NOT NULL,\n\t"Composer" NVARCHAR(220),\n\tPRIMARY KEY ("TrackId")\n)\n/*\n3 rows from Track table:\nTrackId\tName\tComposer\n1\tFor Those About To Rock (We Salute You)\tAngus Young, Malcolm Young, Brian Johnson\n2\tBalls to the Wall\tNone\n3\tMy favorite song ever\tThe coolest composer of all time\n*/', 'stop': ['\nSQLResult:']}    Examples of tracks by Bach include "American Woman", "Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace", "Aria Mit 30 VerÃ¤nderungen, BWV 988 'Goldberg Variations': Aria", "Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude", and "Toccata and Fugue in D Minor, BWV 565: I. Toccata".


in D Minor, BWV 565: I. Toccata".    > Finished chain.    'Examples of tracks by Bach include "American Woman", "Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace", "Aria Mit 30 VerÃ¤nderungen, BWV 988 \'Goldberg Variations\': Aria", "Suite for Solo Cello No. 1 in G Major, BWV 1007: I. PrÃ©lude", and "Toccata and Fugue in D Minor, BWV 565: I. Toccata".'SQL Viewsâ€‹In some case, the table schema can be hidden behind a JSON or JSONB column. Adding row samples into the prompt might help won't always describe the data perfectly. For this reason, a custom SQL views can help.CREATE VIEW accounts_v AS    select id, firstname, lastname, email, created_at, updated_at,        cast(stats->>'total_post' as int) as total_post,        cast(stats->>'total_comments' as int) as total_comments,        cast(stats->>'ltv' as int) as ltv        FROM accounts;Then limit the tables visible from SQLDatabase to the created view.db = SQLDatabase.from_uri(    "sqlite:///../../../../notebooks/Chinook.db",    include_tables=['accounts_v']) # we include only the viewSQLDatabaseSequentialChainâ€‹Chain for querying SQL database that is a sequential chain.The chain is as follows:1. Based on the query, determine which tables to use.2. Based on those tables, call the normal SQL database chain.This is useful in cases where the number of tables in the database is large.from


normal SQL database chain.This is useful in cases where the number of tables in the database is large.from langchain.chains import SQLDatabaseSequentialChaindb = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")chain = SQLDatabaseSequentialChain.from_llm(llm, db, verbose=True)chain.run("How many employees are also customers?")            > Entering new SQLDatabaseSequentialChain chain...    Table names to use:    ['Employee', 'Customer']        > Entering new SQLDatabaseChain chain...    How many employees are also customers?    SQLQuery:SELECT COUNT(*) FROM Employee e INNER JOIN Customer c ON e.EmployeeId = c.SupportRepId;    SQLResult: [(59,)]    Answer:59 employees are also customers.    > Finished chain.        > Finished chain.    '59 employees are also customers.'Using Local Language Modelsâ€‹Sometimes you may not have the luxury of using OpenAI or other service-hosted large language model. You can, ofcourse, try to use the SQLDatabaseChain with a local model, but will quickly realize that most models you can run locally even with a large GPU struggle to generate the right output.import loggingimport torchfrom transformers import AutoTokenizer, GPT2TokenizerFast, pipeline, AutoModelForSeq2SeqLM, AutoModelForCausalLMfrom langchain import HuggingFacePipeline# Note: This model requires a large GPU, e.g. an 80GB A100. See documentation for other ways to run private non-OpenAI models.model_id = "google/flan-ul2"model = AutoModelForSeq2SeqLM.from_pretrained(model_id, temperature=0)device_id = -1  # default to no-GPU, but use GPU


temperature=0)device_id = -1  # default to no-GPU, but use GPU and half precision mode if availableif torch.cuda.is_available():    device_id = 0    try:        model = model.half()    except RuntimeError as exc:        logging.warn(f"Could not run model in half precision mode: {str(exc)}")tokenizer = AutoTokenizer.from_pretrained(model_id)pipe = pipeline(task="text2text-generation", model=model, tokenizer=tokenizer, max_length=1024, device=device_id)local_llm = HuggingFacePipeline(pipeline=pipe)    /workspace/langchain/.venv/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html      from .autonotebook import tqdm as notebook_tqdm    Loading checkpoint shards: 100%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 8/8 [00:32<00:00,  4.11s/it]from langchain import SQLDatabase, SQLDatabaseChaindb = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db", include_tables=['Customer'])local_chain = SQLDatabaseChain.from_llm(local_llm, db, verbose=True, return_intermediate_steps=True, use_query_checker=True)This model should work for very simple SQL queries, as long as you use the query checker as specified above, e.g.:local_chain("How many customers are there?")


many customers are there?")            > Entering new SQLDatabaseChain chain...    How many customers are there?    SQLQuery:    /workspace/langchain/.venv/lib/python3.9/site-packages/transformers/pipelines/base.py:1070: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset      warnings.warn(    /workspace/langchain/.venv/lib/python3.9/site-packages/transformers/pipelines/base.py:1070: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset      warnings.warn(    SELECT count(*) FROM Customer    SQLResult: [(59,)]    Answer:    /workspace/langchain/.venv/lib/python3.9/site-packages/transformers/pipelines/base.py:1070: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset      warnings.warn(    [59]    > Finished chain.    {'query': 'How many customers are there?',     'result': '[59]',     'intermediate_steps': [{'input': 'How many customers are there?\nSQLQuery:SELECT count(*) FROM Customer\nSQLResult: [(59,)]\nAnswer:',       'top_k': '5',       'dialect': 'sqlite',       'table_info': '\nCREATE TABLE "Customer" (\n\t"CustomerId" INTEGER NOT NULL, \n\t"FirstName" NVARCHAR(40) NOT NULL, \n\t"LastName" NVARCHAR(20) NOT NULL,


NOT NULL, \n\t"LastName" NVARCHAR(20) NOT NULL, \n\t"Company" NVARCHAR(80), \n\t"Address" NVARCHAR(70), \n\t"City" NVARCHAR(40), \n\t"State" NVARCHAR(40), \n\t"Country" NVARCHAR(40), \n\t"PostalCode" NVARCHAR(10), \n\t"Phone" NVARCHAR(24), \n\t"Fax" NVARCHAR(24), \n\t"Email" NVARCHAR(60) NOT NULL, \n\t"SupportRepId" INTEGER, \n\tPRIMARY KEY ("CustomerId"), \n\tFOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")\n)\n\n/*\n3 rows from Customer table:\nCustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n1\tLuÃs\tGonÃ§alves\tEmbraer - Empresa Brasileira de AeronÃ¡utica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSÃ£o JosÃ© dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n2\tLeonie\tKÃ¶hler\tNone\tTheodor-Heuss-StraÃŸe 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n3\tFranÃ§ois\tTremblay\tNone\t1498 rue BÃ©langer\tMontrÃ©al\tQC\tCanada\tH2G


rue BÃ©langer\tMontrÃ©al\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n*/',       'stop': ['\nSQLResult:']},      'SELECT count(*) FROM Customer',      {'query': 'SELECT count(*) FROM Customer', 'dialect': 'sqlite'},      'SELECT count(*) FROM Customer',      '[(59,)]']}Even this relatively large model will most likely fail to generate more complicated SQL by itself. However, you can log its inputs and outputs so that you can hand-correct them and use the corrected examples for few shot prompt examples later. In practice, you could log any executions of your chain that raise exceptions (as shown in the example below) or get direct user feedback in cases where the results are incorrect (but did not raise an exception).poetry run pip install pyyaml chromadbimport yaml    huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...    To disable this warning, you can either:        - Avoid using `tokenizers` before the fork if possible        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)    11842.36s - pydevd: Sending message related to process being replaced timed-out after 5 seconds    Requirement already satisfied: pyyaml in /workspace/langchain/.venv/lib/python3.9/site-packages (6.0)    Requirement already satisfied: chromadb in /workspace/langchain/.venv/lib/python3.9/site-packages (0.3.21)


(0.3.21)    Requirement already satisfied: pandas>=1.3 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (2.0.1)    Requirement already satisfied: requests>=2.28 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (2.28.2)    Requirement already satisfied: pydantic>=1.9 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (1.10.7)    Requirement already satisfied: hnswlib>=0.7 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (0.7.0)    Requirement already satisfied: clickhouse-connect>=0.5.7 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (0.5.20)    Requirement already satisfied: sentence-transformers>=2.2.2 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (2.2.2)    Requirement already satisfied: duckdb>=0.7.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (0.7.1)    Requirement already satisfied: fastapi>=0.85.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (0.95.1)    Requirement already satisfied: uvicorn[standard]>=0.18.3 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (0.21.1)    Requirement already satisfied: numpy>=1.21.6 in


Requirement already satisfied: numpy>=1.21.6 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (1.24.3)    Requirement already satisfied: posthog>=2.4.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from chromadb) (3.0.1)    Requirement already satisfied: certifi in /workspace/langchain/.venv/lib/python3.9/site-packages (from clickhouse-connect>=0.5.7->chromadb) (2022.12.7)    Requirement already satisfied: urllib3>=1.26 in /workspace/langchain/.venv/lib/python3.9/site-packages (from clickhouse-connect>=0.5.7->chromadb) (1.26.15)    Requirement already satisfied: pytz in /workspace/langchain/.venv/lib/python3.9/site-packages (from clickhouse-connect>=0.5.7->chromadb) (2023.3)    Requirement already satisfied: zstandard in /workspace/langchain/.venv/lib/python3.9/site-packages (from clickhouse-connect>=0.5.7->chromadb) (0.21.0)    Requirement already satisfied: lz4 in /workspace/langchain/.venv/lib/python3.9/site-packages (from clickhouse-connect>=0.5.7->chromadb) (4.3.2)    Requirement already satisfied: starlette<0.27.0,>=0.26.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from fastapi>=0.85.1->chromadb) (0.26.1)    Requirement already satisfied: python-dateutil>=2.8.2 in


Requirement already satisfied: python-dateutil>=2.8.2 in /workspace/langchain/.venv/lib/python3.9/site-packages (from pandas>=1.3->chromadb) (2.8.2)    Requirement already satisfied: tzdata>=2022.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from pandas>=1.3->chromadb) (2023.3)    Requirement already satisfied: six>=1.5 in /workspace/langchain/.venv/lib/python3.9/site-packages (from posthog>=2.4.0->chromadb) (1.16.0)    Requirement already satisfied: monotonic>=1.5 in /workspace/langchain/.venv/lib/python3.9/site-packages (from posthog>=2.4.0->chromadb) (1.6)    Requirement already satisfied: backoff>=1.10.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from posthog>=2.4.0->chromadb) (2.2.1)    Requirement already satisfied: typing-extensions>=4.2.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from pydantic>=1.9->chromadb) (4.5.0)    Requirement already satisfied: charset-normalizer<4,>=2 in /workspace/langchain/.venv/lib/python3.9/site-packages (from requests>=2.28->chromadb) (3.1.0)    Requirement already satisfied: idna<4,>=2.5 in /workspace/langchain/.venv/lib/python3.9/site-packages (from requests>=2.28->chromadb) (3.4)    Requirement already satisfied:


(3.4)    Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (4.28.1)    Requirement already satisfied: tqdm in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (4.65.0)    Requirement already satisfied: torch>=1.6.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (1.13.1)    Requirement already satisfied: torchvision in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (0.14.1)    Requirement already satisfied: scikit-learn in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (1.2.2)    Requirement already satisfied: scipy in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (1.9.3)    Requirement already satisfied: nltk in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (3.8.1)    Requirement already satisfied: sentencepiece in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (0.1.98)    Requirement already satisfied:


(0.1.98)    Requirement already satisfied: huggingface-hub>=0.4.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from sentence-transformers>=2.2.2->chromadb) (0.13.4)    Requirement already satisfied: click>=7.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (8.1.3)    Requirement already satisfied: h11>=0.8 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.14.0)    Requirement already satisfied: httptools>=0.5.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.5.0)    Requirement already satisfied: python-dotenv>=0.13 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (1.0.0)    Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.17.0)    Requirement already satisfied: watchfiles>=0.13 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.19.0)    Requirement already satisfied: websockets>=10.4


(0.19.0)    Requirement already satisfied: websockets>=10.4 in /workspace/langchain/.venv/lib/python3.9/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (11.0.2)    Requirement already satisfied: filelock in /workspace/langchain/.venv/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence-transformers>=2.2.2->chromadb) (3.12.0)    Requirement already satisfied: packaging>=20.9 in /workspace/langchain/.venv/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence-transformers>=2.2.2->chromadb) (23.1)    Requirement already satisfied: anyio<5,>=3.4.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from starlette<0.27.0,>=0.26.1->fastapi>=0.85.1->chromadb) (3.6.2)    Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /workspace/langchain/.venv/lib/python3.9/site-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (11.7.99)    Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /workspace/langchain/.venv/lib/python3.9/site-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (8.5.0.96)    Requirement already satisfied:


(8.5.0.96)    Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /workspace/langchain/.venv/lib/python3.9/site-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (11.10.3.66)    Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /workspace/langchain/.venv/lib/python3.9/site-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (11.7.99)    Requirement already satisfied: setuptools in /workspace/langchain/.venv/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (67.7.1)    Requirement already satisfied: wheel in /workspace/langchain/.venv/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb) (0.40.0)    Requirement already satisfied: regex!=2019.12.17 in /workspace/langchain/.venv/lib/python3.9/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers>=2.2.2->chromadb) (2023.3.23)    Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from


in /workspace/langchain/.venv/lib/python3.9/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers>=2.2.2->chromadb) (0.13.3)    Requirement already satisfied: joblib in /workspace/langchain/.venv/lib/python3.9/site-packages (from nltk->sentence-transformers>=2.2.2->chromadb) (1.2.0)    Requirement already satisfied: threadpoolctl>=2.0.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from scikit-learn->sentence-transformers>=2.2.2->chromadb) (3.1.0)    Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /workspace/langchain/.venv/lib/python3.9/site-packages (from torchvision->sentence-transformers>=2.2.2->chromadb) (9.5.0)    Requirement already satisfied: sniffio>=1.1 in /workspace/langchain/.venv/lib/python3.9/site-packages (from anyio<5,>=3.4.0->starlette<0.27.0,>=0.26.1->fastapi>=0.85.1->chromadb) (1.3.0)from typing import DictQUERY = "List all the customer first names that start with 'a'"def _parse_example(result: Dict) -> Dict:    sql_cmd_key = "sql_cmd"    sql_result_key = "sql_result"    table_info_key = "table_info"    input_key = "input"    final_answer_key = "answer"    _example = {        "input": result.get("query"),


= {        "input": result.get("query"),    }    steps = result.get("intermediate_steps")    answer_key = sql_cmd_key # the first one    for step in steps:        # The steps are in pairs, a dict (input) followed by a string (output).        # Unfortunately there is no schema but you can look at the input key of the        # dict to see what the output is supposed to be        if isinstance(step, dict):            # Grab the table info from input dicts in the intermediate steps once            if table_info_key not in _example:                _example[table_info_key] = step.get(table_info_key)            if input_key in step:                if step[input_key].endswith("SQLQuery:"):                    answer_key = sql_cmd_key # this is the SQL generation input                if step[input_key].endswith("Answer:"):                    answer_key = final_answer_key # this is the final answer input            elif sql_cmd_key in step:                _example[sql_cmd_key] = step[sql_cmd_key]


= step[sql_cmd_key]                answer_key = sql_result_key # this is SQL execution input        elif isinstance(step, str):            # The preceding element should have set the answer_key            _example[answer_key] = step    return _exampleexample: anytry:    result = local_chain(QUERY)    print("*** Query succeeded")    example = _parse_example(result)except Exception as exc:    print("*** Query failed")    result = {        "query": QUERY,        "intermediate_steps": exc.intermediate_steps    }    example = _parse_example(result)# print for now, in reality you may want to write this out to a YAML file or database for manual fix-ups offlineyaml_example = yaml.dump(example, allow_unicode=True)print("\n" + yaml_example)            > Entering new SQLDatabaseChain chain...    List all the customer first names that start with 'a'    SQLQuery:    /workspace/langchain/.venv/lib/python3.9/site-packages/transformers/pipelines/base.py:1070: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset      warnings.warn(    SELECT firstname FROM customer WHERE firstname LIKE '%a%'    SQLResult: [('FranÃ§ois',), ('FrantiÅ¡ek',), ('Helena',), ('Astrid',), ('Daan',), ('Kara',), ('Eduardo',), ('Alexandre',),


('Kara',), ('Eduardo',), ('Alexandre',), ('Fernanda',), ('Mark',), ('Frank',), ('Jack',), ('Dan',), ('Kathy',), ('Heather',), ('Frank',), ('Richard',), ('Patrick',), ('Julia',), ('Edward',), ('Martha',), ('Aaron',), ('Madalena',), ('Hannah',), ('Niklas',), ('Camille',), ('Marc',), ('Wyatt',), ('Isabelle',), ('Ladislav',), ('Lucas',), ('Johannes',), ('StanisÅ‚aw',), ('Joakim',), ('Emma',), ('Mark',), ('Manoj',), ('Puja',)]    Answer:    /workspace/langchain/.venv/lib/python3.9/site-packages/transformers/pipelines/base.py:1070: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset      warnings.warn(    [('FranÃ§ois', 'Frantiek', 'Helena', 'Astrid', 'Daan', 'Kara', 'Eduardo', 'Alexandre', 'Fernanda', 'Mark', 'Frank', 'Jack', 'Dan', 'Kathy', 'Heather', 'Frank', 'Richard', 'Patrick', 'Julia', 'Edward', 'Martha', 'Aaron', 'Madalena', 'Hannah', 'Niklas', 'Camille', 'Marc', 'Wyatt', 'Isabelle', 'Ladislav', 'Lucas', 'Johannes', 'Stanisaw', 'Joakim', 'Emma', 'Mark', 'Manoj', 'Puja']    > Finished chain.    *** Query succeeded


'Puja']    > Finished chain.    *** Query succeeded        answer: '[(''FranÃ§ois'', ''Frantiek'', ''Helena'', ''Astrid'', ''Daan'', ''Kara'',      ''Eduardo'', ''Alexandre'', ''Fernanda'', ''Mark'', ''Frank'', ''Jack'', ''Dan'',      ''Kathy'', ''Heather'', ''Frank'', ''Richard'', ''Patrick'', ''Julia'', ''Edward'',      ''Martha'', ''Aaron'', ''Madalena'', ''Hannah'', ''Niklas'', ''Camille'', ''Marc'',      ''Wyatt'', ''Isabelle'', ''Ladislav'', ''Lucas'', ''Johannes'', ''Stanisaw'', ''Joakim'',      ''Emma'', ''Mark'', ''Manoj'', ''Puja'']'    input: List all the customer first names that start with 'a'    sql_cmd: SELECT firstname FROM customer WHERE firstname LIKE '%a%'    sql_result: '[(''FranÃ§ois'',), (''FrantiÅ¡ek'',), (''Helena'',), (''Astrid'',), (''Daan'',),      (''Kara'',), (''Eduardo'',), (''Alexandre'',), (''Fernanda'',), (''Mark'',), (''Frank'',),      (''Jack'',), (''Dan'',), (''Kathy'',), (''Heather'',), (''Frank'',), (''Richard'',),      (''Patrick'',), (''Julia'',), (''Edward'',), (''Martha'',), (''Aaron'',),


(''Edward'',), (''Martha'',), (''Aaron'',), (''Madalena'',),      (''Hannah'',), (''Niklas'',), (''Camille'',), (''Marc'',), (''Wyatt'',), (''Isabelle'',),      (''Ladislav'',), (''Lucas'',), (''Johannes'',), (''StanisÅ‚aw'',), (''Joakim'',),      (''Emma'',), (''Mark'',), (''Manoj'',), (''Puja'',)]'    table_info: "\nCREATE TABLE \"Customer\" (\n\t\"CustomerId\" INTEGER NOT NULL, \n\t\      \"FirstName\" NVARCHAR(40) NOT NULL, \n\t\"LastName\" NVARCHAR(20) NOT NULL, \n\t\      \"Company\" NVARCHAR(80), \n\t\"Address\" NVARCHAR(70), \n\t\"City\" NVARCHAR(40),\      \ \n\t\"State\" NVARCHAR(40), \n\t\"Country\" NVARCHAR(40), \n\t\"PostalCode\" NVARCHAR(10),\      \ \n\t\"Phone\" NVARCHAR(24), \n\t\"Fax\" NVARCHAR(24), \n\t\"Email\" NVARCHAR(60)\      \ NOT NULL, \n\t\"SupportRepId\" INTEGER, \n\tPRIMARY KEY (\"CustomerId\"), \n\t\      FOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n)\n\n/*\n\      3 rows from Customer table:\nCustomerId\tFirstName\tLastName\tCompany\tAddress\t\


3 rows from Customer table:\nCustomerId\tFirstName\tLastName\tCompany\tAddress\t\      City\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n1\tLuÃs\tGonÃ§alves\t\      Embraer - Empresa Brasileira de AeronÃ¡utica S.A.\tAv. Brigadeiro Faria Lima, 2170\t\      SÃ£o JosÃ© dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\t\      luisg@embraer.com.br\t3\n2\tLeonie\tKÃ¶hler\tNone\tTheodor-Heuss-StraÃŸe 34\tStuttgart\t\      None\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n3\tFranÃ§ois\t\      Tremblay\tNone\t1498 rue BÃ©langer\tMontrÃ©al\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\t\      None\tftremblay@gmail.com\t3\n*/"    Run the snippet above a few times, or log exceptions in your deployed environment, to collect lots of examples of inputs, table_info and sql_cmd generated by your language model. The sql_cmd values will be incorrect and you can manually fix them up to build a collection of examples, e.g. here we are using YAML to keep a neat record of our inputs and corrected SQL output that we can build up over time.YAML_EXAMPLES = """- input: How


SQL output that we can build up over time.YAML_EXAMPLES = """- input: How many customers are not from Brazil?  table_info: |    CREATE TABLE "Customer" (      "CustomerId" INTEGER NOT NULL,       "FirstName" NVARCHAR(40) NOT NULL,       "LastName" NVARCHAR(20) NOT NULL,       "Company" NVARCHAR(80),       "Address" NVARCHAR(70),       "City" NVARCHAR(40),       "State" NVARCHAR(40),       "Country" NVARCHAR(40),       "PostalCode" NVARCHAR(10),       "Phone" NVARCHAR(24),       "Fax" NVARCHAR(24),       "Email" NVARCHAR(60) NOT NULL,       "SupportRepId" INTEGER,       PRIMARY KEY ("CustomerId"),       FOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")    )  sql_cmd: SELECT COUNT(*) FROM "Customer" WHERE NOT "Country" = "Brazil";  sql_result: "[(54,)]"  answer: 54 customers are not from Brazil.- input: list all the genres that start with 'r'  table_info: |    CREATE TABLE "Genre" (      "GenreId" INTEGER NOT NULL,       "Name" NVARCHAR(120),       PRIMARY KEY ("GenreId")    )    /*    3 rows from Genre table:    GenreId Name


/*    3 rows from Genre table:    GenreId Name    1   Rock    2   Jazz    3   Metal    */  sql_cmd: SELECT "Name" FROM "Genre" WHERE "Name" LIKE 'r%';  sql_result: "[('Rock',), ('Rock and Roll',), ('Reggae',), ('R&B/Soul',)]"  answer: The genres that start with 'r' are Rock, Rock and Roll, Reggae and R&B/Soul. """Now that you have some examples (with manually corrected output SQL), you can do few shot prompt seeding the usual way:from langchain import FewShotPromptTemplate, PromptTemplatefrom langchain.chains.sql_database.prompt import _sqlite_prompt, PROMPT_SUFFIXfrom langchain.embeddings.huggingface import HuggingFaceEmbeddingsfrom langchain.prompts.example_selector.semantic_similarity import SemanticSimilarityExampleSelectorfrom langchain.vectorstores import Chromaexample_prompt = PromptTemplate(    input_variables=["table_info", "input", "sql_cmd", "sql_result", "answer"],    template="{table_info}\n\nQuestion: {input}\nSQLQuery: {sql_cmd}\nSQLResult: {sql_result}\nAnswer: {answer}",)examples_dict = yaml.safe_load(YAML_EXAMPLES)local_embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")example_selector = SemanticSimilarityExampleSelector.from_examples(                        # This is the list of examples available to select from.                        examples_dict,


examples_dict,                        # This is the embedding class used to produce embeddings which are used to measure semantic similarity.                        local_embeddings,                        # This is the VectorStore class that is used to store the embeddings and do a similarity search over.                        Chroma,  # type: ignore                        # This is the number of examples to produce and include per prompt                        k=min(3, len(examples_dict)),                    )few_shot_prompt = FewShotPromptTemplate(    example_selector=example_selector,    example_prompt=example_prompt,    prefix=_sqlite_prompt + "Here are some examples:",    suffix=PROMPT_SUFFIX,    input_variables=["table_info", "input", "top_k"],)    Using embedded DuckDB without persistence: data will be transientThe model should do better now with this few shot prompt, especially for inputs similar to the examples you have seeded it with.local_chain = SQLDatabaseChain.from_llm(local_llm, db, prompt=few_shot_prompt, use_query_checker=True, verbose=True, return_intermediate_steps=True)result = local_chain("How many customers are from Brazil?")            >


many customers are from Brazil?")            > Entering new SQLDatabaseChain chain...    How many customers are from Brazil?    SQLQuery:SELECT count(*) FROM Customer WHERE Country = "Brazil";    SQLResult: [(5,)]    Answer:[5]    > Finished chain.result = local_chain("How many customers are not from Brazil?")            > Entering new SQLDatabaseChain chain...    How many customers are not from Brazil?    SQLQuery:SELECT count(*) FROM customer WHERE country NOT IN (SELECT country FROM customer WHERE country = 'Brazil')    SQLResult: [(54,)]    Answer:54 customers are not from Brazil.    > Finished chain.result = local_chain("How many customers are there in total?")            > Entering new SQLDatabaseChain chain...    How many customers are there in total?    SQLQuery:SELECT count(*) FROM Customer;    SQLResult: [(59,)]    Answer:There are 59 customers in total.    > Finished chain.PreviousUsing OpenAI functionsNextSummarizationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Summarization | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsHow toFoundationalDocumentsPopularAPI chainsRetrieval QAConversational Retrieval QAUsing OpenAI functionsSQLSummarizationAdditionalMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesChainsPopularSummarizationSummarizationA summarization chain can be used to summarize multiple documents. One way is to input multiple smaller documents, after they have been divided into chunks, and operate over them with a MapReduceDocumentsChain. You can also choose instead for the chain that does summarization to be a StuffDocumentsChain, or a RefineDocumentsChain.Prepare Dataâ€‹First we prepare the data. For this example we create multiple documents from one long one, but these documents could be fetched in any manner (the point of this notebook to highlight what to do AFTER you fetch the documents).from langchain import OpenAI, PromptTemplate, LLMChainfrom langchain.text_splitter import CharacterTextSplitterfrom langchain.chains.mapreduce import MapReduceChainfrom langchain.prompts import PromptTemplatellm = OpenAI(temperature=0)text_splitter = CharacterTextSplitter()with open("../../state_of_the_union.txt") as f:    state_of_the_union = f.read()texts = text_splitter.split_text(state_of_the_union)from langchain.docstore.document import Documentdocs = [Document(page_content=t) for t in texts[:3]]Quickstartâ€‹If you just want to get started as quickly as possible, this is the recommended way to do it:from langchain.chains.summarize import load_summarize_chainchain = load_summarize_chain(llm, chain_type="map_reduce")chain.run(docs)


= load_summarize_chain(llm, chain_type="map_reduce")chain.run(docs)    ' In response to Russian aggression in Ukraine, the United States and its allies are taking action to hold Putin accountable, including economic sanctions, asset seizures, and military assistance. The US is also providing economic and humanitarian aid to Ukraine, and has passed the American Rescue Plan and the Bipartisan Infrastructure Law to help struggling families and create jobs. The US remains unified and determined to protect Ukraine and the free world.'If you want more control and understanding over what is happening, please see the information below.The stuff Chainâ€‹This sections shows results of using the stuff Chain to do summarization.chain = load_summarize_chain(llm, chain_type="stuff")chain.run(docs)    ' In his speech, President Biden addressed the crisis in Ukraine, the American Rescue Plan, and the Bipartisan Infrastructure Law. He discussed the need to invest in America, educate Americans, and build the economy from the bottom up. He also announced the release of 60 million barrels of oil from reserves around the world, and the creation of a dedicated task force to go after the crimes of Russian oligarchs. He concluded by emphasizing the need to Buy American and use taxpayer dollars to rebuild America.'Custom PromptsYou can also use your own prompts with this chain. In this example, we will respond in Italian.prompt_template = """Write a concise summary of the following:{text}CONCISE SUMMARY IN ITALIAN:"""PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])chain = load_summarize_chain(llm, chain_type="stuff", prompt=PROMPT)chain.run(docs)    "\n\nIn questa serata, il Presidente degli Stati Uniti ha annunciato una serie di misure per affrontare la crisi in Ucraina, causata dall'aggressione di Putin. Ha anche


la crisi in Ucraina, causata dall'aggressione di Putin. Ha anche annunciato l'invio di aiuti economici, militari e umanitari all'Ucraina. Ha anche annunciato che gli Stati Uniti e i loro alleati stanno imponendo sanzioni economiche a Putin e stanno rilasciando 60 milioni di barili di petrolio dalle riserve di tutto il mondo. Inoltre, ha annunciato che il Dipartimento di Giustizia degli Stati Uniti sta creando una task force dedicata ai crimini degli oligarchi russi. Il Presidente ha anche annunciato l'approvazione della legge bipartitica sull'infrastruttura, che prevede investimenti per la ricostruzione dell'America. Questo porterÃ  a creare posti"The map_reduce Chainâ€‹This sections shows results of using the map_reduce Chain to do summarization.chain = load_summarize_chain(llm, chain_type="map_reduce")chain.run(docs)    " In response to Russia's aggression in Ukraine, the United States and its allies have imposed economic sanctions and are taking other measures to hold Putin accountable. The US is also providing economic and military assistance to Ukraine, protecting NATO countries, and releasing oil from its Strategic Petroleum Reserve. President Biden and Vice President Harris have passed legislation to help struggling families and rebuild America's infrastructure."Intermediate StepsWe can also return the intermediate steps for map_reduce chains, should we want to inspect them. This is done with the return_map_steps variable.chain = load_summarize_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)chain({"input_documents": docs}, return_only_outputs=True)    {'map_steps': [" In response to Russia's aggression in Ukraine, the United States has united with


{'map_steps': [" In response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains.",      ' The United States and its European allies are taking action to punish Russia for its invasion of Ukraine, including seizing assets, closing off airspace, and providing economic and military assistance to Ukraine. The US is also mobilizing forces to protect NATO countries and has released 30 million barrels of oil from its Strategic Petroleum Reserve to help blunt gas prices. The world is uniting in support of Ukraine and democracy, and the US stands with its Ukrainian-American citizens.',      " President Biden and Vice President Harris ran for office with a new economic vision for America, and have since passed the American Rescue Plan and the Bipartisan Infrastructure Law to help struggling families and rebuild America's infrastructure. This includes creating jobs, modernizing roads, airports, ports, and waterways, replacing lead pipes, providing affordable high-speed internet, and investing in American products to support American jobs."],     'output_text': " In response to Russia's aggression in Ukraine, the United States and its allies have imposed economic sanctions and are taking other measures to hold Putin accountable. The US is also providing economic and military assistance to Ukraine, protecting NATO countries, and passing legislation to help struggling families and rebuild America's infrastructure. The world is uniting in support of Ukraine and democracy, and the US stands with its Ukrainian-American citizens."}Custom PromptsYou can also use your own prompts with this chain. In this example, we will respond in Italian.prompt_template = """Write a concise summary of the following:{text}CONCISE SUMMARY IN ITALIAN:"""PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])chain =


= PromptTemplate(template=prompt_template, input_variables=["text"])chain = load_summarize_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True, map_prompt=PROMPT, combine_prompt=PROMPT)chain({"input_documents": docs}, return_only_outputs=True)    {'intermediate_steps': ["\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Gli Stati Uniti e i loro alleati stanno ora imponendo sanzioni economiche a Putin e stanno tagliando l'accesso della Russia alla tecnologia. Il Dipartimento di Giustizia degli Stati Uniti sta anche creando una task force dedicata per andare dopo i crimini degli oligarchi russi.",      "\n\nStiamo unendo le nostre forze con quelle dei nostri alleati europei per sequestrare yacht, appartamenti di lusso e jet privati di Putin. Abbiamo chiuso lo spazio aereo americano ai voli russi e stiamo fornendo piÃ¹ di un miliardo di dollari in assistenza all'Ucraina. Abbiamo anche mobilitato le nostre forze terrestri, aeree e navali per proteggere i paesi della NATO. Abbiamo anche rilasciato 60 milioni di barili di petrolio dalle riserve di tutto il mondo, di cui 30 milioni dalla nostra riserva strategica di petrolio. Stiamo affrontando una prova reale e ci vorrÃ  del tempo, ma alla fine Putin non


una prova reale e ci vorrÃ  del tempo, ma alla fine Putin non riuscirÃ  a spegnere l'amore dei popoli per la libertÃ .",      "\n\nIl Presidente Biden ha lottato per passare l'American Rescue Plan per aiutare le persone che soffrivano a causa della pandemia. Il piano ha fornito sollievo economico immediato a milioni di americani, ha aiutato a mettere cibo sulla loro tavola, a mantenere un tetto sopra le loro teste e a ridurre il costo dell'assicurazione sanitaria. Il piano ha anche creato piÃ¹ di 6,5 milioni di nuovi posti di lavoro, il piÃ¹ alto numero di posti di lavoro creati in un anno nella storia degli Stati Uniti. Il Presidente Biden ha anche firmato la legge bipartitica sull'infrastruttura, la piÃ¹ ampia iniziativa di ricostruzione della storia degli Stati Uniti. Il piano prevede di modernizzare le strade, gli aeroporti, i porti e le vie navigabili in"],     'output_text': "\n\nIl Presidente Biden sta lavorando per aiutare le persone che soffrono a causa della pandemia attraverso l'American Rescue Plan e la legge bipartitica sull'infrastruttura. Gli Stati Uniti e i loro alleati stanno anche imponendo sanzioni economiche a Putin e tagliando l'accesso della Russia alla tecnologia. Stanno anche sequestrando yacht, appartamenti di lusso e jet privati di Putin e fornendo piÃ¹ di un miliardo di dollari in assistenza all'Ucraina. Alla


di un miliardo di dollari in assistenza all'Ucraina. Alla fine, Putin non riuscirÃ  a spegnere l'amore dei popoli per la libertÃ ."}The custom MapReduceChainâ€‹Multi input promptYou can also use prompt with multi input. In this example, we will use a MapReduce chain to answer specific question about our code.from langchain.chains.combine_documents.map_reduce import MapReduceDocumentsChainfrom langchain.chains.combine_documents.stuff import StuffDocumentsChainmap_template_string = """Give the following python code information, generate a description that explains what the code does and also mention the time complexity.Code:{code}Return the the description in the following format:name of the function: description of the function"""reduce_template_string = """Given the following python function names and descriptions, answer the following question{code_description}Question: {question}Answer:"""# Prompt to use in map and reduce stages MAP_PROMPT = PromptTemplate(input_variables=["code"], template=map_template_string)REDUCE_PROMPT = PromptTemplate(input_variables=["code_description", "question"], template=reduce_template_string)# LLM to use in map and reduce stages llm = OpenAI()map_llm_chain = LLMChain(llm=llm, prompt=MAP_PROMPT)reduce_llm_chain = LLMChain(llm=llm, prompt=REDUCE_PROMPT)# Takes a list of documents and combines them into a single stringcombine_documents_chain = StuffDocumentsChain(    llm_chain=reduce_llm_chain,    document_variable_name="code_description",)# Combines and iteravely reduces the mapped documents reduce_documents_chain = ReduceDocumentsChain(        # This is final chain that is called.        combine_documents_chain=combine_documents_chain,        # If documents exceed context for


# If documents exceed context for `combine_documents_chain`        collapse_documents_chain=combine_documents_chain,        # The maximum number of tokens to group documents into        token_max=3000)# Combining documents by mapping a chain over them, then combining results with reduce chaincombine_documents = MapReduceDocumentsChain(    # Map chain    llm_chain=map_llm_chain,     # Reduce chain    reduce_documents_chain=reduce_documents_chain,    # The variable name in the llm_chain to put the documents in    document_variable_name="code",)map_reduce = MapReduceChain(    combine_documents_chain=combine_documents,    text_splitter=CharacterTextSplitter(separator="\n##\n", chunk_size=100, chunk_overlap=0),)code = """def bubblesort(list):   for iter_num in range(len(list)-1,0,-1):      for idx in range(iter_num):         if list[idx]>list[idx+1]:            temp = list[idx]            list[idx] = list[idx+1]            list[idx+1] = temp    return list##def insertion_sort(InputList):   for i in range(1, len(InputList)):      j = i-1      nxt_element = InputList[i]   while (InputList[j] > nxt_element) and (j >= 0):      InputList[j+1] = InputList[j]      j=j-1


= InputList[j]      j=j-1   InputList[j+1] = nxt_element   return InputList##def shellSort(input_list):   gap = len(input_list) // 2   while gap > 0:      for i in range(gap, len(input_list)):         temp = input_list[i]         j = i   while j >= gap and input_list[j - gap] > temp:      input_list[j] = input_list[j - gap]      j = j-gap      input_list[j] = temp   gap = gap//2   return input_list"""map_reduce.run(input_text=code, question="Which function has a better time complexity?")    Created a chunk of size 247, which is longer than the specified 100    Created a chunk of size 267, which is longer than the specified 100    'shellSort has a better time complexity than both bubblesort and insertion_sort, as it has a time complexity of O(n^2), while the other two have a time complexity of O(n^2).'The refine Chainâ€‹This sections shows results of using the refine Chain to do summarization.chain = load_summarize_chain(llm, chain_type="refine")chain.run(docs)    "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The


seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This investment will"Intermediate StepsWe can also return the intermediate steps for refine chains, should we want to inspect them. This is done with the return_refine_steps variable.chain = load_summarize_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True)chain({"input_documents": docs}, return_only_outputs=True)    {'refine_steps': [" In response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains.",      "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs,


gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. Putin's war on Ukraine has left Russia weaker and the rest of the world stronger, with the world uniting in support of democracy and peace.",      "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S.


contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This includes investing"],     'output_text': "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This includes investing"}Custom PromptsYou can also use your own prompts with this chain. In this example, we will respond in Italian.prompt_template = """Write a concise summary of the following:{text}CONCISE SUMMARY IN ITALIAN:"""PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])refine_template = (    "Your job is to produce a final summary\n"    "We


(    "Your job is to produce a final summary\n"    "We have provided an existing summary up to a certain point: {existing_answer}\n"    "We have the opportunity to refine the existing summary"    "(only if needed) with some more context below.\n"    "------------\n"    "{text}\n"    "------------\n"    "Given the new context, refine the original summary in Italian"    "If the context isn't useful, return the original summary.")refine_prompt = PromptTemplate(    input_variables=["existing_answer", "text"],    template=refine_template,)chain = load_summarize_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True, question_prompt=PROMPT, refine_prompt=refine_prompt)chain({"input_documents": docs}, return_only_outputs=True)    {'intermediate_steps': ["\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia e bloccando i suoi piÃ¹ grandi istituti bancari dal sistema finanziario internazionale. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi.",      "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma


sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi piÃ¹ grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo piÃ¹ di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare,",      "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi piÃ¹ grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo piÃ¹ di un miliardo di dollari


russi. Stiamo fornendo piÃ¹ di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare."],     'output_text': "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi piÃ¹ grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo piÃ¹ di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare."}PreviousSQLNextAdditionalCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Using OpenAI functions | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsHow toFoundationalDocumentsPopularAPI chainsRetrieval QAConversational Retrieval QAUsing OpenAI functionsSQLSummarizationAdditionalMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesChainsPopularUsing OpenAI functionsOn this pageUsing OpenAI functionsThis walkthrough demonstrates how to incorporate OpenAI function-calling API's in a chain. We'll go over: How to use functions to get structured outputs from ChatOpenAIHow to create a generic chain that uses (multiple) functionsHow to create a chain that actually executes the chosen functionfrom typing import Optionalfrom langchain.chains.openai_functions import (    create_openai_fn_chain,    create_structured_output_chain,)from langchain.chat_models import ChatOpenAIfrom langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.schema import HumanMessage, SystemMessageGetting structured outputsâ€‹We can take advantage of OpenAI functions to try and force the model to return a particular kind of structured output. We'll use the create_structured_output_chain to create our chain, which takes the desired structured output either as a Pydantic class or as JsonSchema.See here for relevant reference docs.Using Pydantic classesâ€‹When passing in Pydantic classes to structure our text, we need to make sure to have a docstring description for the class. It also helps to have descriptions for each of the classes attributes.from pydantic import BaseModel, Fieldclass Person(BaseModel):    """Identifying information about a person."""    name: str = Field(..., description="The person's name")    age: int = Field(..., description="The


description="The person's name")    age: int = Field(..., description="The person's age")    fav_food: Optional[str] = Field(None, description="The person's favorite food")# If we pass in a model explicitly, we need to make sure it supports the OpenAI function-calling API.llm = ChatOpenAI(model="gpt-4", temperature=0)prompt_msgs = [    SystemMessage(        content="You are a world class algorithm for extracting information in structured formats."    ),    HumanMessage(        content="Use the given format to extract information from the following input:"    ),    HumanMessagePromptTemplate.from_template("{input}"),    HumanMessage(content="Tips: Make sure to answer in the correct format"),]prompt = ChatPromptTemplate(messages=prompt_msgs)chain = create_structured_output_chain(Person, llm, prompt, verbose=True)chain.run("Sally is 13")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for extracting information in structured formats.    Human: Use the given format to extract information from the following input:    Human: Sally is 13    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': '_OutputFormatter', 'arguments': '{\n  "output": {\n    "name": "Sally",\n    "age": 13,\n    "fav_food": "Unknown"\n  }\n}'}}        > Finished chain.    Person(name='Sally', age=13,


> Finished chain.    Person(name='Sally', age=13, fav_food='Unknown')To extract arbitrarily many structured outputs of a given format, we can just create a wrapper Pydantic class  that takes a sequence of the original class.from typing import Sequenceclass People(BaseModel):    """Identifying information about all people in a text."""    people: Sequence[Person] = Field(..., description="The people in the text")chain = create_structured_output_chain(People, llm, prompt, verbose=True)chain.run(    "Sally is 13, Joey just turned 12 and loves spinach. Caroline is 10 years older than Sally, so she's 23.")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for extracting information in structured formats.    Human: Use the given format to extract information from the following input:    Human: Sally is 13, Joey just turned 12 and loves spinach. Caroline is 10 years older than Sally, so she's 23.    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': '_OutputFormatter', 'arguments': '{\n  "output": {\n    "people": [\n      {\n        "name": "Sally",\n        "age": 13,\n        "fav_food": ""\n      },\n      {\n        "name": "Joey",\n        "age": 12,\n


"age": 12,\n        "fav_food": "spinach"\n      },\n      {\n        "name": "Caroline",\n        "age": 23,\n        "fav_food": ""\n      }\n    ]\n  }\n}'}}        > Finished chain.    People(people=[Person(name='Sally', age=13, fav_food=''), Person(name='Joey', age=12, fav_food='spinach'), Person(name='Caroline', age=23, fav_food='')])Using JsonSchemaâ€‹We can also pass in JsonSchema instead of Pydantic classes to specify the desired structure. When we do this, our chain will output json corresponding to the properties described in the JsonSchema, instead of a Pydantic class.json_schema = {    "title": "Person",    "description": "Identifying information about a person.",    "type": "object",    "properties": {        "name": {"title": "Name", "description": "The person's name", "type": "string"},        "age": {"title": "Age", "description": "The person's age", "type": "integer"},        "fav_food": {            "title": "Fav Food",            "description": "The person's favorite food",            "type": "string",


"type": "string",        },    },    "required": ["name", "age"],}chain = create_structured_output_chain(json_schema, llm, prompt, verbose=True)chain.run("Sally is 13")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for extracting information in structured formats.    Human: Use the given format to extract information from the following input:    Human: Sally is 13    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': 'output_formatter', 'arguments': '{\n  "name": "Sally",\n  "age": 13\n}'}}        > Finished chain.    {'name': 'Sally', 'age': 13}Creating a generic OpenAI functions chainâ€‹To create a generic OpenAI functions chain, we can use the create_openai_fn_chain method. This is the same as create_structured_output_chain except that instead of taking a single output schema, it takes a sequence of function definitions.Functions can be passed in as:dicts conforming to OpenAI functions spec,Pydantic classes, in which case they should have docstring descriptions of the function they represent and descriptions for each of the parameters,Python functions, in which case they should have docstring descriptions of the function and args, along with type hints.See here for relevant reference docs.Using Pydantic classesâ€‹class RecordPerson(BaseModel):    """Record some identifying information about a pe."""    name: str = Field(..., description="The person's name")


name: str = Field(..., description="The person's name")    age: int = Field(..., description="The person's age")    fav_food: Optional[str] = Field(None, description="The person's favorite food")class RecordDog(BaseModel):    """Record some identifying information about a dog."""    name: str = Field(..., description="The dog's name")    color: str = Field(..., description="The dog's color")    fav_food: Optional[str] = Field(None, description="The dog's favorite food")prompt_msgs = [    SystemMessage(content="You are a world class algorithm for recording entities"),    HumanMessage(        content="Make calls to the relevant function to record the entities in the following input:"    ),    HumanMessagePromptTemplate.from_template("{input}"),    HumanMessage(content="Tips: Make sure to answer in the correct format"),]prompt = ChatPromptTemplate(messages=prompt_msgs)chain = create_openai_fn_chain([RecordPerson, RecordDog], llm, prompt, verbose=True)chain.run("Harry was a chubby brown beagle who loved chicken")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for recording entities    Human: Make calls to the relevant function to record the entities in the following input:    Human: Harry was a chubby brown beagle who loved chicken    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': 'RecordDog', 'arguments': '{\n  "name": "Harry",\n  "color": "brown",\n  "fav_food":


"Harry",\n  "color": "brown",\n  "fav_food": "chicken"\n}'}}        > Finished chain.    RecordDog(name='Harry', color='brown', fav_food='chicken')Using Python functionsâ€‹We can pass in functions as Pydantic classes, directly as OpenAI function dicts, or Python functions. To pass Python function in directly, we'll want to make sure our parameters have type hints, we have a docstring, and we use Google Python style docstrings to describe the parameters.NOTE: To use Python functions, make sure the function arguments are of primitive types (str, float, int, bool) or that they are Pydantic objects.class OptionalFavFood(BaseModel):    """Either a food or null."""    food: Optional[str] = Field(        None,        description="Either the name of a food or null. Should be null if the food isn't known.",    )def record_person(name: str, age: int, fav_food: OptionalFavFood) -> str:    """Record some basic identifying information about a person.    Args:        name: The person's name.        age: The person's age in years.        fav_food: An OptionalFavFood object that either contains the person's favorite food or a null value. Food should be null if it's not known.    """    return f"Recording person {name} of age {age} with favorite food {fav_food.food}!"chain = create_openai_fn_chain([record_person], llm, prompt, verbose=True)chain.run(    "The most important thing to remember about Tommy, my


verbose=True)chain.run(    "The most important thing to remember about Tommy, my 12 year old, is that he'll do anything for apple pie.")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for recording entities    Human: Make calls to the relevant function to record the entities in the following input:    Human: The most important thing to remember about Tommy, my 12 year old, is that he'll do anything for apple pie.    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': 'record_person', 'arguments': '{\n  "name": "Tommy",\n  "age": 12,\n  "fav_food": {\n    "food": "apple pie"\n  }\n}'}}        > Finished chain.    {'name': 'Tommy', 'age': 12, 'fav_food': {'food': 'apple pie'}}If we pass in multiple Python functions or OpenAI functions, then the returned output will be of the form{"name": "<<function_name>>", "arguments": {<<function_arguments>>}}def record_dog(name: str, color: str, fav_food: OptionalFavFood) -> str:    """Record some basic identifying information about a dog.    Args:        name: The dog's name.        color: The dog's color.        fav_food: An OptionalFavFood object that either contains the dog's favorite food or a null value. Food should be null if it's not known.    """    return f"Recording dog


be null if it's not known.    """    return f"Recording dog {name} of color {color} with favorite food {fav_food}!"chain = create_openai_fn_chain([record_person, record_dog], llm, prompt, verbose=True)chain.run(    "I can't find my dog Henry anywhere, he's a small brown beagle. Could you send a message about him?")            > Entering new LLMChain chain...    Prompt after formatting:    System: You are a world class algorithm for recording entities    Human: Make calls to the relevant function to record the entities in the following input:    Human: I can't find my dog Henry anywhere, he's a small brown beagle. Could you send a message about him?    Human: Tips: Make sure to answer in the correct format     {'function_call': {'name': 'record_dog', 'arguments': '{\n  "name": "Henry",\n  "color": "brown",\n  "fav_food": {\n    "food": null\n  }\n}'}}        > Finished chain.    {'name': 'record_dog',     'arguments': {'name': 'Henry', 'color': 'brown', 'fav_food': {'food': None}}}Other Chains using OpenAI  functionsâ€‹There are a number of more specific chains that use OpenAI functions.Extraction: very similar to structured output chain, intended for information/entity extraction specifically.Tagging: tag inputs.OpenAPI: take an OpenAPI spec and create + execute valid requests against the API, using OpenAI functions under the hood.QA with citations: use OpenAI functions ability to extract citations from text.PreviousConversational Retrieval


with citations: use OpenAI functions ability to extract citations from text.PreviousConversational Retrieval QANextSQLGetting structured outputsUsing Pydantic classesUsing JsonSchemaCreating a generic OpenAI functions chainUsing Pydantic classesUsing Python functionsOther Chains using OpenAI  functionsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Additional | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsHow toFoundationalDocumentsPopularAdditionalAnalyze DocumentSelf-critique chain with constitutional AICausal program-aided language (CPAL) chainElasticsearch databaseExtractionFLAREArangoDB QA chainGraph DB QA chainHugeGraph QA ChainKuzuQAChainNebulaGraphQAChainGraph QAGraphSparqlQAChainHypothetical Document EmbeddingsBash chainSelf-checking chainMath chainHTTP request chainSummarization checker chainLLM Symbolic MathModerationDynamically selecting from multiple promptsDynamically selecting from multiple retrieversNeptune Open Cypher QA ChainRetrieval QA using OpenAI functionsOpenAPI chainOpenAPI calls with OpenAI functionsProgram-aided language model (PAL) chainQuestion-Answering CitationsDocument QATaggingVector store-augmented text generationMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesChainsAdditionalAdditionalğŸ“„ï¸� Analyze DocumentThe AnalyzeDocumentChain can be used as an end-to-end to chain. This chain takes in a single document, splits it up, and then runs it through a CombineDocumentsChain.ğŸ“„ï¸� Self-critique chain with constitutional AIThe ConstitutionalChain is a chain that ensures the output of a language model adheres to a predefined set of constitutional principles. By incorporating specific rules and guidelines, the ConstitutionalChain filters and modifies the generated content to align with these principles, thus providing more controlled, ethical, and contextually appropriate responses. This mechanism helps maintain the integrity of the output while minimizing the risk of generating content that may violate guidelines, be offensive, or deviate from the desired


minimizing the risk of generating content that may violate guidelines, be offensive, or deviate from the desired context.ğŸ“„ï¸� Causal program-aided language (CPAL) chainThe CPAL chain builds on the recent PAL to stop LLM hallucination. The problem with the PAL approach is that it hallucinates on a math problem with a nested chain of dependence. The innovation here is that this new CPAL approach includes causal structure to fix hallucination.ğŸ“„ï¸� Elasticsearch databaseInteract with Elasticsearch analytics database via Langchain. This chain builds search queries via the Elasticsearch DSL API (filters and aggregations).ğŸ“„ï¸� ExtractionThe extraction chain uses the OpenAI functions parameter to specify a schema to extract entities from a document. This helps us make sure that the model outputs exactly the schema of entities and properties that we want, with their appropriate types.ğŸ“„ï¸� FLAREThis notebook is an implementation of Forward-Looking Active REtrieval augmented generation (FLARE).ğŸ“„ï¸� ArangoDB QA chainOpen In CollabğŸ“„ï¸� Graph DB QA chainThis notebook shows how to use LLMs to provide a natural language interface to a graph database you can query with the Cypher query language.ğŸ“„ï¸� HugeGraph QA ChainThis notebook shows how to use LLMs to provide a natural language interface to HugeGraph database.ğŸ“„ï¸� KuzuQAChainThis notebook shows how to use LLMs to provide a natural language interface to KÃ¹zu database.ğŸ“„ï¸� NebulaGraphQAChainThis notebook shows how to use LLMs to provide a natural language interface to NebulaGraph database.ğŸ“„ï¸� Graph QAThis notebook


to NebulaGraph database.ğŸ“„ï¸� Graph QAThis notebook goes over how to do question answering over a graph data structure.ğŸ“„ï¸� GraphSparqlQAChainGraph databases are an excellent choice for applications based on network-like models. To standardize the syntax and semantics of such graphs, the W3C recommends Semantic Web Technologies, cp. Semantic Web. SPARQL serves as a query language analogously to SQL or Cypher for these graphs. This notebook demonstrates the application of LLMs as a natural language interface to a graph database by generating SPARQL.\ğŸ“„ï¸� Hypothetical Document EmbeddingsThis notebook goes over how to use Hypothetical Document Embeddings (HyDE), as described in this paper.ğŸ“„ï¸� Bash chainThis notebook showcases using LLMs and a bash process to perform simple filesystem commands.ğŸ“„ï¸� Self-checking chainThis notebook showcases how to use LLMCheckerChain.ğŸ“„ï¸� Math chainThis notebook showcases using LLMs and Python REPLs to do complex word math problems.ğŸ“„ï¸� HTTP request chainUsing the request library to get HTML results from a URL and then an LLM to parse resultsğŸ“„ï¸� Summarization checker chainThis notebook shows some examples of LLMSummarizationCheckerChain in use with different types of texts.  It has a few distinct differences from the LLMCheckerChain, in that it doesn't have any assumptions to the format of the input text (or summary).ğŸ“„ï¸� LLM Symbolic MathThis notebook showcases using LLMs and Python to Solve Algebraic Equations. Under the hood is makes use of SymPy.ğŸ“„ï¸� ModerationThis


hood is makes use of SymPy.ğŸ“„ï¸� ModerationThis notebook walks through examples of how to use a moderation chain, and several common ways for doing so. Moderation chains are useful for detecting text that could be hateful, violent, etc. This can be useful to apply on both user input, but also on the output of a Language Model. Some API providers, like OpenAI, specifically prohibit you, or your end users, from generating some types of harmful content. To comply with this (and to just generally prevent your application from being harmful) you may often want to append a moderation chain to any LLMChains, in order to make sure any output the LLM generates is not harmful.ğŸ“„ï¸� Dynamically selecting from multiple promptsThis notebook demonstrates how to use the RouterChain paradigm to create a chain that dynamically selects the prompt to use for a given input. Specifically we show how to use the MultiPromptChain to create a question-answering chain that selects the prompt which is most relevant for a given question, and then answers the question using that prompt.ğŸ“„ï¸� Dynamically selecting from multiple retrieversThis notebook demonstrates how to use the RouterChain paradigm to create a chain that dynamically selects which Retrieval system to use. Specifically we show how to use the MultiRetrievalQAChain to create a question-answering chain that selects the retrieval QA chain which is most relevant for a given question, and then answers the question using it.ğŸ“„ï¸� Neptune Open Cypher QA ChainThis QA chain queries Neptune graph database using openCypher and returns human readable responseğŸ“„ï¸� Retrieval QA using OpenAI functionsOpenAI functions allows for structuring of response output. This is often useful in question answering when you want to not only get the final answer but also supporting evidence, citations,


in question answering when you want to not only get the final answer but also supporting evidence, citations, etc.ğŸ“„ï¸� OpenAPI chainThis notebook shows an example of using an OpenAPI chain to call an endpoint in natural language, and get back a response in natural language.ğŸ“„ï¸� OpenAPI calls with OpenAI functionsIn this notebook we'll show how to create a chain that automatically makes calls to an API based only on an OpenAPI  spec. Under the hood, we're parsing the OpenAPI spec into a JSON schema that the OpenAI functions API can handle. This allows ChatGPT to automatically select and populate the relevant API call to make for any user input. Using the output of ChatGPT we then make the actual API call, and return the result.ğŸ“„ï¸� Program-aided language model (PAL) chainImplements Program-Aided Language Models, as in https://arxiv.org/pdf/2211.10435.pdf.ğŸ“„ï¸� Question-Answering CitationsThis notebook shows how to use OpenAI functions ability to extract citations from text.ğŸ“„ï¸� Document QAHere we walk through how to use LangChain for question answering over a list of documents. Under the hood we'll be using our Document chains.ğŸ“„ï¸� TaggingThe tagging chain uses the OpenAI functions parameter to specify a schema to tag a document with. This helps us make sure that the model outputs exactly tags that we want, with their appropriate types.ğŸ“„ï¸� Vector store-augmented text generationThis notebook walks through how to use LangChain for text generation over a vector index. This is useful if we want to generate text that is able to draw from a large body of custom text, for example, generating blog posts that have an understanding of


draw from a large body of custom text, for example, generating blog posts that have an understanding of previous blog posts written, or product tutorials that can refer to product documentation.PreviousSummarizationNextAnalyze DocumentCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.