Package: promptflow
Prompt flowWelcome to join us to make prompt flow better by
participatingdiscussions,
openingissues,
submittingPRs.Prompt flowis a suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications, from ideation, prototyping, testing, evaluation to production deployment and monitoring. It makes prompt engineering much easier and enables you to build LLM apps with production quality.With prompt flow, you will be able to:Create and iteratively develop flowCreate executableflowsthat link LLMs, prompts, Python code and othertoolstogether.Debug and iterate your flows, especially theinteraction with LLMswith ease.Evaluate flow quality and performanceEvaluate your flow's quality and performance with larger datasets.Integrate the testing and evaluation into your CI/CD system to ensure quality of your flow.Streamlined development cycle for productionDeploy your flow to the serving platform you choose or integrate into your app's code base easily.(Optional but highly recommended) Collaborate with your team by leveraging the cloud version ofprompt flow in Azure AI.InstallationEnsure you have a python environment,python=3.9is recommended.pipinstallpromptflowpromptflow-toolsQuick Start ⚡Create a chatbot with prompt flowRun the command to initiate a prompt flow from a chat template, it creates folder namedmy_chatbotand generates required files within it:pfflowinit--flow./my_chatbot--typechatSetup a connection for your API keyFor OpenAI key, establish a connection by running the command, using theopenai.yamlfile in themy_chatbotfolder, which stores your OpenAI key:# Override keys with --set to avoid yaml file changespfconnectioncreate--file./my_chatbot/openai.yaml--setapi_key=<your_api_key>--nameopen_ai_connectionFor Azure OpenAI key, establish the connection by running the command, using theazure_openai.yamlfile:pfconnectioncreate--file./my_chatbot/azure_openai.yaml--setapi_key=<your_api_key>api_base=<your_api_base>--nameopen_ai_connectionChat with your flowIn themy_chatbotfolder, there's aflow.dag.yamlfile that outlines the flow, including inputs/outputs, nodes,  connection, and the LLM model, etcNote that in thechatnode, we're using a connection namedopen_ai_connection(specified inconnectionfield) and thegpt-35-turbomodel (specified indeployment_namefield). The deployment_name filed is to specify the OpenAI model, or the Azure OpenAI deployment resource.Interact with your chatbot by running: (pressCtrl + Cto end the session)pfflowtest--flow./my_chatbot--interactiveContinue to delve deeper intoprompt flow.Release History1.5.0 (2024.02.06)Features Added[SDK/CLI][azure] Support specify compute instance as session compute in run.yaml[SDK/CLI][azure] Stop support specifyingidle_time_before_shutdown_minutesfor automatic runtime since each session will be auto deleted after execution.Bugs Fixed[SDK/CLI] The inputs of node test allows the value of reference node output be passed directly in.[SDK/CLI][azure] Fixed bug for cloud batch run referencing registry flow with automatic runtime.[SDK/CLI] Fix "Without Import Data" in run visualize page when invalid JSON value exists in metrics.[SDK/CLI][azure] Fix azureml serving get UAI(user assigned identity) token failure bug.[SDK/CLI] Fix flow as function connection override when node has default variant.Improvements[SDK/CLI] Forpf run delete,pf connection delete, introducing an option to skip confirmation prompts.[SDK/CLI] Move pfs extra dependency to required dependency.1.4.0 (2024.01.22)Features Added[Executor] Calculate system_metrics recursively in api_calls.[Executor] Add flow root level api_calls, so that user can overview the aggregated metrics of a flow.[Executor] Add @trace decorator to make it possible to log traces for functions that are called by tools.[Tool] InputSetting of tool supports passing undefined configuration.[SDK/CLI][azure] Switch automatic runtime's session provision to system wait.[SDK/CLI] Add--skip-open-browseroption topf flow serveto skip opening browser.[SDK/CLI][azure] Support submit flow to sovereign cloud.[SDK/CLI] Supportpf run deleteto delete a run irreversibly.[SDK/CLI][azure] Automatically put requirements.txt to flow.dag.yaml if exists in flow snapshot.[SDK/CLI] Supportpf upgradeto upgrade prompt flow to the latest version.[SDK/CLI] Support env variables in yaml file.Bugs FixedFix unaligned inputs & outputs or pandas exception during get details against run in Azure.Fix loose flow path validation for run schema.Fix "Without Import Data" in run visualize page results from invalid JSON value (-Infinity,InfinityandNaN).Fix "ValueError: invalid width -1" when show-details against long column(s) in narrow terminal window.Fix invalid tool code generated when initializing the script tool with icon.Improvements[SDK/CLI] Forpfazure flow create:If used by non-msft tenant user, use user name instead of user object id in the remote flow folder path. (e.g.Users/<user-name>/promptflow).When flow has unknown attributes, log warning instead of raising error.Use local flow folder name and timestamp as the azure flow file share folder name.[SDK/CLI] Forpf/pfazure run create, when run has unknown attribute, log warning instead of raising error.Replacepyyamlwithruamel.yamlto adopt YAML 1.2 specification.1.3.0 (2023.12.27)Features Added[SDK/CLI] Supportpfazure run cancelto cancel a run on Azure AI.Add support to configure prompt flow home directory via environment variablePF_HOME_DIRECTORY.Please set before importingpromptflow, otherwise it won't take effect.[Executor] Handle KeyboardInterrupt in flow test so that the final state is Canceled.Bugs Fixed[SDK/CLI] Fix single node run doesn't work when consuming sub item of upstream nodeImprovementsChangeruamel.yamllower bound to 0.17.10.[SDK/CLI] Improvepfazure run downloadto handle large run data files.[Executor] Exit the process when all async tools are done or exceeded timeout after cancellation.1.2.0 (2023.12.14)Features Added[SDK/CLI] Supportpfazure run downloadto download run data from Azure AI.[SDK/CLI] Supportpf run createto create a local run record from downloaded run data.Bugs Fixed[SDK/CLI] Removing telemetry warning when running commands.Empty node stdout & stderr to avoid large visualize HTML.Hide unnecessary fields in run list for better readability.Fix bug that ignores timeout lines in batch run status summary.1.1.1 (2023.12.1)Bugs Fixed[SDK/CLI] Fix compatibility issue withsemantic-kernel==0.4.0.dev0andazure-ai-ml==1.12.0.[SDK/CLI] Add back workspace information in CLI telemetry.[SDK/CLI] Disable the feature to customize user agent in CLI to avoid changes on operation context.Fix openai metrics calculator to adapt openai v1.1.1.0 (2023.11.30)Features AddedAddpfazure flow show/listto show or list flows from Azure AI.Display node status in run visualize page graph view.Add support for image input and output in prompt flow.[SDK/CLI] SDK/CLI will collect telemetry by default, user can usepf config set telemetry.enabled=falseto opt out.Addraise_on_errorfor stream run API, by default we raise for failed run.Flow as function: consume a flow like a function with parameters mapped to flow inputs.Enable specifying the default output path for run.Usepf config set run.output_path=<output-path>to specify, and the run output path will be<output-path>/<run-name>.Introduce macro${flow_directory}forrun.output_pathin config, which will be replaced with corresponding flow directory.The flow directory cannot be set as run output path, which meanspf config set run.output_path='${flow_directory}'is invalid; but you can use child folder, e.g.pf config set run.output_path='${flow_directory}/.runs'.Support pfazure run create with remote flow.For remote workspace flow:pfazure run create --flow azureml:<flow-name>For remote registry flow:pfazure run create --flow azureml://registries/<registry-name>/models/<flow-name>/versions/<flow-version>Support set logging level via environment variablePF_LOGGING_LEVEL, valid values includesCRITICAL,ERROR,WARNING,INFO,DEBUG, default toINFO.Remove openai version restrictionsBugs Fixed[SDK/CLI] Fix node test with dict node input will raise "Required input(s) missing".[SDK/CLI] Will use run name as display name when display name not specified (used flow folder name before).[SDK/CLI] Fix pf flow build created unexpected layer of dist folder[SDK/CLI] Fix deploy prompt flow: connections value may be noneImprovementsForce 'az login' if using azureml connection provider in cli command.Add env variable 'PF_NO_INTERACTIVE_LOGIN' to disable interactive login if using azureml connection provider in promptflow sdk.Improved CLI invoke time.Bumppydashupper bound to 8.0.0.BumpSQLAlchemyupper bound to 3.0.0.Bumpflaskupper bound to 4.0.0,flask-restxupper bound to 2.0.0.Bumpruamel.yamlupper bound to 1.0.0.1.0.0 (2023.11.09)Features Added[Executor] Addenable_kwargstag in tools.json for customer python tool.[SDK/CLI] Supportpfazure flow create. Create a flow on Azure AI from local flow folder.[SDK/CLI] Changed column mapping${run.inputs.xx}'s behavior, it will refer to run's data columns instead of run's inputs columns.Bugs Fixed[SDK/CLI] Keep original format in run output.jsonl.[Executor] Fix the bug that raise an error when an aggregation node references a bypassed nodeImprovements[Executor] Set the outputs of the bypassed nodes as None0.1.0b8 (2023.10.26)Features Added[Executor] Add average execution time and estimated execution time to batch run logs[SDK/CLI] Supportpfazure run archive/restore/update.[SDK/CLI] Support custom strong type connection.[SDK/CLI] Enable telemetry and won't collect by default, usepf config set cli.telemetry_enabled=trueto opt in.[SDK/CLI] Exposed functionfrom promptflow import load_runto load run object from local YAML file.[Executor] SupportToolProviderfor script tools.Bugs Fixedpf config set:Fix bug for workspaceconnection.provider=azuremldoesn't work as expected.[SDK/CLI] Fix the bug that using sdk/cli to submit batch run did not display the log correctly.[SDK/CLI] Fix encoding issues when input is non-English withpf flow test.[Executor] Fix the bug can't read file containing "Private Use" unicode character.[SDK/CLI] Fix string type data will be converted to integer/float.[SDK/CLI] Remove the max rows limitation of loading data.[SDK/CLI] Fix the bug --set not taking effect when creating run from file.Improvements[SDK/CLI] Experience improvements inpf run visualizepage:Add column status.Support opening flow file by clicking run id.0.1.0b7.post1 (2023.09.28)Bug FixedFix extra dependency bug when importingpromptflowwithoutazure-ai-mlinstalled.0.1.0b7 (2023.09.27)Features Addedpf flow validate: support validate flowpf config set: support set user-level promptflow config.Support workspace connection provider, usage:pf config set connection.provider=azureml://subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace_name>Support override openai connection's model when submitting a flow. For example:pf run create --flow ./ --data ./data.jsonl --connection llm.model=xxx --column-mapping url='${data.url}'Bugs Fixed[Flow build] Fix flow build file name and environment variable name when connection name contains space.Reserve.promptflowfolder when dump run snapshot.Read/write log file with encoding specified.Avoid inconsistent error message when executor exits abnormally.Align inputs & outputs row number in case partial completed run will breakpfazure run show-details.Fix bug that failed to parse portal url for run data when the form is an asset id.Fix the issue of process hanging for a long time when running the batch run.Improvements[Executor][Internal] Improve error message with more details and actionable information.[SDK/CLI]pf/pfazure run show-details:Add--max-resultsoption to control the number of results to display.Add--all-resultsoption to display all results.Add validation for azurePFClientconstructor in case wrong parameter is passed.0.1.0b6 (2023.09.15)Features Added[promptflow][Feature] Store token metrics in run propertiesBugs FixedRefine error message body for flow_validator.pyRefine error message body for run_tracker.py[Executor][Internal] Add some unit test to improve code coverage of log/metric[SDK/CLI] Update portal link to remove flight.[Executor][Internal] Improve inputs mapping's error message.[API] Resolve warnings/errors of sphinx build0.1.0b5 (2023.09.08)Features Addedpf run visualize: support lineage graph & display name in visualize pageBugs FixedAdd missing requirementpsutilinsetup.py0.1.0b4 (2023.09.04)Features addedSupportpf flow buildcommands0.1.0b3 (2023.08.30)Minor bug fixes.0.1.0b2 (2023.08.29)First preview version with major CLI & SDK features.Features addedpf flow: init/test/serve/exportpf run: create/update/stream/list/show/show-details/show-metrics/visualize/archive/restore/exportpf connection: create/update/show/list/deleteAzure AI support:pfazure run: create/list/stream/show/show-details/show-metrics/visualize0.1.0b1 (2023.07.20)Stub version in Pypi.